EVALUATION OF THE MEDIA/MATERIALS CLEARINGHOUSE Elisa Knebel August 2004 Submitted by: LTG Associates, Inc. Social & Scientific Systems, Inc. Submitted to: The United States Agency for International Development Under USAID Contract No. HRN–C–00–00–00007–00 This document is available in printed or online versions (POPTECH Publication Number 2004– 186–026). To review and/or obtain a document online, see the POPTECH web site at www.poptechproject.com. Documents are also available through the Development Experience Clearinghouse (www.dec.org). Printed copies and additional information about this and other POPTECH publications may be obtained from: The Population Technical Assistance Project 1101 Vermont Avenue, NW, Suite 900 Washington, DC 20005 Telephone: (202) 898-9040 Fax: (202) 898-9057 admin@poptechproject.com Evaluation of the Media/Materials Clearinghouse was made possible through support provided by the United States Agency for International Development (USAID) under the terms of Contract Number HRN– C–00–00–00007–00, POPTECH Assignment Number 2004–186. The opinions expressed herein are those of the authors and do not necessarily reflect the views of USAID. ACKNOWLEDGMENTS The consultant would like to express her gratitude to the M/MC and HCP staffs, Melanie Kindsfather of POPTECH, survey questionnaire respondents, and those individuals who participated in the interviews. The evaluation benefited greatly from the time and effort contributed by these individuals. The consultant would especially like to thank the hard work and support offered by Mary Beda Andourou. Her assistance in survey design and administration greatly added to the findings for this report. ACRONYMS AED Academy for Educational Development CA Cooperating agency CDC Centers for Disease Control and Prevention CHID Combined Health Information Database DEC Development Experience Clearinghouse GH/PRH Bureau for Global Health, Office of Population and Reproductive Health HCMD Health Communication Materials Database HCMN Health Communication Materials Network HCP Health Communication Partnership HIPNet Health Information and Publications Network HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome IEC Information, education, and communication INFO Information and Knowledge for Optimal Health Project (JHU/CCP) IP Internet protocol JHU/CCP Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs M/MC Media/Materials Clearinghouse (JHU/CCP) NGO Nongovernmental organization NLM National Library of Medicine NPIN National Prevention Information Network (CDC) PHIL Public Health Image Library PIP Population Information Program POPLINE® Bibliographic database of family planning and related health and development issues (JHU/CCP) REPROLINE Reproductive Health Online (JHPIEGO) RH Gateway Reproductive Health Gateway (managed by INFO on behalf of HIPNet) STD Sexually transmitted disease UNAIDS The Joint United Nations Programme on HIV/AIDS UNICEF United Nations Children’s Fund UNIFEM United Nations Development Fund for Women USAID United States Agency for International Development WHO World Health Organization CONTENTS Page Executive Summary........................................................................................................ I I. Introduction...............................................................................................................1 Purpose of the Evaluation ...........................................................................................1 Overview of the Media/Materials Clearinghouse.......................................................1 Methodology...............................................................................................................2 II. Findings......................................................................................................................4 Use of the M/MC ........................................................................................................4 Efficiency of the M/MC............................................................................................10 User Satisfaction With the M/MC ............................................................................16 Utility of the M/MC..................................................................................................21 Conclusion ................................................................................................................24 Recommendations.....................................................................................................26 FIGURES 1. Geographic Distribution of M/MC Clients.................................................................4 2. Number of Searches per Month ..................................................................................5 3. Number of Distinct IP Addresses to Online Database................................................6 4. Request by Occupation and Region, 2002..................................................................7 5. Materials by Target Region and Health Topic, August 2004 ...................................18 6. Number of Materials Catalogued and Added to the Web Site..................................20 TABLES 1. Top Referring Sites to the M/MC by Frequency ........................................................9 2. User Satisfaction Measures.......................................................................................17 3. The 20 Most Popular Search Terms by Frequency of Search ..................................18 APPENDICES A. Scope of Work B. Survey and Responses of Users C. Staff Interview Guide D. Donor Interview Guide E. Persons Interviewed F. Documents Reviewed EXECUTIVE SUMMARY OVERVIEW This report contains an external assessment of the Media/Materials Clearinghouse (M/MC) for the United States Agency for International Development (USAID), Bureau for Global Health, Office of Population and Reproductive Health (GH/PRH). The M/MC is based at the Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs (JHU/CCP), in Baltimore, Maryland. Since 2002, the M/MC has been managed by the Health Communication Partnership (HCP), which is held by JHU/CCP and its partners, the Academy for Educational Development (AED), Save the Children Federation, Inc., the International HIV/AIDS Alliance, and Tulane University’s School of Public Health and Tropical Medicine. The mandate of the M/MC is to collect project materials, such as films, videos, posters, pamphlets, and other items designed to convey health messages, and to share these widely with the developing country health communication community, including USAID, cooperating agencies (CAs), and international donors. Funding for the M/MC from USAID comes under USAID Population and Reproductive Health Strategic Objective 1. The purpose and objectives of this assessment, as noted in the scope of work (see appendix A) are to ƒ better understand the scope and usefulness of the M/MC’s collection of information and ƒ explore how the M/MC can best serve the mandate of HCP. The assessment examines the achievements and challenges of the M/MC in terms of its use, efficiency, satisfaction, and utility: ƒ Use: amount or extent to which the M/MC is accessed and contributed to by users ƒ Efficiency: the use of resources in providing the M/MC, whether the financial model provides for ongoing operations as well as growth, and whether the M/MC provides any duplication with any other services or organizations ƒ Satisfaction: how well the services and outputs of the M/MC satisfy users ƒ Utility: how well the M/MC met the needs of users, USAID, and HCP i The consultant reviewed the web-based M/MC information as well as key documents capturing historical data and usage statistics. Telephone and onsite interviews were conducted regarding the areas noted above. In total, 32 individuals were interviewed, which included USAID, M/MC, and HCP staffs; frequent users; partners; and donors. Additionally, a survey was conducted, of which 194 health communication professionals responded, 139 users and 54 nonusers (1 nonresponse on use). The following section summarizes the assessment’s key findings and conclusions and the principal recommendations for the future. SUMMARY OF FINDINGS Use ƒ Survey respondents report high use of the M/MC. ƒ There is a steady number of submissions from CAs, international organizations, and others. Users report a desire to see that their items have been posted to the database. ƒ Use of the M/MC’s online database has remained stagnant since January 2004 and likely decreased since mid−2003 (this figure is an estimate because web statistics are not reliable for this period). Evidence suggests that the lack of promotion, a weak brand/identity, and a hard-to-find web presence have contributed to this decline. Current web use does not match previous use of the M/MC when it was primarily a paper-based operation. Efficiency ƒ The M/MC does not have sufficient funds to support mailing items to country resource centers nor individuals without charging them for the materials. It also does not have the funds to support its virtual presence nor its online community, the Health Communication Materials Network (HCMN). ƒ The M/MC is providing a unique collection and service to the health communication community. No other site is as comprehensive and accessible. Health communicators see the M/MC as the only place to provide a long-term, accessible place for preserving their projects’ materials past the duration of the often short-term timeframe of projects. ƒ Without accurate statistics on the number of users and use of the materials, it is impossible to calculate cost-effectiveness. Satisfaction ƒ Satisfaction with the quality, quantity, relevance, topics, and languages of the materials is extremely high among current users of the M/MC. ƒ Satisfaction with the customized information solutions provided by staff and the overall responsiveness of staff is very high. M/MC’s ability to connect professionals to enable building collaborative partnerships is appreciated. ƒ Users are equally interested in the topics of HIV/AIDS and reproductive health, and the make up of the materials collection reflects this. ii Utility ƒ Through the regular and extensive application by HCP staff, the M/MC is extremely valuable in supporting HCP’s mission to develop the capacity of health communicators. However, M/MC’s role as part of HCP’s larger strategy is not clearly articulated in the HCP framework, nor are there measures by which to judge its performance. ƒ The M/MC is effectively serving USAID’s mission of harnessing the development experience by actively creating partnerships with other organizations and individuals and making this experience widely accessible through the use of web-based tools. ƒ The materials are used in their original form for immediate use or are adapted when users design their own materials. Saving time and a desire not to redo work that has already been done were the two most frequently cited reasons for accessing the collection. ƒ The M/MC wishes to complement its comprehensive collection with a more dynamic presentation of materials to include lessons learned about their design, production, and impact, while also maintaining the most up-to-date and relevant collection possible. SUMMARY OF RECOMMENDATIONS Use ƒ The M/MC should explore adding a registration page to capture background information when users download items from the site and/or a shopping cart feature or cookie system in order to track which materials are downloaded. This will facilitate knowing the extent of use and which information and under which formats it is accessed. ƒ The M/MC needs to develop a marketing and branding strategy. Part of this process requires obtaining an address that is easy for users to remember. There is a need to remind former users and inform new users of the M/MC’s services and the added value of the materials to their work. ƒ A new, easy-to-complete online submission form with an option for sending electronic file attachments as well as a revised paper submission form are needed. This will stimulate regular submissions and up-to-date collection. ƒ Incorporating accountability language on the submission of health communication materials to the M/MC into CAs’ cooperative agreements and contracts would support exhaustive submission of materials. iii Efficiency ƒ The M/MC needs financial support to ensure the continuity of its existing programs. ƒ The M/MC should investigate other CAs’ experiences in recovering costs from charges for materials or annual organizational membership fees for the use of the database. ƒ In light of the need to reach those individuals who are not connected to the Internet, funding should be delegated to providing shipments of samples to resource centers that are likely to have the widest reach in sharing the materials in the field. Satisfaction ƒ The M/MC should establish a feedback system to continually monitor satisfaction with the materials and the web site as well as the relevance of the collection to users’ needs. ƒ The M/MC should clearly articulate criteria for the submission of materials. Utility ƒ HCP management needs to clearly articulate how the M/MC fits into its strategic framework and assist in the development of indicators for monitoring. ƒ The M/MC should invest in actively engaging the HCMN and HCP staffs so that they become the channels through which to identify the lessons learned and best practices of its collection. The M/MC needs a paid, high-level leader/facilitator who will provide intensive support for HCMN. ƒ The M/MC should merge the HCMN site and the M/MC database as one to create a hub around the shared knowledge and community of practice. Forming partnerships with other similar sites or organizations should be explored to mitigate any redundancies. ƒ USAID/Washington and CA staffs should promote the products and resources of the M/MC when working with USAID Missions and in-country partners. Monitoring and Evaluation ƒ The M/MC needs to develop a monitoring and evaluation strategy that regularly monitors content, use, utility, and satisfaction, collecting only data specifically related to the mission of HCP and only that data that will be analyzed and reported. The M/MC should obtain whatever is necessary to allow for the automatic generation of accurate web usage statistics to its pages. iv ƒ In light of current staffing, at a minimum, the librarian should keep a monthly request log with the most essential information in tabular format and enter these data into the computer at the end of each month. v I. INTRODUCTION PURPOSE OF THE EVALUATION The purpose and objectives of this assessment, as noted in the scope of work (see appendix A) are to ƒ better understand the scope and usefulness of the Media/Materials Clearinghouse’s (M/MC) collection of information and ƒ explore how the M/MC can best serve the mandate of the Health Communication Partnership (HCP). OVERVIEW OF THE MEDIA/MATERIALS CLEARINGHOUSE Since 1983, the mission of the M/MC has been to improve the quality of health communication programs in developing countries by collecting and sharing examples of health communication materials. Such materials include, but are not limited to, pamphlets, posters, audiotapes, films, images, photos, documents, and job aids. By systematically housing these materials in one area, the M/MC intends users to have ready access to the wealth of experience so as to be more efficient in their work. The M/MC’s main audience includes communication specialists, writers, marketing staff, publishers, trainers, graphic designers, and photographers. From 1997 until 2002, the U.S. Agency for International Development’s (USAID) Office of Population and Reproductive Health in the Bureau for Global Health (GH/PRH) funded the M/MC under the Population Information Program (PIP). Since 2002, USAID has funded the clearinghouse under the bureauwide HCP. The M/MC received additional funding in the last year of the Centers for Disease Control and Prevention (CDC) contract and the CORE Initiative. Until the late 1990s, the M/MC’s work primarily consisted of sending printed materials via mail in response to users’ orders from a printed catalogue. Via mail, the M/MC also regularly distributed the What’s New newsletter that documented the latest editions to the collection and a PCS Packet Series containing actual sample materials and guidelines for developing materials. As the Internet and other technology became more accessible in the late 1990s, M/MC’s work shifted its attention to managing what is known as the Health Communication Materials Database (HCMD), which contains over 44,000 samples of health communication materials, many of which can be downloaded free of charge in PDF and other digital formats. The Internet has proven to be more cost-effective than using regular mail for the M/MC, and audiences now have a better opportunity to see what is available in the collection by viewing the online images on the site. Additionally, the M/MC has also shifted its catalogue to CD−ROM, and the print newsletter has been converted into a regular e-mail announcement. The M/MC’s other primary activity is the hosting of a Health Communication Materials Network (HCMN), which serves the community of 1 developers and designers of health materials. Additionally, the M/MC performs extensive networking services by connecting users and developers of materials. M/MC’s primary focus area is reproductive health, but the collection has evolved to include dozens of distinct topic areas, including HIV/AIDS, environmental health, infectious disease, and child survival. The Health Communication Partnership, held by the Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs (JHU/CCP) and its partners, the Academy for Educational Development (AED), Save the Children Federation, Inc., the International HIV/AIDS Alliance, and Tulane University’s School of Public Health and Tropical Medicine, currently manages the M/MC. METHODOLOGY Data and information upon which the M/MC assessment is based came from three data collection methods: a web-based survey, individual and group interviews conducted either in person or by telephone, and a record review. Web-Based Survey An e-mail was sent to approximately 29,485 health communication professionals who subscribe to the list serves of the Health Information and Publications Network (HIPNet), HCMN, and The Communication Initiative’s Drum Beat. Both M/MC and USAID staffs suggested the list serves. This was not a random sample but rather a way to comprehensively target the universe of individuals working in health communication. The e-mail included the web address of the survey questionnaire and a background letter about the survey. The Drum Beat invitation was shorter and less detailed than the original letter that was e-mailed. Additionally, a pop-up window was created on the M/MC site to invite users to take the online survey. The Communication Initiative also posted a banner on its web page with a brief description and link to the survey. The staff at The Communication Initiative also sent out the original invitation letter to several thousand members it identified as a “niche audience.” The web-based survey was created using a service that permits subscribers to create their own questionnaires. The survey consisted of 24 items based on questions in the scope of work. A copy of the online survey and related results appears in appendix B. The survey was completed by 194 individuals. Responses were received from people in 42 countries. Respondents included program officers, students, directors of organizations, program associates, consultants, nurses, trainers, policy advisers, and several other positions. The breakdown by type of institution was 27.5 percent associated with international NGOs, 22.2 percent with local nongovernmental organizations, and 17.5 percent with educational institutions. The remaining responses were received from individuals from bilateral/multilateral donor agencies, the commercial sector, and national/local governments as well as individuals not affiliated with any organization. 2 Interviews Interview guides were developed for both staff and donors that were based on the topics featured in the scope of work. Because not all respondents were knowledgeable about all aspects covered in the guides, not all respondents were asked the same number of questions. Interviews typically lasted about 30 minutes; several were longer, depending on the respondent’s experience with the M/MC. (Appendix C contains the staff interview guide and appendix D contains the donor interview guide.) Between August 15 and September 2, 2004, the consultant completed 32 interviews. These included 8 respondents from USAID/Washington, 10 staff from M/MC and HCP in Baltimore, 7 from partner/funding organizations, and 7 users. The names of individuals proposed for interviews were derived from suggestions from the M/MC and USAID. (Appendix E contains the persons interviewed.) Record Review The consultant conducted a record review of related M/MC and USAID documents and publications related to the last 3 years of its functioning. These included historical documentation records, client usage records, web tracking statistics, the M/MC web site, and other major health information sites. (A list of documents and publications is included in appendix F.) 3 II. FINDINGS USE OF THE M/MC The following section explores the users of M/MC, which information is being accessed, and the extent to which the M/MC is being used, both online and physically, throughout the health communication community. This use includes both requests and submissions. Overall, when asked the extent to which they accessed the M/MC when designing health communication materials, 61.2 percent of 193 respondents reported that they either always (11.4 percent), regularly (25.4 percent), or occasionally (24.4 percent) use the M/MC. Among the 54 respondents (28 percent) who never use the M/MC when designing health communications materials, a majority (64.2 percent) reported that it was because they did not know about it. Of the 21 respondents (11 percent) who reported that they rarely use the M/MC when designing health communications materials, 8 cited that they use other sources of materials, the remaining 4 reported that they forget about the M/MC, and the remaining 9 do not design materials as part of their jobs. These findings are difficult to relate to web usage and request data. Under the PIP project, the M/MC kept very detailed records of every request, visits to the web site, and visits to the Baltimore office. This was partly facilitated by a full-time staff of 11 and a regularly maintained request database. Such statistics are available for the period until mid−2003. Figure 1, from the final PIP report, provides the most recent and available estimate for the geographic distribution of users (note that the data sources used to calculate this estimate are unknown). Figure 1 Geographic Distribution of M/MC Clients (April 1, 1997 to March 31, 2003) Europe + USA 5% Asia 20% Africa 35% Eastern Europe 10% Near East 15% Latin America 15% 4 After mid−2003, the M/MC’s monitoring and evaluation of usage, both physical and online, declined. In light of staffing reductions, staff members report that cataloguing and fulfilling requests and their desire to be service oriented took precedent over tracking the actual services provided. Additionally, as part of joining the HCP, the web pages of the M/MC were incorporated under the broader HCP web site, making it difficult to gather usage of the M/MC online database without writing specialized code to do so. Sensing that such statistics were important to capture, the M/MC did ask an Internet systems developer in January 2004 to write specialized tracking queries so that some basic data could be gleaned separately from the larger HCP site. Online Usage No reliable data are available on the use of the M/MC web site for the last three years. Data collection procedures for the site changed in mid−2003, when M/MC was placed under HCP. The incompatibility of the current collection procedure with the previous one makes any observations on use over time questionable. However, one can see a steep decline in the number of searches conducted per month on the online database since January 2004. Figure 2 Number of Searches per Month (January to June 2004) 8,593 4,423 1,461 2,643 4,981 4,333 0 2,000 4,000 6,000 8,000 10,000 January February March April May June Month N u m ber of Searches For the period that data are available, the number of distinct Internet protocol (IP) addresses accessing the M/MC’s online database per month has remained the same, at an average of 960 (see figure 3 on the following page). An IP address is a numeric address that is given to servers and computers connected to the Internet and may represent a unique person, but often many people share one IP address, especially in the developing world. Therefore, there may be a higher number of individuals visiting the site than the number of users counted. There is no way to know whether this is higher or lower than the past, as this measure was not previously monitored. 5 Figure 3 Number of Distinct IP Addresses to Online Database (January to June 2004) 939 1,059 981 968 831 983 0 200 400 600 800 1,000 1,200 January February March April May June Month N u m ber of I P A d dresses It is important to note that without a login feature or cookie system in place, it is impossible to know exactly who is using information from the M/MC, the exact number of users, the country of origin, and the formats and content that are being downloaded. Offline Usage At its peak funding in 2000, the M/MC was distributing nearly 200,000 materials a year, which included specialized subject bibliographies, M/MC Packet Series, What’s New at the M/MC, CD−ROMs, videos, videoshare catalogues, and sample materials. These outputs were in response to demand. By the end of the Population Information Program (PIP), for example, requesters had ordered over 10,500 copies of the M/MC’s Mini M/MC: A Selection of Media for Health Communication. For the Condom CD−ROM, which had 292 posters, 111 pamphlets, and 600 promotional items related to condoms, there were over 2,000 requests. For every specialized collection on CD−ROM, the M/MC could not keep enough in stock. Data from 2000 show that there were 2,140 requests from users. Since 2000, in light of the expansion of the Internet, the M/MC has made efforts to convert its collection to digital format and reduce the physical delivery of materials. This has prompted an associated decline in requests. In 2002, there were 16,441 items mailed to requesters, dropping to 3,700 in 2003. Regardless of the decline, staff estimates that the distribution of the requests would be about the same in terms of occupation and region as in 2002 (see figure 4 on the following page). Without a monitoring system in place, the percentage of use that is online or offline cannot be discerned. Additionally, the level of past offline use that has been recaptured through current online use cannot be estimated. Several of the interviewees and 14 survey respondents expressed concern over the move to being completely virtual. Ten survey respondents commented that the lack of sample materials and/or the lack of rural resource 6 centers where physical materials could be obtained were what they least liked about the M/MC. Four respondents reported that they wished the M/MC would offer additional physical materials. Figure 4 Request by Occupation and Region, 2002 Policymakers 8% Educators/ Trainers 10% Health Professionals 30% Communications Professionals 21% Others 31% Near East 5% North Africa America 21% 27% Asia 9% Latin America 6% Europe/NIS 1% International 31% 7 Submissions by Users The M/MC receives hundreds of materials a month. It no longer actively solicits materials from non−HCP projects. Regardless, unsolicited materials are constantly being submitted. In the past, special mailings and visits to cooperating agencies (CAs) were made to ensure the exhaustive collecting of their materials, but this outreach is now being performed by occasional solicitations to the HCMN list serve. Staff acknowledges that submissions could be strengthened with additional outreach. In interviews with health communication professionals, all developers stressed that they submit their health materials to the M/MC and believed that their colleagues did as well. Many see the M/MC as the only place to provide a long-term, accessible location for preserving their projects’ materials past the duration of the often short-term timeframe of projects. Another motivation cited was their desire to share their work with the wider health communication community; they looked to the M/MC to act as the portal for this function. Based on interviews with HCP senior management and program officers, all members of the HCP know to submit their materials to the M/MC. The M/MC has longstanding relationships, some informal and some formulated in memoranda of understanding, with other CAs, donor agencies, and foundations to ensure that all their materials are sent to the M/MC. All the communication materials from the United Nations Children’s Fund (UNICEF), The Joint United Nations Programme on HIV/AIDS (UNAIDS), and CARE are automatically submitted. Many of the mini M/MCs (field-based resource centers that have sought M/MC’s assistance in development) send their locally developed materials to the Baltimore M/MC, which then puts these materials in the larger collection for universal use. The submission form for users is not online nor does it have fields to prompt for background or evaluation information about the materials. Awareness of M/MC Although awareness is not an indicator of use, the lack of awareness of the M/MC seems to be one of the reasons why the use of the M/MC is in decline. Thirty-four out of the 54 respondents (62.9 percent) who had never accessed the M/MC did not do so because they did not know about it. Of the 21 respondents who reported that they rarely used the M/MC, 4 stated that it was because they just forgot about it. Prior to becoming part of HCP, the M/MC sent out a regular paper-based, What’s New in the M/MC, as well as targeted brochures for audiences with specific needs (e.g., a brochure listing key HIV/AIDS materials). Since joining HCP, it has not engaged in any advertising or promotion for fear of not being able to meet the demand of its users for physical samples of materials. Additionally, the online database was never formally marketed to the health community. This is evident in the low number of links to the site from other health organization sites. 8 When typing in www.jhuccp.org/mmc and http://www.hcpartnership.org/mmc/ in LinkPopularity.com and filtering for any duplicate links from the same organization, there were only 35 distinct sites linking to the M/MC. Even among those users who access the M/MC site, the vast majority of users come to the M/MC site by either going to the JHU/CCP or the CORE Initiative sites first (see table 1). Based on interview responses, this indicates two possibilities: some users associate the M/MC with JHU/CCP and visit the JHU/CCP site to look for the database, or some users are at the JHU/CCP site already and stumble upon the M/MC site while there. The fact that a search engine is not a top referrer indicates that users are not looking for “M/MC” in a search engine. Again, based on interview responses, this supports the idea that neither “M/MC” nor “HCMD” is a name that resonates with users. Table 1 Top Referring Sites to the M/MC by Frequency January to June 2004 Web Site/Page User Sessions http://www.jhuccp.org/ 243 http://www.aed.org/health/health_publications.html 60 http://www.usaid.gov/cgi-bin/goodbye?http://www.coreinitiative.org/index.php 47 http://www.aed.org/ 34 http://www.jhuccp.org/pressroom/2003/05-27.shtml 30 http://www.jhuccp.org/jobs/ 19 http://www.jhuccp.org 19 http://www.coreinitiative.org/index.cfm 18 http://www.jhuccp.org/index.shtml 17 http://mail.bagan.net.mm/cgi-bin/openwebmail/openwebmail￾read.pl?action=readmessage&firstmessage=1&sessionid=care21-session￾0.859083038129942&status=V&folder=INBOX&sort=date&keyword=&searchtype =subject&headers=simple&message_id=%3C200308292204.h7TM4rZ2116 16 http://www.jhuccp.org/pressroom/2003/09-04.shtml 14 http://www.jhuccp.org/quality/ 13 http://www.jhuccp.org/admin/web_admin/ 13 http://www.jhuccp.org/pressroom/ 13 http://www.jhuccp.org/asia/contacts.shtml 12 http://www.jhuccp.org/pressroom/2003/05-28.shtml 12 http://coreinitiative.org/core.php?sp=evaluation_core 10 http://www.jhuccp.org/sp/index.shtml 9 http://search.yahoo.com/search?p=usaid+faith+core&ei=UTF-8&fr=fp-tab-web￾t&n=20&fl=0&x=wrt 9 http://www.jhuccp.org/topics/heath_com.shtml 9 In the online survey, respondents also provided information on awareness. When asked how they found out about the M/MC, 31.4 percent reported that it was from a colleague, 30.8 percent through a link from another site, and 26.5 percent reported that they found out about the M/MC via a newsletter (respondents were free to choose more than one answer). Only 11.9 percent found M/MC through a search engine. The M/MC knows that by advertising, usage and requests will skyrocket. For example, requests to the M/MC rose by 25 percent in response to a brief mention in Population Reports about the availability of materials on ending violence against women. 9 Respondents’ Comments on Awareness of the M/MC I would say that my colleagues probably don’t use the M/MC that much. Mostly because they are busy but also because they probably don’t know about it. I cannot really recall the M/MC ever advertising itself so probably many people don’t know anything about it or what it can do for them. They really need a marketing campaign to clarify the mission, what they do and don’t do, how the web works, and how and why people can submit their materials. Director, Nongovernmental Organization (NGO) I would like to see the M/MC be more autonomous. It does not have its own URL. It is now difficult to find on HCP site. I would like to see it more like a clearinghouse with its own distinct identity. Librarian, CA I think the M/MC is really under recognized, probably because of our past and present reluctance to publicize our work. We never had a communications strategy, partly because we were afraid that we could not meet the demand if we ever aggressively advertised. We are constantly flooded for requests for TA [technical assistance] and materials. In the last year, the M/MC has been hiding for fear of disappointing people. Staff, M/MC Frankly, I did not even realize what the M/MC was even though I was using it. I thought that this was a Hopkins center. I am not sure how recognized the name “M/MC” is in the field. I think people associate these materials with JHU. Field Officer, USAID Frankly, the M/MC is just too humble. They need to profile themselves more, build up more of a name in the regions, and position themselves as the leader in KM [knowledge management] that they are currently performing. Project Manager, CA EFFICIENCY OF THE M/MC The following explores the use of USAID resources in providing the M/MC, whether the financial model provides for ongoing operations as well as growth, and whether adequate stewardship is provided on behalf of donors, beneficiaries, and partners. This section documents the current partnership activities in which the M/MC is engaging. Whether the M/MC provides any duplication with any other services or organizations that provide competing or complementary information is also explored. Current USAID Funding When the M/MC was awarded in the first year of HCP, a budget of $134,000 was set. Staff reports that this figure was set based upon a misreading of the request for application that USAID was no longer interested in the M/MC. By the second year, this had been clarified by USAID, and an accurate budget was agreed upon that would support basic dissemination of materials at an effective level ($437,000). This budget was then reduced by $50,000 after the second year had begun, resulting in a projected deficit of $50,000 for the beginning of the third year. Thus, third year costs presented in the HCP planning phase were $495,000 (the agreed-upon figure of $437,000, $8,000 for inflation, and $50,000 to eliminate the deficit from the unplanned second year decrease). This $495,000 annual budget (less then half the peak M/MC budget under PIP) was to enable the M/MC to maintain its unique leadership role in providing sample health communication materials to the public health community, both online and physically. Approximately 75 percent of these costs would cover 3.5 full-time staff, and the remaining 25 percent would provide them with the means to disseminate sample materials and provide services. However, the budget was set at $338,970. 10 Use of USAID Funding The current budget covers three full-time staff, materials, supplies, and other personnel costs (e.g., labels, shelves, hiring time of information technology specialists for database and web maintenance). The breakdown of services involved in the M/MC’s operations and the associated staff time follow. ƒ Library Assistant (75 percent) z Filing and Duplication: Organizes, labels, and files all items that are submitted in the physical collection. Refiles all items that are loaned or displayed. When the inventory of materials is low, duplicates items. z Fulfillment: For requests that can pay for shipping, duplicates and mails materials, either by sea or DHL. ƒ Assistant Librarian (100 percent) z Cataloguing: Creates database record that includes type of material, originating region, title, producer, dimensions in centimeters, physical description and background summary of the related project, contact information, date of publication, target audience, subject, language, abstract, and distributor. Assigns a list of key words to each item. (These terms are standardized with POPLINE®.) Often, items arrive with no associated background or information, which involves calling or writing the producer and probing for more information. If the item is in a foreign language, translates this information or seeks translation assistance. On average, it takes about 30−40 minutes to catalogue each item. z Converting to Digital Form: Once the librarian decides that an item should be converted to digital form, must secure rights from copyright holders for permission to copy and convert materials to digital form for nonprofit uses. z Technical Assistance: Offers regular library tours to visitors and creates material displays according to their needs. Also conducts specialized searches for materials specific to users’ needs. ƒ Librarian (50 percent) (position vacant since July 9, 2004) z Cataloguing: The librarian manages the M/MC online and onsite catalogue database through classifying and writing abstracts for new materials, ensuring quality control of database records, making collection development decisions, and maintaining a subject heading thesaurus and authority files related to the database. Also manages intraorganization library and web projects, responds to reference requests from health 11 professionals worldwide, and provides training in database searching and management and general orientations to the collection and web resources. ƒ Chief (75 percent) z Management: Supervises staff; sets policies for distribution; guides web site, database, and list serve development; elaborates work plans; monitors budget; and manages content. z Technical Assistance: Answers specialized requests for materials specific to donors’ needs and acts as a broker for less developed countries’ networking of users of materials with distributors and developers of materials. z Capacity Building: Provides technical assistance in media/materials development and resource center management to various countries on call. (The M/MC receives USAID Mission and other organizational funding to perform this work for the initial launch of resource center development. However, this funding is typically used for one-time development costs and has not factored in the recurrent costs of providing ongoing support in answering questions regarding clearinghouse maintenance and implementation, which are numerous. USAID core funds are used for this to be responsive to USAID Mission needs.) z Promotion and Partnership Building: When in the field and at key dissemination events, executes all promotion and partnership development activities to expand the reach of and funding for the M/MC. An employee from the Information and Knowledge for Optimal Health (INFO) project provides the leadership for the HCMN. In return, the M/MC assists INFO with assistance on Photoshare, which it built and managed under PIP. Partnerships To Increase Efficiency and Support Operations The M/MC regularly seeks partnership opportunities wherever possible. The M/MC has created successful partnerships with several organizations—other sectors of USAID as well as international organizations and foundations working inside and outside reproductive health—to gain additional resources and to incorporate new materials into its collection. Partnerships To Create New Tools and Expand the Use of Materials Currently, the CORE Initiative has contracted with the M/MC to develop a specialized database of selected tools for the needs of its community and faith-based organization users. Through the partnership, CORE showcases the M/MC database through its own site, bringing in new users from faith-based organizations and agencies working in HIV/AIDS. 12 Recently, the World Bank contracted with the M/MC to produce an HIV/AIDS Media for Youth CD−ROM, of which 5,000 copies were disseminated at the Bangkok World AIDS Conference. The World Bank approached the M/MC because it heard that the M/MC had the largest collection of such materials at its immediate disposal. In 2000, the Packard Foundation contracted with the M/MC ($27,000) to distribute 10,000 kits of postabortion care communication materials throughout Africa. The Packard Foundation thought that the M/MC was best poised to duplicate, store, and distribute the materials as the M/MC’s mailing list already included individuals and organizations for marketing purposes. The M/MC has performed similar services for many USAID projects. Partnerships To Build Institutional Capacity In a review conducted by USAID in 1993, the M/MC was asked to decentralize its operations to in-country organizations so that they could disseminate, use, and share their materials at the country level. The M/MC has made considerable efforts to use its extensive expertise in designing, planning, and establishing media resource centers and to transfer this knowledge to those wishing to set up and operate their own centers. M/MC staff has assisted in developing country health resource centers in Bolivia, Egypt, Ethiopia, Ghana, Haiti, India, Jordan, Nigeria, the Philippines, Rwanda, and Turkey, and has also assisted resource center managers in 24 other countries. A noteworthy example is the CDC−funded Ethiopian AIDS Resource Center, which serves as a repository of print and audiovisual materials and informational resources on HIV/AIDS, sexually transmitted diseases (STDs) and tuberculosis prevention for Ethiopian journalists, politicians, religious leaders, community leaders, and the general public. The M/MC is currently planning centers in Namibia and Zambia. The assistance provided includes database design, setting up processes for publication distribution and information distribution networks, and designing Internet services. The M/MC also offers 1 and 2−week study tours for database designers, librarians, project managers, and others by request. Participants from 13 countries in Africa, Latin America, and Asia have come to Baltimore to learn resource center management. Resource centers in Egypt and Jordan are in the process of translating the M/MC database into Arabic for their use. Partnerships To Make the Collection Comprehensive and Up To Date As noted above, the M/MC has longstanding relationships, some informal and some formulated in memoranda of understanding, with other CAs, donor agencies, and foundations to ensure that all their materials are sent to the M/MC. These include longstanding relationships with UNICEF, UNAIDS, and CARE. A recently completed contract (January 1998 to July 2004) was with Analytical Sciences, Inc. (now the Constella Group), funded by CDC, to develop and manage the National Prevention Information Network (NPIN)’s Educational Materials Database, a reference service for people and organizations working to prevent HIV/AIDS, STDs, tuberculosis, and other diseases. As part of this contract, the M/MC received $200,000 per year over six years to design, collect, and manage the processes needed to support the collection. 13 Through the collaboration, the M/MC catalogued thousands of materials for NPIN, and in the process, catalogued duplicate copies for its own online database and physical library. This enabled visitors to Baltimore to use the M/MC as a dedicated NPIN library during the life of the contract. The partnership ended in 2004 when the Constella Group was no longer eligible to renew the contract. In the process, however, the M/MC acquired over 12,000 HIV/AIDS materials and helped them distribute over 10,000 HIV/AIDS materials to USAID. Similarly, in 2000, the United Nations Development Fund for Women (UNIFEM) contracted with the M/MC to create a web site on violence against women. UNIFEM wanted to use the already large collection of violence against women materials that M/MC had and M/MC’s mechanisms for soliciting, cataloguing, creating digital materials, and storing materials. Additionally, UNIFEM estimated that the M/MC would provide better exposure to the site through its stream of regular users. The M/MC set up the “End Violence Against Women” web site (www.endvaw.org), adding new materials to its own collection as they came in from UNIFEM, which now total 1,924. Other Cost-Recovery Options The M/MC has explored other cost recovery options, including charging for materials, but has found that it is challenging for overseas agencies to pay for items. Using credit cards over the Internet is not feasible for their target users, and sending money orders with differing currencies requires too many resources. Staff reports that the volume would have to be very high to justify the administration involved in cost recovery and setting up a budget. Additionally, the issue of customs duties and levies once items arrive in-country is difficult to control. Using the data above, it is impossible to determine the cost-effectiveness (e.g., per user reached) and allocation of USAID funding per year per user. Again, without a login function, there is no true estimate of the number of users, and without steady data collection on the extent of use and what users are doing with the materials, it is impossible to quantify the reach. Additionally, there is an inherent pass-on readership of the materials that it is difficult to quantify without extensive monitoring and evaluation. One past JHU/CCP study showed that each Packet Series produced was passed on to nine people. Pass-on viewing of some videos can be over 1,000,000 people if the video is shown on national television stations. Staff report distributing items that have been adapted from country to country and videos that were developed for one organization that were then seen by an entire country. Extent of Duplication With Other Programs/Sites As supported by interview and survey responses as well as an exhaustive search on the web, the M/MC offers a truly unique collection and is one of the few sites offering health communication professionals actual free accessible materials for their work. M/MC staff has made special efforts over the years not to duplicate the efforts of others but to offer only useful and relevant materials for the specific target audience of health communicators. As such, the M/MC has not added such items as technical reports or data sets to its database. 14 There are two smaller databases that systematically capture health materials but these are both primarily focused on materials produced in the United States: the Public Health Image Library (PHIL), which houses only CDC’s materials; and the Combined Health Information Database (CHID), which houses materials produced by federal agencies and includes documents, data sets, guidelines, and other noncommunication materials. The M/MC is the only database capturing the health communication materials of USAID−funded CAs. Although USAID’s Development Experience Clearinghouse (DEC) systematically collects, catalogues, and shares some of the reports and documents associated with its projects, it does not collect or provide access to any of the communication materials that are produced as part of its projects worldwide. The M/MC’s library and database are the only places where USAID−funded communication materials, dating back to the late 1980s, can be accessed. DEC currently has no plans to start collecting such materials. Although there are several sites cataloguing reproductive health information, there are no sites systematically cataloguing reproductive health communication materials. The USAID−funded POPLINE provides scientific articles, reports, books, and academic papers on reproductive health. The USAID−funded Reproductive Health Online (REPROLINE) provides tools and information for reproductive health trainers, and the Reproductive Health Gateway (RH Gateway) acts as a portal to various links for all those working in reproductive health. In HIV/AIDS, another comparable site would be the CDC’s NPIN. Although the M/MC has always had reproductive health as its primary focus, through its partnerships with UNAIDS, NPIN, the CORE Initiative, and the World Health Organization (WHO), the M/MC now has amassed the largest accessible collection of international HIV/AIDS information, education, and communication (IEC) materials, featuring free samples from the earliest days of HIV prevention in the 1980s to the present. The USAID−funded Synergy Project, which manages the online database of HIV/AIDS technical information, does not offer health communications materials. These findings are supported by survey responses. When asked how the M/MC compared with other sources for health communications materials, 87 percent (100) of respondents either rated the M/MC as the best (32.2 percent) or better than other sources (54.8 percent). Comments of respondents when probed for comparisons with other sites are provided on the following page. The National Library of Medicine (NLM) houses one of the world’s largest history of medicine collections. Its mandate is archival. The M/MC has a relationship with the NLM to send outdated materials to be archived for posterity, although such shipments are irregular due to the lack of time to purge the collection. 15 Comments on M/MC’s Comparative Advantage The M/MC is the first place I go to for such materials, though there is Lovelife, Smooth City, and UNAIDS that have other HIV/AIDS stuff, but it is not easy to access and they don’t have very many images. Project Manager, CA The M/MC is our primary database, as it is the most comprehensive. We also use the NPIN and Healthlink. But M/MC is the best of all the digital libraries. Librarian, Resource Center Sometimes, I look at CDC, Johns Hopkins, UCLA, UNAIDS, and SIDA NET for information but the M/MC is the best and most comprehensive for our needs. Librarian, Resource Center I cannot think of another site like the M/MC. Maybe CRIPS in Paris but that is in French. Academic, University I don’t use any other site because I don’t know of any site that offers the depth and breadth of materials. Director, Communications Center I cannot think of any other site that functions like the M/MC. I have seen other sites with specialized collections but you would really have to go from site to site to find what you are looking for. Librarian, CA There are a number of other sites in the region that have sample materials and I believe that people use UNICEF as well. However, I don’t think these have as many USAID−funded materials. USAID Field Officer I cannot think of another site like the M/MC. There is not another clearinghouse. Sometimes I go to Video Fair Trust for videos but that is about it. Director, Center for Communication I don’t know of any other site or place like the M/MC. Nobody else is doing it. Sure, there are plenty of sites with theory and tools but a place where you can see the actual materials does not exist. I want to see real materials that people have developed. The M/MC is really facilitating some heavy duty South-to-South exchanging of materials. Project Manager, CA USER SATISFACTION WITH THE M/MC This section addresses the satisfaction of users with the quality of the materials and the relevance of the materials to users’ jobs. This includes whether the make up of the collection in terms of topics and languages meets users’ needs and whether the materials are perceived as credible to users. Relevance of the Materials The majority of survey respondents either agreed or strongly agreed that the M/MC has an up-to-date collection of health materials, has a highly useful collection of materials, offers materials in the language they need, and offers materials in their topic of interest (see table 2 on the following page for findings). 16 Table 2 User Satisfaction Measures Strongly Disagree Disagree Neither Disagree nor Agree Agree Strongly Agree N/A Question % n % n % n % n % n % n Response Average M/MC has an up-to-date collection of health communication materials. 4 4 4 5 6 7 40 46 41 47 4 5 4.17 M/MC has a highly useful collection of materials. 4 5 0 0 4 4 35 39 55 62 2 2 4.39 M/MC offers materials in the language that I need. 4 5 4 5 11 12 33 38 41 47 6 7 4.09 M/MC offers materials in my topics of interest. 4 5 1 1 6 7 32 36 53 59 4 4 4.32 Total Respondents 115 (No. who skipped this question) 79 Additionally, when asked what they most liked about the M/MC, 21 respondents reported that they appreciated the variety of topics offered. Figure 5 on the following page illustrates the distribution of materials by target country and health topic. The distribution of topics in the collection, with reproductive health at 40 percent and HIV/AIDS at 34 percent, reflects the audience’s current interests— according to the survey respondents, there is almost an equal interest in HIV/AIDS (81.7 percent) and reproductive health (82.6 percent). These are also the top two search subjects on the M/MC’s online database. Table 3 on the following page shows the 20 most popular search terms as of August 25, 2004, emphasizing the growing popularity of HIV/AIDS. Among survey respondents, the most popular format that users are seeking is training materials and manuals (78.4 percent), followed by brochures (65.5 percent), posters (59.5 percent), and videos (48.3 percent). This interesting trend needs to be explored further. When asked what more the M/MC should offer, five survey respondents listed HIV/AIDS. Other health topics cited included advocacy, elderly, gender, communicable disease, sanitation, and drug abuse. Suggestions for formats of materials included training manuals, job aids, electronic devices, and CD−ROMs. 17 Figure 5 Materials by Target Region and Health Topic August 2004 Asia 17% North America NIS 30% 1% LAC 14% Near East 6% Africa 23% Europe 9% Infectious Disease Child Survival 9% 2% Reproductive Health and Population 31% HIV/AIDS and STDs 27% Maternal Health 7% Environmental Health 24% Table 3 The 20 Most Popular Search Terms by Frequency of Search (August 21, 2003 to August 25, 2004) 18 AIDS 215 Condom 185 HIV Prevention 180 JHU/CCP 173 Family Planning 168 JHU/PCS 168 Child Health 146 Enter−Educate 145 Logos 142 HIV Testing 135 Campaigns 134 Adolescents 133 Communication 132 Condom Use 130 Female Genital Mutilation 128 Contraceptive Methods 127 Sexually Transmitted Disease Prevention 126 Prenatal Care 126 Maternal Health 122 Family Planning Benefits 120 Quality of the Materials Among survey respondents, over 45 percent strongly agreed and 35 percent agreed that the M/MC materials had been tested in the field, indicating a high level of trust and perception of quality of the materials. Although evaluation and testing of pieces was important to survey respondents, background information on the project from which the materials originated and the intended audience for the pieces was rated nearly the same. According to the interviews, there is a wide consensus that few health communication materials are evaluated in isolation. It is often rare to find evaluations of entire media campaigns. No person interviewed could recall a material being evaluated in isolation and did not expect the M/MC to limit its collection to only evaluated materials. Interviewees cited that “word of mouth” about an item’s impact as often the best evaluation respondents could hope for in terms of judging quality. Staff reports that all materials produced under the Population Communication Services or HCP projects were pretested with the intended audience. Whenever an evaluation report about a material or the project under which it was produced is available, a link is made between the record of the material in the HCMD and the report. When users do submit items for cataloguing at the M/MC, the M/MC immediately follows up with an evaluation form that asks more information about how the item was used, whether it was evaluated, and what the results of its use were. The M/MC is dependent upon people’s willingness to contribute this information and whether they documented this information. In the past, the M/MC would catalogue everything that arrived at the office and try to create digital versions as much as possible. In the desire to focus on quality rather than quantity, the M/MC has begun to implement criteria for converting materials to a digital format. A material has to be innovative, replicable, and/or precedent setting. Innovative would be a format that has never been used before; replicable is something that even settings with limited resources could replicate; and precedent setting is a different variation on an old topic or unique health message. Although the M/MC is trying to be more selective about what is posted on the online database, those who submit materials are very keen to see their items posted on the site. If the M/MC does not post an item, staff members fear that people will stop sending materials. This is an issue that the staff has not been able to address, and as such, has loosely interpreted the above criteria. To provide a better idea of quality, an “Add your comments” feature was recently added to every item in digital format on the M/MC web site. The idea was taken from the very successful Amazon web site as a way of engaging users and collecting feedback on the quality of materials, but it is too early to determine its effectiveness. 19 If a material’s message is deemed irrelevant, the material is supposed to be sent to archival resources at NLM, although as stated earlier, the manpower to regularly purge the collection has not been available. As figure 6 illustrates, the number of items catalogued and for which digital versions were created since 2000 has steadily declined, due to staffing shortages and stricter requirements about what should be catalogued. Figure 6 Number of Materials Catalogued and Added to the Web Site (2000 to June 2004) 2,376 2,131 1,228 872 62 758 4,581 3,736 5,582 1,700 0 1,000 2,000 3,000 4,000 5,000 6,000 2000 2001 2002 2003 2004 Year N u m ber of Materials Number of Materials Catalogued Number of Materials Added to the Web Site Quality of the Service Respondents from both interviews and the survey stressed that it was important to note that the M/MC offers a real service component that is not advertised. When respondents were asked whether they thought that the M/MC offered excellent customer service, 71 percent agreed or strongly agreed. Twelve people cited the staff as what they liked most about the M/MC. Numerous comments were offered by interviewees on the helpful service provided by staff regarding specialized queries and technical assistance. Comments on Service Quality of the M/MC Staff at the M/MC helped us design the whole site and taught us how to design systems for cataloguing and acquisition. They were amazingly competent and professional. So tremendously helpful. President, Communications Consulting Firm The staff there are extremely helpful. It is not just a matter of them just handing you some materials. They really think about your needs and try to connect you with others that might be of service. It is a huge undertaking what they do—acquisitions, cataloguing, labeling, uploading—really a lot of manpower is needed to do this. Manager, NGO I had to make a video and did not know how. I came to the M/MC and they sat with me, found sample for me, had materials waiting for me…They are top service. You cannot find that kind of support from anyone else. Project Officer, CA I am highly impressed by the responsiveness and courtesy of the staff. MMC is a gem! Trainer 20 UTILITY OF THE M/MC The following section explores whether the M/MC supports the goals of HCP and USAID and how and to what extent the collection and its services are accessible and provide immediate knowledge to perform effectively. Utility of the M/MC for Users Overall, 78 percent either agreed (40 percent) or strongly agreed (38 percent) that the M/MC helps them work better. Survey respondents and interviewees cited four main ways they work with the materials of the M/MC: directly using materials as they are to distribute to target populations, adapting materials or taking certain elements for new ideas in creating their own, featuring materials in training sessions, and studying them as a reference when learning about health communications. Many users order and download materials to share information and health messages with target populations. The range of uses includes ordering a wall chart for a clinic, distributing brochures to churches, distributing items in an anti−AIDS campaign, and showing videos to youth. Some of these users report having to perform the role of being health communicators but having neither the time nor the resources to design their own materials. Others cite having to perform health communication duties although it is not their primary job (e.g., doctor at a clinic). They look to the M/MC for ready-to-use materials that they know have been used with some success in other settings. Use of M/MC’s Materials Without Adaptation I am a nurse in India. My mother tongue is different from that of the people who I care for so most of the time a descriptive poster will come in handy to me. Most of the time, I modify the wordings with the help of the staff who know the people’s language, as even though I can understand people and talk to them, I am not fluent enough. I can only make up with my passion and visual aids. Nurse I work with dozens of small NGOs and CBOs [community-based organizations]. They don’t have the capacity to design their own materials and are just desperate for things to distribute to their communities. They, for example, order physical brochures from the M/MC as is and just place stickers over the addresses written at the bottom and add their own address. These kind of groups would never be able to design things on their own, even with an example. Project Manager, CA A number of clients use the M/MC materials as is with little modification. They would never be able to develop their own. These are small NGOs who want to serve their users but just don’t have the time, resources, or ability to sit down and design communication materials. You can take a limited USAID project in Mexico, for example, and take the same materials and leverage these for use in Guatemala, Bolivia, etc. It is an efficient use of resources. Director, CA My library largely serves students and journalists looking for health communication materials. The students are looking for materials to use for their presentations. Some health workers want materials for their clinic. Librarian, Resource Center Our center largely serves journalists, students, and professional schools for nurses and doctors. They use materials to develop their own projects, prevention activities, and conference presentations. They largely use the materials as is without modifying them. Librarian, Resource Center Printing PDFs or small images from the site is not useful for the groups I work with. They need ready-made materials to get the messages out. Project Manager, NGO 21 From the survey, the majority of respondents (72.3 percent) reported that they used the M/MC for ideas in designing new communication materials. Respondents from both the survey and the interviews cited that learning from another project’s experience and seeing the actual products can save a great deal of time and expense in project planning and can help them work more efficiently. Use of M/MC’s Materials in Design and Development I received several brochures from M/MC and identified messages and pictures I could use in Guinea. As a consequence, we have developed some of the best brochures for youth on HIV, abstinence and unwanted pregnancy. It was the same for maternal care material. Material Developer I show [materials] to MOH or NGO colleagues in other countries to give ideas of what other places have developed; gather ideas of how to illustrate difficult concepts (e.g., rapid or difficult breathing). Developed malaria, nutrition, and diarrhea materials in several countries using M/MC materials to give ideas. M/MC resources contributed new ideas to increase creativity in our teams and to confirm that what we were developing was on the right track. We developed in Kenya, for example, materials that led to a statistically significant improvement in malaria treatment knowledge and practice. In Benin, we used radio serial drama cassettes we had gotten from the M/MC to give us ideas as we developed theatre and radio programs that contributed to increases in knowledge and practice in family planning, malaria, and maternal health. Material Designer I often refer to the materials at the M/MC to see what has already been produced in the field. It gives me a good idea of what has been done and gives me ideas for the materials I need to create. Material Designer I think the M/MC serves a critical function to the health communication community. There are always new ideas, new formats developed, and it is important to have a center that keeps abreast of these innovations and makes them easily accessible like the M/MC does. Manager, NGO There are many poor materials produced; many who do not pretest. Looking at the M/MC before beginning any design should really be a requirement. Director, Communications Center I know of so many instances where people downloaded materials from the M/MC and then created their own. Sometimes I come across entire campaigns that are almost exact replicas of something they found on the M/MC. They knew it has been pretested and could not afford to do that themselves. Staff, M/MC Apart from the training manuals featured on the site, trainers cite the DVDs, videos, and CD−ROMs as the most useful in their work. Students and academics access the M/MC as part of their studies of health communication. Several academics cited the M/MC as the source to which they send their students and assign course work around the collection. Utility of the M/MC for HCP Staff As the use of the M/MC by HCP is not systematically captured, it is difficult to quantify the extent of its use among staff, although through informal observations and interviewees, it appears to be very high. Both Baltimore and field-based HCP staffs report that they routinely make use of the M/MC to aid them in a broad spectrum of technical assistance activities for governments and other clients. As each HCP client has different needs in terms of language, health topic, or type of campaign, staff members see the M/MC as a complete knowledge repository where they can find any type of material that they would need to customize solutions for their clients. During the proposal stage of projects, staff reports accessing the collection to review work that has been done by HCP, JHU/CCP, and others to identify trends and best practices. Staff members also frequently examine materials of past programs that have 22 proven successful and incorporate or adapt these in new projects. Staff members also access the M/MC collection to obtain materials for presentations, workshops, and other training activities, both in Baltimore and abroad. Of the documented exchanges that are available, illustrative examples include the M/MC providing support to HCP program staff in identifying and accessing birthspacing materials in preparation for a programmatic review and visit by an assessment team in January and March 2004. Another example of HCP staff accessing the M/MC was to find materials to aid in the development of logos and graphics for the Sports for Life project. In training, the M/MC recently provided various sample materials for the creation of youth HIV/AIDS training curricula for Haiti. Through personal and professional contacts, staff and partners regularly refer other health communication professionals to the M/MC. The many visitors who come from abroad, both from HCP and partner field offices and other organizations, are routinely provided with an orientation to the M/MC and its services. In this regard, the M/MC presents the public image for HCP. Supporting a Community of Practice Among Health Communicators USAID’s development experience and knowledge resides not only in documents and databases, but also in individuals. As exemplified in USAID’s knowledge management strategy, one method to tap into such knowledge is to facilitate a community of practice that enables individuals to share best practices and lessons learned, thus reducing the duplication of efforts and stimulating the creation of partnerships for future collaboration. The M/MC is supporting a community of practice, the Health Communication Materials Network (HCMN), a virtual organization of nearly 700 health communication professionals from 65 countries who share knowledge they have about how to develop, pretest, and use communication materials. These individuals come from a number of USAID partners, NGOs, and grantees as well as others outside USAID worldwide. HCMN is facilitated through an e-mail list serve as well as a web site, www.hcmn.org. HCMN members receive e-mails about the latest materials catalogued by the M/MC— including their own—complete with the names and addresses of the producers and a detailed description and image of the item. Colleagues often use the list serve to request help from their colleagues in pretesting their products and to share success stories in the field. In addition to facilitating the network, M/MC staff responds to many requests by referring one local organization to another that is doing similar work, thus fostering collaboration. The fact that it has attracted so many subscribers with no promotion indicates that there is some unrealized potential for expansion. 23 Future Directions Interviewees and survey respondents expressed a desire for M/MC to include theory, lessons learned, and history around the materials so that users could learn more about the effort that went into designing choice materials and the impact of the materials in the field. Several expressed a need to highlight selected materials as part of larger campaigns. People especially wanted to know more about failures so that mistakes are not repeated. The M/MC did previously provide this role under the INFO project when it produced thematic What’s New guides, the Packet Series of how-to topics, thematic CD−ROMs, and training guides for health communicators. Its recent short-term contracts with the World Bank, the Discovery Channel, UNIFEM, and others demonstrate that it has the experience to gather its collection together in highly usable formats for immediate on-the￾job use. The M/MC would like to continue this work for the entire M/MC audience by showcasing a best practices collection on the site and providing additional information about the campaigns that featured them by presenting an overview of the campaign, including the theoretical framework, showcasing the entire set of materials for a campaign along with related evaluations and background notes. Staff sees its role shifting from a physical content provider to an online knowledge manager serving the health communication community, specifically the members of the HCMN. A number of staff and interviewees expressed worries, however, about the potentially negative impact of the M/MC’s increasing emphasis on online products and services, including the potential for the ƒ decline in the availability of printed materials, especially the decline in the very popular how-to tools that were produced; ƒ neglect of populations who currently lack easy access or are not responsive to new digital technology; and ƒ neglect of populations that would like to use the materials as they are but lack a printer to print the electronic files. (Comments on future directions for the M/MC appear on the following page.) CONCLUSION This evaluation reveals that through the extensive use by HCP staff, the M/MC is extremely valuable in supporting HCP’s mission to develop the capacity of health communicators. The evaluation also shows that the M/MC is effectively serving USAID’s mission of harnessing the development experience and collective memory of the field by actively establishing partnerships with other organizations and individuals and making this experience widely accessible though the use of web-based tools. Users take advantage of this collection to avoid mistakes of the past and gain efficiencies in performing their own 24 work. Additionally, the M/MC serves as a model in systematically cataloguing and organizing materials for easy retrieval, as exemplified by the number of field-based organizations that seek M/MC’s assistance in developing their capacity in this area. Comments on Future Directions for the M/MC I would also like that the M/MC include more history, lessons learned, and information on the application of the materials. It would also be useful to have a search function that shows which materials are high resolution. Librarian, CA It would be useful to see the materials as part of a large campaign. Perhaps the M/MC could provide links to the whole set and background information on the campaign? Academic If the M/MC is scaling back, it would be interesting for them to catalogue the pieces in a “best of” collection and make the materials a thematic set. It might be interesting if they could review the materials and filter them into sets with more background information around how they were developed. Director, NGO There are a number of organizations that are entering HIV/AIDS prevention that don’t have the competencies to design health communication materials or messages. Ideally, these organizations should get materials along with how to use them, lessons learned in their application, and even failed health communication materials and why they failed. This incorporates the M/MC materials with intensive TA for those who often enter this field unequipped. Manager, NGO Regarding the training of future health communicators, I wish the M/MC had more video streaming capability so that the students could see some sample films. I understand, though, that this is hard and the technology might not be there yet. Professor, Health Communications If we migrate to an all digital approach, the developing world is going to be left behind. I get handwritten letters from people all the time. They don’t have e-mail. Some people are lucky enough to visit the center, but there are very few. Staff, M/MC Though it might seem like the Internet is spreading rapidly, very few people in the field have Internet access. You still need to get the physical materials to people. The idea that all these people can be serviced by “mini M/MCs” is not realistic, as there are so few. These “mini M/MCs” need the Baltimore M/MC to even start and then function over time. Manager, NGO Satisfaction is extremely high among current users of the M/MC with the quality, quantity, relevance, topics, and languages of the materials; the customized information solutions that staff provides; and M/MC’s ability to connect professionals to build collaborative partnerships. Users also report that the M/MC is providing a unique collection and service to the health communication community. However, use of the M/MC is declining. Use of the M/MC’s online database has decreased among health communication professionals, with the number of searches dropping since January 2004 and the number of users stagnating. Evidence suggests that the lack of promotion, a weak brand/identity, and a hard-to-find web presence has contributed to this decline. The M/MC would like to complement its exhaustive collection with a more dynamic presentation of materials to include lessons learned about their design, production, and impact. This vision supports disseminating the most useful knowledge and places increased emphasis on partnerships and results, which reinforces USAID’s own 25 commitment to do a better job of harnessing past experience instead of redoing what has already been done. The M/MC has the leadership, relevant knowledge base, and comparative advantage to achieve this vision. It also has the collection, cataloguing, and storage systems in place. Additionally, it has a dynamic community of health communicatorsthe HCMNwhich is already actively contributing to and supporting the development and expansion of this knowledge base. The M/MC faces constraints, however, in achieving this vision. No clearly articulated strategic plan has been elaborated, and M/MC’s role as part of HCP’s larger strategy is not clearly established in the HCP framework. The vision does not have any explicit, measurable indicators, without which the contribution of materials to HCP’s mandate cannot be assessed nor monitoring processes put in place to support such measurement. The M/MC does not have the funded staff to support this endeavor. This includes funding for a dedicated web/database manager as well as a dedicated facilitator for the HCMN. Although the M/MC will continue to migrate toward a virtual system as much as possible and assist in the development of field-based resource centers, there is a large percentage of M/MC’s target users that are not connected to the Internet or to a reliable printer. There is still a need to reach this population with the same knowledge base but through physical means. RECOMMENDATIONS Use ƒ The M/MC should explore adding a registration page to capture background information when users download items of the site and/or a shopping cart feature or cookie system in order to track which materials are downloaded. This will facilitate knowing the extent of use and which information and under which formats it is accessed. ƒ The M/MC needs to develop a marketing and branding strategy. Part of this process requires obtaining a distinct address that is easy for users to remember. A marketing campaign is needed to remind former users and inform new users of M/MC’s services and the value that is added to their work. The M/MC needs to clearly articulate the services it provides and does not provide. ƒ A new, easy-to-complete online form, with an option for sending electronic file attachments, and a revised paper submission form are needed. This will stimulate regular submissions and up-to-date collection. ƒ Incorporating accountability language on the submission of health communication materials to the M/MC into CAs’ cooperative agreements and contracts would support exhaustive submission of materials. 26 Efficiency ƒ The M/MC needs a financial strategy to ensure the continuity of its existing programs. ƒ The M/MC should investigate HIPNet and other CAs’ experiences in recovering costs from charges for materials or annual organizational membership fees for the use of the database. ƒ In light of the need to reach those individuals not connected to the Internet, funding should be delegated to providing shipments of samples to resource centers, which are likely to have the widest reach in sharing the materials in the field. Satisfaction ƒ The M/MC should establish a feedback system to continually monitor satisfaction with the materials, the web, and the relevance of the collection to users’ needs. ƒ The M/MC should clearly articulate criteria for submission of material. ƒ Online and paper submission forms (see above section, Use) with fields for background and evaluation information are needed that clearly articulate criteria for acceptance. Communicating these criteria to users would mitigate any expectations they might have for what is posted on the online database. Obtaining additional background information will contribute toward developing a wider knowledge base around the collection. Utility ƒ HCP management needs to clearly articulate how the M/MC fits into its strategic framework and to assist in the development of indicators to monitor this. ƒ The M/MC should invest in actively engaging the HCMN and HCP staffs so that they become the channels through which the lessons learned and best practices of its collection are selected. ƒ The M/MC needs a paid high-level leader/facilitator who will provide intensive support for HCMN. ƒ The M/MC should merge the HCMN site and the M/MC database so that a health communication hub is created around the shared knowledge. Creating partnerships with HealtheCommunication, Photoshare, REPROLINE, and INFO should be explored to mitigate any redundancies. To provide people true lessons about a campaign and lessons learned, such assets should be housed in one area. 27 ƒ USAID/Washington and CA staffs should promote the products and resources of the M/MC when dealing with USAID Missions and in-country partners. Monitoring and Evaluation ƒ The M/MC needs to develop a monitoring and evaluation strategy that regularly monitors content, use, utility, and satisfaction, collecting only that data that produces measures specifically related to the mission of HCP and only that data that will be analyzed and reported. The M/MC should obtain whatever software or programming is necessary to allow for the automatic generation of accurate web usage statistics. The following includes measures that should be collected: z Use: Distinct number of IP addresses to site (number of unique visits), source of referrals to site, most downloaded items from site, total number of page requests per month, requester by region and type of organization, and number of onsite visitors. z Satisfaction: Most popular search terms; distribution of materials by topic; number of materials catalogued per year; number of materials put into digital format per year; and number of materials discarded and submitted. z Utility: Set a target number of case studies to illustrate the application of M/MC materials in developing communication capacity. This could be greatly facilitated by HCP management requiring staff to outline how the use of M/MC materials helped them carry out key activities as part of the writing of progress, annual, or end-of-activity reports. Additionally, collect data from the HCMN on how they have used the M/MC and HCMN, documenting any added value on the job, assistance in facilitating problem solving, and/or the extent to which new partnerships have been made as a result of use. z Satisfaction: The new “Add your comments” feature placed next to each of the materials on the web site is a great first step toward engaging users and collecting feedback. This should be promoted as well as solicitations to the HCMN for comments. ƒ The M/MC should benchmark the monitoring and evaluation processes and databases used at Photoshare. ƒ In light of current staffing, at a minimum, the librarian should keep a monthly request log with most essential information in a tabular format and then enter these data into the computer at the end of each month. The table on the following page is an example log. 28 Request Name E-mail Type of Organization Region Type and Quantity of Request (poster, referral to producer) Health Topic Comments on Use 1. 2. 3. 4. 5. 6. 29 APPENDICES A. SCOPE OF WORK B. SURVEY AND RESPONSES OF USERS C. STAFF INTERVIEW GUIDE D. DONOR INTERVIEW GUIDE E. PERSONS INTERVIEWED F. DOCUMENTS REVIEWED APPENDIX A SCOPE OF WORK (from USAID) Scope of Work for Evaluation of Media Materials Clearinghouse by POPTECH Task External Evaluation of Media Materials Clearinghouse by 1 POPTECH consultant, August 16-September 3, 2004. Background USAID’s Bureau for Global Health (GH) intends to undertake an external assessment of the Media Materials Clearinghouse (MMC) at JHU/CCP. The Office of Population and Reproductive Health (GH/PRH) funded the MMC for many years under the Population Information Program. In 2002, USAID funding for the MMC was transferred to the GH Bureau-wide Health Communication Partnership (HCP), held by JHU/CCP and its partners, AED, Save the Children, the International HIV/AIDS Alliance and Tulane University. To date under HCP, the MMC has received funding from SO1 core (USAID/W Population and Reproductive Health Strategic Objective) only. As part of the HCP management review, the USAID HCP management team concluded that an assessment was warranted before deciding whether significant additional funds, possibly from other strategic objective teams, will be invested. The management review states that: “The Media Materials Clearinghouse (MMC) was established in 1982 as part of the Population Communication Services project (PCS) and is led by Hugh Rigby. From 1997 until 2002, USAID’s funding for the MMC was through the Population Information Program (PIP). In 2002, funding for the MMC was transferred to HCP with the intent of strengthening the connection between the MMC and field program implementation. The MMC collects materials and makes them available to the interested public. Its main function used to be to duplicate materials and package and mail them out. Now, most publications are digitized in order to be faster with the information and more cost-effective. The MMC has three criteria for deciding what material to digitize: whether the materials are innovative, creative and precedent-setting; whether there is evidence of positive impact of the materials; and whether the materials have potential for replication. In moving away from hardcopy material distribution, Rigby feels that the MMC has lost the ability to reach out to individuals and smaller grass-roots organizations in the field, for example high school teachers, but is better at reaching larger organizations with information. Smaller groups are referred to the web site and/or can be sent CD-ROMS. The MMC’s management language is English. A Spanish website was developed but was not able to be maintained due to lack of resources. The Communications Initiative will link to the MMC database, which should expand the reach of the MMC. In addition to the MMC at Hopkins, there are mini-MMCs in several countries, including Bolivia, Egypt, Ethiopia, India, and Jordan.1 The current MMC budget is less than half of what it was under PIP and USAID’s share of MMC funding has declined as the organizations who contribute to the MMC have grown in number. Other sources of funding include the CDC, and the CORE Initiative. Most of the MMC’s acquisition of TB and HIV/AIDS materials has been funded by CDC. Use of the MMC peaked last year with a series of over 20,000 hits each day on the website. It has more than 2000 written requests every year, and over 1000 walk-in visitors to HCP premises each year, most from developing countries. Due to a smaller budget, the MMC carries out less evaluation and record keeping than it did in the past. For example, the MMC does not use software to track hits and use of its web-based resources. 1 These mini-MMCs will not be included in this assessment. 1 USAID supported the MMC for many years as a component of PIP, where its focus on collecting and disseminating health communication materials and information about them was very appropriate. Now that the MMC is part of HCP, USAID [wants] to better understand the scope and usefulness of the collection as well as how the MMC can best serve the mandate of HCP.” Information Sources • HCP Management Review, 2003 (selected sections); • MMC website - (http://www.hcpartnership.org/mmc/); • HCP website - (http://www.hcpartnership.org); • MMC section of 19xx PIP Cooperative Agreement (attached). Purpose of the Assignment The purpose of this assignment is to better understand the scope and usefulness of MMC’s collection of information, as well as how the MMC can best serve the mandate of HCP. Specific questions that should be answered by the review include: 1. How much of the collection addresses child survival, infectious disease, HIV/AIDS, maternal health and environmental health issues relative to reproductive health and population issues? 2. Who is using information from MMC, which information is being accessed, and under which formats (e.g. digitized or hard copy)? This information should include characteristics of users, country of user (if possible), website hits and length of visit, downloads, hard copy mailings, and other inquiries. 3. The MMC website invites users to “Add your comments about materials you view.” How are comments read and used? Is this an effective way to collect feedback? How else is MMC engaging users? Summarize and highlight relevant comments received from visitors to the site including % positive, % negative, and comments of note. (Depending on what information is or is not available on the website about usefulness of the materials, the consultant may wish to interview selected MCC users regarding usefulness of materials). 4. What are trends in the collection itself over the past 3 years (types of information, size of collection) and use (number of users, type of users, type of information requested)? Do any of the materials have an evaluation of effectiveness (may be from other sources)? 5. How much USAID funding has been used annually for MMC over the past 5 years? For what are USAID funds being used and what is the cost effectiveness (e.g. per user reached)? 6. How do other funders perceive the MMC? What is their interest in the MMC? Why do they fund it? For how long do they plan to fund it? What is the MMC’s value added to their programs? 7. How is the collection being updated and maintained (including how new materials are identified and incorporated and how outdated or redundant information is discarded)? Present a chart identifying by year, the type and quantity of material on the MMC website. 2 8. What are other information resources that provide either competing or complementary information (e.g. Reproline; PHNIP; PopLine, CDC, NLM)? Does the MMC have a comparative advantage over any/all of these? The MMC website mentions a Health Communication Materials Network – include an overview of the Network including numbers and countries. 9. How does the MMC complement HCP’s mandate? How could it do so better? The consultant should develop an evaluation report describing the assessment, findings, and relevant analysis. This will be used by GH office to determine the amount and sources of future USAID funding for the MMC, and by the MMC to make management decisions. Methodology of the Assignment Information should be collected through visits to the MMC, interviews with MMC staff, and review of relevant documents. MMC will be asked to provide access to the collection, software, and relevant financial information. Information on other resources should be done via the internet. Tasks to be Accomplished Over the course of the three-week assignment, the consultant should: Review background documents 1 day Meet with USAID team ½ day Meet with MMC staff and visit collection 4 days Media research for other information 3 days Prepare report and analysis 3 days Present report and findings to GH HCP management team & HCP staff ½ day Incorporate GH comments/changes and submit final report 3 days Team Composition and Participation One consultant with the following skills and qualifications: • Experience doing evaluations of international PRH programs • Experience or training in library science, health communication and/or knowledge management as they relate to health issues in developing countries • Familiar with current GH CAs, information resources, journals, and electronic databases • Excellent analytical and writing skills • Excellent computer navigation and database skills 3 Schedule and Logistics The evaluation is expected to take place August 16 - September 3, 2004. This task is based in Baltimore, MD at the Johns Hopkins, MMC offices. HCP will provide space with a computer and access to office equipment (printer, copier, fax, telephone) during this three week period for the consultant’s use. The consultant will be paid for and should record local travel expenses. GH/PRH contact will arrange presentation times and locations for the meetings with the consultant before and after the evaluation. These will both be in Washington, DC at the Ronald Reagan Building. Estimated Level of Effort • PRH core funds will be used to pay for the assignment. • The consultant is needed full time (8 hours/day) for three weeks (fifteen business days). • The consultant should live near or have easy transportation to Baltimore, MD. • The consultant will need to pay for transport to and from the MMC facility. Deliverables • The consultant is expected to deliver an evaluation report to Ellen Starbird at the USAID GH/PRH office at the end of the assignment (COB September 3). • The consultant should deliver three hard unbound copies of the document, and two electronic copies (by e-mail or on disk in Microsoft Word) which should not exceed 20 pages. Electronic versions should be delivered to both Ellen Starbird and Hugh Rigby (HCP/MMC). If delivered by e-mail, send it to estarbird@usaid.gov and hrigby@jhuccp.org Report Time Line The evaluation report will not be distributed widely. It is expected to be delivered as a final working draft at completion of the evaluation on September 3. A final, edited version should be delivered from POPTECH no later than September 17. 4 APPENDIX B SURVEY AND RESPONSES OF USERS (Tables generated by SurveyMonkey.com) SURVEY AND RESPONSES OF USERS INVITATION AND SURVEY OF USERS Dear Fellow Health Professional: If you design, use, or have an interest in health communication or training materials, the Media/Materials Clearinghouse (M/MC) is striving to meet your needs. As you may know, the M/MC hosts the world’s largest, most comprehensive collection of health communication materials—pamphlets, posters, audiotapes, videos, training materials, electronic media, and other items—both at the onsite library in Baltimore and online at the Health Communication Materials Database (www.hcpartnership.org/mmc). I have been engaged by the U.S. Agency for International Development (USAID) to undertake an assessment of the M/MC, which is supported in large part through the USAID-funded Health Communication Partnership (HCP). Among other things, the assessment will advise on ways to improve the collection, database, and services. Your input is critical to this effort. Whether you have ever used the M/MC or not, I would like to hear from you about your use of and preferences regarding health communication materials. I would greatly appreciate your taking a few minutes to answer this brief on-line survey. The survey will only take 10 minutes, and will be extremely valuable in determining the future direction of the M/MC. You can find it by simply clicking on the link below before Wednesday, September 1, 2004: http://www.surveymonkey.com/s.asp?u=55245601130 Thank you for your assistance. Your input will feed into the assessment report that will be shared with USAID and HCP staff. However, no attribution to individuals will be made. Sincerely, Elisa Knebel Consultant Office of Population and Reproductive Health Bureau for Global Health USAID SURVEY RESULTS 1. I am currently affiliated with the following type of organization: Other: ƒ U.S.-based nonprofit training organization ƒ Consultant ƒ Nonprofit publisher ƒ Communicational (radio) ƒ Local radio station ƒ CDC ƒ Nonprofit HIV/AIDS service organization ƒ Freelance journalist ƒ London School of Hygiene and Tropical Medicine (2) ƒ Not affiliated with any organization but still seeking affiliation ƒ Individual ƒ Educational media ƒ Volunteer/researcher ƒ Local educational organization ƒ Giving health education to students in secondary schools ƒ Churches ƒ Commercial sex workers centers and seminars ƒ Communication consultancy firm ƒ Nongovernmental organization (2) ƒ Government public hospital, Australia ƒ Funding through SOUL CITY institute for health communication ƒ Student at the University of Washington ƒ Media ƒ USAID cooperating agency (2) ƒ Lintas Ghana (advertising agency) ƒ Freelance 2. My current position is: ƒ Program officer/manager/coordinator/assistant 53 ƒ Communications professional (graphic, TV, video, publicity, advocacy, publications, media, social marketing) 39 ƒ Director/CEO/President 33 ƒ Student 12 ƒ Academic (professor, educational officer, lecturer) 10 ƒ Consultant (undefined) 8 ƒ Medical or nursing professional 8 ƒ Journalist/editor 7 ƒ Other/undefined 7 ƒ Librarian/ resource center manager 6 ƒ Trainer 5 ƒ Researcher 3 ƒ Volunteer 3 3. The country where I am currently located is: USA 65 India 29 Nigeria 17 Bangladesh 9 Ghana 6 Pakistan 4 Uganda 4 UK 4 Cameroon 3 Ethiopia 3 Kenya 3 Mexico 3 Philippines 3 China 2 Morocco 2 Sierra Leone 2 Tanzania 2 Thailand 2 Australia 1 Cambodia 1 Canada 1 Egypt 1 Eritrea 1 French Guiana 1 Germany 1 Guinea (Conakry) 1 Indonesia 1 Jordan 1 Lesotho 1 Moldova 1 Mozambique 1 Nepal 1 Peru 1 Qatar 1 Republic of Kiribati 1 Scotland 1 Senegal 1 South Africa 1 The Gambia 1 Uruguay 1 Yemen 1 Zimbabwe 1 4. Regarding M/MC services, I have: (check all that apply) Other: ƒ Attended a conference workshop ƒ Former employee of JHU/PCS ƒ None of the above ƒ Received materials (3) ƒ Attended globalization and health course this week ƒ It is new to me, but I have profound interest ƒ Attended global health and communications course this week ƒ Assessing communication materials for vulnerable groups, HIV/AIDS programs in Bangladesh ƒ Still looking, thanks ƒ Accessed other related web sites affiliated with this program ƒ Contacted new members of HCMN when they first joined and, with a few, established working partnership(s) ƒ Had an e-mail contact with other members ƒ Work with JHU in Nigeria ƒ Used the Photoshare database, accessed RHGateway, POPLINE and other materials collections ƒ Received and viewed CD 5. I found out about the M/MC through a: (check all that apply) Other: ƒ Formerly or current employee at JHU/CCP (8) ƒ Your survey e-mail message (4) ƒ Through an e-mail from a researcher ƒ The Communication Initiative web site ƒ JHU/CCP newsletters ƒ Internet ƒ Never ƒ Please give some detail information ƒ Communications Working Group ƒ E-mail from M/MC ƒ By e-mail ƒ Yahoo Australia ƒ Request to include one of our reports ƒ USAID Communications Working Group ƒ JHU direct ƒ Through JHU/CCP web site 6. When designing health communication materials, I use the M/MC: Note: Respondents who chose ‘always’, ‘regularly’, or ‘occasionally’ on the above question were directed to question 13 and completed the rest of the survey from that question forward. Those who chose ‘never’ were asked question 7, and then 8 or 9, if appropriate. Those who chose ‘rarely’ were directed to question 10, and then question 11 or 12, if appropriate, and then completed from question 13 onwards. 7. I do not use the M/MC because (only those respondents who chose ‘Never’ for question 6 were asked this question): Other: ƒ I do not design health communication materials ƒ I have no need for it ƒ I do not know what it has or how it works ƒ Not had a chance to assess yet ƒ This is the first time of hearing this and I am very much interested in this ƒ Haven’t had an opportunity yet ƒ I forgot about it 8. My field/area of work is (only those who responded to question 7 that they never use M/MC because “I do not think it has materials in my field or area” were asked to complete this statement): ƒ Advocacy, reproductive health, human rights ƒ Reproductive health ƒ Abortion ƒ Behavior change: environmental health, nutrition, child’s health, mothers’ health, social marketing ƒ Training ƒ Population and reproductive health 9. The source I use for health communication materials is (only those who responded to question 7 that they never use M/MC because “I usually use other sources for my communications materials” were asked to complete this statement): ƒ How to develop IEC materials ƒ Those generated by my own organization 10. I rarely use the M/MC because (only those who responded to question 6 that they ‘rarely’ use the M/MC were asked to complete this statement): Other: ƒ Not enough material in Spanish ƒ Just neglect it ƒ Lack of awareness ƒ We don’t develop a lot of materials ƒ I forget the existence of M/MC. M/MC and its services need more promotion ƒ Have not thought of using it ƒ Our materials are usually aimed at high￾level decision makers, not the general public ƒ Not involved in development of materials 11. My field/area of work is (this statement was to be completed only by those who stated that they rarely use M/MC because “It does not have materials in my field or area.” However, no respondents completed this.) 12. The source I use for health communication materials is (only those who reported that they rarely use M/MC because “I usually use other sources” were asked to complete this statement): ƒ PSI ƒ Many specialized, e.g., adapt from MAQ ƒ Government/nongovernmental AIDS/HIV organizations ƒ Stock art; original work 13. Please rate the following elements of the materials and services available from the M/MC: Strongly Disagree Disagree Neither Disagree nor Agree Agree Strongly Agree N/A Response Average M/MC provides excellent customer service. 4% (4) 1% (1) 15% (17) 36% (41) 35% (39) 10% (11) 4.08 I refer my colleagues to the M/MC to see health communication materials. 5% (6) 5% (6) 2% (2) 37% (41) 44% (49) 7% (8) 4.16 M/MC Web site and online database are easy to navigate. 2% (2) 5% (6) 13% (15) 53% (60) 24% (27) 3% (3) 3.95 M/MC has an up-to-date collection of health communication materials. 4% (4) 4% (5) 6% (7) 40% (46) 41% (47) 4% (5) 4.17 M/MC has a highly useful collection of materials. 4% (5) 0% (0) 4% (4) 35% (39) 55% (62) 2% (2) 4.39 I can trust that the materials in the M/MC have been tested in the field. 3% (3) 1% (1) 12% (13) 35% (40) 45% (51) 4% (5) 4.25 M/MC offers materials in the language that I need. 4% (5) 4% (5) 11% (12) 33% (38) 41% (47) 6% (7) 4.09 M/MC offers materials in my topics of interest. 4% (5) 1% (1) 6% (7) 32% (36) 53% (59) 4% (4) 4.32 M/MC helps me to do better work. 3% (3) 0% (0) 15% (16) 40% (44) 38% (42) 5% (5) 4.16 Total Respondents 115 (skipped this question) 79 14. I have used sample M/MC materials (check all that apply): Other: ƒ To review/get updated on available materials on specific issues, especially RH topics, violence against women, and FGM [female genital cutting] as well as to review existing training curricula and videos on related issues ƒ I do not see any of M/MC yet ƒ In writing project work ƒ Use M/MC photos in our journals and other publications ƒ Not used ƒ For publication ƒ Planning, implementing and evaluating behavior change campaign ƒ Proposal writing 15. Compared to other sources for health communications materials, I would rate the M/MC as: 16. Please rate the importance to you of knowing the following features of M/MC materials: Very Important Important Somewhat Important Not Important I Don't Know N/A Response Average Intended audience for the piece 48% (54) 38% (43) 11% (12) 0% (0) 0% (0) 3% (3) 1.61 Background information on the project it was a part of 29% (33) 54% (62) 11% (13) 3% (3) 0% (0) 3% (3) 1.87 Information about the rest of the campaign and other materials used 25% (28) 51% (58) 18% (20) 3% (3) 1% (1) 3% (3) 2.01 Whether or not the specific material has been field tested 41% (46) 40% (45) 12% (14) 2% (2) 2% (2) 3% (3) 1.80 Results of any field tests 36% (41) 47% (53) 11% (12) 2% (2) 2% (2) 3% (3) 1.83 Total Respondents 114 (skipped this question) 80 17. When using M/MC, I am most often looking for samples of (check all that apply): Other: ƒ Curricula and trainer’s guides ƒ I do not see any of M/MC yet ƒ T Shirt ƒ I only use photographs ƒ IPC materials for audiences ƒ Web-based communications ƒ Software ƒ Technical content samples ƒ Project reports ƒ Statistical data ƒ Materials available online 18. When using the M/MC, I am usually looking for materials on (check all that apply): Note: Questions 19–23 were open ended. Responses are grouped into categories for ease of reading and organization. 19. Other: ƒ STI ƒ Substance abuse ƒ Female genital mutilation (FGM) ƒ Prevention of violence against women (2) ƒ Tuberculosis, malaria and other viral diseases, such as Nipah (3) ƒ Contraceptives ƒ Hygiene ƒ Reaching communities of color ƒ Youth ƒ Medical emergency ƒ Communication ƒ New technologies, diagnostics, drug resistance, minimal access surgery, pain management ƒ Adolescent health ƒ Counseling ƒ Parent education and support 20. I wish the M/MC offered more Materials Format ƒ Motivation oriented posters and flip charts ƒ More training manuals on reproductive health and child survival issues and job aids to go with them as well1 ƒ Research findings/reports ƒ Visual aids like posters ƒ Electronic devices to teach easily ƒ Electronic support like televisions and video to developing countries ƒ Relating to training adult non-alphabets ƒ Training material manual ƒ CD and hard copy materials 1 Items marked with an asterisk (*) are repeated in more than one section. Materials Subject Area ƒ HIV/AIDS (4) ƒ Book on advocacy. It’s very important for NGOs and others today to advocate fundraising, changing of law, policy and others. Many of them lack skills on advocacy. ƒ Nonmedical determinants of health ƒ Gender-specific materials ƒ More training manuals on reproductive health and child survival issues and job aids to go with them as well* ƒ ICT [Information Communications and Technology] and population ƒ Communication tools and methodologies ƒ Health of the elderly ƒ Communicable diseases ƒ Sexual health material for men ƒ Medical emergency ƒ Community development linked to above topics ƒ Alcohol, cigarettes, drugs ƒ Checklist prevention information for infectious diseases in local languages ƒ Health communications and social marketing ƒ Water and health ƒ Reproductive health communication in Africa ƒ Population and HIV/AIDS issues ƒ Gender related HIV/AIDS ƒ Domestic U.S. campaigns Services ƒ Translated versions of materials ƒ Financial aids ƒ Interaction with members of HCMN as in workshops in developing countries to discuss different communication media, how to adapt material to suit local flavors, how to communicate effectively ƒ Training others to produce local materials for their countries; technical assistance ƒ Informal seminars and online forums ƒ Opportunity to have real training and help more in the area of physical monitoring of our programs ƒ Opportunity to read Masters in Public Health ƒ Training on preparation of IEC materials Other ƒ Of what it is offering now ƒ Your materials ƒ Very true ƒ Copies of samples ƒ Free materials ƒ Info on how the materials were used, and evaluation of their effectiveness ƒ Learning material for local health professionals ƒ Cost-effective ƒ Campaign materials ƒ Private space for viewing videos (multisystem facilities) ƒ Samples 21. Please briefly describe how you used material from the M/MC in your work. What did you do with it? What were the results or outcomes you achieved based on the availability of the M/MC materials? ƒ To teach persons, especially the chart about reproductive health is helping us to teach families ƒ The materials are used in our library for the general public consumption and to also serve as tool for developing programs for youths, health workers, etc. ƒ Showing and describing to the public what it is, how it works. Another thing is that we can compare it to what we already have, or look for some alternatives. ƒ I used them to educate students, commercial sex workers, women in the churches and at seminars for student nurses. I gave out some copies of text books to school libraries and some to church resource centres. I gave some copies to some teachers in the schools visited. ƒ We distribute the materials at field offices and the health workers use the materials for their own learning and share with other health professionals/organizations. The attitude is being changed towards using new media like CDs. ƒ Use in anti-AIDS campaign ƒ I used M/MC materials to distribute to clinic, friends and posted to some clinic far away to where I lived. I achieved a lot people a round always came to ask question about what they read and request for more what you send to me. ƒ We have received videos from M/MC. We used them in our family health outreach activities, and they were really useful. I wonder what we could (have) achieved without them. ƒ The materials…come in handy for our rural health education initiatives ƒ Il est indispensable pour l’education: IEC ?Ainsi pour repondre aux questions de la population et pour la marche des programmes de sante surtout dans les pays en developpement. ƒ Public Enlightenment Programme Youth Campaign against HÏV/AIDS Education and curriculum development on preventive education ƒ In fact as a secondary school teacher especially in a mixed school, I am using my videotape they sent to me last two yrs to teach them about family planning, reproductive health, etc. I need help blcs they pay much attention during the programme. ƒ I have applied materials from M/MC in BCC strategy development, media campaign, campaign for events like World AIDS Day, Human Rights day, Women’s Rights day, Radio Programme, etc. Adapted Use ƒ Materials on HIV/AIDS and hepatitis (A, B, C) were part of one-year educational campaign in Qatar. We adapted into Arabic, Malbali and Urdu most of our materials. Our work was made easy with the availability of the M/MC materials. ƒ Materials used to design or adapt to educational material in LAC countries. Results are good and it saves a lot of time. ƒ I am a nurse in India. My mother tongue is different from that of the people who I care for so most of the time a descriptive poster will come in handy to me. And most of the time I modify the wordings with the help of the staff who know the people’s language as even though I can understand people and talk to them I am not fluent enough. I can only make up with my passion and visual aids. Thank you a lot. ƒ As educator, use it as a model that can be followed in designing culturally oriented health education materials, apply principles of media and material planning/developing and evaluation as well as monitoring its effect. As community health educator, I refer and use it with some translation or modifications. Usually it gives excellent results as most of the community and colleagues are not adequately exposed to this material. But I think efforts and steps have been taken to produce more of M/MC in Egypt compared to before but still not enough community coverage or focus. Ideas for Designing Own Materials ƒ I used IEC materials from another campaign to design one in southern Africa. ƒ I get the material to be inspired and adapt them in local context. Dealing with youth for instance, I received several brochures from MMC and identified messages ad pictures I could use in Guinea. As a consequence, we have developed some of the best brochures for youth on HIV, abstinence and unwanted pregnancy. It was the same for maternal care material. ƒ M/MC CD ROMs and other materials have been very helpful in providing support information on similar programs to what I am engaged in producing. ƒ Show to MOH or NGO colleagues in other countries to give ideas of what other places have developed; gather ideas of how to illustrate difficult concepts (e.g., rapid or difficult breathing). Developed malaria, nutrition and diarrhea materials in several countries using M/MC materials to give ideas. M/MC resources contributed new ideas to increase creativity in our teams and to confirm that what we were developing was on the right track. We developed in Kenya, for example, materials that led to a statistically significant improvement in malaria treatment knowledge and practice. In Benin, we used radio serial drama cassettes we had gotten from the M/MC to give us ideas as we developed theatre and radio programs that contributed to increases in knowledge and practice in family planning, malaria and maternal health. ƒ I often refer to the materials at the MMC to see what has already been produced in the field. It gives me a good idea of what has been done and gives me ideas for the materials I need to create. ƒ Used as a guide to develop health communication material. Good outcome. ƒ Materials used to design or adapt to educational material in LAC countries. Results are good and it saves a lot of time.* ƒ Materials coming from M/MC are seen as authoritative, well researched, and very popular among my colleagues here in the Philippines. Almost always, we search for documents/reports that we could use in designing campaigns, developing materials, and designing curricula on communication and leadership. ƒ Use training materials to help me develop my own training materials, tools, checklists. ƒ Surveyed alternative approaches and strategies for development of a mass media campaign to encourage testing for HIV. Used messages as stimuli for focus group discussions. Learned from experience of others. ƒ I adapted pieces of other HIV/AIDS educational campaigns to design a new workshop focused on having compassion for PLWHA. ƒ Examples of job aids for contraceptive methods ƒ Read the materials to get the insight then design parallel ones tailored to the local environment and cultural issues. The results have always been phenomenal. ƒ Ideas for designing materials ƒ I have used materials from the M/MC to help prepare myself to design training and outreach materials on RH, FGM prevention and related issues. ƒ Our designer searches for ideas in fact to illustrate our materials. ƒ I primarily use material from M/MC as a resource for ideas that can be adapted to the local context, alternative approaches to sex education/ HIV prevention, etc. In terms of an information resource, M/MC is particularly helpful, unfortunately most of my colleagues do not have ready access to the Internet to benefit more from the site. In addition, much work is needed to change to incorporate a more participatory approach to prevention activities. Training Others ƒ I usually use information got from M/MC during trainings and to share with other network partners and communities. This has really added value to our work and network. ƒ Since I conduct training full time, the MMC has been a great resource. I use their videos, DVDs and photo bank in my presentations. I bring training participants in my Baltimore workshops to MMC when I can. I am highly impressed by the responsiveness and courtesy of the staff. MMC is a gem! ƒ Got the material I wanted and trained people in the same topics in the local language ƒ Materials used in training, seminars, workshops. ƒ Mostly I will be using the material for teaching and training purposes. It is valuable and very timely, available at one place. For field training of paramedical health workers and community workers I am using. *Material available through M/MC is having “standards” and more knowledge base. ƒ I have always used them as references to develop training modules. I have also provided some of the materials as handouts after the training. The outcomes have been very positive. ƒ AIDS journalism training ƒ Used for training and updating knowledge on HIV/AIDS, the result was excellent as other nurses depended on the information for their work ƒ For training activities they help to have good guidelines, objectives, and to evaluate the training. The result is encouraging, that’s why I still use the M/MC. ƒ Improved on the training of the literates ƒ I used them to help me teach. My students were able to learn more effectively. ƒ In Bangladesh, HASAB (HIV/AIDS and NGO Alliance Bangladesh) is offering VCT counselor training to staff in agencies working with high-risk groups. In creating the curriculum, I suggested that we should seek more counseling scenarios in video or CD–ROM. Also, we needed to acquire samples of behavior change aids. I had picked up materials when visiting Dhaka that we are using in the training as well as the CABI international health CD–ROM on HIV/AIDS. Previously, I had picked up materials that are cataloged in the ICDDRB library in Dhaka that we also borrowed to examine. Fortunately, BCCP also had a companion collection of materials in Dhaka, which were immediately accessible. ƒ Used the materials for presentations and training. ƒ Training students in the field of community health and development, health communication is an important component towards the propagation of the same. Using already made materials like newsletters and videos and others on folk media, it places a suitable avenue to jump start students towards the development of the same, targeting community-based communication tools that can be sustained at the community level with locally available resources. The results have been impressive, and my students are yearning to get health communication and promotion as a special track. Thank you very much. ƒ Used in training sessions. It was useful for the trainees to understand the concept in a better way. ƒ I have also shared examples of existing materials on these issues with colleagues in countries where communications resources (and Internet facilities) are sparse, and they find themselves working in a vacuum, without access to samples of what others have done so as to help them build upon these. Finally, I have used videotapes from the collection, such as “Scenarios du Sahel” and “Pourquoi Donc L’Excision” (with permission) to help show participants in Communication for Change (C4C) participatory video workshops the range of approaches that can be used in addressing health and development issues through media. ƒ I used to integrate the materials I found on M/MC in my training for some stakeholders and counterparts. Reference ƒ As a student of population studies, these materials are very useful for me to gain knowledge about the issues related to the reproductive health, AIDS, women’s emancipation, etc. ƒ Availability has enabled me to learn more and hence teach more. It has enabled me with the information to open a resource centre. I hope you will continue to assist us. ƒ Helped improve my own knowledge on certain issues such as BCC. ƒ Used the site as an educational tool, to become more familiar with project and campaign ideas. ƒ To learn more about the themes for which I am more concerned about. ƒ Used also as reference in making project proposals or during presentations ƒ Generally used for understanding and reference. ƒ Collected materials on media advocacy to help our field office write RFPs on the subject... collected materials on HIV and support for people living with HIV/AIDS to provide NGOs with examples of support materials. ƒ As a librarian I refer users to the M/MC site. I have visited the actual premises of M/MC and found it very useful and interesting and always tell people looking for information in health related issues about it. ƒ I use those materials every time I’m planning an activity. Especially they help to have a correct objective, to choose strategies those are proved to be efficient and to plan monitoring and evaluation. ƒ I do not have a cheap Internet access, so the ability to quickly download and study materials on￾line is what I like about M/MC. I have used M/MC materials to develop some grant proposals and am still awaiting results. ƒ I use materials from M/MC to update information on health issues. The results are usually wonderful. ƒ To investigate the feasibility of development or adaptation of print or other IEC materials for socially sophisticated (urban) Spanish-speaking gay men and/intravenous drug users and their partners. A specific topic of interest is re-infection, resulting in a drug-resistant form of the virus. I’ve found very little (none) bilingual or Spanish-language materials on this subject. ƒ Actually we’re starting the adolescent RH program in rural areas and we are at the stage of start￾up phase of the program. Therefore, we are using as a reference for our new radio program. ƒ Contraceptive Technology Booklet, develop material Population Reports, aware and use of latest information, innovation, progress, etc., on FP/RH. Other materials: design, implement and evaluation of health communication campaign and programs ƒ We get very good information about health from MMC. It is very helpful for us to communicate in the field of health. Other ƒ I rarely use the materials in my work. Fact being I am not well informed about the kind of work done ƒ Not used so far only read a few ƒ From the HCMN site, I have downloaded many materials through the links provided. I have even ordered AIDSCAP material free. It is wonderful to have this resource and the links to the last detail. ƒ I utilized the materials to help me as a manual while adapting materials that were pre-tested for the country. The step-by-step guide helped me to follow a certain module and my objectives were simplified in the long term. Responses Not About M/MC ƒ My role is fairly limited in the project planning so I’m not looking at other organizations’ materials for ideas. I produce our print and Web projects after the projects have been conceived. I use Photoshare as my first resource for photographs appropriate for our journals and other publications. Because of the royalty free and cost-free availability of the photos we are able to use more photos in our printed pieces and on our Web site. Everyone likes photos! ƒ We use the images from the MMC as illustrations to PopReports. We are usually successful in finding the type of picture we need. ƒ We use visuals to illustrate our own publications, which are targeted to developing country health providers. ƒ Built a resource library of several substance abuse guideline reports; also researched photos from database. Did not use. 22. What do you like best about the M/MC? Staff/Service ƒ Friendly assistance from staff ƒ Anwar is terrifically helpful. ƒ The staff is very responsive ƒ Promptness ƒ They never miss sending the request. ƒ The people are extremely helpful. Nothing can beat that! ƒ Extremely helpful staff, who work with you to help you find what you are looking for and even suggest things you hadn’t thought of that would be helpful for your work. ƒ The staff is very professional. They have patience to deal with even French-speaking people even when communication is not easy. ƒ Staff are also extremely helpful. ƒ The helpfulness and knowledge of its staff ƒ The timely fashion with which new findings and materials are posted. ƒ Helping the developing countries in providing resource and supporting them Variety/Selection of Materials ƒ Breadth ƒ Many topics, many examples ƒ The variety of materials ƒ The diversity of material ƒ Selection ƒ Range of content covered ƒ Variety of materials ƒ The variety of cultures represented. Impressive collection of materials. ƒ Large database and resources ƒ The diversity of information from across the world. ƒ The variety of materials, including those from outside my focus ƒ MMC offers a diversity of material. There is always something you can learn from in this store. ƒ Lots of great resources and can see the posters/brochures on the Web ƒ Vast amount of materials—it’s virtually endless. ƒ The wide range of issues covered, and the depth of treatment. ƒ Wide range of materials available ƒ Examples from across the globe and successful examples compiled and presented. ƒ Variety and range (in terms of both topic and language) of materials and videos. ƒ Very rich with ideas from around the world, very comprehensive database. ƒ Providing samples from all over the countries, and that helps a lot in my work International/Developing Country Focus ƒ They give attention to nonprivileged people and the undeveloped nations. ƒ The focus on developing countries ƒ International focus ƒ Your passion to cover the whole world and the knowledge you bring to it that people are almost same everywhere Quality of Materials Up to Date ƒ What I like best about M/MC is its up-to-date information ƒ Its up-to-date and comprehensive information ƒ It gives me the update on issues related to the subject. Simple ƒ The information is in simple language and usable. ƒ Materials use simple or easy to understand words. ƒ The easily communicative aspects of its materials and pamphlets ƒ Their ease of use Tested ƒ I feel very confident about using the materials as the credibility is very high. Being a nonclinical person, I find these materials very useful as a guide to developing other needed specific modules and manuals. ƒ High quality and tested ƒ Their proven use in other similar settings ƒ Most of the materials are designed and produced with the target group in mind. ƒ Comprehensiveness, field based, effect tested. Relevant ƒ That it is a health and development communication tool that reaches all sectors of the audience through producing relevant and appropriate materials ƒ Most of the topics are relevant and of interest ƒ Database of graphics Specific Topics or Formats Training ƒ Collection of training materials* ƒ It is my bank for teaching aids Particular Health Topics ƒ HIV/AIDS materials ƒ Reproductive health, maternal health, brochures, population report, what’s new, HIV/AIDS and environmental health ƒ Materials on population and environmental health ƒ The focus on maternal health ƒ Information dissemination on a radio program for adolescent RH issues ƒ Information on family planning ƒ Health communication and reproductive health Formats ƒ CD–ROMs. It is very helpful to get examples in detail. ƒ Presentations ƒ Videos, posters, brochures, training materials, stickers, etc. ƒ Lists ƒ Photos and posters ƒ Video, posters, CD–ROM ƒ That it offers us a great proportion of the videos we use in our work ƒ Samples of materials, videos Organization ƒ Sorting options for searching ƒ It’s incredibly well organized. ƒ It is a highly organized system. ƒ Searchability of the database and background notes Accessibility ƒ It is easy to access by Internet. ƒ Its accessibility* ƒ Ability to visit the center and view everything Other ƒ Excellent promotion of services ƒ I like what it has to offer. ƒ I had the opportunity to receive materials from M/MC (e.g., Health Communication, Essentials of Contraceptive Technology, AV materials on campaign ads, population reports). They are sent to us on a very timely basis. Almost anything we request, M/MC either has the materials or we get information where we can get them. ƒ It’s a great effort to collate and showcase important artworks and communication material that can serve as a muse for others to develop their material from. ƒ Links. Resource materials. Ease of use. Contact. ƒ None ƒ It’s a wonderful resource and I can always find something there that is helpful to my work. ƒ All ƒ Availability ƒ Everything about them because their materials are educative. Responses Not About M/MC ƒ The Photoshare customer service staff is extremely prompt and helpful when I have questions (David Alexander especially). The images have informative captions that tell you exactly what you are looking at and where the image was taken. The search works well too. 23. What do you like least about the M/MC? Availability of Material Samples ƒ Samples are not available. ƒ When I cannot take a sample of a material because there are no more available. ƒ There needs to be a system of checking out materials... right now the system allows people to borrow materials on an honor system basis: it is NOT working. I have looked for materials and have not found them because people have checked them out and never returned them. So, materials seem to be disappearing... a system to manage materials is critical. Content of Materials ƒ There is less material in French. There was a time I was looking for a widely known video in French on STDs; we could not find reference. ƒ Not available in local language like in Bengali ƒ Not enough on child survival, mostly on FP and HIV ƒ Audio recordings, banners/signs, and novelty items ƒ Seems to be primarily dominated by JHU/CCP produced materials ƒ Calendars ƒ Lack of materials specifically related to African-American health communications issues ƒ Availability of African-based materials ƒ Accessibility for educational and training purposes. Also for community education purposes, particularly those in Arabic language. ƒ Limited on topics and not enough domestic information Want More Services ƒ They are not on a toll–free line in case an individual may want to talk with them. They don’t usually write personally to people that write them or they don’t usually acknowledge reports and letters sent to them. ƒ Definitely not enough info on context, how materials were used by whom, whether they were pretested and whether their impact was evaluated. That’s where I have to use my own judgment, since I am not informed about whether these materials were effective. I would like to know if a given material was tested and found to be NOT effective, or was effective in combination with some other intervention. ƒ The members should get a downloadable directory listing of other members with contact names, addresses, email addresses, telephone. Absence of that leads to saving of all files but does not result in end users gaining useful material at one glance. ƒ That it does not yet have a branch in my country ƒ Linkages and networking for sharing of M/MC materials, especially for rural based community health education initiatives who have no postal or Internet access to the M/MC ƒ They don’t normally send me materials knowing fully well that I am from a developing country and personally I would like them to help me because I want to go further in this my field [B.Sc. Health Education] ƒ Not disseminating the learning through existing forums Nothing/Don’t Know ƒ No, all are good, all one step higher than the least ƒ None (6) ƒ In overall view, I cannot say the least about M/MC as it plays its part in all the dynamics. ƒ Nothing (9) ƒ Never felt so ƒ Cannot say as I have not experienced ƒ I like the content of material ƒ I could not think of anything as of the moment. ƒ I have not addressed my mind to that yet. ƒ Well I have just started using your help I may be able tell something in the due course, hihi. Just kidding this time. ƒ I really don’t know. Web Site and Database ƒ I find the navigation tool to search difficult to use sometimes and to then navigate through the results. ƒ The search engine is cumbersome; the word wheel is sometimes ineffective for searching. ƒ The new Web site; lots of content has disappeared. ƒ Sometimes I don’t know what key words are best to use in the database. Things like, if a handbook is categorized as a “book” or a “pamphlet.” ƒ Search engine on the computer is a bit complicated to use ƒ Inability to easily copy the visuals directly from the Web site Other ƒ That it is free and easy to download ƒ Least writings, but more images and signs, or symbols ƒ Can be more user friendly Responses about Photoshare ƒ It would be helpful to be able to download an image just to see how it looked in a publication before committing to a full request for usage rights. But, this is a small complaint. I’m happy to continue working with the current system. If the proof image could be shown a little larger on the Web site that would help too. ƒ Some photos are not good enough quality. ƒ Having to get clearance/copyright approval in order to use some of the materials. 24. How could the M/MC be improved to meet your needs? The Materials Collection ƒ Even more information related to HIV/AIDS ƒ More images and signs in communication materials ƒ Having worked at the MMC, I know most of the material contained in Baltimore. I wish more of this material was accessible over the Web (some great manuals are not always posted on the Web). ƒ Materials in Arabic language ƒ If it could have more information on communication tools and methodologies ƒ Widen collection to include health issues other than reproductive health and HIV/AIDS. People can learn a lot from health campaigns in other fields and issues. ƒ Add more issue specific training manuals and job aids on reproductive health and child survival issues. Videos on it would also be appreciated. ƒ More information on advocacy ƒ By coming out with materials about the real people, i.e., by showing the real people as such instead of posers ƒ M/MC should deal more on diseases that are common in the Sub–Saharan region of Africa. ƒ Expand database on African American issues ƒ Please engage in serious documentation and development of African-based materials, and I place myself at reach if requested to assist in the development of the same. Information About the Material ƒ Make more materials available online full text. Include background info about the materials, the campaign with the materials. ƒ Include evaluations of campaigns, programs that used the materials ƒ Training and technical assistance ƒ They should be inviting us for seminars and training. ƒ Provide assistance so that M/MC materials can be translated into local Nigerian languages and disseminated. Provide support to developing the digital resources for such translation work, and make it scalable for other African rural communities. ƒ I will be very happy if some small organisations like ours could be monitored and assisted to achieve our goal. Literacy Advocacy Project, P.O. Box 45, Agbado Railway Station, Ogun State, Nigeria ƒ More online collaboration and offering courses online or internships for developing countries ƒ We have an MMC in Bangladesh and a visitor exchange program by MMC would benefit us. ƒ Some kind of exchange program where M/MC staff can visit our place to do a needs assessment and to know the activities we are doing here and to help us to do it better. Similarly we visit your place to the activities you are doing and to be trained in material production. Lists of Materials ƒ 1. Through e-mail alert of new arrivals. 2. Communication through “Population Reports” ƒ Please send us the list of health communication materials which you have. ƒ By providing lists of new materials collected ƒ Provide links to the Photoshare database. Announce new materials if clearly in a high needs area through HIPNET or APLIC–I. Other Services ƒ By sending us up-to-date information, and the additional charts. And it is better if we can get videos to show documentary films to our audiences. It is nice sending computers not only CD– ROMs. By the way, I was interviewed by one of the Johns Hopkins workers when she met me in the Ethiopia HIV/AIDS resource center in Addis Ababa, Dembel city center. ƒ It can give the issues in particular of any country where the subscriber is from. ƒ Get a staff with French background to respond to particular needs. ƒ Provide more resources and database besides JHU/CCP work, send regular updates of sites to all interested, can become more easy to navigate and access ƒ Tailor the e-mail format more specifically to each member’s needs. ƒ Materials need to be sent more quickly. ƒ If I obtain the hard and CD copy of the materials ƒ Presenting the materials and methods to NGOs working for the subject and incorporation of greater learning ƒ Packet/packages by goal ƒ Enjoy the updates from the e-mail announcements. Would eliminate all “new” people to the list. Expand Centers to New Countries ƒ Establishing local M/MC at country level to assess the needs to be focused. ƒ Not yet for this moment. When I finish school, I will have more time to work on it. ƒ By sustaining its present tempo and programme of activities, and opening up a branch in my country Other ƒ So far, everything is good ƒ Have not used it before, but will subscribe NOW ƒ Not sure ƒ Not yet for this moment. When I finish school, I will have more time to work on it. ƒ Everything is OK. ƒ We have submitted a directory of population experts and colleagues here in the Phils. We hope M/MC will continue using the list for mailing materials that are regularly published by it. JHU M/MC has already gained a very good reputation for information dissemination. We hope this will not change. ƒ By insuring the materials requested reach us for proper utilization. ƒ By emphasizing more on population health promotion and public health. ƒ I wish I had more informative comments. Sorry to be so sparse in my replies. ƒ There seems to be a database through the CCP Web site, and a different one IN the center. I don’t really understand why these aren’t the same. Or are they? Are they just different in layout? ƒ Don’t know ƒ By address ƒ Research activities and communication on time to colleagues ƒ I want to study them first ƒ As an international organization, the sky is their limit. ƒ I’m not sure. ƒ Better navigation system ƒ Be more culturally focused, allow legal permission to translate and make it in hands of whoever will make use of it: educators, trainers, health professionals and clients. ƒ It’s already meeting my needs. Should keep it up. ƒ Sometimes it’s hard to screen a video, when the existing room is in use. Answers About Photoshare ƒ It would be helpful to be able to download an image just to see how it looked in a publication before committing to a full request for usage rights. But, this is a small complaint. I’m happy to continue working with the current system. If the proof image could be shown a little larger on the Web site that would help too. ƒ Improve photo quality ƒ More photo and video share materials and ability to download them into printable formats APPENDIX C STAFF INTERVIEW GUIDE STAFF INTERVIEW GUIDE 1 What is your position at the M/MC? 2 What is the mission of the M/MC? What are its strategic goals and associated metrics? 3 What are the main services? 4 When did you start working at the M/MC? 5 How is the M/MC advertised? What kind of outreach activities do you perform? 6 Who are you primary audiences? From what country? What occupations? What organizations? What else do you know about your users? What would you like to know but don’t? 7 How do new users hear about the M/MC? What are the number and type of referral sites per year? What are the most common keywords used to find site? 8 What are the Web-based usage monitoring software or approaches currently in use? 9 What are the specific Web usage metrics and definitions in use? 10 What are the most recent copies (or URLs) of any recent, illustrative reports or data analyses on Web usage? Specifically, do you have the following: ƒ Number of user sessions per month/year for last 3 years ƒ Number of unique visits per month/year for last 3 years ƒ Average length of user visit to M/MC ƒ % increase in number of repeat users per year ƒ Number of document downloads/requests per month/year for last 3 years ƒ Most accessed downloads/requests per month/year for last 3 years ƒ Number of requests fulfilled/delivered ƒ Most frequently requested format of materials (i.e., digital or paper) per month/year 11 What are you thoughts or findings on what is working best so far regarding Web usage and Web site evaluation and what issues need attention? 12 What are the short and medium-term plans for the M/MC in extending your reach? 13 Who are the collaborative partners for this program? What do they provide? What do they expect? 14 What is the budget for this program? What is the allocation of spending? 15 How is the budget developed? 16 How many staff members does the program have? 17 What are the roles and responsibilities of each staff member? 18 What do the funders want to know? 19 How does external funding affect the program? 20 Do you anticipate changes in sources of support for the M/MC? Please describe. 21 What is the annual amount of USAID funding for M/MC for past 5 years? 22 What is the allocation of USAID funding per year per user? 23 Can you estimate the average cost of providing site per user session? 24 Do you think you are providing a unique service or do you overlap with any other existing database or clearinghouse (probe for Reproline, PHNIP, PopLine, CDC, NLM, etc.)? 25 What is the process to decide if materials are digitized? What percent is hard copy? 26 Who participates in this process? 27 How does the M/MC fit in with the mission of HCP? 28 How does the M/MC support the work of the HCP? 29 How do you describe the M/MC to people inside and outside HCP? 30 What is the number of total and new materials on site per year for the last 3 years (chart identifying by year, the type and quantity of material on the M/MC Web site) as compared to other leading database sites? 31 What is the percentage of materials that address child survival, infectious disease, HIV/AIDS, maternal health and environmental health issues relative to reproductive health and population issues for the last 3 years? 32 What percentage of materials has undergone an evaluation of effectiveness? 33 Who decides if the materials are innovative, creative and precedent-setting? 34 What are the working definitions for innovative, creative and precedent-setting materials? 35 What constitutes evidence of positive impact of the materials? 36 What is the process for removing materials from the M/MC? 37 Do you have copies of any recent, noteworthy reports or data analyses on completed Web evaluation activities? 38 How does the M/MC undertake and support program evaluation? How do you use your evaluation data? 39 Do you keep a log of visitors to the office? 40 Do you collect anecdotal information? What kinds of comments do you have? 41 Overall, how effective are your current evaluation approaches? Why? 42 What problems or difficulties do you encounter in reaching your clients? What is your response time in regular mail? 43 What makes it hard for clients to access your services or get information? 44 How do you update and maintain the collection? 45 Have you had a Web audit of the search and navigation functions of the site? 46 What do you want M/MC to be known for? 47 Can you recall a time when you learned how the M/MC had influenced one of its users? What user comments do you have regarding quality, impact, or use? 48 What would you say are the benefits of the M/MC for its users? 49 How has the program improved over time? 50 What else can be done to improve it? 51 What, if any, barriers exist to implementing these changes? 52 How are changes made to the program? 53 Who should participate in the evaluation of M/MC? 54 What would you like to know about the program that you don’t know now? Is there anything else that you would like to add? Thank you. APPENDIX D DONOR INTERVIEW GUIDE DONOR INTERVIEW GUIDE 1 What kind of information do you receive about the utility and scope of M/MC? 2 What kinds of information would you like to receive? 3 What are you thoughts or findings on what is working best so far regarding Web usage and Web site evaluation and what issues need attention? 4 What do you currently provide the M/MC? 5 What is the planned budget for next year and following years? 6 What do you currently expect of the M/MC? 7 Do you anticipate changes in sources of support for the M/MC? Please describe. 8 How does the M/MC fit in with your own organizational goals? 9 What do you think M/MC is known for? 10 Can you recall a time when you learned how the M/MC had influenced one of its users? 11 What would you say are the benefits of the M/MC for its users? 12 How has the program improved over time? 13 What else can be done to improve it? 14 What, if any, barriers exist to implementing these changes? 15 What would you like to know about the program that you don’t know now? Is there anything else that you would like to add? Thank you. APPENDIX E PERSONS INTERVIEWED PERSONS INTERVIEWED UNITED STATES Johns Hopkins University Bloomberg School of Public Health Media/Materials Clearinghouse Hugh Rigby, Chief Lori Rosman, Librarian, CORE Intiative, Clearinghouse Supervisor Anwar Singletary, Librarian Assistant Health Communication Partnership Marcela Arguila, Program Officer Peter Gottert, Deputy Project Director Susan Leibtag, Knowledge Manager Kim Martin, Senior Public Affairs Officer Robert McPeak, Internet Systems Developer Jose Rimon, Project Director Doug Storey, Associate Director INFO Project Peggy D’Adamo, Manager, Networking and Best Practices Lavina Velasco, Librarian, Photoshare Ward Rinehart, Project Director Center for Communication Programs Aaron Brady, Senior Multimedia Specialist U.S. Agency for International Development, Bureau for Global Health (GH) Office of Population and Reproductive Health (GH/PRH) Margaret Neuse, Director Jim Shelton, Senior Medical Advisor Gloria Coe, Cognizant Technical Officer (CTO), Health Communication Partnership, Policy Evaluation and Communication Division Ellen Starbird, Chief, Policy, Evaluation and Communication Division Joanne Grossi, former CTO, Population Communication Services Project Elizabeth Fox, CTO, TASC II Retu Singh, former technical advisor, Health Communication Partnership Matt Sattah, Knowledge Management Advisor Constella Group Jesse Milan, Vice President CORE Initiative Antje Becker-Benton, Behavior Change Communication Advisor Patricia Mechael, Project Manager Georgetown Institute for Reproductive Health Jeanette Cachan, Field Program Specialist Rollins School of Public Health, Emory University/Global Dialogues Kate Winskell, Professor, Department of International Health INTERNATIONAL PROJECTS African AIDS Research Network, Senegal Modou Fall Sall, Librarian, Information and Documentation Center AIDS Resource Center, Ethiopia Gashaw Mengitsu, Coordinator Bangladesh Center for Communication Programs, Bangladesh Shahjahan Mohammad, Director Center for Communication Programs, Zambia Integrated Health Programme Uttara Bharath, Deputy Chief of Party Population Council, Kenya Karusa Kiragu, Behavior Change Specialist, Horizons Program USAID/Serbia Montenegro Sergej Anagnosti APPENDIX F DOCUMENTS REVIEWED DOCUMENTS REVIEWED Health Communication Partnership. “Annual Workplan-Health Communication Partnership, Year 3: July 1, 2004–June 30, 2005.” Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, Health Communication Partnership, 2003. ____________. “Health Communication Partnership.” Web document found at http://www.hcpartnership.org/index.php. Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, Health Communication Partnership, 2003. ____________. HCP Program Accomplishments, July 1, 2002–September 30, 2003. Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, Health Communication Partnership, 2003. ____________. “M/MC Clearinghouse.” Web document found at http://www.hcpartnership.org/mmc/. Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, Health Communication Partnership, 2003. Maxfield, A. “Information and Communication Technologies for the Developing World.” Health Communication Insights #1. Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, Health Communication Partnership, 2004. Population Information Program (PIP). Final Report. Baltimore: Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs, June 30, 2003. USAID. “Development Communities of Practice at USAID: Perspective and Issues, BTEC KM/KS Pilot Communities Project.” Web document found at http://knowledge.usaid.gov/documents/cop_pilot.pdf. May 27, 2003. ____________. Report of an Expanded Management Review of the Population Information Program (PIP). June 15, 2000. Wheeler, T., Population Information Project (PIP). “As Is” Basis for the Information and Knowledge for Optimal Health (INFO) Project. Analytical Sciences, Inc., 2003. POPULATION TECHNICAL ASSISTANCE PROJECT 1101 Vermont Ave., NW Suite 900 Washington, DC 20005 Phone: (202) 898-9040 Fax: (202) 898-9057 www.poptechproject.com