KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION: KNOWLEDGE MANAGEMENT LEADERSHIP AND SOCIAL MEDIA JUNE 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Siobhan Green and Nancy Kaplan through the Global Health Technical Assistance Bridge Project. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION: KNOWLEDGE MANAGEMENT LEADERSHIP AND SOCIAL MEDIA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-01-045) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00004. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION i CONTENTS ACRONYMS ....................................................................................................................................... iii GLOSSARY .......................................................................................................................................... v EXECUTIVE SUMMARY .................................................................................................................. xi Purpose and Background ........................................................................................................... xi Methods and Analysis ................................................................................................................. xi Findings .......................................................................................................................................... xi Conclusions and Recommendations ..................................................................................... xvi I. INTRODUCTION ....................................................................................................................... 1 Scope and Methodology ............................................................................................................. 1 Purpose of the Evaluation .......................................................................................................... 1 Consultant Introduction ............................................................................................................. 2 Data Collection ............................................................................................................................ 2 Analysis ........................................................................................................................................... 4 II. FINDINGS ..................................................................................................................................... 7 Social Media in KM Leadership and Global Health: The Context ..................................... 7 Social Media within Global Health ........................................................................................... 8 K4Health’s Approach to Social Media...................................................................................13 Definition of KM Leadership ...................................................................................................15 K4Health’s Approach to KM Leadership..............................................................................19 Rationale for KM for Health ....................................................................................................23 KM and Social Media .................................................................................................................24 Evolution of KM via Social Networking ................................................................................26 Voices and Value of Social Networks via Social Media .....................................................28 Definition of Intangible Assets ................................................................................................29 III. CONCLUSIONS AND RECOMMENDATIONS ...............................................................33 Social Media .................................................................................................................................33 Intangible Assets .........................................................................................................................37 IV. FUTURE THINKING ................................................................................................................39 Social Media .................................................................................................................................39 KM Leadership For Public Health ...........................................................................................40 ii KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION APPENDICES APPENDIX A. SCOPE OF WORK ..............................................................................................43 APPENDIX B. PERSONS INTERVIEWED ..................................................................................53 APPENDIX C. REFERENCES .........................................................................................................55 APPENDIX D. SEARCH ENGINE OPTIMIZATION ...............................................................59 APPENDIX E. LITERATURE REVIEW MATERIALS .................................................................61 APPENDIX F. INTERVIEW QUESTIONS ..................................................................................63 FIGURES Figure 1. Map of KM Services and Products ...............................................................................25 Figure 2. Knowledge Management as a Tree Metaphor...........................................................27 TABLES Table 1. Top 10 Low-income Developing Countries on Facebook ......................................10 Table 2. Top 10 Countries for Facebook Mobile Penetration ...............................................11 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION iii ACRONYMS ACE App for Contraceptive Eligibility CLA Collaborating, Learning and Adapting COP Community of Practice DEC Development Experience Clearinghouse eLearning Online learningGHeLGlobal Health eLearning Platform HIFA Healthcare Information for All HIPNet Health Information and Publications Network IBP Implementing best practices INFO USAID INFO project (previous version of K4Health) ICT Information communication technologies IT Information technology JHU∙CCP Johns Hopkins Bloomberg School of Public Health Center for Communication Programs K4Health Knowledge for Health KM Knowledge management mHealth Mobile health M&E Monitoring and evaluation MSH Management Sciences for Health NGO Non-governmental organization OTI Office of Transition Initiatives PAHO Pan American Health Organization PCV Peace Corps Volunteer PSI Population Services International RFA Request for Application RFP Request for Proposal SEO Search Engine Optimization SHARE Southern Africa HIV/AIDS Regional Exchange USAID United States Agency for International Development WHO World Health Organization iv KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION v GLOSSARY Active listening (online) “Active listening is a communication technique that requires the listener to feed back what he hears to the speaker, by way of re-stating or paraphrasing what he has heard in his own words to confirm what he has heard and moreover, to confirm the understanding of both parties.”1 In an online environment, the communication can come from direct conversations between users and the organization or from listening to conversations between users on a topic that affects the organization (i.e., on its products and services or on the problem the products and services address). Active online listening is a technique associated with sentiment analysis and opinion mining and can been observed in a number of ways: A moderator facilitating an online conversation, including summarizing the feedback of a group discussion (including the points of disagreement); A public awareness team focusing on emerging concerns of core users and speaking directly to those audiences; or Improvements made to products and services stemming from feedback gleaned from conversations by users. Agile development “Agile software development is a group of software development methods based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change.”2 In the context of this evaluation, “agile” refers to the development process by which the feedback collected from the sentiment analysis can quickly turn into changes in the web infrastructure and products, with testing and tweaking built into the process. Audience engagement Audience engagement refers to the patterns of audience contributions and investment in a community and website. Positive reputation and brand awareness are requirements before audience engagement can occur, but these two assets only refer to passive views of the project. Audience engagement focuses on active interaction by a core audience, such as downloading documents and adapting them for local usage, commenting on blogs and discussion for a, recommending content and sites on- and offline, using Twitter hashtags, contributing to working groups, and contributing content to data repositories. Audience engagement is the hardest asset to build and sustain, and it is the core of social media and knowledge management (KM) projects. Content and data repositories and taxonomy Content and data repositories and associated taxonomy refer to the databases of structured or unstructured content generated by KM and social media projects, and the tagging/taxonomy that evolves to help manage that content. Examples include community of practice (COP) discussion 1Active Listening. Wikipedia. Available athttp://en.wikipedia.org/wiki/Active_listening. 2 Agile Software Development. Wikipedia. Available at http://en.wikipedia.org/wiki/Agile_software_development. vi KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION archives, COP contact information (names, titles, email addresses), blog articles and associated content, databases of peer-reviewed journals organized by taxonomy, and document/photo/multimedia repositories. Brand awareness Brand awareness refers to awareness in the targeted user population of offerings and contact information (URL, search terms) of a project or organization. For example, one target population of Knowledge for Health (K4Health) is USAID global health’s implementing partners. The brand awareness refers to this population’s awareness of the project, its focus and purpose, its toolkits and other web-based products, and its working groups—as well as how to find these resources easily. Brand awareness also refers to the use of hashtags associated with the project when used by audience members outside of the project staff. Brand awareness is an important precursor and component in building a strong reputation and thought leadership. Edge case An edge case is when one or more parameters of one’s approach (such as literacy, information and communications technology access, or demographics) impacts a minority of users and thus must be modified from what works in the majority of situations. The parameter impact often occurs at the parameter’s minimum or maximum, hence the name “edge”; it refers to situations on the edge of the requirements. For example, if 90% of the time a solution works, but 10% of the time (for the poorest or most remote, etc.) it does not, identification of this 10% constitutes an edge case that may require designing an alternative approach. Implementing partners U.S.-based non-governmental organizations (NGOs) who receive funding from the United States Agency for International Development (USAID) are implementing partners. In this report, implementing partners often refer to the community of NGOs who receive support from the Bureau for Global Health to provide health-related development assistance. Intangible assets Intangible assets are “identifiable non-monetary assets that cannot be seen, touched or physically measured, which are created through time and/or effort and that are identifiable as a separate asset.3 These assets have real value and are essential to a project or organization’s success. For knowledge-based projects, these assets are even more common and more central to their successes. Examples of intangible assets include organizational ability, research and development, brand equity, customer databases, exclusivity within a particular market or geographic area, software, drawings, special expertise, customer satisfaction, the speed at which companies are able to bring new products and services to market, and more. Such assets usually involve information and are knowledge based, focusing on products, services, and organizational systems. Knowledge-based, intangible assets are sometimes referred to as intellectual capital.4 3 Intangible Asset. In Wikipedia. [online] Available at: . [Accessed May 2012]. (Undated). 4 Intangible Assets. Free Encyclopedia of Ecommerce, 2012. http://ecommerce.hostip.info/pages/592/Intangible￾Assets.html#ixzz1wxFFg1uF. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION vii The specific types of assets focused on in this report are brand awareness, reputation, audience engagement, content and data repositories and taxonomy, technology investments, and thought leadership/special expertise. Knowledge management Knowledge management (KM) is “a set of approaches that value and support generating, capturing, organizing, and adapting knowledge, and making it accessible and usable in different formats for different people.”5 KM aims to improve productivity, improve performance, decrease time expended, and create and sustain a learning culture where best practices and lessons learned continually support process improvement and data-based decision-making. KM requires involvement of people, processes, and technology. Knowledge management leadership KM leadership facilitates the use of KM science and practice by donors and implementing partners to improve global health and other international development outcomes. The main objectives of KM leadership for health are to: Facilitate KM within health organizations and USAID-funded projects so that organizations use KM effectively to improve their own practices and outcomes and to share their experiences and lessons learned with others. Advance the science and practice of KM as a field via improving measurement methods and sharing learning about KM for continual process improvement. Modified grounded theory Modified grounded theory is a research method that starts the research process with data collection instead of a hypothesis.6 Once the data are collected, key points are identified and coded. Similarly coded groups are joined into categories. Categories form the basis for the creation of a theory, thus creating a hypothesis in reverse. Reputation Reputation refers to the sentiment within the targeted user group toward the project or organization. Brand awareness is a necessary precursor but only refers to general awareness, not positive or negative feelings about the project, its services, its products, or its management. Reputation focuses on judgment about trustworthiness, importance, value to the user, value to the broader community, relevance, and ideas about sustainability. An organization’s thought leadership activities can have a critical impact on its reputation. Special expertise Special expertise is an intangible asset that relates to rare or unique operational know-how or subject matter experience that directly promotes the products and/or services of an organization or project. The expertise may be in an emerging field where there is growing interest but not yet a great deal of experience by others. To be counted as an organizational intangible asset, the expertise has to be experience held by multiple employees on multiple activities at the organization. Evidence of special expertise can include publications on the topic, 5 Knowledge for Health. “About K4Health.” http://www.k4health.org/about-k4health. (Undated) 6 Grounded Theory. Wikipedia. Available at http://en.wikipedia.org/wiki/Grounded_theory. Accessed June 2012. (Undated) viii KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION technical assistance offered to third parties, and limited competition on the topic. Special expertise also links directly to reputation and is combined with thought leadership for the purposes of this report. Sentiment analysis “Sentiment analysis refers to the application of natural language processing, computational linguistics and text analytics to identify and extract subjective information in source material.”7 Sentiment analysis can be performed by humans using techniques of active listening—meaning actively analyzing social media conversations for relevant feedback, examples of application, and emotional sentiment toward the topic. Large businesses use sentiment analysis to track positive and negative feedback on their products and services in order to keep on top of needed changes, issues, and opportunities. Sentiment analysis can also be referred to as opinion mining.8 Social media and social networking Social media and social networking enable new relationships among organizations, formal and informal groups, and individuals. Social media is not defined by specific sites (such as Facebook or Twitter) but is digitally enabled engagement that both drives and is driven by four key considerations: Community: Communities comprise individuals or groups with a common interest. Communities can be highly focused on a specific issue, such as the effectiveness of contraceptive methods in developing countries, or on broader issues, such as family planning and reproductive health. Communities share information and serve as support networks for members. Multidirectional communication: Until recently, communication has involved only two people or entities. Social media, however, has changed that dynamic. Social media communication is multidirectional, a many-to-many conversation that the initiator cannot control. Mass collaboration: Social networking and social media enable a broad range of perspectives and a variety of insights to inform new ideas and facilitate joint activities. Mass collaboration can occur in a structured forum built for the purpose or can arise spontaneously as a result of active listening. Building relationships: The essence of social media is the building of relationships. Numerous studies have demonstrated that people respond to and trust people within their social network significantly more than they trust organizations or unknown parties. Social media enables people to scale relationship-building and put names and faces to organizations. Social media utilizes a dizzying array of electronic channels and platforms, with new channels continuing to emerge. Facebook, Twitter, LinkedIn, YouTube, blogs, discussion forums, and private networks such as Yammer all play a role in social networking, but they do not define it. 7 Sentiment Analysis. Wikipedia. http://en.wikipedia.org/wiki/Sentiment_analysis. (Undated) 8Whatley, Simon. “Qualitative Social Media Monitoring Tools (Sentiment Monitoring).” July 21, 2011. Available at http://www.simonwhatley.co.uk/qualitative-social-media-monitoring-tools-sentiment-monitoring. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION ix Technology investments Technology investments are the intangible assets related to technology developed and designed through a project. Even when there is no dollar valuation to the software (such as when it is developed under open-source licensing), the performance enhancements or increased service offerings provided by the technology have a real value. Technology investments also involve trade secrets for implementing and using the technology to achieve those performance enhancements and service offerings. Trade secrets also can overlap into special expertise. Thought leadership Thought leadership is an emerging intangible asset referring to the recognition by peer organizations and audiences of an organization’s knowledge and forward-thinking ideas. Thought leadership is developed by generating publications (formal and informal), sharing research and development discoveries, contributing to communities of practice, and coordinating/collaborating in the field. Thought leadership is an essential part of reputation and brand awareness.9 9 Thought Leadership. Wikipedia. Available at http://en.wikipedia.org/wiki/Thought_leadership. (Undated) x KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION xi EXECUTIVE SUMMARY PURPOSE AND BACKGROUND The USAID K4Health external evaluation examined the role of social media and KM leadership within the K4Health project. The main questions were: What state-of-the-art social media and social networking practices (including Web 2.0) do K4Health and other companies and organizations use to reach audiences in the developing world? What role has K4Health played as a KM leader? What recommendations would K4Health and other companies and organizations offer for the future use of social media and social networking as part of an overall KM strategy? What intangible assets does K4Health offer regarding social media and KM? METHODS AND ANALYSIS The team performed literature reviews of K4Health’s documents and other associated documents and websites. Forty individuals—representing the K4Health team, USAID (Washington and field), members of the implementing partner community, and other development organizations—were interviewed. This evaluation was a learning evaluation for K4Health and USAID; the role of the team was to explore the above questions with K4Health representatives and other interviewees and identify strengths, challenges, and potential areas for new exploration. Statements of challenges and/or recommendations are not intended as criticism of K4Health’s activities unless explicitly stated as such. As a learning evaluation and part of USAID’s emerging Collaborating, Learning, and Adapting (CLA) policy, a major underlying purpose of this evaluation has been to share these strengths and challenges with USAID and implementing partners so they can use and integrate social media, KM leadership, and concepts of intangible assets in their ongoing work. FINDINGS K4Health’s Approach to Social Media K4Health’s Strengths in Social Media The K4Health team has three particular strengths: K4Health both develops and participates in a variety of active and useful COPs that take advantage of discussion forums, blogs, and other means of engaging with the community. The K4Health team has taken a leadership position in developing good uses of social media for family planning and reproductive health, as well as for other international development considerations. By leading the newly formed Social Media Working Group, K4Health is enhancing its position as a leader in social media, and in KM overall. xii KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION The team stays abreast of social media trends, but evaluates and tests before embracing the latest sites. Given the rapid evolution and change among social media sites, this “rational openness” of the K4Health team is a great strength. Other social media activities occur at a lower level, such as the recently established LinkedIn groups, sentiment monitoring and active listening.10 Google Alerts, use of video and webinars, and engagement in others’ social media activities are the major methods by which K4Health engages in active listening. Although social media was not part of the original scope of the K4Health project, the project team has recognized its importance within KM and initiated additional activity in social media channels over the past two years. K4Health’s Challenges in Social Media Please note that these challenges do not represent failures by the K4Health team, as social media is a newly emerging field. In fact, K4Health is at the forefront of exploring the role of social media in KM and global health. These challenges and issues should be viewed as areas where K4Health and USAID may want to pay particular attention in the future as they all learn from K4Health’s experience. Overarching Issues Is K4Health a definitive source of facts, or should experiential sharing also be included? The confusion of these two perspectives tends to polarize views on the use of social media and has impeded K4Health from establishing clear directives and processes. K4Health still needs to embrace social media as a means of community engagement, not simply one-way or bidirectional communication. Social media needs to be integrated throughout the project, into everyone’s role, and across products (such as k4health.org, GHeL, POPLINE, Photoshare, and the COPs). The next stage for K4Health is using social media to drive many of the intangible assets of KM leadership, such as brand awareness and reputation, audience engagement, thought leadership, and community strength. Specific Applications Broadband access varies significantly across countries, and customizing social media for regions is critical—albeit challenging. With a few exceptions, much of K4Health’s social media activity is currently focused in North America. However, the COPs are very active in developing countries and may be a good source of information on how to expand participation in developing countries. K4Health currently “controls” most of its social media activity. Taking advantage of social media, however, will require K4Health to listen more effectively and participate more on others’ sites—for instance, by commenting on others’ blogs, by monitoring comments and sentiment across social media channels, and by acknowledging negative and controversial comments found on others’ sites. Conversations on non-K4Health sites display a range of 10 See glossary for sentiment analysis and active listening definitions. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION xiii perspectives and insights; the comments expressing the greatest negativity or concerns are not often posted on those sites maintained or managed by K4Health. K4Health has had great success with its eLearning activities. The programs, however, operate largely as solo activities for individuals. Social media technologies—including videos, discussion forums, and Twitter chats—could greatly enhance the efficacy of the eLearning programs. K4Health is actively working on this. Methods to measure the outcomes and value of social media activities are in the early development phase. Increased monitoring of the reach of and engagement driven by social media activities will help to create a responsive culture. K4Health can then develop feedback cycles to allow the information to quickly improve services and products, and even define new ones. K4Health’s Approach to KM Leadership The team found that K4Health’s approach to KM leadership has three major elements: Supporting and promoting KM professionals: KM Leadership involves supporting organizations where they are right now in their KM evolution, as well as helping them with the transition to the next stages. Modeling and experimenting: KM Leadership involves modeling good practices, being on the cutting edge of new techniques, and trying out new approaches with the expectation that some of these will not succeed. Championing and advocating: KM Leadership helps demonstrate the value and return on investment of KM initiatives for improved global health outcome. K4Health’s Strengths in KM Leadership Overall, interviewees had strong positive feelings about the work of K4Health in KM generally and as a KM leader. There is strong brand awareness of the project, its working groups, website, toolkits, and other projects. Interviewees stated that K4Health has played the role of preeminent KM facilitator in the global health community, serving as an honest broker among partners, while recognizing the subject matter expertise of the various players. Interviewees also appreciated K4Health’s efforts coordinating and organizing the KM community to accomplish many important tasks. K4Health’s Challenges in KM Leadership Lack of Knowledge about K4Health’s Internal KM Several interviewees mentioned that they considered the modeling of internal (i.e., within organization/project) and external (i.e., in the delivery of products and services to target beneficiaries) KM practices as a key element of KM leadership. As many members of the KM working group are primarily focused on internal KM activities, they are actively looking for good practices that use KM to improve internal project performance. K4Health mentioned that, to date, the project had intentionally not shared details about its internal KM systems out of concern that it might interfere with K4Heath’s roles of facilitator and honest broker. However, now that the working group is nearly two years old, there will be a good opportunity for K4Health to share its internal KM experiences. xiv KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Role of K4Health in Content Curation Both within USAID and among implementing partners there was some confusion and disagreement about whether K4Health should compile all content, regardless of quality, or vet and “stamp with approval” certain types of content. Interviewees viewed toolkits as uneven in quality due to different standards of the implementing partners who managed them. There were concerns about differences between evidence-based information (and the derivative tools that come from it) and experiential information, which social media excels at producing. There were concerns that K4Health had tried to integrate social media tools (such as rating and comments) into the K4Health website that were considered unsuccessful and as a result, discontinued. Technology Platform In several cases, K4Health has inherited or been directed to use existing technology platforms, which has caused some limitations both in dealing with the inevitable obsolescence of platforms and responding to emerging technology changes, such as social media integration. Decisions about which platform to use and when to move to a new system need to be responsive to the changing nature of the technology world. As technology becomes obsolete, migration of existing communities and data needs to be built into the project planning. For example, K4Health has been using the Management Sciences for Health (MSH)-based Global Health eLearning (GHeL) platform to deliver eLearning courses for several years. However, K4Health, MSH, and USAID have collectively recognized that the platform has come to the end of its lifespan; as a result, K4Health is building a new eLearning platform that integrates social media elements and reflects current technological standards. Rationale for KM for Health While KM is applicable to international development as a field, having a specific project that is focused on KM leadership in global health provides the following benefits: Uses the health community’s expertise and emerging experiences in KM and related fields to support and drive leadership activities, which would be diluted in a broader scope. Narrows the scope of demonstration of KM interventions to make them more manageable. Makes explicit K4Health’s leadership among other KM leaders as a “go-to” project for health-related KM. Allows the KM tools and activities modeled and offered by the K4Health’s KM leadership to be answerable to public health goals and initiatives. Links K4Health KM leadership focus areas to global health activities and impact measurements. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION xv KM and Social Media When social networking is fully integrated with information systems as part of a KM system, many interconnecting benefits can emerge. Some possibilities include identifying good practices and lessons learned, translating scientific evidence into implementable practice, capturing real world feedback on existing challenges and opportunities, identifying edge cases and exceptions to good practices, and exposing emerging patterns and trends that need additional investigation. It is important to remember that COP participants are subject matter experts of their own experiences and contexts. Their expertise must be tapped into for an effective KM system. Intangible Assets Brand Awareness and Reputation Examples of positive brand awareness and reputation by the K4Health project include: Core audience awareness of URL, toolkits, and content topics focused on by K4Health. Recognition for being an honest broker to work with diverse implementing partners. Reputation for quality content. Association with USAID and WHO, which reinforces reputation. High search engine optimization (SEO) status for many core searches. Audience Engagement and Community Strength Examples of positive engagement by K4Health include: COP development. Work group commitment, as evidenced by development of deliverables and toolkits. Technology Experience and Code Base The examples for K4Health include: Its main website, www.k4health.org. Toolkits application. POPLINE website. Photoshare website. Implementing Best Practices (IBP) platform (the improvements for usability that have been undertaken by WHO). Southern Africa HIV/AIDS Regional Exchange (SHARE), an online collaboration platform built locally in South Africa. GHeL’s new platform (currently under development). Sites4Dev/OpenAid (soft launched and moving forward). The mobile app, ACE (App for Contraceptive Eligiblity. xvi KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Content and Data Repository and Taxonomy Examples of valuable content and data repository assets include: IBP COP archives (700+ communities). K4Health website (50+ toolkits, blogs). POPLINE (350,000+ articles). Photoshare (20,000+ images). GHeL (50+ courses, 60,000+ users having completed one or more courses). Special Expertise and Thought Leadership Examples of these assets include: White papers on good practices/lessons learned and how to manage community forums. Peer-reviewed publications on information needs assessment and social network mapping. Publications such as the monitoring and evaluation guide. Authors of blog posts and contributors to KM COPs. Tacit knowledge by staff about KM development/management, including in-depth field experience by key staff on how to nurture COPs and KM in the field. CONCLUSIONS AND RECOMMENDATIONS Social Media Organizational and Strategic Define and treat social media as integral to the definition of KM. Continue to support a social media expert, but disperse both the strategy and execution of social media by making it part of everyone’s job. Establish an internal social media network. Tactical Begin active listening to understand both where people are talking online and what is being said about issues related to family planning and reproductive health as well as about the K4Health project and products. Have K4Health staff engage with their communities on others’ sites: comment on blog posts, participate in LinkedIn discussions of non-K4Health groups, follow and respond to tweets, ask questions and post thoughtful responses on wikis and discussion forums. Establish a means to place information along the knowledge continuum, from results of controlled studies to novice experiences. Continue and expand blogging to establish expertise; include guest bloggers to expand the community and add credibility; write guest posts on blogs of other well-regarded community members. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION xvii Research and document the access and availability to both mobile and fixed line data communication in field locations. To the extent possible, distinguish the access and availability for those community members who are the major users of K4Health information (e.g., medical personnel, Ministry of Health staff) from the access and availability for the broader populace in a geographic area. Build social media into existing K4Health successes by, for instance, incorporating more social networking into eLearning. KM Leadership Continue addressing a real need. Protect and expand role as honest broker. Clarify but integrate different types of information. Manage the life cycle of technology platforms, including migration and retirement. Increase the voices from the field. Key Intangible Assets to Retain Brand awareness and reputation. Audience engagement. Technology investments. Data and content migration or archive. Methods for creating/maintaining taxonomy and tagging. xviii KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 1 I. INTRODUCTION In April 2012, USAID Global Health, Office of Population and Reproductive Health, requested a learning evaluation of KM and social media for health, focusing on the work and role of the K4Health project, run by the Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (JHU∙CCP) and supported by FHI360 and MSH. A team under leaders Siobhan Green and Nancy Kaplan conducted the evaluation from April 17–June 22, 2012 in the Washington, DC, metro area. This report details the evaluation’s scope and methodology, findings, conclusions, and recommendations, as well as some future thinking for USAID. SCOPE AND METHODOLOGY Scope The evaluation examined what leadership role K4Health plays in the KM for global health field and how K4Health leverages social media and networking practices to further the project’s objectives. The data were collected via literature reviews and interviews with key stakeholders from K4Health and leading global health/international development companies/organizations. The analysis: Outlined the current KM landscape in global health; Captured experiences, good practices,11 and lessons learned by individuals/organizations employing social media and networking practices to reach audiences in developing countries; Identified K4Health’s role as a KM leader; Identified recommendations for future growth to allow K4Health to lead the rapidly evolving field of KM; and Identified K4Health’s intangible assets related to the social media and KM fields. PURPOSE OF THE EVALUATION This report is the end result of a learning evaluation for K4Health and USAID, as described in the USAID evaluation policy. Therefore, the role of the evaluation was to explore the key questions with K4Health representatives and other interviewees, and determine strengths, challenges (for K4Health and similar KM for development initiatives), and suggested new areas of exploration. As a result, the evaluation was very forward focused and didn’t examine whether K4Health was doing the “right” things or not; rather, the team observed what the project is currently doing well and what areas K4Health and/or USAID should focus on in the future. Statements of challenges and/or recommendations in this report are not intended as criticism of K4Health’s activities unless explicitly stated as such. In many cases, the evaluation team found that K4Health had already started to experiment or think about the challenges and recommendations the evaluation team proposed. 11 While the current term used is best practices, in the world of rapidly evolving KM and social media, the phrase good practices is preferred, as “best” often implies “fixed,” “perfect,” “cannot be improved,” and “universal,” which is not true and contrary to the philosophy of KM. 2 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION As a learning evaluation and part of USAID’s draft CLA policy, a major underlying purpose of this evaluation has been to share these strengths and challenges with USAID and implementing partners so they can use and integrate social media, KM leadership, and concepts of intangible assets into their ongoing work. CONSULTANT INTRODUCTION Two consultants led the evaluation, Siobhan Green and Nancy Kaplan. Ms. Green oversaw all aspects of the evaluation, including organization of technical interviews, documentation, and data collection and analysis. She wrote sections of the report, incorporating fellow consultant Nancy Kaplan’s contributions, and presented the conclusions and recommendations to USAID. Ms. Green leveraged her technical expertise on current good practices regarding KM leadership for global health/international development along with her knowledge regarding the cultural context of USAID. Ms. Kaplan attended all technical interviews, during which she focused on social media and social networking, particularly as they relate to an organization’s promotion, knowledge-sharing, and audience-building activities. Ms. Kaplan wrote sections of the report and co-hosted the briefings with USAID and K4Health. Ms. Kaplan brought industry perspective and expertise on the role and use of social media. DATA COLLECTION The evaluation employed qualitative research methods. Interviews with key informants were a substantial component of the evaluation. USAID technical team Madeleine Short Fabic and Peggy D’Adamo provided the initial list of suggested interviewees. Suggested individuals worked in a variety of roles in/across three subject areas: KM in global health, KM in international development, and social media. There was considerable overlap among subject areas given their analogous nature. The interviewees represented different global health stakeholders including USAID and its implementing and external partners. Interviewees were recognized as experts in KM and/or the use of social media based on their professional work in those areas. After finalizing the master list of interviewees, the consultants selected a sampling of individuals from each subject area to contact based on their experience, knowledge, and background in those subject areas. In-person meetings were arranged (preferred), or over-the-phone conversations took place if logistics required. Individuals who did not respond to the initial email message were only re-contacted if their subject area was underrepresented in the list of completed interviews. During the course of their own interviews, some interviewees suggested other individuals relevant to the evaluation because of their expertise on the subject matter. The consultants contacted some of those individuals and scheduled interviews as needed. The consultants interviewed a total of 40 individuals for this evaluation. The primary interviews with key informants were coupled with other methodologies, including literature reviews, reviews of K4Health websites, and analysis of KM working group activities and the KM toolkit to explore the main questions. In the initial workplan, focus groups and online surveys were included as potential data collection mechanisms. However, those instruments were ultimately not utilized due to time constraints and the fact that sufficient data were collected through interviews and literature reviews. Methodologies were employed based on the specific research question and are detailed below. Limitations The major limitation to the evaluation was the compressed time line. A total of 10 weeks was available, including 2 weeks for kick-off and workplan development and 2 final weeks of KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 3 presentation and incorporation of feedback. Therefore, only six weeks in total were available for reviewing background documents, conducting interviews, collecting and analyzing data, and writing. In addition, over 80 documents (of approximately 100 pages each) were provided to the two-person team to review, with an additional 20 documents presented by the K4Health team at the first interview. The team also received a list of 70 potential interviewees to speak with during this time. Due to the nature of the overlaps in topics and the subject matter expertise of the evaluators, it was determined that both interviewers needed to be present for interviews. As a result, the team was unable to return to interviewees for further clarification, research additional documentation, or interview all recommended organizations and individuals (especially those located overseas). The team was able to allow USAID and K4Health the opportunity to review some of our preliminary findings for clarification and fact-checking. In addition, as this is a learning evaluation of a rapidly emerging field, there are elements in this report that are subjective; the evaluators focused on being evidence-based where possible, but due to the nature of the field, there are many areas where experiential knowledge is used. The fact that the team is not evaluating the performance of K4Health but rather a rapidly evolving area of social media and KM allowed it the latitude to use experiential knowledge for this evaluation. The team’s goal was to highlight emerging trends, good practices, and ideas for the future, rather than to be definitive on this topic. It is hoped that this approach will result in more conversation among key players. Question 1: What state-of-the-art social media and social networking practices (including Web 2.0) are K4Health and other companies/organizations using to reach audiences in the developing world? To explore this question, the consultants conducted literature reviews of a range of KM sources including: K4Health products, World Bank materials, the Microlinks website, the AIDSTAR-One website, and the Healthcare Information for All (HIFA) 2015 website and mailing list (see Appendix C for list of resources). The consultants also interviewed experts in the use of social medial/social networking tools (see Appendix B for complete list of interviewees). Questions included (see Appendix F for complete list of questions): Which social media hold the most promise for improving information dissemination and knowledge-sharing among health workers in developing countries? How are social media and social networking used to enhance or expand knowledge sharing, including the promotion of good practices and their scale up? For both knowledge sharing and promotion, what are the elements of a successful blog related to global health? Of a successful Twitter feed? What are the gender differentials in social media access and use within the developing world and how should these differentials be factored into a social media strategy? How are K4Health and other organizations effectively using Facebook, LinkedIn, and other social networking sites to facilitate knowledge sharing, information dissemination, and promotion? What social media opportunities has K4Health taken good advantage of and what gaps remain? Question 2: What role has K4Health played as a KM leader? 4 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION To explore this question, the consultants reviewed the K4Health website, the KM working group activities, and the KM toolkit. The consultants also interviewed stakeholders and users, asking the following questions: What other organizations are providing strong KM leadership in health and development? Is there a strong rationale for having distinct global health-focused KM leadership? What are K4Health’s core strengths in KM leadership as recognized by different stakeholders, including USAID/Washington, USAID/Missions, K4Health staff, K4Health clients (i.e., toolkit contributors and eLearning authors), and K4Health users? To what degree are K4Health’s KM leadership activities represented on www.k4health.org? Discuss the benefits and drawbacks to housing KM information in the same platform as health information. Question 3: What recommendations would K4Health and other companies/organizations offer for the future use of social media and social networking as part of an overall KM strategy? To explore this question, the consultants conducted interviews with stakeholders and website users. The consultants proposed the following questions in the interviews: How can social media/networking tools effectively engage and address the needs of audiences like health program managers and health service providers? Should KM leadership be incorporated into a follow-on project? If so, what level of effort should be applied? Question 4: What intangible assets does K4Health offer regarding social media and KM? This question required a different approach given its unquantifiable nature. Therefore, the consultants extrapolated such assets from interviewees’ responses to interview questions. ANALYSIS The evaluation used an iterative qualitative analysis process. The iterative component involved the team moving forward with just enough information to start the next stage of the evaluation, a process that allowed for continual refinement of the evaluation’s focus and instruments. The consultants shared ideas, perspectives, and outstanding questions to identify gaps in knowledge, highlight unexpected findings, reveal interesting perspectives, and identify new questions. Ideas, recommendations, and lessons learned were grouped according to common themes and patterns. The methods of qualitative analysis used included content analysis and modified grounded theory. Such an analysis provided a framework that facilitated sorting collected data (i.e., ideas and perspectives) and outstanding questions into ideas, recommendations, and lessons learned. Together, the iterative and qualitative components created the following analysis process. Information Gathering/Question Refinement The team used the primary information sources (the reference documents and K4Health staff and the USAID technical team members, Madeleine Short Fabic and Peggy D’Adamo) to gather KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 5 the detailed context, understand key elements, and refine the interview questions. The structure involved independent review and interviews with key staff. The consultants then produced a summary context for the main evaluation areas, a list of key questions to be answered, and interview summaries. Extended Interviewing Using this summary context, the consultants interviewed the extended stakeholders (which included the primary information sources). The consultants refined interviews instruments in conjunction with topics raised in these interviews. Ideas, Perspectives, and Outstanding Questions On a weekly basis, the team met to discuss gaps in knowledge, highlight unexpected findings and interesting new perspectives, and identify new questions to incorporate into the interviews. The team kept a list of these findings and shared them with both USAID and K4Health as appropriate. The consultants used the following three questions12 to guide this process: What are the data telling me? (Explicitly engaging with theoretical, subjective, and field understandings) What is it I want to know? (According to research objectives, questions, and theoretical points of interest) What is the dialectical relationship between what the data are telling me and what I want to know? (Refining the focus and linking back to research questions) Refinement of Findings into Recommendations and Lessons Learned During the final analysis period, the team analyzed these findings to identify common themes and patterns. They wrote this final report on recommendations and lessons learned, including areas identified for future analysis or monitoring. 12 Srivastava, Prachi and Nick Hopwood. “A Practical Iterative Framework for Qualitative Data Analysis.” International Journal of Qualitative Methods, 8(1). 2009.Available at http://ejournals.library.ualberta.ca/index.php/IJQM/article/download/1169/5199. 6 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 7 II. FINDINGS Question 1: What state-of-the art social media and social networking practices (including Web 2.0) are used by K4Health and other companies and organizations to reach audiences in the developing world? SOCIAL MEDIA IN KM LEADERSHIP AND GLOBAL HEALTH: THE CONTEXT Social media is a key pathway to developing a position of leadership in KM. Social media enhances engagement and communication among relevant communities; allows knowledge and data to be shared more broadly and quickly, and gives individuals the opportunity to participate in those discussions of greatest personal interest. Social media creates opportunities for K4Health to highlight established and new information, as well as to exhibit thought leadership through blog posts and commentary. By hosting sites and providing content curation, K4Health receives a significant benefit in brand awareness, name recognition, and reputation as a go-to site. The use of social media in businesses, governments, NGOs, and other organizations is rapidly becoming standard practice. For instance, a 2011 study by Booz & Co. reported that 96% of companies have an overall social media strategy.13 Social media has also begun to influence the field of international development. InterMedia’s study, Building Support for International Development (March 2012), divided its findings on “influentials” into “established influentials” and “new generation influentials.”14 The researchers define new generation influentials as including “institutional bloggers—individuals working for recognized development organizations such as Oxfam or the Center for Global Development— as well as independent bloggers who discuss development issues online but do not have development related careers outside the blogosphere.”15 “Influentials” are individuals whose opinions and comments are highly regarded and tend to create conversations about issues and at times convert others to their points of view. “New generation influentials” have created their impact through social networking activities, particularly blogging, rather than through more traditional channels. That many of these new generation influentials do not have careers in international development is particularly interesting to note. On June 15, 2012, the Philippine Daily Inquirer noted “social media [sic] have become critical tools in coping with natural disasters.”16 The article continued to point out that “the use of the social network [sic] also allows for instant reporting of health issues, making immediate 13 Vollmer, Christopher and Karen Premo. Campaigns to Capabilities: Social Media & Marketing 2011. New York: Booz & Company, October 2011. Available at http://www.booz.com/media/file/BoozCo-Campaigns-to-Capabilities-Social￾Media-and-Marketing-2011.pdf. 14 Debeljak, Klara. Building Support for International Development: Results and Recommendations from a Multi￾Country Study Aimed at Understanding and Communicating with Key Policy Constituencies. Washington, DC: InterMedia, 2012. Available at http://www.audiencescapes.org/sites/default/files/FinalReport.pdf. 15 Ibid. 16 Balana, Cynthia D. “Social Media Major Tool in Disaster Response.” Philippine Daily Inquirer. June 15, 2012. Available at http://technology.inquirer.net/12167/social-media-major-tool-in-disaster-response. 8 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION response absolutely critical, as in the cholera outbreak in Haiti and the dengue flareups [sic] in Thailand and Indonesia.”17 While social media is increasingly establishing its importance through various channels and types of engagement, it is still new and evolving. Although most organizations and individuals use social media to some extent, many government agencies, NGOs, and other organizations are still struggling to define how to use it. The findings and recommendations in the InterMedia study highlight both the acceptance of and the hesitancy to use social media in international development. For example, the study notes the importance of bloggers and recommends “sharing of online links to development issues and campaigns through social media [sic]” while at the same time declaring that “social media [sic] may not be the best tool for engaging influentials as most of them consider information conveyed via Twitter to be less trustworthy and more difficult to verify.”18 SOCIAL MEDIA WITHIN GLOBAL HEALTH Although still early in development, numerous organizations have begun to integrate social media into global health initiatives. Mobile health (mHealth), COPs, blogging, eLearning, and especially texting are among the most widely incorporated activities. Within the developed nations, the wide availability of bandwidth and Internet access has accelerated the use of blogging and COPs, as well as the use of Twitter. Of note, however, is the finding from the 2011 Pew Research Center Survey19 that people in lower income nations who have Internet access use social networking as much as or more than people in affluent countries. Interviewees commented that they find people in the field (USAID and implementing partners) are often more engaged than those in Washington in using social media to access otherwise unavailable information. Interviewees commented on how impressed they were at field staff and partners taking the time (one hour+) to watch videos and send in comments, especially from Asia. One example given was a filmed presentation shared with field staff, who watched the video and made comments to creators by email. The following quote, from the K4Health India interviews, is another example of the value people in the field place on video: I really like TED talks. I get a lot out of TED talks, you know short five minute lectures from eminent people in the field? . . . I personally find pages and pages of technical texts difficult to absorb, unless I have hours of time to distill it down to the key points. I like graphical presentations, trend analysis, I like things extrapolated.20 Mobility In developing countries, significant penetration of mobile phones has created a fertile environment for more widespread and effective use of social media. Research from the GSM Association21 reported that 65% of the African population has access to mobile phones; the 649 17 Ibid. 18 Debeljak, Klara. Building Support for International Development. 19 Pew Research Center, Global Attitudes Project. Global Digital Communication: Texting, Social Networking Popular Worldwide. December 20, 2011. Available at http://www.pewglobal.org/files/2011/12/Pew-Global-Attitudes￾Technology-Report-FINAL-December-20-20111.pdf. 20 K4Health User Experience Study, June 2012. 21 African Mobile Observatory (GSMA, A. T. Kearney, and Wireless Intelligence). Driving Economic and Social Development through Mobile Services. 2011. Available athttp://www.gsma.com/publicpolicy/wp￾content/uploads/2012/04/africamobileobservatory2011–1.pdf. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 9 million subscribers in the fourth quarter of 2011 are expected to rise to 735 million by the end of 2012. Africa is now the second largest mobile device market in the world, second only to Asia, and is the continent with the highest ratio of mobile to total telephone subscribers of any region in the world—over 85% (260 million) of the 280 million total telephone subscribers are mobile cellular subscribers. According to the 2011 Pew Research Center Survey,22 texting is most common among cell phone owners in two of the poorest nations surveyed: Indonesia and Kenya. In Kenya, 74% of the population owns a cell phone, and 90% of cell phone owners text. In light of the importance of mobility in Africa and other developing nations, along with its population’s affinity for social media, it is important that K4Health actively incorporate social media in its mobility planning. mHealth and social media should not be regarded or managed as distinct services or products. Mobility defines how information is distributed and increases K4Health’s ability to reach the intended audiences at the time information is needed, regardless of fixed broadband access. “Social media” implies the linking of the content and the channel. Current mHealth activities largely target the community population, and K4Health could likely improve its reach and efficacy by adapting similar approaches. One mobile strategy that K4Health could adapt for its use is Project Masiluleke, a South African project that broadcasts an HIV awareness message in unused text message space. According to Health Affairs, this effort produced a fourfold increase in calls to the national AIDS helpline.23,24,25,26 K4Health could similarly use texting to share information, highlight new findings, or create a quick search and find capability. State-of-the-art mHealth projects increasingly seek to ensure that technology and applications are available beyond single country borders and do not rely on smart phones. Text4Baby, for instance, is a program that provides education to pregnant women and new parents; the program reaches more than 20 million people in China, India, Mexico, Bangladesh, South Africa, and Nigeria. FrontlineSMS, particularly MedicMobile (formerly called FrontlineSMS: Medic), uses open-source software to support health services across the globe.27 Frontline SMS defines its target regions as “worldwide.” FrontlineSMS version 2 also recognizes the importance of Social Media capabilities that allow people to identify and converse with others with similar interests. The SmartGroup feature enables groups to be created dynamically based on specific criteria. “Users 22 Pew Research Center, Global Attitudes Project, 2011. 23 PopTech. “Project Masiluleke: A Breakthrough Initiative to Combat HIV/AIDS Utilizing Mobile Technology & HIV Self-Testing in South Africa.” Available at http://poptech.org/system/uploaded_files/27/original/Project_Masiluleke_Brief.pdf. (Undated) 24 Hyett, Chad. “Mobile Health in Developing Countries.” http://www.kevinmd.com/blog/2010/10/mobile-health￾developing-countries.html (Undated) 25 Johnson, Bobbie. “Text Messages Could Help Turn the Tide of HIV And Aids In South Africa.” The Guardian. October 24, 2008. Available at http://www.guardian.co.uk/technology/2008/oct/24/hiv-aids-text-message-project￾masiluleke. 26 Rawlings, Lauren. Project Masiluleke: Fighting HIV/AIDS through Mobile Phones. BroadReach Healthcare and Center for Health Market Innovations. April 2011. Available at http://healthmarketinnovations.org/sites/healthmarketinnovations.org/files/FINAL_ProjectM042011_0.pdf. http://healthmarketinnovations.org/sites/healthmarketinnovations.org/files/FINAL_ProjectM042011_0.pdf. 27 Frontline SMS. The Frontline SMS: Medic Story. Available at http://medic.frontlinesms.com/. 10 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION are encouraged to share views, experiences and ideas in an online forum and to provide feedback.”28 Twitter, Facebook, and Other Social Media Sites Recognizing that bandwidth remains an issue, the social media sites are providing service to developing nations using low-bandwidth technologies. Access to Twitter in Kenya, for instance, is via an inexpensive SMS or texting gateway that does not require a smart phone. Social network penetration in the Philippines has reached 95%, the highest in the world, and the country’s most popular website is not Google but Facebook, which boasts a penetration rate of almost 94%.29 The popularity of Facebook is the major driver behind a 46% increase in photo sharing in the Philippines in one year. The Philippines is also the eighth most popular country for Twitter use on a global scale, with a penetration rate of 16.1%. Internet use in the Philippines is at 29.7%. Of the approximately 140 million Africans on the Internet, 40 million are on Facebook.30 Table 1. Top 10 Low-income Developing Countries on Facebook31 Country (by rank) Users on Facebook Online Population D.R. of the Congo 60,500 365,000 Afghanistan 52,980 1,000,000 Mozambique 45,420 612,500 Mali 42,420 250,000 Burkina Faso 36,200 120,000 Niger 14,240 115,900 Sierra Leone 8,780 14,900 Burundi 6,740 65,000 Central African Republic 3,040 22,600 Chad 1,540 187,800 In 2012, 8 of the top 10 countries for Facebook Mobile penetration were developing nations.32 28 Vital Wave Consulting. mHealth for Development, The Opportunity of Mobile Technology for Healthcare in the Developing World. Washington, DC and Berkshire, UK: UN Foundation-Vodafone Foundation Partnership, 2009. Available at http://www.globalproblems-globalsolutions￾files.org/unf_website/assets/publications/technology/mhealth/mHealth_for_Development_full.pdf. 29 Russell, Jon. “Philippines Named Social Networking Capital of the World.” Asian Correspondent. May 15, 2011. Available at http://asiancorrespondent.com/54475/philippines-named-the-social-networking-capital-of-the-world￾indonesia-malaysia-amongst-top-10/. 30 Social Bakers. “Developing Countries on Facebook—TOP 10.” May 24, 2010. Available at http://www.socialbakers.com/blog/61-developing-countries-on-facebook-top-10/. 31 Ibid. 32 Social Bakers. “Facebook Hits 488 Million Mobile Users.” May 8, 2012. Available at http://www.socialbakers.com/blog/554-facebook-hits-488-million-mobile-users-infographic/. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 11 Table 2. Top 10 Countries for Facebook Mobile Penetration33 Country (by rank) Penetration of Facebook Mobile Nigeria 81.2% Brunei 80.8% South Africa 80.5% Malawi 78.3% Papua New Guinea 78.3% Namibia 76.7% Botswana 75.4% Zambia 73.9% Japan 72.1% Singapore 71.8% Gender Differences Studies that address the differential use of the Internet by gender consistently report greater usage by men than women, with less developed countries often having larger gaps. However, the cause of the difference is not clear. “Measuring the Information Society”34 (2011) postulates that lower educational levels, lower economic positions, and lower levels of Internet awareness among women in developing countries play an important role in women participating in the digital world at lower rates than men. The International Telecommunication Union notes, “Empirical evidence has shown that, under equal conditions of education or employment, there are relatively minor differences between men and women in terms of the use of technologies.”35 The data show especially strong correlation between awareness of technologies and usage. Activities A number of efforts are underway to increase the engagement of community members and to share information more effectively using social networking. Some of the initiatives involving COPs and other social media channels include: IBP Knowledge Gateway, initiated by WHO and USAID – The IBP Knowledge Gateway had wide recognition among our interview group. Although K4Health neither manages the technical running of the IBP platform nor controls its user interface/functionality, some people think K4Health owns and runs IBP. Others simply see the two as closely allied. In spite of technical “clunkiness,” the IBP platform hosts over 700 COPs, is widely known, and is seen to have highly valuable content and audience engagement. Several people commented that they would like to 33 Ibid. 34 International Telecommunications Union. “Measuring the Information Society.” Geneva, Switzerland, 2011. Available at http://www.itu.int/net/pressoffice/backgrounders/general/pdf/5.pdf. 35 Ibid. 12 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION see the Knowledge Gateway be more interactive. Specific suggestions included establishing blogs on the Knowledge Gateway, having additional content moderators, and having guests pose questions for discussion each month. Our interviewees were generally anxious to improve the IBP Knowledge Gateway and make it more social. mHealth Alliance (founded by Rockefeller Foundation, UN Foundation, Vodafone) – A well-funded endeavor with “lots of clout” according to our interviewees, the mHealth Alliance works mostly at the policy level and with national health plans, and is now focusing on maternal and child health care. A recent UN report on mHealth for development notes that “mobile technology has the potential to transform the approach to healthcare challenges in the developing world.”36 Development of mobile applications is key to the effective use of social media where Internet access is limited. The K4Health-led mHealth Working Group has reached out to the mHealth Alliance to seek a closer working relationship. WASHplus – WASHplus, a water and sanitation project, exemplifies good blogging practices and the use of social media to drive community engagement. People from WASHplus whom we interviewed mentioned that Twitter has been very effective in expanding subscriptions to WASHplus Weekly, a weekly online newsletter. An upcoming “big project” in Zambia will focus on Facebook, interviews, and podcasts. The team has also embraced blogging and will soon use Facebook as well as WordPress for blogs. WASHplus encourages readers to post pictures of water-related innovation on Facebook. Three other blogs are news aggregator sites, all of which appear on the WASHplus.org site. A still early stage blog called Innovation Exchange encourages sharing and collaboration among partners. AIDSTAR-One – AIDSTAR-One.com has incorporated numerous KM and social media good practices and offers realistic development time lines based on its experience. A senior technical advisor at AIDSTAR commented, “It takes three to fouryears to generate a user base; each quarter now sees big increases in subscribers and users.” The Promising Practices section of the site is both a good example of the emerging use of social media in a content-vetted environment and an illustration of the need to match funding duration with the time required to build active, engaged communities. Promising Practices provides a venue for implementing partners to discuss their work and opportunities to establish their technical expertise in HIV programming. User-generated content, however, did not begin to appear in reasonable volume until year 3. More interaction and conversation was anticipated before the loss of funding at the end of the year. AIDSTAR-One’s Promising Practices Database claims to broaden the definition of a “best practice” by including emerging, innovative approaches as well as proven practices. Nonetheless, all content was highly vetted according to our interviews. Over time, 36 Engelbrecht, Lezette. “Developing Apps, Developing Lives.” ITWeb Limited. June 12, 2012. Available at http://www.itweb.co.za/index.php?option=com_content&view=article&id=55696:developing-apps,-developing-lives. [Accessed June 2012]. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 13 AIDSTAR-One has also found that the percentage of users referred by Twitter and Facebook has continued to increase. The key factors used in evaluating good social media practices often have little relationship to the baseline “counting metrics” often quoted, such as the number of Facebook likes or Twitter followers. Instead, each instance is evaluated based on considerations such as the level of engagement of participants, the quality of the Social Network (for instance, does the network include influential members of the broader subject community?), the use of the most appropriate social media channels for the articulated objective, and the established synergies across social media channels. Ultimately, social media should drive outcomes. K4HEALTH’S APPROACH TO SOCIAL MEDIA Strengths The K4Health team has particular strength in three areas: K4Health both develops and participates in a variety of COPs that take advantage of discussion forums, blogs, and other means of engaging with the community. Many of our interviewees identified the IBP Knowledge Gateway with K4Health, although K4Health only manages the content and does not operate the technology platform. The SHARE is also a good example of a vibrant COP that integrates social media through the eSHARE forums and its blogs. The K4Health team has taken a leadership position in developing good uses of social media for family planning and reproductive health, as well as for other international development considerations. By leading the newly formed Social Media Working Group, K4Health is enhancing its position as a leader in social media, and in KM overall. Given the rapid evolution and change among social media sites, the “rational openness” of the K4Health team is a great strength. The team stays abreast of current social media trends, but evaluates and tests before embracing the latest sites. The K4Health team shows willingness to continue to evaluate new opportunities and sites, such as Pinterest, in the rapidly evolving and changing social media environment. They have two dedicated staff with social media experience who are trying new approaches and usages, including partnering with Jhpiego for the Mother’s Day “Tweet Up” (aimed to raise awareness of maternal mortality issues around the world). Other social media activities occur at a lower level, such as the recently established LinkedIn groups, sentiment monitoring and active listening.37 Google Alerts, use of video and webinars, and engagement in others’ social media activities are the major methods by which K4Health engages in active listening. Challenges K4Health’s challenges with social media involve both overarching issues of how to use social media and more practical concerns regarding specific applications. Please note that these challenges do not represent failures by the K4Health team; social media is a newly emerging field and K4Health is already on the forefront of experimentation. Instead, these challenges and 37 See glossary for sentiment analysis and active listening definitions. 14 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION issues should be viewed as areas where K4Health and USAID may want to pay particular attention as they learn from K4Health’s experience. Overarching Issues A major concern focuses on the unanswered question about whether information shared over social media is, or needs to be, vetted and how it links to K4Health’s role. Is K4Health a definitive source of facts, or should experiential sharing also be included? The confusion of these two perspectives tends to polarize views on the use of social media and has prevented K4Health from establishing clear directives and processes. This issue affects not only toolkits, but also all of the information shared across various channels. For instance, is a blog post a personal opinion or important experiential insight that contributes to the body of knowledge? Does the reputation of the person who is sharing make a difference? Do the comments on the post affect its importance or validity? K4Health has made good progress and shown openness in changing its perspective on social media from being a part of the promotional strategy to being a means to distribute information to more people. However, K4Health still needs to embrace social media as a means of community engagement, not of simply one-way or bidirectional communication. Social media is currently a centralized, distinct activity within K4Health. In order to benefit fully from its potential, social media needs to be integrated throughout the project, into everyone’s role, and across products. The next stage for K4Health is the integration of social media with the concept of KM leadership by using social media to drive many of the intangible assets, such as brand awareness and reputation, audience engagement, thought leadership, and community strength. Specific Applications Broadband access varies significantly across countries, and customizing social media for regions is critical, albeit challenging. With a few exceptions, much of the social media is currently focused in North America. However, the COPs are very active in developing countries and may be a good source of information on how to transfer their participation into multiple channels. K4Health currently “controls” most of its social media activity. Deriving value from social media will require K4Health to listen more effectively and to participate more on others’ sites—for instance, by commenting on others’ blogs, by monitoring comments and sentiment about issues, and by embracing negative and controversial comments found on others’ sites. Listening to conversations on non-K4Health sites will help to bring a range of perspectives and insights. Moreover, the comments expressing the greatest negativity or concerns about K4Health will not often be posted on those sites maintained or managed by the project. K4Health has had success with its eLearning activities. The programs, however, operate largely as solo activities for individuals. Social media technologies, including video, discussion forums, Twitter chats, and other channels could greatly enhance the efficacy of the eLearning programs. To paraphrase one interviewee, “Learners could meet other people KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 15 who are also taking this class and find out who in their country is an expert in the topic. It would also help learners with finding partners who are doing work in that area.”38 Methods to measure the outcomes and value of social media activities are in the early development phase. In addition, increased monitoring of the reach and engagement driven by social media activities will help to create a responsive culture. K4Health can then develop feedback cycles to allow the information to quickly improve services and products, and even define new ones. Question 2: What Role has K4health Played as a KM Leader? DEFINITION OF KM LEADERSHIP There is no standard definition of KM leadership but the one that the evaluation team settled on as most appropriate to the spirit of K4Health and KM practices at USAID is the following: KM is emerging as a potentially powerful public health intervention and a new skill set required for implementers by USAID and other donors.39 Therefore, KM leadership for health will facilitate the use of KM science and practice to improve global health outcomes by donors and implementing partners. The main objectives of KM leadership for health should be to: Facilitate KM within health organizations and USAID-funded projects so that they use KM to improve their own practices and outcomes and to share their experiences and lessons learned with others. Advance the science and practice of KM as a field, via improving measurement methods and sharing learning about KM for continual process improvement. There were more differences of opinion between interviewees (implementing partners and USAID) on how KM leadership should be implemented, with the main determinant being the relative knowledge the interviewee had with KM as a field. Those who were more used to traditional global health interventions focused on KM leadership as defining the best practices and scientific documentation related to KM. For example, one USAID interviewee expressed concern about the non-scientific nature of social media and asked, “What is the evidence telling us about the impact of KM on public health interventions?”40 Those who are more used to capacity-building and KM projects focused more on facilitating learning environments and iterative processes, and leading by example. For example, one highly experienced KM practitioner defined KM leadership as “making connections and creating spaces to communicate.”41 Growth of KM KM emerged in the 1970s as a management theory and gained a great deal of attention in the 1990s and 2000s as information technology (IT) made knowledge capture and access increasingly 38 Interview with USAID staff member. 39 While the impact of KM on public health is yet to be measured or proven, it is increasingly being asked for in USAID RFPs and RFAs, which indicates that implementing partners need to address it. 40 Paraphrased from USAID interview. 41 Paraphrased from interview with K4Health staff. 16 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION easy and cost effective. These emerging technological advances in decentralized data collection and analysis have made many KM processes easier to manage with lower infrastructure costs; the advances have also raised expectations for continuous learning for performance. As a result, most businesses, and in turn government agencies and non-profits, increasingly must incorporate KM processes and tools in order to remain competitive. Within the USAID community, there is increasing awareness among donors and implementing partners (as evidenced by the increase in references to KM in Requests for Applications [RFAs] and Requests for Proposals [RFPs]) that global health projects and organizations must engage in KM in order to be effective.42 KM at USAID As part of the Quadrennial Diplomacy and Development Review and initiatives under USAID Forward, USAID is focused on improving its internal performance through KM and learning activities. The goal is to improve USAID operations and impact by developing CLA methods within the missions (and eventually Washington as well).43 The strategy states that USAID knowledge is a development resource. This strategy is still in development stages; however, USAID has been using and experimenting with different forms of KM and social media tools. For example, the USAID intranet holds “developedia,” a wiki to help define core terms at USAID, modeled on state department’s “diplopedia” wiki. Various offices manage intranet blogs and COPs for internal communications and KM objectives. USAID is also now using Google Docs for collaborative document management. The internal KM approach is still fragmented and usually driven by bureau, office, or mission investments. For example, the Office of Transition Initiatives (OTI) hosts OTI Anywhere, its KM platform that allows its staff to communicate and collaborate anywhere in the world. The Bureau for Economic Growth, Education, and Environment has developed an expert locator that has been expanded to include experts from the Bureau for Democracy, Conflict, and Humanitarian Assistance and the Bureau for Global Health. The Office of Afghanistan and Pakistan Affairs has developed PakInfo, which tracks projects and performance information in Pakistan. The Bureau for Global Health has the Knowledge Management Services project, which serves its internal KM needs such as report writing and analysis, technology development and support, data management, and communications support. In addition to internal KM activities, USAID is also experimenting with external KM and social media tools, such as Facebook pages, USAID blogs, Twitter feeds, and LinkedIn groups. However, much of the external KM and social media activities focus on communications and advocacy efforts, rather than collaboration. The exceptions to this are program-focused activities, such as Microlinks, FRAMEweb, and WASHplus, which are intended to support collaboration among implementing partners and USAID. Other USAID KM Projects The evaluation team performed a very quick review of the three other KM projects mentioned to identify ideas and recommendations for K4Health. We also interviewed the KM managers for WASHplus, AIDSTAR, and Measure/Evaluation to identify areas of interaction between those 42 Interviews with USAID/implementing partners. 43 U.S. Agency for International Development. “Program Cycle Learning Guide.” Washington, DC: USAID. (Draft) KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 17 projects and K4Health; the results of those interviews are not reviewed here but rather scattered throughout this report. The WASHplus project has four different websites focused on specific topics, one of which is co-managed with the International Red Cross in the Netherlands. The three others are news aggregator sites, including content generated by WASH projects at USAID. One blog focuses on innovation and WASH topics. The main WASH website also includes Google Translate on all of its pages (similarly to K4Health). Microlinks is a longstanding project focusing on microenterprise learning information and knowledge sharing. Managed by QED, the approach and content venues overlap with K4Health, including eLearning, publications and tools, online discussion, and news aggregation. However, Microlinks also sponsors periodic in-person seminars that are then filmed and posted on its website, as well as field stories and a wiki on the value chain. Microlinks is one of the most well￾known USAID KM projects due to its long history and its frequent events. FRAMEWeb is a peer-to-peer collaboration network for Natural Resource Management practitioners. Run by DAI, FRAMEWeb features discussion forums, Twitter feeds on the front page, a tag cloud, Geographical Information System mapping of activities, and translation into French and Spanish. KM at Other Development Organizations The team’s analysis included reviewing other development organizations and their use of KM internally and externally. Due to the limitations of time, combined with the relatively new nature of KM, the team was unable to select organizations of comparable size or focus to K4Health. Instead, researchers asked interviewees to suggest which organizations they thought did KM well. The team then explored those organizations’ KM initiatives in order to get a sense of what other KM leaders were actually doing. The following is a snapshot of other KM activities run by these organizations, to offer some context for K4Health’s work and to give ideas for good practices to USAID and K4Health. The World Bank The World Bank is working to improve its internal performance via KM and social media through several methods, including a combination of SharePoint for document collaboration and Scoop, a social media platform. These tools are meant to address internal silos and separation between staff (especially when physically distant, but also within the same building); silos within Washington staff are more pronounced than within the regional offices. The goal of Scoop is to allow people to “narrate their work.”44 It also is meant to reduce the learning curve for new staff as it is used as a lifeline to help find internal expertise and support on a topic. Scoop is intended to replicate the “coffee break” conversations between professionals. As of now, there are approximately one thousand users of Scoop at the World Bank, and this number is growing. There is a significant culture gap in users that often (though not always) follows demographics; younger users are more likely to experiment and “get” the technology, meanwhile, the older users (especially those with longstanding institutional memory) who do become active are often seen as more valuable as they can contribute the most useful information. Many longstanding World Bank staff noted reluctance in changing their work patterns, while newer staff were open to newer approaches and saw the value more quickly. 44 Interview with World Bank staff. 18 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Field staff seemed to embrace it more readily than Washington staff (even though the siloing problem is felt to be more pronounced in Washington). Key elements the support team identified as part of their pathways to success include: Reverse mentoring, by which younger staff mentor older staff on how to incorporate a particular technology into their everyday work habits, replacing existing methods (including email and saving documents onto their hard drive) and demonstrating the value of these new methods. Champions at senior levels, and many opportunities to reward social behavior. In one division, during Friday breakfast meetings, the most senior staff person will highlight the “best” contribution by a member of their division that week. Identifying social contributions as an expected part of staff members’ daily work, not something extra. Training and reinforcement by senior management are critical to allowing this mindset to occur. Identifying cultural barriers, especially the “knowledge is power” thinking among some staff. The culture needs to change to support individuals being identified as experts through their sharing of knowledge with others. Blogs by staff and Q&A sessions where the best answer is ranked and rewarded are two ways they promote this culture. Pan American Health Organization (PAHO) In April 2008, PAHO45 launched its integrated KM and communications strategy, which reorganized its web content and templates to improve usability and promote the use of social media (YouTube, Twitter, blogs, and wikis), virtual conferencing, and user portals. Unfortunately, due to time constraints, the consultants were unable to interview anyone from PAHO to discuss this topic in more depth. Peace Corps The Peace Corps is an interesting organization for KM as it has a large network of highly committed volunteers who are technologically savvy, geographically dispersed, and highly motivated to reach out to each other during and after their Peace Corps experiences. However, a 2010 study46 demonstrated that despite an identified need and potential for usage, KM approaches are significantly lacking in the Peace Corps and among Peace Corps Volunteers (PCVs). Further evidence is shown by the fact that the Peace Corps wiki,47 a highly popular and active site about the Peace Corps for current and returning PCVs, is completely independent and maintained by PCVs on a volunteer basis, funded by donations from the community. The content is sourced from the Freedom of Information Act requests and contributions from the community. There is also the Peace Corps Journal,48 where PCVs can submit stories about their 45 D’Agostino, Marcelo. PAHO/WHO Web 2.0 Strategy: Connecting People, Institutions and Content in the Information Society. Available at http://www.slideshare.net/marcelodagostino/paho-web-20-at-web-for-development￾conference. 46 Gilbert, Engels, Joseph Morabito, and Edward A. Stohr. “Knowledge Sharing and Decision Making in the Peace Corps.” Knowledge and Process Management: The Journal of Corporate Transformation, 17(3):128–144. 21 July/September 2010.. 47 Peace Corps Wiki, 2012. Available at http://www.peacecorpswiki.org/Peace_Corps_Wiki. Accessed May 2012. 48 Peace Corps Journals. Available at http://www.peacecorpsjournals.com/?GetStarted. Accessed May 2012. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 19 experiences and read about others activities. K4Health also supports multiple Peace Corps toolkits. K4HEALTH’S APPROACH TO KM LEADERSHIP We found that K4Health’s approach to KM leadership has three major elements: Supporting and promoting KM professionals within the implementing partner community. Modeling and experimenting, especially in field research and activities. Championing and advocating for KM within the USAID community. Supporting and Promoting KM Professionals Most KM initiatives begin with information management processes and tools—data collection, document repositories, and contact details. The transformation occurs when organizations recognize that knowledge is applied information and that just having the information available is not enough to effect change. This differing definition also shows that global health organizations are in different stages of KM implementation and reveals that different individuals/roles need different types of knowledge and information in their day-to-day work.49,50 KM leadership involves supporting organizations where they are right now as well as helping them with the transition to the next stages. Several interviewees remarked that the first few K4Health KM working group meetings had many new attendees with the title of “knowledge manager” of their organization or project, yet some did not entirely know what that meant. They were also unsure how to get buy-in from other team members or their organization on moving KM forward. As a result, much of the KM working group’s focus in the first 18 months was on reaching a general consensus on definitions and advocacy tools among the KM community within USAID global health community. As one K4Health staff member stated, “KM is hot—where do we start?”51 According to the K4Health team, the KM working group is responding to this need for clarification in the global health community by offering professional support for KM leaders, including centrally gathering KM resources and identifying the missing pieces in KM. The project collects current resources in KM for health and international development within the K4Health website toolkits. The team has different sub-working groups that identify case studies and illustrative examples, gather job descriptions for knowledge managers, and help develop guidelines and tools on KM strategy documents. The KM working group is now focusing on identifying applied information through the capture and use of case studies. Modeling and Experimenting Another crucial factor in supporting and promoting KM mentioned by interviewees was the facilitation of an environment that is welcoming to innovation and not overly focused on failure 49 Okimoto, Jennifer. “Industry Trends: The Evolution of Knowledge Management (KM 1.0 vs. KM 2.0).” Slideshow. IBM Corporation, September 2007. Available at http://www.slideshare.net/elsua/the-evolution-of-knowledge￾management-km-10-vs-km-20 50 Griffiths, David. “The Evolution of Knowledge Management? No, time to Evolve the D-I-K-W Hierarchy!” The Knowledgecorp’s Blog. February 3, 2012. Available at http://theknowledgecore.wordpress.com/2012/02/03/the￾evolution-of-knowledge-management-no-time-to-evolve-the-d-i-k-w-hierarchy/. 51 Interviews with K4Health staff. 20 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION rates or risks. Experimentation and course correction are important features for leadership in emerging technologies and processes, especially as many implementing partners do not have the resources or the freedom to experiment with different techniques. KM leadership involves modeling good practices, being on the cutting edge of new techniques, and trying out new approaches with the expectation that some of these will not succeed. Through the KM working group and other working groups such as mHealth and social media, K4Health identifies gaps in current knowledge and facilitates filling those gaps. For example, several interviewees mentioned the K4Health monitoring and evaluation (M&E) guide as an important resource for helping create a standard set of indicators for KM/information projects. Through field-based pilots (such as Malawi, Nigeria, Bangladesh, Southern Africa), K4Health models KM for public health using a variety of different tools and approaches. For example, the Nigeria project is focusing on online learning (eLearning), while Bangladesh is experimenting with giving community health workers netbooks containing documents and other resources. Through the field initiatives, K4Health also has the opportunity to perform analyses on KM in the field—how to design KM programs, how to measure these programs’ impacts, and how to course correct when necessary. Championing and Advocating The final element of KM leadership is demonstrating and coordinating high-level visible support for KM initiatives within USAID global health projects. Many individuals interviewed remarked that it can be hard to advocate for KM within their organizations or projects without high-level champions demonstrating its value and benefits. This need was especially crucial in a context of shrinking budgets, increasing need for impact, and relative lack of familiarity with KM initiatives—especially related to social media and technology investments. KM leadership helps demonstrate the value and return on investment by KM initiatives for improved global health outcome. The K4Health team is active in blogging, publishing, and presenting emerging trends and new research related to KM and health. They have started experimenting with using social media platforms like SHARE to expand their communications. The proposed new eLearning platform will continue to expand the role of social media for increased engagement of learners and exerts. K4Health, through its management of the Health Information and Publications Network (HIPNet), has been central to the development of the M&E guidance document, which offers the global health and international development community a standard approach for evaluate KM and information projects. Several interviewees referenced the M&E guide as highly important to their development of M&E indicators for their work. Strengths Overall, interviewees had strongly positive feelings about the work of K4Health, in KM generally and as a KM leader. They stated that K4Health has played the role of preeminent KM facilitator in the global health community, serving as an honest broker among partners. Interviewees found K4Health to be an honest broker between potential competitors, while recognizing the subject matter expertise of partners. Interviewees appreciated the role that K4Health played in coordinating and organizing the KM community to accomplish many KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 21 important tasks. As one interviewee said, K4Health is “bringing partners together around public health through KM, which no one had done before.”52 The K4Health team demonstrates leadership in the form of committing personnel, technology platforms, and their communications networks to address major questions with the KM community. Explicit examples given included: Organizing and facilitating working group meetings, such as KM, social media and mHealth. Generating support, advocacy, collaboration, and champions for KM among implementing partners. Ensuring and contributing to KM tools and resources through K4Health working groups. Writing peer-reviewed publications on KM and information for development. Offering space and sharing ownership of online toolkits. Experimenting with new dissemination methods such as mobile applications, Facebook pages and Tweet-ups. Serving as the aggregator/compiler of information on FP/PRH. Providing original research on information needs, such as social network mapping and needs assessment. Within the core KM/global health community, there is strong brand awareness of the K4Health project and its working groups, website, toolkits, and other products. The project also has a strong, positive reputation for being an honest broker and curator of good content. Technology Platform Management As technology becomes obsolete, migration of existing communities and data needs to be built into the project planning, something that K4Health has been aware of and working on. For example, K4Health has been using the MSH-based GHeL platform to deliver eLearning courses for several years. However, K4Health, MSH, and USAID have collectively recognized that the platform had come to the end of its lifespan; as a result, K4Health is building a new eLearning platform that integrates social media elements, as well as current technological standards. K4Health is also aware of the current limitations of the IBP Knowledge Gateway platform and has been advocating for technological improvements to the site, which is managed by the World Health Organization (WHO). K4Health has been active in promoting usability and functionality improvements as part of its involvement with the IBP. In addition, K4Health has explored alternative platforms for similar types of COPs, such as the South African developed and hosted SHARE platform in Drupal. These efforts are a form of KM leadership as they are about keeping on top of the technological changes that support KM efforts. 52 Interview with implementing partner staff member. 22 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Challenges Lack of Knowledge about K4Health’s Internal KM Several interviewees mentioned that they saw the modeling of internal and external KM practices as a key element of KM leadership. As many members of the KM working group are primarily focused on internal KM activities, many are actively looking for good practices for using KM to improve internal project performance. Some interviewees mentioned the Population Services International (PSI) internal KM approach and SocialCast as good examples of one model for internal KM. While examining K4Health’s internal KM system was clearly out of scope of this evaluation, the linkage to KM leadership made us explore this element a little further. When we asked some interviewees about their perceptions of K4Health’s internal KM, they were unaware of how K4Health used KM for their internal project management. In some cases, this lack of awareness lead to a perception that K4Health does not offer a good model for internal KM. When we discussed this finding with K4Health, they shared with us some details of their very robust internal KM processes and the work they have been doing over the past few years to improve them. They also mentioned that, to date, they had intentionally not shared details about their internal KM systems with the KM working group or other implementing partners. They were concerned that it might interfere with their role as facilitator and honest broker if they were seen as focusing too much on their own practices and efforts. However, they acknowledged that since the KM working group is now more seasoned and relationships have been established, it may be a good opportunity for K4Health to share their internal KM experiences as a case study or possible model for others to explore. Role of K4Health in Content Curation Within both implementing partners and USAID, there was some confusion and disagreement around whether K4Health should compile all content, regardless of quality, or vet and “stamp with approval” certain types of content. USAID senior leadership seemed to view K4Health as a “one place for best-available information” on global health, especially information related to family planning and reproductive health.53 As one interviewee in India stated: “[V]ery few websites, including K4Health, have a benchmark regarding the quality of documentation. So that is a barrier. If you require information then you have to shift through the entire gamut. It could be a poorly documented report or it could be a refined one.”54 Other interviewees, however, when presented with the idea of K4Health playing a vetting role, thought strongly that K4Health’s role was to aggregate all content and let users sort out what they thought was useful.55 Interviewees viewed toolkits as being uneven in quality—some are hard to use and include items of dubious quality, while others are highly vetted and more usable. When probed, many interviewees and K4Health staff acknowledged that the reason for this variance was likely due to K4Health sharing the responsibility for managing the toolkits with implementing partners, some of whom use different criteria for inclusion in the toolkit. One acknowledged challenge stated by 53 Interview with USAID staff member. 54 K4Health User Experience Study India, June 2012. 55 Interviews with implementing partners. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 23 K4Health and other interviewees is that this delegation of management responsibility, while supporting K4Health’s role as “honest broker,” also led to inconsistency between toolkits. 56 It should be noted that during the period of the evaluation, the K4Health website was undergoing a redesign; the main part of the website had been transitioned to the new format, but the toolkits were still in the old format. The mHealth toolkit was relaunched during the drafting of this report and preliminary inspection showed improvements in usability. However, it is not clear whether the redesign of the toolkit templates will improve the quality of the content where it was found to be inconsistent. There were some strong concerns by a few USAID interviewees about the difference between evidence-based information (and the derivative tools that come from it) and experiential information, which social media excels at producing. A job aid for community health workers on the use of injectable contraception is an example of an evidenced-based tool, while personal accounts of how individual workers used the job aid or the modifications they had to make to their work or the document as a result is an example of experiential information. There were a few concerns that integrating social media into K4Health tools and products would lead to a conflation of the two types of information unless done thoughtfully. There were also references to past attempts to integrate social media (such as rating and comments) into the K4Health website that were considered unsuccessful due to users’ lack of use and as a result, discontinued. Technology Platform K4Health has inherited or been directed to use existing technology platforms in several cases, which has caused some limitations in both dealing with the inevitable obsolescence of platforms and responding to emerging technology changes, such as social media integration. While understanding that there are political and historical reasons—some of which may be highly beneficial to the project as a whole—for these decisions, USAID and K4Health recognize that there can be challenges associated with using an existing platform that is outside of the control of the project. RATIONALE FOR KM FOR HEALTH One of the questions asked of us by USAID was about whether there is a strong rationale for having distinct global health-focused KM leadership. The answer by interviewees is unanimously yes. The reasons given center on the need to ground the activities of KM leadership—such as its examples, tools, case studies, and community network—in a specific purpose and focus area. While KM is applicable to the international development as a field, having a specific project which focused on providing KM leadership in global health provides the following benefits: Uses the expertise and emerging experiences of the health community in KM and related fields to support and drive leadership activities, something that would be diluted in a broader scope. Narrows the scope of demonstration of KM interventions to make them more manageable. 56 Evaluating the quality of resources within each toolkit was outside the scope of this evaluation and should be performed by subject matter experts in the topics addressed by the toolkits. However, cursory review of the toolkits showed significant differences in layout and quantity of information that didn’t seem to be related to the topic area. The gender toolkit was mentioned by several interviewees as being particularly exemplary of the “kitchen sink” approach. 24 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Makes explicit K4Health’s leadership among other KM leaders as a “go-to” project for health-related KM. Allows the KM tools and activities modeled and offered by the K4Health’s KM leadership to be answerable to public health goals and initiatives. Links K4Health KM leadership focus areas to global health activities and impact measurements. However, it was also explicitly mentioned that while the KM working group should focus on KM for health, it should not be restricted to only those working in KM for health, as it is potentially a useful place for other KM practitioners to collaborate cross-sectorally. The idea is that while the KM working group is focused on KM for health, inviting members from other sectors and other KM support projects, like KM4Dev, will enrich the experiences of the working group. Therefore, K4Health has been reaching out to other groups and KM experts to present and share information on KM initiatives in other sectors. One leading development-focused KM group is KM4Dev, based out of Europe and founded in 2000. The KM4Dev model is more of a COP, made up of experts who have been working in KM for years. The KM4Dev community focuses more on cross-sector (i.e., water, sanitation, agriculture) problem solving.57 KM4Dev’s approach is inclined to be more social because they focus on arranging events that to bring people together. Effectively, they serve as “a convener for people looking for solutions.”58 One interviewee explained that the difference between the approach of the K4Health team and KM4Dev group is that K4Health addresses KM by creating and facilitating the KM working groups, with the focus on developing products and documents around KM. The goal of the working group is to primarily create and identify tools and resources for the community, with the secondary outcome of developing a community of practitioners. There was a sub-question related to the appropriateness of the KM toolkit being housed on the K4Health website. Findings indicate that since there is a strong rationale for a targeted KM leadership for health project, having the KM leadership resources on the K4Health website is a good idea, as much of the content may be cross-linked with toolkits or blogs. It marks the KM leadership work as firmly grounded in health, something that is often important for improving the question of KM’s validity as a whole. However, there are further recommendations in the next section on how to make the toolkits more robust and accessible. Question 3: What recommendations would K4Health and other companies and organizations offer for the future use of social media and social networking as part of an overall KM strategy? KM AND SOCIAL MEDIA KM and social media are not distinct from each other. Rather, KM includes social media and social networking practices and technology, and these practices and technology will be more and more integrated into successful KM processes in the future. 57 Knowledge Management for Development. “About km4dev.org.” January 10, 2009. Available at http://www.km4dev.org/notes. 58 Ibid. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 25 KM is an evolving field. Traditionally focused on core, vetted, structured information—such as document repositories, databases holding quantitative data, and summary descriptions of activities—current practitioners of KM understand that social networking extends, adds to, feeds, and reinforces this core data set. Figure 1 demonstrates the current evolution of the K4Health KM approach—starting with the information systems created by INFO and other predecessor projects. The next stage after creating these central information systems entails communications, advocacy, training and technical assistance—a proactive outreach strategy to target audiences with the collected information and apply this information for development objectives. The third stage is where social networking is added in. While an aspect of social networking has been a part of this and previous projects in the form of COPs, social networking was not a central element of the projects. Figure 1. Map of KM Services and Products Conversation (peer-to-peer, Social Media & Social Networking, autonomous) • COPs • Working groups • Informal/F2F Communications & Advocacy • Blogs • Social Networking channels • Newsletters and mailing lists Information Management • Library/repository Training & Technical Assistance eLearning Social Media discussions via forums, Twitter, or other channels Capture/starting point Sharing/synthesis Conversation 26 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION EVOLUTION OF KM VIA SOCIAL NETWORKING When social networking is fully integrated with information systems as part of a KM system, many interconnecting benefits can emerge. Some possibilities include the identification of formal good practices and lessons learned, translation of scientific evidence into implementable practice, capture of real world feedback on existing challenges and opportunities, identification of edge cases59 and exceptions to “good practices,” and exposure of emerging patterns and trends that need additional investigation. To illustrate this interaction and the different roles of different types of information in an effective KM system, we use a tree as our metaphor (see figure 2). The trunk of the tree is our core data set; this core data set comprises information that is key to our KM system. It may be a toolkit of documents, a database of peer-reviewed journals, indicator data on project performance, and so on. The crucial element is that this core data set is vetted and managed following explicit standards and practices, which here are represented as the roots of our tree. The leaves of our tree represent experiential knowledge, usually generated by social networking. These are informal comments, observations, and rankings related to different topics and the core data set, brought together by formal lessons learned and good practices (indicated by the branches). The aspect of the metaphor that is most important is that each part of our KM tree feeds into the other parts. The roots nourish the tree and keep it grounded. The trunk keeps it upright, and the branches organize and reach outwards, while the leaves capture experiences in the world and translate them into energy to keep growing. The connection between the elements is essential, or the entire tree will not grow. 59 An edge case is when one or more parameter (such as literacy, ICT access, or demographics) in your approach impacts a minority of users and would require an alternative to what works for the majority of situations. The parameter impact often occurs at the parameter’s minimum or maximum, hence the name “edge”; it refers to situations on the edge of the requirements. For example, if 90% of the time, a solution works, but 10% of the time (for the poorest or most remote, etc.), it does not, identification of this 10% constitutes an edge case that may require the design of alternative approaches. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 27 Figure 2. Knowledge Management as a Tree Metaphor Limbs Formal best practices/lessons learned The transition between experiential and core data and the path by which both core and experiential knowledge feed each other. Roots The process for grounding and vetting the core data in structured, scientific approaches and other criteria. Trunk Core, vetted knowledge Information derived from scientific evidence and official analyses/data, which are fed by lessons learned, vetted practices, and experiential knowledge. Leaves Experiential knowledge via social networking Knowledge based on experience, which feeds lessons learned and ultimately impacts core data and core data processes. 28 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION VOICES AND VALUE OF SOCIAL NETWORKS VIA SOCIAL MEDIA Recent studies60 of social media in developing countries have found higher levels of involvement in countries with great civil unrest and dictatorial control. One possible reason for this fact is that humans have a strong need to feel heard and have their experiences acknowledged. In time of civic conflict, one of the few avenues available to express this need is social media; a review of blogging in Iran and Iraq supports this theory.61 People also feel empowered when their experiences cause changes in others’ actions and beliefs. Interviewees commented on how COP participants from developing countries actively contributed to discussions at a higher rate than their Western counterparts, despite having less access, less familiarity, and more cost associated with their contributions. This fact shows the relative value and importance of these COPs in the professional lives of the targeted audiences. Surveys and other analyses bear out these findings; a survey held after a COP discussion on the Knowledge Gateway discovered that 90% of respondents reported being very or somewhat satisfied with the content; 80% reported they would use the information in their work; 43% reported reading materials recommended in the discussion; and 54% forwarded forum postings to others outside of the discussion.62 It is important to remember that COP participants are subject matter experts of their own experiences and contexts. Their expertise must be tapped into for an effective KM system. Social Networks also allow those of us working in international development to have unprecedented abilities to communicate and interact with individuals and communities at a fraction of the investment and time of traditional methods. There must always be an audience analysis to value and engage the right individuals as part of the social networking of any KM process. The audience value comes from the location and positions/roles of those participating. For example, a project may have four thousand likes on Facebook, but most of those may be from individuals who are not part of the targeted audience. A COP can have only 45 participants, but if those participants are both part of the targeted audience and/or have wider reach into those audiences, those 45 individuals are more valuable than the four thousand “friends.” Question 4: What intangible assets does K4Health offer regarding social media and KM? 60 ReliefWeb. Program on Humanitarian Policy and Conflict Research. 2009–2012. Available at http://reliefweb.int/organization/hpcr. 61 Al-Ani, Ban, Gloria Mark, and Bryan Semaan. “Blogging through Conflict: Sojourners in the Age of SSocial MMedia.” Paper prepared for International Conference on Intercultural Collaboration, Copenhagen, Denmark, August 2010. [online] Available at: . [Accessed June 2012]. Paper prepared for International Conference on Intercultural Collaboration, Copenhagen, Denmark, August 2010. 62O’Brien, Megan and Catherine Richey. “Knowledge Networking for Family Planning: The Potential for Virtual Communities of Practice to Move Forward the Global Reproductive Health Agenda.” Knowledge Management & E￾Learning: An International Journal, 2(2). 2010. Available at http://www.kmel-journal.org/ojs/index.php/online￾publication/article/viewFile/60/42> KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 29 DEFINITION OF INTANGIBLE ASSETS According to Wikipedia, intangible assets are “identifiable non-monetary assets that cannot be seen, touched or physically measured, and are created through time and/or effort and that are identifiable as a separate asset.63 Key elements include the following: Elements of the asset may exist legally in the public domain or be owned by another party but the asset in totality involves processes, know-how, and reputation that are a core part of the asset’s value. These assets can be transferred, extended, nurtured, and preserved; conversely, they can be lost if not nurtured or protected. The nurturing and extension of these assets requires an investment from the organization in terms of money, level of effort, and other resources. The valuation of these intangible assets is measured in either the cost of recreation from scratch or cost of lost productivity, impact, and/or other negative outcomes of the loss of use/access to the asset. Roughly speaking, there are five elements of intangible assets relevant to this evaluation. They are: Brand awareness and reputation. Audience engagement and community strength. Technology investment. Content and data repository. Special expertise and thought leadership. Brand Awareness and Reputation This intangible asset is also sometimes referred to as “good will” in the for-profit valuation world. In social media, marketing, and communications, brand awareness and reputation refer to how well known a product or service is to its target audiences, and how favorably the target audience views it. Thought leadership is an element of brand awareness and reputation (in combination with content and data repository) as the purpose of thought leadership is to associate forward thinking with the organization. Examples of positive brand awareness and reputation by the K4Health project include: Core audience awareness of the URL, toolkits, and content topics focused on by K4Health. Recognition for being an honest broker that works with diverse implementing partners. Reputation for quality content. Association with USAID and WHO, which reinforces reputation. High SEO status for many core searches. 63 Intangible Asset. Wikipedia. http://en.wikipedia.org/wiki/Intangible_asset. (Undated) 30 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Valuation/Loss The valuation of this asset relates to how long it would take to build awareness and reputation from scratch. It takes on average between 18 months and 3 years64 to build a strong audience for a content-rich website, and even longer to build a dedicated audience needed for engagement (see next item). Reputation also takes an equally long time to build, especially when the opportunities for engagement with users are sporadic. Audience Engagement and Community Strength This asset relates to how habituated the core audience is with participating in community￾building and how strong the community is. This asset is one of the most important intangibles for KM and social media, since they rely directly on community contributions and management for content creation and synthesis. Examples of positive engagement by K4Health include: COP development. Workgroup commitment, as evidenced by development of deliverables and toolkits. Valuation/Loss The valuation of this asset relates to the value of contributors, especially from developing countries and from professions/demographics being targeted by the project. The other source of input from these contributors is usually limited to focus groups (expensive and small scale), surveys (limited in scope), and personal experiences of project staff who have worked overseas. With audience engagement and strong communities, the ability for a project to be more responsive—and even partially directed by the needs of the audience—increases greatly. Technology Experience and Code Base This asset relates to the investments K4Health has made in technology platforms that are not necessarily owned by K4Health (due to copyright, open source, or public domain reasons). The investments by K4Health include: Its main website, www.k4health.org. Toolkits application. POPLINE website. Photoshare website. IBP platform (the improvements for usability that have been undertaken by WHO). SHARE (online collaboration platform, built locally in South Africa). GHeL’s new platform (currently under development). 64 As reported by interviewees and evaluators’ personal experiences. The building of an audience is highly variable based on many factors, including topic, goal for the size of audience (and their demographics), saturation of market, and marketing strategy. The outcome of online research we reviewed cited anywhere from six months to seven years to build an audience, but with very divergent types of content and goals, making the citations not useful for this report. One interviewee with a great deal of content experience in global health mentioned that she found it takes three years on average to get a good audience for any content site for USAID global health topics, and much longer to get engagement. That estimate is also presuming that there is an active content management strategy to keep content fresh and useful to the audience during those three years. This finding mirrored our own professional experience, though there are few published resources to validate this experience. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 31 Sites4Dev/OpenAid (soft launched and moving forward). The mobile app, ACE. Valuation/Loss The valuations of these assets are multifold. Investment in technology offers reduced cost and increased opportunity for expansion. By developing a platform that meets most requirements of project sites, K4Health has greater ease of expansion and duplication. K4Health can take one successful application and replicate it for another project or site with much lower cost, higher speed, and higher quality.65 K4Health also has experience with adjusting the platform to accommodate real-world needs and then measuring the impact of those improvements on user involvement. This iterative feedback cycle gives the team greater insight into the needs of users and future site designs. Since K4Health sites are built on the same platform, the ability for platforms to be interoperable is much greater. For example, sites have the same underlying code to manage permissions by users for access to different content, editing roles, etc. . (Even if the actual models are different, the architecture being the same makes migrating user access a lot easier to manage.) They also have the same data model for core content, making sharing content a lot easier. Content and Data Repository and Taxonomy One of the easiest assets to measure, content and data repository are often overlooked, especially when the information is a combination of original content and aggregated data from other sources. Another element of this asset is the thought leadership content that is offered by K4Health in the form of the blogs, toolkits, and international reproductive health taxonomy defined by POPLINE. Examples of content and data repository assets include: IBP COP archives (700+ communities). The content of K4Health website (50+ toolkits, blogs). The records in POPLINE (350,000+ articles). The Photoshare image collection (20,000+ images). GHeL (50+ courses, records of 60,000+ users who completed one or more courses). Valuation/Loss There are four elements to this set of assets: 1. Capture of content, data, and information in a centralized location. In some cases, the data are unique to K4Health. In others, the data may originate elsewhere but be is harder to find and gain access to without the K4Health repositories. The K4Health repositories also protect some data and information, which may be lost if the original goes offline. 2. Organization of the data, including taxonomy, description, authorship, source, and so on. Organized data are extremely important for access and cross-integration with other data sets, especially as we move into a world of “big data” (see future thinking for more details). 65 Caveat: There’s a cost of any project taking on too much of IT role, which can undermine a KM role—too much resource diversion, too much of a website production focus. 32 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 3. Access to the data. The above two elements lead to the third, which is increased access for development experts, researchers, and professionals to a rich repository of information. 4. Standard setting. The K4Health repositories, as part of K4Health’s KM leadership for health activities, set the standards for what information should be captured and how it should be organized. For example, POPLINE taxonomy is the defacto taxonomy for the international population and reproductive health libraries.66 This role shows the extreme value of the POPLINE database—and the database managers’ work on taxonomy—to the global health community. The valuation of this asset is made more extreme due to the ongoing challenge USAID has had in preserving these digital content and data repositories when contracts and cooperative agreements end or transition to new implementers. There are multiple examples of rich data and content repositories disappearing at the conclusion of a project, with perhaps a copy of the data housed on a CD-Rom with a USAID project manager or the implementing partner, resulting in data that is inaccessible—and in many cases unknown—to the rest of the development community. As of the date of this evaluation, USAID has been limited to capturing project documents in its Development Experience Clearinghouse (DEC) and has not had the capacity to centrally store and make available multimedia, especially software and databases. The USAID/Management Bureau will soon launch a database repository, but there is limited information on how this work will address the issue of loss of content and data.67 Special Expertise and Thought Leadership The final asset relates to the KM systems, processes, and approaches that K4Health uses and has identified as good public relations. How it operates, manages, nurtures, and extends these various assets—and shares that information with others—are key parts of these assets. Examples of these assets include: White papers on how to manage community forums and other good practices/lessons learned. Peer-reviewed publications on information needs assessment and social network mapping. Publications such as the M&E guide. Authors of blog posts and contributors to KM COPs. Tacit knowledge by staff about KM development/management, including in-depth field experience by key staff on how to nurture COPs and KM in the field. Valuation/Loss While K4Health has been prolific in its publication and sharing of best practices and findings related to KM in formal and informal settings, there is still a great deal of tacit knowledge held by the team, especially those members working in the field. As this is an emerging field where there are vastly more new knowledge managers than experienced ones, the value of K4Health’s experience remains high. 66 INFO evaluation findings, personal notes of Siobhan Green from interviews with POPLINE staff. 67 The evaluation team reached out to the USAID staff in charge of this new database repository but did not receive any response during the time period of the evaluation. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 33 III. CONCLUSIONS AND RECOMMENDATIONS SOCIAL MEDIA K4Health’s Use of Social Media Strengths Creating and supporting various virtual communities. Assuming leadership position in determining how to get value from social media and how to execute social media. Openness to emerging social media opportunities and willingness to explore and experiment. Recommendations Organizational and Strategic Although K4Health is currently exploring social media, it remains somewhat peripheral to the overall thrust of the team. We suggest more assertively defining and treating social media as integral to the definition of KM. Social media recognizes that knowledge does not change in finite, discrete steps, but evolves over time. When social media is fully integrated operationally and organizationally, the organization becomes a “social business,” not an organization using social media channels or tools. K4Health currently incorporates some of the communications capabilities of social media in selected parts of the KM logic model; we suggest that social media be infused throughout. Back in 2010, Umair Haque of Harvard defined the social business and distinguished it as follows: Most social media [sic] strategies have one or more of three goals: to push product, build buzz, or engage consumers. None of these lives up to the Internet’s promise of meaning. They’re just slightly cleverer ways to sell more of the same old junk. But the great challenge of the 21st century is making stuff radically better in the first place.68 K4Health staff should continue to support a social media expert, but make social media part of everyone’s job. K4Health staff should engage with their communities and include how best to use social media to achieve their objectives within the K4Health project in their plans. Specific actions will vary across project results and individuals’ roles. Illustrative considerations include possibilities such as enhancing eLearning with community engagement (for example, via tweet-ups of students taking a common course, discussion forums, or virtual conferences about a topic); contributing to the KM leadership position by establishing a blog that features an array of experts; posting on others’ blogs; and actively listening on Social Networks as an additional input to research, measurement, and evaluation. K4Health should establish an internal social media network. Internal social media networks enable staff to understand the value of this type of engagement, learn to use the channels most effectively, and develop the skills necessary for impact. Internal networks also help to 68 Haque, Umair. “From Social Media to Social Strategy.” Harvard Business Review. April 1, 2010. Available at http://ocvets4pets.com/archive21/From_Social_Media_to_Social_Strategy_-_Umair_Haque_- _Harvard_Business_Review.pdf. 34 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION eliminate silos and create a more “social” culture; effective external social media often reflects the internal culture. A few organizations have implemented new, customized platforms, such as The World Bank’s “Scoop.” Initial steps, however, require little investment and can be built on freeware that provides the basic functionality needed. The greatest focus should be on the processes and behavioral changes required to create a more social culture. Tactical K4Health should incorporate active listening69 techniques to understand both where people are talking online and what is being said both about topics related to family planning and reproductive health and the K4Health project and products. For example, the K4Health Social Media team has set up a number of Google Alerts. Additional information could be gleaned broader listening tools that sweep multiple channels to more active engagement and “lurking” on more sites to hear and understand the conversation. K4Health staff should engage with their communities on other groups’ sites. For example, staff could comment on blog posts, participate in LinkedIn discussions of non-K4Health groups, follow and respond to tweets, ask questions, and post thoughtful responses on various social networking sites. However, listening is not enough; K4Health has to implement change and improvements based on this information. It should also show its users that it is listening to their needs and concerns. K4Health has already demonstrated experience with this approach through its blog series on upcoming changes on the K4Health website. Extending that approach throughout the K4Health products and services will help users feel heard. Social media time lines are highly compressed, highly responsive, and usually use agile development methods. As a result, traditional analysis, requirements, and development processes are not always appropriate to meet these needs.70 One alternative is implementing rapid iteration and constant experimentation processes. These processes reduce the time between capture of feedback, experimental changes in response, and implementation of improvements.71 K4Health should establish the means to place information along the knowledge continuum, from results of controlled studies to novice experiences. As examples, techniques could include sharing community members’ backgrounds and including summaries or links to other posts or responses made by a particular commenter. K4Health could ask people to rate the value of comments, by using a one- to five-star rating to indicate if readers found a comment helpful or consistent with their experiences. Cumulative ratings for individuals can also be shown next to their comments or responses. K4Health staff should continue to blog to establish expertise. In addition to writing the K4Health blog, guest posts on blogs of other well-regarded community members could help to expand the reputation, impact, and following of the K4Health bloggers and blog. Including a wider range of guest bloggers on the K4Health blog and introducing their posts with a few 69 See glossary for definition of active listening. 70 See glossary for definition of agile development. 71 The team did not investigate the internal K4Health operational practices for implementation of evaluation analysis. These suggestions are to be taken as examples of good practices and emerging trends rather than weaknesses by K4Health. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 35 lines describing why K4Health invited them to post could also help to expand the community and add credibility. For example, guest bloggers could be private sector representatives, USAID project vendors, academics; the variety among posters would provide different perspectives on KM and other issues of interest. K4Health staff should continue to research and document as much as possible the availability of both mobile and fixed line access, and should distinguish access for target community members (e.g., medical personnel, Ministry of Health staff) from general community availability. They should note social media channels currently used, such as Facebook or IBP Gateway, by geographic region. K4Health should build social media into existing K4Health successes by incorporating more social networking into eLearning, trying multiple approaches, and continuing to experiment. For example, eLearning programs could incorporate more video in small modules that could be referenced again at a later time; online tweet-ups or discussion threads could bring together course participants from diverse locales to discuss the information and share their own experiences and perspectives. Discussions can be asynchronous to accommodate different time zones, geographies, and the calendar period in which a course is viewed. KM Leadership The following recommendations are not meant to suggest that K4Health has not addressed these concerns or tried some of the suggestions. The recommendations are rather meant to emphasize important tasks and areas of attention (with appropriate resources) for the project. Addressing a Real Need The K4Health project fills a need for KM leadership that is currently being experienced in the USAID global health community and beyond. Several implementing partners interviewed mentioned that they felt only a project with dedicated resources for KM and health could maintain the motivation and focus to move the field forward. They valued the contributions of the K4Health team, from organizing the diverse community to giving them focus and guidance. The evaluation team recommends that the financial and programmatic support that USAID offers implementing partners for improving their KM abilities via the K4Health project be continued, especially in light of USAID’s new prioritization of KM. Honest Broker Key to its role as KM leader has been K4Health’s reputation as an honest broker among the implementing partner community, especially in getting partners to contribute content and expertise to the K4Health repository (mainly the toolkits, but also blogs and working groups). K4Health should continue to spend effort on this, and USAID should include this role in the follow-on procurement. However, K4Health may also want to address the issue about vetting content by creating common standards among the global health community for appropriate content and makeup of the different toolkits. Clarify but Integrate Different Types of Information As a repository for a variety of types of information, K4Health should continue to make room for both vetted and experiential knowledge, and make clear the difference between the types. All experiential knowledge is not equal; social media—via blogs, rankings, and so on—enables us to distinguish one-off and very early thinking on a subject from more established ideas that eventually get vetted. Placement along the KM continuum is determined by factors such as the 36 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION reputation of the person speaking, the number of people responding to an idea, and the passion and supporting detail. While integrating social media tools into vetted content has been tried in the past, it is important for K4Health and USAID to understand that building social media audiences can take time. Some suggestions for improving the social media component (including some things K4Health is already doing) include: Continue honing social media components to make it extremely easy and intuitive for users to add comments and rankings. Tie existing audiences together. For example, K4Health has hosted discussions of publications on existing COPs, providing links with “latest post/discussion” abstracts on the publication page; connecting the COP to the other products is a great way to build an audience. Twitter or Facebook can also host the discussion, with modules that pull the content references on the publication page. Think about the intrinsic rewards and motivations for raters/rankers, and allow users to either build their online brand or be anonymous. Some people may feel like they should not publicly comment on a publication created by their own organization or that of a competitor, while others may want to make a name for themselves as arbitrators of good content. Technology Platforms as KM Leadership Continue to manage the lifecycle of technology platforms, including migration of data and audiences and possible retirement of products or tools. Continue to use and maintain an iterative development style and explore/share with other KM practitioners how this style can lead to better user integration and better KM systems. Integrate experience of life cycle management as part of KM leadership—as a model of how to maintain content, audiences, and data over multiple project lifetimes. Increase the Voices from the Field K4Health has a unique opportunity to capture and integrate field-based experience into its activities, with faster iterative processes between field staff and core project staff. K4Health, with its growing field experience, has been and continues to be a model and support mechanism for the integration of KM processes and tools into USAID-funded health projects. This opportunity becomes even more important in light of the new local procurement requirements of USAID Forward. USAID is expected to procure up to 30% of its programming budget directly from local partners, many of whom have never before received direct U.S. funding. KM should and can be central in capacity-building and accountability efforts for local organizations that are performing development assistance support. These firms will need more direct support on how to implement and integrate KM processes and tools into their organizational structures, which will help build their capacities. On top of this new requirement, KM within field-based projects supports USAID’s new focus on developing a learning culture. USAID will be relying on its implementing partners to provide many of the lessons learned and impact evidence (tacit and explicit) on how USAID funds are achieving objectives. By developing good KM processes, implementing partners are better placed to provide USAID with that information and feedback. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 37 The “Inside” USAID vs. Outside Divide Needs to be Clearly Addressed One area that was slightly outside the scope of our evaluation but did emerge on several occasions related to the role that K4Health and other such KM projects would have for USAID staff in implementing their work. Several USAID staff members interviewed use K4Health tools and products (either knowingly or unknowingly) such as POPLINE, toolkits, and GHeL. In addition, several USAID staff members showed interest in using KM for their own work, which is out of the scope of the K4Health project, but is something that K4Health may be able to support if it can work with other projects that support USAID internal KM initiatives. As USAID staff also need to be recipients of KM for health information, more interaction and integration between K4Health and other parts of USAID KM will help USAID capitalize on investments made in K4Health and other projects. INTANGIBLE ASSETS Key Intangible Assets to Retain The standard five-year project cycle offers new opportunities for changes to the approach and structure of projects. Especially in light of tightened budgets, USAID needs to maximize the investments it has made in the development of intangible assets in existing projects, especially during the project transition or close-out period. Areas to focus on: Brand awareness and reputation: The existing audience for a website or other online publication, or a product or service, relies on awareness of a particular brand (i.e., the name of the product or service, what it does, how to find/use it, and whether it is valuable). This asset can also include high search engine rankings (see addendum for current SEO analysis) and being on partner organization websites. Products such as POPLINE have large audiences and wide usage because they have been around for decades; they enjoy a dedicated audience of users and a strong reputation for quality. It can take upwards of three years to build an audience from scratch. – Some things that can affect this asset: changing the project or product name, URLs, hashtags. Audience engagement: When audiences contribute via COPs or other social media tools, it represents a level of trust and commitment to the community that is hard to create and replicate, especially in cases where the communities offer insight and perspective into needs and behavior hard to capture elsewhere. The COPs and working groups managed by K4Health represent years of work and investment by the community; they offer a unique perspective of high value to USAID. However, communities require a lot of work and time to build, and a certain level of support to maintain. In reality, there are more examples of failure than success in developing COPs for international development. Audience engagement can be fragile to changes in the platform; active communities have died rapidly due to web redesign efforts or platform changes. – Some things that can affect this asset: changing COP platform (design, features, technology), reducing support for moderation/facilitation. Technology investments: While technology keeps moving forward, recreating established systems (even moving hosts and URLS) can take more time and money than 38 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION expected and can have negative impacts on audience engagement and brand (see above). Those technology investments (in software, hardware, and technological know-how) can be replicated and shared by other projects, and possibly expanded to existing audiences as the needs of the project warrant. – Some things that can affect this asset: Replacement of an existing system with a new one. Data and content: Imagine being able to data mine 10 years of COP archives to track changes over time on different topics. Imagine being able to review 15 years of project descriptions and indicators. Imagine being able to compare usage statistics. When a project shuts down, what happens to the data in its databases? Are the data migrated into a new system? Are they archived in some fashion that can be used by researchers down the road? Or are the data lost on a hard drive or CD-ROM in someone’s drawer? – Some things that can affect this asset: Not migrating data into a new system, not providing for data archival in a format that can be easily accessed. Methods for creating/maintaining taxonomy and tagging: Structuring data is highly important as big data and semantic web technology is becoming more and more central to how organizations learn. When projects take the time to structure their data, either through traditional database methods, taxonomies, or free-form tagging, their content is much more machine-readable and sharable with other database systems. – Some things that can affect this asset: Not investing time during the project in structuring data collected, not ensuring that structure data is preserved and shared with others. One USAID interviewee suggested that USAID-funded activities with large KM/social media components, like K4Health, should perform a transition evaluation (probably with an outside evaluator) during the final year of the project to identify the key intangible assets produced by the project. The evaluation would develop recommendations on which assets should be transferred, which archived, and which retired, and how to transfer those assets to a new project or archive them (regardless of prime). This evaluation would also be an opportunity for those intangible assets to be recorded with USAID. The transfer recommendations would then be included in the follow-on RFA to be addressed by all bidders. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 39 IV. FUTURE THINKING SOCIAL MEDIA In the near term, social media and social networking expand the reach of KM systems; they enable people to access, aggregate, and update information as it best suits them. While one person might like to stay current via Twitter, another prefers to follow select blogs or research reports. Social networking also broadens information gathering outside of prescribed and predefined communities. Information consumption becomes personalized. Fairly quickly, however, the vetted fact base of a centralized KM system will collide with the experiential knowledge prevalent in social networking. K4Health and similar organizations need to determine how to distinguish “facts” from anecdotal reports—in practice, the two can look much alike. Some of those interviewed have asked how K4Health can maintain its position as the leading reliable source for information on family planning and reproductive health if submissions and comments are not screened? Clearly identifying links to researched data is important. Allowing discussion of scientific content, implementation experiences, and alternative ideas is equally important in a vibrant KM strategy. Information is no longer static but continuously evolving as more people contribute, converse, and comment. Social media technologies have begun to shift how information is developed. Wikis, for instance, can replace much of the traditional curated content. Blog posts help to establish reputation and expertise while also providing the opportunity to apply curated facts to particular circumstances. Sharing new insights or detailed implementation experiences expands the knowledge base. The Journal of Pakistan Medical Students (JPMS) Blogs,72 which went live online on May 25, 2012, is a good example of how Social Networks can enhance traditional KM and work to share information more broadly. The purpose of the blogs “is to discuss the scientific content of medical research in a reader friendly manner.” JPMS Blogs includes among its objectives: “Spread awareness in students, doctors, policy makers regarding the updates in the medical literature and healthcare related issues.” “Bridge the gaps among the researchers, students, doctors, policy makers and common people.” “Developing countries have limited access to research journals. If correct and accurate information is given to the doctors from the developing countries, their needs for the scientific knowledge can be addressed via this blog.”73 Blogging becomes a means for the discussion, dissemination, and advancement of knowledge. Social interaction yields new insights and knowledge; social media and KM fuse together. The UN Global Pulse initiative relies heavily on social media to gain a real-time understanding of collective behavior in order to generate information that can be used in a timely manner. The project is built on the belief that new digital networks and tools have contributed to a highly 72 Journal of Pakistan Medical Student Blogs. Available at http://blogs.jpmsonline.com/. Accessed May 2012. 73 Journal of Pakistan Medical Student Blogs. “About Us.” http://blogs.jpmsonline.com/our-team/. 40 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION accelerated pace of change. Development is no longer a slow, linear process; knowing what is happening at the time it is happening is critical. Listening to Social Networks helps keep knowledge fresh and relevant to immediate situations. Sentiment analysis and social media analytics help to identify emerging trends and issues. The knowledge gained is immediately actionable. KM LEADERSHIP FOR PUBLIC HEALTH Changing Nature of Audience Engagement with Digital Telecommunications The web as a public form of interaction has only been widely available and accessible for around 15 years, and Web 2.0 (i.e., highly peer-to-peer based interactions) since the early 2000s. Many communities that USAID works in have only recently been able to use web-based technology with any frequency, and large percentages of the population still have no or limited access. Mobile technology became available in the commercial sector in the 1990s, with exponential growth in developing countries in the 2000s, leapfrogging over landline technology in many places. Now with Internet-enabled smart phones and tablets, and more advanced usage of text messaging, we are at the start of truly mobile Internet connectivity age. These technologies, similar to the revolutions in transportation, are fundamentally changing how we manage our social existences and do business, education, and government. Many questions remain about what the impact will be, including how we will interact with each other. For example, there have been studies that showed that by “outsourcing” our memory for facts to databases and networks, we retain fewer facts but we gain access to more information as a whole. We also gain additional social and educational benefits (even in those above the age of 65) from access to these resources. These facts will change how we cognitively learn and process information. 74,75,76 Anthropological Differences in Technology Engagement We also should not assume that cultural and historical differences around the world will not have different impacts on how we interact with these technologies. Certainly, in the United States, there is acknowledgment of the differences between the digital immigrants (i.e., those born before 1985) and digital natives (those born after 1985), who have never known a world without the Internet, PC computing, mobile telephony, and digital media. There are also differences in how men and women interact with technology and those of different socioeconomic groups in the United States. For example, women in developing countries have significantly lower access than men to technology (mobile phones, Internet, PCs and software) and training and support for information communication. There are also lower social 74 Sparrow, Betsy, Jenny Liu, and Daniel M. Wegner. “Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips.” Science, 333(6043). 2011. Available at http://www.sciencemag.org/content/333/6043/776.abstract. 75 Baker, Russell, Erika Matulich, and Raymond Papp. “Teach Me In the Way I Learn: Education and the Internet Generation.” Journal of College Teaching and Learning, 4(4). 2007. Available at KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 57 Okimoto, Jennifer. “Industry Trends: The Evolution of Knowledge Management (KM 1.0 vs. KM 2.0).” Slideshow. IBM Corporation, September 2007. Available at http://www.slideshare.net/elsua/the-evolution-of-knowledge-management-km￾10-vs-km-20. Peace Corps Journals. Available at http://www.peacecorpsjournals.com/?GetStarted. Accessed May 2012. Peace Corps Wiki, 2012. Available at http://www.peacecorpswiki.org/Peace_Corps_Wiki. Accessed May 2012.Pew Research Center, Global Attitudes Project. Global Digital Communication: Texting, Social Networking Popular Worldwide. December 20, 2011. Available at http://www.pewglobal.org/files/2011/12/Pew-Global-Attitudes-Technology￾Report-FINAL-December-20-20111.pdf. PopTech. “Project Masiluleke: A Breakthrough Initiative to Combat HIV/AIDS Utilizing Mobile Technology & HIV Self-Testing in South Africa.” Available at http://poptech.org/system/uploaded_files/27/original/Project_Masiluleke_Brief.pdf. (Undated) Qualam, Eric. “10 New 2012 Social Media Stats = WOW!” Socialnomics. June 6, 2012. Available at http://www.socialnomics.net/2012/06/06/10-new-2012-social-media-stats-wow/. Rawlings, Lauren. Project Masiluleke: Fighting HIV/AIDS through Mobile Phones. BroadReach Healthcare and Center for Health Market Innovations. April 2011. Available at http://healthmarketinnovations.org/sites/healthmarketinnovations.org/files/FINAL_Projec tM042011_0.pdf. ReliefWeb. Program on Humanitarian Policy and Conflict Research. 2009–2012. Available at http://reliefweb.int/organization/hpcr. Russell, Jon. “Philippines Named Social Networking Capital of the World.” Asian Correspondent. May 15, 2011. Available at http://asiancorrespondent.com/54475/philippines-named-the￾social-networking-capital-of-the-world-indonesia-malaysia-amongst-top-10/. Sentiment Analysis. Wikipedia. http://en.wikipedia.org/wiki/Sentiment_analysis. (Undated) Simmons, Bright B. “African Lessons for the City-State of Facebook.” HBR Blog Network. May 29, 2012. Available at http://blogs.hbr.org/cs/2012/05/what_the_city￾state_of_faceboo.html. Social Bakers. “Developing Countries on Facebook—TOP 10.” May 24, 2010. Available at http://www.socialbakers.com/blog/61-developing-countries-on-facebook-top-10/. Social Bakers. “Facebook Hits 488 Million Mobile Users.” May 8, 2012. Available at http://www.socialbakers.com/blog/554-facebook-hits-488-million-mobile-users￾infographic/. Social Media. Wikipedia. Available at http://en.wikipedia.org/wiki/Social_media. (Undated) Sparrow, Betsy, Jenny Liu, and Daniel M. Wegner. “Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips.” Science, 333(6043). 2011. Available at http://www.sciencemag.org/content/333/6043/776.abstract. 58 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION Srivastava, Prachi and Nick Hopwood. “A Practical Iterative Framework for Qualitative Data Analysis.” International Journal of Qualitative Methods, 8(1). 2009.Available at http://ejournals.library.ualberta.ca/index.php/IJQM/article/download/1169/5199. Thought Leadership. Wikipedia. Available at http://en.wikipedia.org/wiki/Thought_leadership. (Undated) U. S. Agency for International Development. Gender and Information & Communications Technology: Mobile Phones and Women. April 3, 2012. Available at http://transition.usaid.gov/our_work/cross￾cutting_programs/wid/ict/case_study_mobile_phones_women.html.. U.S. Agency for International Development. Gender and Information & Communications Technology: Overview. April 3, 2012. Available at http://transition.usaid.gov/our_work/cross￾cutting_programs/wid/ict/gender_ict.html. U.S. Agency for International Development. “Program Cycle Learning Guide.” Washington, DC: USAID. (Draft) Vital Wave Consulting. mHealth for Development, The Opportunity of Mobile Technology for Healthcare in the Developing World. Washington, DC and Berkshire, UK: UN Foundation￾Vodafone Foundation Partnership, 2009. Available at http://www.globalproblems￾globalsolutions￾files.org/unf_website/assets/publications/technology/mhealth/mHealth_for_Development _full.pdf. Vollmer, Christopher and Karen Premo. Campaigns to Capabilities: Social Media & Marketing 2011. New York: Booz & Company, October 2011. Available at http://www.booz.com/media/file/BoozCo-Campaigns-to-Capabilities-Social-Media-and￾Marketing-2011.pdf. Whatley, Simon. “Qualitative Social Media Monitoring Tools (Sentiment Monitoring).” July 21, 2011. Available at http://www.simonwhatley.co.uk/qualitative-social-media-monitoring￾tools-sentiment-monitoring. Zickuhr, Kathryn and Mary Madden. “Older Adults and Internet Use.” June 6, 2011. Available at http://www.pewinternet.org/Reports/2012/Older-adults-and-internet-use.aspx. Zickuhr, Kathryn, and Aaron Smith. “Digital Differences: Overview”. April 13, 2012. Pew Internet Available at http://pewinternet.org/Reports/2012/Digital-differences/Overview.aspx KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 59 APPENDIX D. SEARCH ENGINE OPTIMIZATION Search Term* Response Date KM global health #4 June 25, 2012 Knowledge for health #1–7 June 25, 2012 Knowledge management global health #10 June 25, 2012 Global health family planning knowledge #1, #2 June 25, 2012 Global health contraception knowledge #3 June 25, 2012 Knowledge management for public health #4 June 25, 2012 *As entered into google.com 60 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 61 APPENDIX E. LITERATURE REVIEW MATERIALS K4HEALTH RESOURCES Request for Application Agreement Performance Management Plan Yearly Work Plans Yearly Management Reviews Bi-yearly Progress Reports Malawi Monthly Reports Information Needs Assessment Reports eLearning Evaluation Website Usability Evaluation Website and Related Products Facebook, Twitter, Blog, and Social Media Accounts Web Statistics Review of K4Health Social Media and KM Data USAID RESOURCES Guide to Conducting Needs Assessments Desk Review of Promising Social Media Approaches Evaluation Policy Evaluation Checklist Microlinks website: http://microlinks.kdid.org/ FrameWeb: http://frameweb.org/ EXTERNAL RESOURCES John Snow, Inc. AIDSTAR-One website: http://www.aidstar-one.com/ Healthcare Information for All (HIFA) 2015 HIFA 2015 website: http://www.hifa2015.org/ 62 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION INTERNATIONAL FINANCE CORPS Kleemeier, Elizabeth L., and Joy Kazadi. “Rural Water Supplies Collaborative: Best Ways to Build Social Collaboration.” SmartLessons. Washington, DC: IFC and Water Partnership Program, August 2011. Munoz, Javier. “Business Collaboration Meets Social Collaboration: Building a Community for SharePoint Users.” SmartLessons. Washington, DC: IFC, August 2011. OTHER Thorpe, Ian. KM on a Dollar a Day, [blog]. Available at: . [Accessed: May 2012]. KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 63 APPENDIX F. INTERVIEW QUESTIONS KM LEADERSHIP QUESTIONS 1. How does K4Health define KM leadership? 2. What is K4Health’s overall approach to KM leadership? a. Discuss the main methods and approaches (KM working groups, HIPNET, IBP, etc.) b. Discuss the investment and return on that investment related to KM leadership for USAID (K4Health and more broadly) and for international development/global health. c. Share major successes with KM leadership. 3. What do you see as K4Health’s core strengths in KM leadership? 4. Where do you see future growth and/or gaps that need to be addressed in KM leadership (by K4Health or others)? 5. What is your rationale for having a distinct global health-focused KM leadership? 6. How are you measuring the benefits of your role in KM leadership to your different stakeholders, including: a. USAID/Washington b. USAID/Missions c. K4Health staff d. K4Health clients (i.e., toolkit contributors and eLearning authors) e. K4Health users 7. What other organizations are providing strong KM leadership in health and development? Are these also partners and collaborators? Is there a strong rationale for having distinct global health-focused KM leadership? 8. What are K4Health’s core strengths in KM leadership as recognized by different stakeholders, including USAID/Washington, USAID/Missions, K4Health staff, K4Health clients (i.e., toolkit contributors and eLearning authors) and K4Health users? 9. To what degree are K4Health’s KM leadership activities represented on www.k4health.org? Discuss the benefits and drawbacks to housing KM information in the same platform as health information. Social Media Questions 1. Please identify your current use of social media and digital platforms for personal use. 2. Please identify your current use of social media and digital platforms for professional use. 3. What is your relationship/role with K4Health? 4. Have you used social media with any other health organizations? If so, which organizations? Was your usage in or related to a developing country? If so, which countries/locales? 64 KNOWLEDGE FOR HEALTH (K4HEALTH) PROJECT EXTERNAL LEARNING EVALUATION 5. For what purposes have you used social media? a. What social media did you use? Public or private platforms? b. What was the greatest benefit? c. What did not work as you wanted? 6. What are the greatest difficulties using social media in developing locations? 7. For what purposes would you envision using social platforms? 8. For what purposes would you find it useful to share information with other professionals in other locales? 9. Which social media hold the most promise for improving information dissemination and knowledge sharing among health workers in developing countries? How are social media and social networking used to enhance or expand knowledge sharing, including the promotion of good practices and their scale-up? 10. For both knowledge sharing and promotion, what are the elements of a successful blog related to global health? Of a successful Twitter feed? 11. What are the gender differentials in social media access and use within the developing world and how should these differentials be factored into a social media strategy? 12. How are K4Health and other organizations effectively using Facebook, LinkedIn, and other social networking sites to facilitate knowledge sharing, information dissemination, and promotion? 13. What social media opportunities has K4Health taken good advantage of and what gaps remain? Intangible Assets 1. How do you measure and capture value of: a. Brand awareness and reputation, b. Good will and engagement, c. Audience and community development, d. Technology investment, e. Content and data repository, and f. Organizational processes? KM Strategy 1. How can social media/networking tools effectively engage and address the needs of audiences like health program managers and health service providers? Should KM leadership be incorporated into a follow-on project? If so, what level of effort should be applied? For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com