Midterm Performance Evaluation of the Higher Education Solutions Network (HESN) April 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Dr. Joy Amulya, Team Leader, Mr. James Chung, Senior Technical Advisor, Ms. Stephanie Monschein, Evaluation Specialist, and Ms. Charu Vijayakumar, Evaluation Specialist (Dexis Consulting Group). Opinions expressed herein may not necessarily represent official perspectives of the U.S. Government. ii iii HIGHER EDUCATION SOLUTIONS NETWORK MIDTERM PERFORMANCE EVALUATION April, 2016 AID-OAA-I-15-00019/AID-OAA-TO-15-00043 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iv CONTENTS Contents ............................................................................................................................ iv Executive Summary .......................................................................................................... 1 I. Introduction .................................................................................................................... 9 II. Project Background ....................................................................................................10 III. Evaluation Purpose and Methodology .....................................................................13 Evaluation Purpose .................................................................................................................................... 13 Evaluation Methodology ........................................................................................................................... 13 Limitations ................................................................................................................................................... 16 IV. Findings and Conclusions ..........................................................................................18 Evaluation Question 1. To what extent has HESN been successful in achieving the outcomes of which it may be expected?.................................................................................................................. 18 Evaluation Question 2. What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? ..................................... 25 Evaluation Question 3. To what extent has HESN led to changes at HEIs that may increase their impact on international development? ........................................................................................ 31 Evaluation Question 4. To what extent has HESN influenced or assisted USAID operating units other than the U.S. Global Development Lab? ......................................................................... 38 Evaluation Question 5. How can HESN modify its strategy and structure to improve its efficiency and effectiveness? .................................................................................................................... 42 Evaluation Question 6. What, if any, unanticipated positive and negative consequences have occurred as a result of the HESN Project? .......................................................................................... 48 V. Recommendations ......................................................................................................53 v Acronyms AOR Agreement Officer’s Representative BMC Business Model Canvas CITE Comprehensive Initiative on Technology Evaluation, MIT ConDev Conflict and Development, Texas A&M University CSO Civil Society Organization DARPA Defense Advanced Research Projects Agency DIL Development Impact Lab, UC Berkeley DRC Democratic Republic of Congo EU European Union GH Global Health GIS Geographic Information System GCFSI Global Center for Food Systems Innovation, Michigan State University HEI Higher Education Institution HESN Higher Education Solutions Network IDDS International Development Design Summit IDIN International Development Innovation Network, MIT IGERT Integrative Graduate Education and Research Traineeship IRB Internal Review Board M&E Monitoring and Evaluation MIT Massachusetts Institute of Technology MOU Memorandum of Understanding NGO Non-Governmental Organization NIH National Institutes of Health NSF National Science Foundation PCARI Philippine California Advanced Research Institute PEER Partnerships for Enhanced Engagement in Research PI Principal Investigator PO Program Officer RAN ResilientAfrica Network, Makerere University RCT Randomized Control Trials R&D Research and Development RFA Request for Applications RI Lab Resilience Innovation Lab SE Social Entrepreneur SEAD Social Entrepreneurship Accelerator at Duke TechCon Technical Convening UN United Nations UNDP United Nations Development Program WFP World Food Program 1 EXECUTIVE SUMMARY This report is the midterm evaluation commissioned by USAID from October 2015 to April 2016 to provide an evidence-based assessment of the Higher Education Solutions Network (HESN). In 2012, USAID awarded five-year Cooperative Agreements to eight HESN Development Labs housed in seven higher education institutions (HEIs) managed by a core coordinating body at USAID. The program began in November 2012 and will end in fall 2017.1 The life of project budget was $140,559,741 and obligations of $93,426,962 have been made through FY2016. The HESN Request for Applications (RFA) asked for novelty and a diversity of approaches and as a result, the HESN Labs that were selected operate in different areas with distinct approaches. The eight HESN Labs are:  AidData Center for Development Policy, College of William and Mary (AidData) concentrates on high resolution geospatial data, conducts analysis and applies imagery as decision support tools that enable the global development community to more effectively target, coordinate, deliver, and evaluate their aid investments.  Comprehensive Initiative on Technology Evaluation, Massachusetts Institute of Technology (CITE) is developing methods for product evaluation in global development in order to provide evidence for data-driven decision-making by development workers, donors, manufacturers, suppliers, and consumers.  Center on Conflict and Development, Texas A&M University (ConDev) seeks to improve the effectiveness of development programs and policies for conflict-affected and fragile countries through multidisciplinary research and education aimed at reducing armed conflict, sustaining families and communities during conflict, and assisting states to rapidly recover from conflict.  Development Impact Lab, University of California, Berkeley (DIL) combines engineering and the natural sciences with insights from economics and the social sciences to generate sustainable, technology-based solutions to development challenges.  Global Center for Food Systems Innovation, Michigan State University (GCFSI) is developing and testing new approaches to overcome the problems of shrinking farm land in developing countries, help under-resourced farmers deal with less rainfall due to climate change, and improve food systems from production and storage to packaging and distribution.  International Development Innovation Network, Massachusetts Institute of Technology (IDIN) is made up of 660+ innovators from around the world who create technologies hand-in-hand with local communities and — with access to funding, training, and mentorship — go on to turn their prototypes into products designed to make a difference.  ResilientAfrica Network, Makerere University (RAN) innovates and accelerates science and tech-based development tools in concert with USAID and a diverse set of stakeholders to strengthen African resilience to its greatest challenges, such as environmental variability, the effects of urbanization, government transparency, chronic civil conflict, and disease.  Social Entrepreneurship Accelerator, Duke University (SEAD) mobilizes a community of practitioners, investors, policymakers, faculty, staff, and students to identify, assess, help develop, build capacity of, and scale solutions, technologies, and business models for healthcare delivery and preventive services in developing countries around the world. HESN aims to improve data quality, access, and analytics to advance evidence-based development decision making (Objective 1); accelerate the creation, testing, and scaling up of transformative 1 Some end in September 2017, others in November 2017, depending on when the agreement was signed. 2 innovations, technologies, and approaches (Objective 2); and catalyze a global interdisciplinary environment where individuals and institutions can share knowledge, promote learning, and build mutual capacity (Objective 3). The results of the evaluation will be used to inform USAID’s immediate and future program planning and implementation.2 The evaluation was designed to address the following six evaluation questions: 1. To what extent has HESN been successful in achieving the outcomes of which it may be expected? 2. What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? 3. To what extent has HESN led to changes at HEIs that may increase their impact on international development? 4. To what extent has HESN influenced or assisted USAID operating units other than the U.S. Global Development Lab? 5. How can HESN modify its strategy and structure to improve its efficiency and effectiveness? 6. What, if any, unanticipated positive and negative consequences have occurred as a result of the HESN Project? The evaluation team consisted of a Senior Evaluation Advisor (Team Lead), Senior Technical Advisor, two Evaluators, and a Project Manager. The evaluation methodology included document and literature review, interviews with HESN Lab partners and beneficiaries, and site visits to each HESN lead campus consisting of interviews with campus leaders and interviews and focus groups with HESN Lab leaders and team members. Two additional sets of interviews were conducted with lead staff from similar USAID higher education and innovation initiatives, and USAID and non-USAID experts in higher education and development innovation. Finally, longitudinal monitoring and evaluation (M&E) indicators from FY2013-FY2015 were analyzed to assess performance on the HESN Results Framework. A widely used tool in the field of innovation, the Nonprofit Business Model Canvas (BMC), anchored the data collection strategy.3 This tool allows diverse kinds of transformative methodologies, approaches, and research activities to be assessed on nine common areas4 critical to scalable impact. It is applicable to any kind of product, service, technology, tool, approach, or activity at any stage of development or implementation. In the HESN Midterm Evaluation, the Nonprofit BMC provided a framework for creating a detailed, aggregate picture of the actual and potential value generated by the HESN program to date. Responses on this tool provided by each of the HESN Labs were used to identify the primary beneficiary groups and key partners to target for interviews. This report presents the findings, challenges, and recommendations based on analyses of the data relevant to each evaluation question. The overall performance of the HESN Labs on the three objectives outlined in the Results Framework was high over the first three years. Collectively, HESN Labs met or surpassed targets for 27 out of 34 (79 percent) standard HESN indicators. On average across all indicators, HESN Labs surpassed targets by 66 percent. The strongest performance has occurred in the area of improving data quality, access, and analytics to advance evidence-based development decision￾making. In addition to analysis of the HESN indicators, stakeholder interviews were conducted in order to assess whether or not the opportunity or service provided by the HESN Lab was needed and beneficial to the recipient. Ninety percent of those interviewed validated that the intended benefit provided by the HESN Lab was, in fact, beneficial to them in their work. These key informants included representatives from 2 It should be noted that during the contracting and execution of this evaluation, there were shifts to the strategic priorities of the U.S. Global Development Lab, the USAID Operating Unit where HESN is housed. This evaluation is not intended to make definitive recommendations related to those shifts, and only speaks to the overall program of HESN. 3 Retrieved from http://www.innovativenonprofit.com/2015/08/the-nonprofit-business-model-canvas-2/#.VkuXXXudn9Q 4 Beneficiary/User Segments, Value Proposition (product, service, other benefit), Distribution Channels, Beneficiary/User Relationships, Impact Metrics, Key Partners, Key Activities, Key Resources, Cost Structure. 3 USAID Missions and operating units (12), developing country innovators (16), NGOs (4), developing country local or private sector organizations (2), multilateral organizations (1), impact investors (2), developing country universities (2), graduate students (19), and researchers (10). The high rate at which HESN Labs have been able to deliver benefit to their target beneficiaries reflects the fact that they have been able to engage directly with these groups during the first three years of the HESN award and have been able to adjust their services/activities to better fit the needs of specific beneficiary groups. Higher education and development innovation experts5 see HESN as successful in creating incentives (funding, institutional recognition, opportunities for publication) and interdisciplinary projects that faculty, post doctorate, and graduate students need in order to do more than limited development innovation research. According to one informant, HESN has achieved outcomes comparable to those observed for the NSF Innovation Corps initiative (I-Corps), which translates results from NSF-funded, basic research projects into technologies with potential for impact in the commercial world. A benefit of initially allowing the HESN Labs to address multiple objectives is that the work of each has been more fully integrated into the mission of the university compared to a standalone research institute or training grant. The three objectives align with what universities are best at: developing datasets, creating new tools and approaches, and training students and researchers. Furthermore, the fact that multiple HESN Labs are addressing each HESN objective has led to a collective breadth and richness of activity under each objective. Research studies help establish partnerships with development actors and reveal needs for new solutions. Students are a key resource in delivering on research and innovation objectives. Beneficiaries of HESN Lab services become valuable partners. Compared to other USAID initiatives, HESN program staff have been effective in supporting HESN Labs to understand how to fit what they are offering to USAID Missions and operating units, in facilitating relationship building between university partners and USAID operating units, in helping HESN Labs develop a comparable number of projects and partnerships relative to more established initiatives, in placing a large number of graduate researchers into fellowships with developing country partners, and in encouraging HESN Lab faculty researchers to partner with developing country researchers. Additional strategies that could increase HESN’s effectiveness include inviting USAID Missions to articulate upcoming challenges and needs and allowing HESN Labs to submit applications for meeting those needs; holding regional gatherings in developing countries to support exchange between HESN Labs, USAID Missions, and regional ecosystem actors; and establishing a system for identifying developing country researchers funded by USAID’s PEER initiative6 who are engaged with HESN-affiliated projects. The awards to the eight HESN Labs have had wider effects on the seven lead HESN campuses. Several HESN campuses have had NSF Integrative Graduated Education and Research Traineeship (IGERT) awards; several faculty interviewees commented that HESN Labs are more supportive of interdisciplinary research because they incentivize cross-department collaboration among faculty (not just students and because they are focused on a specific set of challenges and research opportunities. Overall, HESN has resulted in bringing international perspectives and research to more departments and research units on campus, increased interdisciplinary collaboration, and greater connection to USAID Missions and other operating units. The specific wider campus impacts documented through the midterm evaluation include:  Increases in the number of academic and research activities for undergraduate and graduate students related to international development;  A greater number of development-focused collaborations across academic disciplines, research institutes, and student service units;  Increased hiring of faculty with interests in development challenges; and 5 Twelve USAID and non-USAID informants 6 Partnerships for Enhanced Engagement in Research 4  Greater awareness and visibility of the role of science and engineering in improving conditions for people living in poverty. To date, the eight HESN Labs have developed an extensive network of partnerships with faculty and students at developing country higher education institutions (HEI). A total of 72 developing country HEIs and research institutes were working with HESN Labs as of FY2015. The primary pathways through which capacity building has taken place are: strengthening research capability, promoting development innovation through curricula and educational materials, partnership on HESN Lab projects, and connecting to wider research and development networks. RAN plays a unique role in building the capacity and ecosystem for innovation throughout Africa. As the lead campus for RAN, Makerere University is both a developing country HEI partner for US-based HESN Labs as well as a capacity￾building catalyst for RAN-affiliated and other African universities. RAN is the only HESN Lab to establish a deliberate partnership strategy for strengthening research and innovation capacity of a strategically￾located system of developing country HEI. HESN Labs are actively seeking funding sources outside of USAID and creating sustainability plans. HESN Lab leaders do not believe their work can be maintained in its current form without some level of USAID funding. A primary challenge is the lack of US science funding sources for early stage development innovation research. Without USAID funding, few if any HESN Labs believed they would be able to remain a cohesive Lab or Center, instead retaining particular activities that might be possible to fund in some other way. Competing needs for funding for HESN Labs include “buying out” faculty members’ time to ensure an ongoing level of dedicated time for carrying out the HESN Lab’s work; funding for graduate students to carrying out core research functions; funding for operating staff functions such as project management, M&E, student engagement, communications, etc.; and travel for field research, community partnerships, and knowledge exchange with developing country partners. Unanimously, HESN Labs agreed that the buy-in capacity of the HESN mechanism is by far the most critical and important component that they hope stays in place long term. HESN Labs and USAID HESN staff have worked to establish and grow relationships between HESN Labs and USAID operating units. The challenging and time-consuming nature of this process was apparent from interviews with USAID HESN staff, HESN Lab leadership and team members, and USAID operating unit staff. The interviews documented that partnerships with USAID Missions were not an initial focus of HESN. Once it became a priority for HESN Labs to influence decision making and provide other kinds of assistance to Missions, USAID HESN program staff played a critical role in establishing relationships with Missions, helping Missions become aware of the work of the eight HESN Labs, and seeking out opportunities for the HESN Labs and Mission staff to become familiar with one another through visits, providing feedback, meeting at regional events, etc. This high level of “customer service” to Missions by USAID HESN staff was informed by a deliberate relationship-building strategy. As of December 2015, HESN Lab activities were taking place in more than 50 countries covered by USAID Missions and there were 55 collaborations in development, in operation, or completed between the eight HESN Labs and 34 USAID operating units (six Bureaus and 28 Missions). Despite the challenges of establishing and maintaining these working relationships, key informant interviews with USAID operating unit staff made it apparent that there is enthusiasm for the work of HESN Labs. USAID staff who have worked with HESN Labs believe working with the HESN Labs is helping operating units advance their thinking and build up the base of research used for decision making. Factors cited by USAID interviewees that support successful collaboration between Missions and HESN Labs include: opportunities for Mission and HESN Lab researchers to meet and become familiar with each other’s work; HESN Labs’ familiarity with the country strategies (CDCSs7); the opportunity for further conversations about the potential gains from working with HESN Labs on 7 Country Development Cooperation Strategy 5 specific development challenges; the timing of the HESN Labs work relative to the Mission’s planning cycle; and a commitment by the Mission director to evidence-based decision making. For their part, HESN Lab leaders are positive about how HESN has engaged universities in a collegial, rather than authoritarian, manner. Contributions of HESN Labs to USAID operating units were particularly strong in the area of data￾driven methodologies, tools, or analytics. By the end of FY2015, 12 USAID operating units were using geographic analysis to design, implement, monitor, and evaluate development projects. In addition, HESN Labs have conducted 11 projects aimed at generating analytic data to inform decisions by USAID operating units. The extent to which USAID operating units leveraged the HESN Labs to accelerate their creation, testing, and scaling up of transformative innovations has been slow to develop. Currently, there are four partnerships aimed at leveraging the development of innovations. HESN Labs produced 200 white papers, articles, assessments, analyses, and evaluations on development challenges, innovations, technologies, approaches, and contexts, an unknown number of which were provided to USAID operating units. Finally, HESN Labs and USAID operating units have also begun to engage in collaborative problem-solving, knowledge sharing, and learning through exchanges, competitions, and other forms of learning-focused engagement. A total of 22 interns from HESN Labs have been placed in USAID Bureaus and Missions to date. Two USAID Mission staff to date have served in "residencies" at UC Berkeley, providing insight and technical input for DIL. A total of 40 USAID operating unit and Mission staff participated in the first two HESN Technical Convenings (TechCon). Based on the analysis of key informant interviews, there are four areas to which the HESN program should devote more attention: support for early stage innovation, ensuring consistent multi-year funding for graduate student researchers, deliberate capacity building of developing country researchers (both HEI-based and development institution-based), and advancing the evidence base for HESN Lab approaches. Existing efforts in all of these areas should be shared across the network for other HESN Labs to learn from and adapt. The three areas identified by the midterm evaluation that should be de￾emphasized were: a narrow definition of scaling, maximizing the number of objectives and activities each HESN Lab engages in, and exclusively performance-focused M&E. Seven factors appear to have been the most critical in achieving HESN’s objectives to date:  Faculty interest and excitement in working on development challenges and on interdisciplinary teams  HESN Lab leaders and faculty with knowledge and experience of USAID and developing country ecosystems  Commitment to capacity building in the field  Strong engagement from key university champions  Ability to grow the core HESN Lab management team and adapt roles in response to the development of the work  Relationship building emphasis of USAID HESN program staff  Mechanism allowing Missions or Bureaus to easily enter into working relationships with HESN Labs Changes to the design of a follow-on program that would better align with factors supporting the success of university innovation ecosystems fall into four areas: partnerships with development actors, operational improvements, enhancement of university environments, and leverage of existing network potential. Recommendations In framing a set of recommendations for future HESN programming, the evaluation team has made an effort to balance two competing realities. On one hand, the investment made by USAID over the first five years is difficult to sustain. On the other hand, the HESN Labs are still evolving as complex systems 6 of innovation and capacity building and, although no longer in startup mode, have not fully stabilized around a coherent set of high-impact activities and target beneficiaries. After five years, HESN will be positioned to leverage its biggest achievements: building an internal and external infrastructure for impacting development challenges; implementing an effective mode of direct “thinking and designing with” engagement with development actors; generating a body of data and knowledge seen as valuable to practitioners and policymakers; generating early stage innovations (both approaches and solutions); laying a groundwork of networks and partnerships for systemic change; and aligning HESN Lab services with priority development objectives and stakeholder needs. The recommendations outlined below are aimed at building on these achievements while also addressing the shortcomings identified through the midterm evaluation.8 Requirements and supports for scalability and replicability are not reliably introduced at the level of individual HESN Lab projects and/or broader HESN Lab activities. Except for RAN, HESN Labs lack a strategy for building research capacity in developing country higher education institutions. Uptake by USAID Missions and other operating units has been slow. There is a wide audience for learning and results from HESN Labs within USAID, other US Government innovation initiatives, and university innovation researchers but dissemination has not reached that audience. HESN Labs are different from technology companies and social innovation startups that design scalable innovations, but their role in generating data, science, and human resources for systemic change is poorly understood. 1. HESN Labs should streamline activities, adjust resource allocations, and increase synergies based on the insights gained through the first five years.  HESN Labs should identify the subset of activities, pilots, approaches, and partnerships with the greatest potential for generating sustainable impact, including likely high-potential activities currently under development. They should articulate a set of criteria based on this assessment that can be used to evaluate opportunities going forward. The utility of the activities, pilots, approaches, and partnerships should be assessed alongside Mission priorities and the existing evidence base (and number of other players) should be taken into consideration as well to avoid unnecessary duplication of efforts.  HESN Labs should identify the core critical operating functions and costs that will be needed to support this subset of activities, pilots, approaches, and partnerships. They should also identify the practices and structures that have created synergy and coherence among different value propositions and activities so as to maximize impact and resources.  HESN Labs should update or replace the Business Model Canvases created during the midterm evaluation to correspond to the streamlined set of activities and use these as the basis for monitoring, evaluation, and learning (see Recommendation 6). 2. USAID should extend funding to HESN Labs needed for core critical operating functions and costs, and should maintain the capacity for Missions and other operating units to use buy-ins to facilitate partnerships with HESN Labs. In addition, partnerships with USAID Bureaus, such as SEAD’s partnership with the Global Health Bureau, should be cultivated, potentially including funding.  One option that USAID can consider is a phased system for HESN Lab activities and innovations (technologies or approaches).9 Once an HESN Lab develops and validates an innovation or approach as beneficial to target beneficiaries/users, the next phase would involve working with implementation partners to build capacity, conduct further testing, and identify funding sources. 8 It should be noted that during the contracting and execution of this evaluation, there were shifts to the strategic priorities of the U.S. Global Development Lab, the USAID Operating Unit where HESN is housed. This evaluation is not intended to make definitive recommendations related to those shifts, and only speaks to the overall program of HESN. 9 This recommendation is based on the 2012 report by Monitor Group, From Blueprint to Scale (http://acumen.org/content/uploads/2013/03/From-Blueprint-to-Scale-Case-for-Philanthropy-in-Impact-Investing_Full-report.pdf) 7  Once an approach developed by an HESN Lab is funded, support from USAID would be shifted to a “buy-in only” model. Additional university development innovation partners would be brought into HESN and given seed funding for a designated period of time with which to prototype, pilot, and evaluate the viability of an approach or solution for use by a development partner. 3. USAID and HESN Labs should work together to hone an effective strategy for creating partnerships with USAID Missions, including a mechanism for USAID Missions and operating units to articulate challenges and needs to HESN Labs. At the same time, HESN should retain the relationship-building, “supply side” strategy it has used to date. USAID HESN program staff have spent considerable time effort helping US researchers understand how to fit what they are offering to USAID Mission requirements. The development of a more robust structure that would invite USAID Missions to articulate upcoming challenges and needs and allow HESN Labs to submit applications for meeting those needs would allow the HESN program staff to focus more time and attention on the relationship-building activities that have played an important role in the success of HESN Lab￾Mission partnerships. 4. USAID should centralize (or contract to centralize) the dissemination of research and policy briefs, academic publications, and other knowledge products produced by HESN Labs for better access by the broader academic and development community. A model suggested for this by one university￾based innovation expert is that used by Abdul Latif Jameel Poverty Action Lab (J-PAL). A USAID￾HESN Lab working group on dissemination could meet at the next TechCon to develop a strategy for centralized knowledge sharing. Centralized knowledge management and dissemination would also support Missions to become familiar with HESN Labs. 5. Each Lab should create an ongoing partnership with one or more strategically-located developing country HEIs to access in-country researchers, communities, and innovators who can identify critical challenges and contextual needs and support capacity for research and innovation in HEI partners. These collaborations could also lead to the development of regional hubs, based on RAN’s framework and learning. To the extent possible, eligible developing country researchers involved in these partnerships should be facilitated to apply for PEER funding under the mentorship of an HESN￾affiliated faculty member who is funded by a US government science agency. Leveraging PEER funding will eliminate the need for HESN Labs to fund developing country researchers. In addition, developing country researchers involved with HESN Labs who are funded by PEER should be tracked to increase the effectiveness of both programs. Better monitoring of the developing country researchers involved with HESN Labs would make it possible for PEER and HESN to compare the types of capacity building and other benefits to developing country researchers and HEI that occur within and outside of the HESN Lab structure.  PEER could capture the university affiliation of the US science agency-funded researcher and whether or not the US researcher is affiliated with an HESN Lab.  HESN Labs could report the number of PEER-funded developing country researchers working with HESN-affiliated faculty. 6. USAID should reduce the number of indicators required for reporting and allow individual HESN Labs to use a focused set of indicators that correspond to a core set of value propositions (based on updated HESN Lab Business Model Canvases as described above). HESN Labs should provide USAID with information about the indicators that are most relevant to their work, and USAID should analyze the extent to which individual indicators are comparable across Labs. In addition, best practices in compiling and managing M&E datasets should be implemented to allow the investment by HESN Labs in collecting and reporting on M&E indicators to feed into semi-annual dashboards and be more feasible for evaluators to use. 8 7. HESN should explore linkages with related USG efforts to share knowledge and increase understanding of the overlaps between university research and technology entrepreneurship, such as the State Department’s Global Innovation through Science and Technology (GIST) initiative10 (State Department) and NSF’s Innovation Corps (iCorps). Creating these linkages would connect HESN to other USG efforts to foster innovation capacity building and measuring its impact. 8. USAID and HESN Labs should develop a strategy, such as conferences and/or communities of practice, to engage development innovation stakeholders outside of HESN in directions both synergistic and complementary with the current set of research areas and value propositions. This strategy should be aimed at identifying gaps and opportunities for collaboration and partnership beyond what can be resourced through USAID and HESN Labs. 10 GIST consists of two programs that foster innovation and technology entrepreneurship through social networking, skill development, and financing. 9 I. INTRODUCTION This report is the midterm performance evaluation of the Higher Education Solutions Network (HESN), a five-year USAID program administered through the U.S. Global Development Lab with an initial programmatic ceiling of $140,559,741. Eight HESN Labs housed at six universities across the United States and one university in Kampala, Uganda (Makerere University) have received cooperative agreements to solve development challenges through three primary objectives: improve data quality, access, and analytics; accelerate the creation, testing, and scaling up of transformative innovations, technologies, and approaches; and catalyze a global ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity. The purposes of the midterm evaluation are to:  Assess progress toward the objectives outlined in the HESN Results Framework  Provide recommendations for future HESN programming Specific evaluation questions are listed in the evaluation scope of work in Annex I. The results of this evaluation will be used to inform USAID’s immediate and future program planning and implementation. This report is organized as follows: project background; evaluation approach; findings for each evaluation question; and recommendations. It should be noted that during the contracting and execution of this evaluation, there were shifts to the strategic priorities of the U.S. Global Development Lab, the USAID Operating Unit where HESN is housed. This evaluation is not intended to make definitive recommendations related to those shifts, and only speaks to the overall program of HESN. 10 II. PROJECT BACKGROUND The Higher Education Solutions Network (HESN) was formed in November 2012 to harness the ingenuity and passion of scientists, students, faculty, and entrepreneurs to solve some of the world’s most pressing development challenges. HESN consist of eight HESN Labs, each of which is led by a higher education institution (HEI), and supported by a core coordinating body at USAID. The goal of HESN is to create an interdisciplinary network of Development Labs11 to solve distinct development challenges. Partnering institutions will foster innovations in science and technology, and engage and inspire a new and broader community of scholars and students involved in the complex issues of international development planning, execution, and assessment. Specifically, through the resultant cooperative agreement(s), the academic community will support USAID and other development organizations to improve their analytical capabilities in understanding development challenges and core barriers to addressing them, catalogue solutions to different challenges, catalyze and bring forward novel approaches to addressing development problems, and encourage the development and application of new tools within science, technology, and engineering to improve the efficacy and decrease cost of development interventions. (HESN Request for Applications, 2012, p. 6) This goal is supported by three objectives: 1. Improve data quality, access and analytics to advance evidence-based development decision making 2. Accelerate the creation, testing, and scaling up of transformative innovations, technologies, and approaches 3. Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge promotes learning and builds mutual capacity. HESN Labs are intended to be virtual centers of knowledge that will help USAID and the larger development community better understand problems and evaluate and develop new solutions to development challenges. HESN Labs and their partners seek to foster innovation in science and technology, and engage and inspire a new and broader community of scholars and students involved in the complex issues of international development planning, execution, and assessment. The HESN Labs support USAID and other development organizations to improve their analytical capabilities in understanding development challenges and core barriers to addressing them, catalog solutions to different challenges, catalyze and bring forward novel approaches to addressing development problems, and encourage the development and application of new tools within science, technology, and engineering to improve the efficacy and decrease the cost of development interventions. In consultation with the HESN Labs, USAID/HESN created a results framework to align the HESN Labs activities with USAID’s strategic objectives. In addition to the aforementioned goal and three objectives, intermediate results (IRs) were selected that tied into each objective. HESN Results Framework: 11 Renamed “HESN Labs” after the creation of the U.S. Global Development Lab 11 Thirty-six (36) standard indicators have been tracked to monitor HESN Labs’ progress.12 Because each HESN Lab is working in different areas, not all of the indicators are applicable to every HESN Lab. Most HESN Labs report on about 10-20 of the standard indicators. The HESN Labs also monitor performance against indicators that are custom to each HESN Lab.13 The HESN Labs provide monitoring data to USAID on a semi-annual basis. Description of Individual HESN Labs: The HESN RFA asked for novelty and a diversity of approaches and, as a result, the HESN Labs that were selected operate in different areas with distinct approaches. The eight HESN Labs, based at seven universities, are described below.  AidData Center for Development Policy, College of William and Mary (AidData) concentrates on high resolution geospatial data, conducts analysis and applies imagery as decision support tools that enable the global development community to more effectively target, coordinate, deliver, and evaluate their aid investments.  Comprehensive Initiative on Technology Evaluation, Massachusetts Institute of Technology (CITE) is developing methods for product evaluation in global development in order to provide evidence for data-driven decision-making by development workers, donors, manufacturers, suppliers, and consumers. 12 Based on the number of standard HESN indicators reported in the HESN PITO Report 2013 - 2015. Note that Attachment 2 of the RFTOP mentions 35 indicators but one indicator was duplicated and two others that are reported in the PITO report (but not listed in Attachment 2) were included in the midterm evaluation analysis. 13 A total of 67 lab-specific indicators are tracked in addition to the 36 standard HESN indicators. These were not analyzed for the midterm evaluation. 12  Center on Conflict and Development, Texas A&M University (ConDev) seeks to improve the effectiveness of development programs and policies for conflict-affected and fragile countries through multidisciplinary research and education aimed at reducing armed conflict, sustaining families and communities during conflict, and assisting states to rapidly recover from conflict.  Development Impact Lab, University of California, Berkeley (DIL) combines engineering and the natural sciences with insights from economics and the social sciences to generate sustainable, technology-based solutions to development challenges.  Global Center for Food Systems Innovation, Michigan State University (GCFSI) is developing and testing new approaches to overcome the problems of shrinking farm land in developing countries, help under-resourced farmers deal with less rainfall due to climate change, and improve food systems from production and storage to packaging and distribution.  International Development Innovation Network, Massachusetts Institute of Technology (IDIN) is made up of 660+ innovators from around the world who create technologies hand-in-hand with local communities and — with access to funding, training, and mentorship — go on to turn their prototypes into products designed to make a difference.  ResilientAfrica Network, Makerere University (RAN) innovates and accelerates science and tech-based development tools in concert with USAID and a diverse set of stakeholders to strengthen African resilience to its greatest challenges, such as environmental variability, the effects of urbanization, government transparency, chronic civil conflict, and disease.  Social Entrepreneurship Accelerator, Duke University (SEAD) mobilizes a community of practitioners, investors, policymakers, faculty, staff, and students to identify, assess, help develop, build capacity of, and scale solutions, technologies, and business models for healthcare delivery and preventive services in developing countries around the world. 13 III. EVALUATION PURPOSE AND METHODOLOGY EVALUATION PURPOSE This section describes the purpose of the midterm performance evaluation and the methodology used by the team to answer the six key evaluation questions. The primary purpose of this midterm evaluation is to assess the progress of the Higher Education Solutions Network (HESN) towards its objectives outlined in the HESN Results Framework. HESN is comprised of eight HESN Labs housed in seven universities, and a core coordinating body at USAID. The network aims to:  Improve data quality, access and analytics to advance evidence-based development decision making  Accelerate the creation, testing, and scaling up of transformative innovations, technologies, and approaches  Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity Evaluation Questions: 1. To what extent has HESN been successful in achieving the outcomes of which it may be expected? 2. What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? 3. To what extent has HESN led to changes at HEIs that may increase their impact on international development? 4. To what extent has HESN influenced or assisted USAID operating units other than the US Global Development Lab? 5. How can HESN modify its strategy and structure to improve its efficiency and effectiveness? 6. What, if any, unanticipated positive and negative consequences have occurred as a result of the HESN project? EVALUATION METHODOLOGY The evaluation request for task order proposals (RFTOP) envisioned that the evaluation would be conducted using mixed methods for data collection including (but not limited to) a literature review, the examination of project documents and other secondary source materials, and interviews with USAID staff, implementing partners and other stakeholders on an individual or group basis, as appropriate. Data analysis using both quantitative and qualitative analysis was anticipated. The evaluation team’s approach largely conformed to these guidelines. The main difference between the approach actually used and what the RFTOP envisioned involved incorporating the Business Model Canvas (BMC) as an analytic tool, as explained below. The evaluation was organized into four major activities: Planning, Data Collection, Analysis, and Report Writing & Presentation. This section describes how the evaluation was conducted. It first describes the overall approach or structure for the evaluation, which was important in informing the selection of the specific methods used for each aspect of the evaluation. The evaluation methods used in this study were developed on a question-by-question basis using a “getting to answers matrix” to identify the types of data needed to answer each question and thus the appropriate data collection and analysis methods to 14 be used. This process is described in further detail in Annex B. In summary, the methods used to conduct this evaluation included: Document Review: The team conducted an initial review of HESN Labs’ applications, cooperative agreements, annual workplans, and annual reports. The review helped the team understand the focus of each HESN Lab, its own objectives and how they mapped to the objectives in the HESN Results Framework, and the types of activities and partners involved in the work of each HESN Lab. For a complete list of documents reviewed, please see Annex XII. Performance data analysis: HESN indicators reported to USAID by universities were cleaned,14 aggregated, and analyzed for each results area in the HESN Results Framework. These data were used to make determinations in Question 1 regarding the level of achievement towards HESN objectives. Dev Results Tables: In addition to requiring HESN Labs to report the HESN indicators, USAID required them to report individual activities in seven categories: HESN Classes and Disciplines, HESN Partners, HESN Fellowships and Practica, HESN Innovations, HESN Outputs, Communications, and Travel. The evaluation team used this information to gain additional detail related to the indicator counts and greater breadth as context for the site visit interview data.15 Use of the Business Model Canvas Tool: In order to address additional questions about the HESN theory of change, this evaluation piloted an adaptation of the nonprofit BMC,16 a widely used tool in the field of innovation. Figure 1. Adaptation of Nonprofit Business Model Canvas Used for HESN Midterm Evaluation Value Propositions: Primary beneficiary/user of services/activities: Secondary beneficiary: Key activities/products/services: Key partners: Delivery channels: Impacts: The BMC allows diverse kinds of transformative innovations to be assessed on a set of common areas17. It is applicable to any kind of product, service, technology, tool, or approach at any stage of development or implementation. The evaluation team used the tool to address the challenges of 14 Data cleaning included assessing missing targets and actuals, replacing zeros with missing values, and cross-checking anomalous values with individual HESN Lab reports. We also created a reduced dataset for the performance analysis that consisted of only the data for HESN Labs with both targets and actuals for a given year. 15 Although the Dev Results data tables included the activities aggregated in the HESN indicators, the counts did not always match, suggesting that the systems were not cross-linked or cross-checked. However, the evaluation team made the assumption that the distribution of activities across sub-categories within each table (e.g., approaches and technologies in the Innovations table) was a reasonably close approximation of actual activities. 16 Retrieved from http://www.innovativenonprofit.com/2015/08/the-nonprofit-business-model-canvas-2/#.VkuXXXudn9Q 17 The adaptation of the Nonprofit BMC used for the HESN midterm evaluation used six of the nine sections included in the standard version of the Nonprofit BMC that were feasible for the HESN Labs to complete and appropriate for the HESN context. 15 evaluating the effectiveness of innovation-generating initiatives. A key challenge is assessing results in cases when metrics have not yet been developed for an innovation or new approach. The BMC tool identified the value that each HESN Lab intended to provide through its activities, the beneficiaries or users it targeted would gain from HESN Lab activities, and the downstream impacts that could result. Led by the Senior Technical Advisor, the evaluation team conducted a webinar to introduce the HESN Lab teams and HESN program staff to the BMC and its purpose for the HESN midterm evaluation. Most HESN Labs had not used the tool in the past, but several had heard of it or had prior experience with it. The tool was completed by a team from each of the HESN Labs with technical assistance from the evaluation team. HESN Labs were requested to complete one canvas for each major type of beneficiary, user, or recipient of their activities or services. The evaluation team used the information from the HESN BMCs to develop a sampling frame of beneficiaries/users and partners for interviews to check whether the intended benefit identified on the BMC tool was needed and useful to the recipient. All eight HESN Labs made a good faith effort to complete draft versions of their canvases in advance of site visits, and assistance was provided as requested. Originally, the evaluation team had hoped to complete beneficiary and partner interviews for each HESN Lab and bring a compilation of results and themes for discussion at the site visit. However, this was not possible given the time needed to draft the canvases and schedule the interviews. Key Informant Interviews: The evaluation team developed interview protocols to generate the body of data needed to address the six evaluation questions and sub-questions (Annex II: Evaluation Methods and Limitations). Campus leader, HESN Lab leadership, and HESN Lab team member interviews were conducted as part of site visits to the eight HESN Labs and are described under Site Visits below. Beneficiary interviews (68) followed up on the information provided in the HESN Lab Canvases, asking whether or not the intended benefit was provided and found useful, and identifying any additional value not anticipated by the HESN Lab. Partner interviews (27) focused on the purposes of the partnership, motivations, and expected and unexpected benefits or negative effects. Interviews with beneficiaries and partners were conducted primarily by phone and Skype, except when it was possible to schedule them at site visits. For data collection instruments, see Annex III. USAID Site Visit: In preparation for the HESN Lab site visits, interviews were carried out with 10 USAID/HESN program staff to capture perceptions and learning on their experiences across the different HESN Labs, relationships between HESN Labs and intended beneficiaries, and other cross￾network activity and experience. Site visits to HESN Labs: To understand the work of each HESN Lab more deeply, two-day site visits were done with all eight HESN Labs (The team spent 4.5 days at Makerere University in Uganda to allow sufficient time to understand the diverse nature of RAN’s cross-Africa consortium.) The Evaluation Team Leader carried out a set of advanced preparation steps to work with the point person at each HESN Lab on the types of interviews and focus groups to set up, the types of individuals who should attend each, and the schedule for the visit. Site visits were originally planned to take place in December and January, but this was not feasible given the academic calendar and availability of campus leaders and HESN Lab team members during these months. One site visit took place in mid-December Malawi Market Interview. Photo Credit: GCFSI. 16 and the remaining six visits were scheduled between mid-January and mid-February. Two to three evaluation team members performed each site visit. As stated above, Campus leader, HESN Lab leadership, and HESN Lab team member interviews were conducted as part of site visits. Campus leader interviews (16) addressed questions regarding campus commitment to the HESN Lab’s sustainability through resources or other means, and the value and impact of the HESN Lab more widely on campus. HESN Lab leadership interviews (19) and team member interviews (68) focused on key achievements as a result of the HESN award to date, factors supporting and challenging progress on HESN Lab objectives, unanticipated outcomes, and how the HESN Lab would be sustained following USAID funding. In addition to interviews, focus groups with the core HESN Lab staff and team members addressed engagement with USAID Mission personnel and other beneficiaries, critical decisions that shaped the work, practices developed through learning from HESN Lab activities, and gains from network membership and activities. Additionally, a focus group session at each site visit was allocated for the review of the BMCs and needs for clarification and completion. Data Analysis: Due to the schedule of the site visits and the short time frame for conducting beneficiary and partner interviews, analysis was not conducted until the site visits were completed. The team entered all interview notes into an Excel database broken down by interviewee type and interview question. Once all of the site visits and key informant interviews were completed, the team identified patterns in responses and major themes, specifically those that answered or were relevant to key evaluation questions. Data were coded using key words and then moved to a separate Excel file broken down by evaluation question to identify patterns, count numbers of similar responses across HESN Labs, and begin to answer evaluation questions. Once broad themes were identified, the team began pulling out specific data to form the findings and ultimately the conclusions within each evaluation question. LIMITATIONS The timeline was a primary limitation during this evaluation. As mentioned above, site visits were originally planned to take place in December and January, data analysis in February, and report writing and additional analysis as needed in March. However, all but one of the HESN Labs were unavailable able for site visits until after mid-January. The revised schedule allowed for more time to be spent on the document review, preparation of tools and methods, facilitating the HESN Labs to complete the BMC, and conducting interviews with other USAID initiatives for Evaluation Question 2b. The remaining seven site visits took place between January 20th and February 20th. This left only a short time for analysis of the rich database of qualitative data generated from the large number of interviews that were completed during site visits. The BMC provided the sampling frame for key informant interviews with HESN Lab beneficiaries and partners. Due to the holiday break, the HESN Labs were not able to complete a draft of the BMC until mid- to late January. This led to a delay in scheduling interviews. Over 100 key informant interviews were conducted parallel to the site visits. The short timeframe eliminated the possibility of using snowball sampling to identify additional individuals to interview based on the need to fill in missing perspectives. It also impacted the ability of the evaluation to represent the full range of beneficiaries and partners for each HESN Lab. However, the evaluation team sought to sample key beneficiary groups (e.g., students, researchers, development practitioners, USAID Mission and operating unit staff) across the eight HESN Labs to the extent possible. Another limitation in answering questions about achievement of HESN objectives was the inconsistent indicator reporting due to changing M&E requirements, lack of data quality control for the HESN M&E system, and problems within the reporting system. The evaluation team received the final version of the consolidated FY2013-FY2015 dataset in mid-February and completed a re-analysis of the indicator data alongside analysis of the site visit and key informant interview data. 17 The evaluation team would have normally conducted data analysis on a daily, weekly, and bi-weekly basis (using different levels of analysis at each interval). However, given the tight schedule of site visits and quantity of interviews per day, there was not time for the team to begin coding, analysis, and development of findings until data collection was complete. The evaluation team also would have normally coded the interview data thoroughly before beginning the analysis. However, because of the tight turnaround time between finishing data collection and submitting the draft report, the team coded only interview data that specifically answered each evaluation question. USAID provided a one-month extension, which allowed the evaluation team more time to analysis the data and develop findings and recommendations. The analysis of the interview data focused on comparing the effectiveness of HESN Labs to other USAID initiatives in managing costs and staff (Evaluation Question 2b) was narrowed to focus on comparisons of strategies related to the management of costs and staffing at the level of the USAID HESN program. The data required to assess the effectiveness of the individual HESN Labs in managing costs and staff was not feasible to collect in the evaluation timeframe, nor did the evaluation team feel that it was appropriate to compare individual Labs to other USAID initiatives. We believe the analysis reported under Evaluation Question 2b provides useful information about the types of strategies related to managing costs and staff that can inform the next stage of HESN programming. Finally, the evaluation team determined that due to the wide range of activities and beneficiary types within and across the HESN Labs, a survey would not be an efficient or feasible approach to collecting qualitative and quantitative data needed for addressing each evaluation question. Instead, the key informant interview for HESN Lab stakeholders (beneficiaries and partners) was honed to 30 minutes and when necessary, completed in writing via email. This allowed data collection from a broad sample of key informants without the need to pilot-test a survey and with a relatively high response rate. 18 IV. FINDINGS AND CONCLUSIONS This section presents the findings for each of the six HESN midterm evaluation questions outlined in the scope of work. EVALUATION QUESTION 1. TO WHAT EXTENT HAS HESN BEEN SUCCESSFUL IN ACHIEVING THE OUTCOMES OF WHICH IT MAY BE EXPECTED? a. To what extent has HESN achieved the outcomes outlined in its Results Framework? b. To what extent has HESN achieved the outcomes expected by key stakeholders? c. To what extent have the outcomes achieved by HESN been consistent with the evaluators’ understanding of what may be expected of universities working on development innovation? Evaluation Question 1.a.: To what extent has HESN achieved the outcomes outlined in its Results Framework? The primary data source for this evaluation question was the standard HESN indicators. The overall performance of the HESN Labs on the three objectives outlined in the Results Framework was high over the first three years. Collectively, HESN Labs met or surpassed targets for 27 out of 34 (79 percent) standard HESN indicators.18 High levels of indicators showing targets achieved represents the wide breadth of the HESN Labs’ performance across all three objectives of the Results Framework. While a third of goal-level indicators fell short of targets, overall there is no weak area of performance in the Results Framework. Table 1.1 Percent of HESN standard indicators that met or exceeded targets Description # of indicators for which targets set # of indicators that met or exceeded targets % of indicators on which targets were met or exceeded Goal: Create a global interdisciplinary network of Development Labs to solve distinct development challenges 12 8 67% Objective 1: Improve data quality, access, and analytics to advance evidence-based development decision making 619 5 83% Objective 2: Accelerate the creation, testing, and scaling of transformative innovations, technologies, and approaches 5 4 80% Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that 11 10 91% 18 Performance against indicators was assessed for the indicators for which targets were set and actuals were reported for a given HESN Lab. In other words, if a Lab reported actuals for any indicators for which it had not set a target, these figures were not included in the performance analysis (but are reported in Annex VI as part of the Overall Total). Two indicators did not have targets. 19 There are a total of 8 indicators for Objective 1 but targets were set for only 6 of them. 19 Description # of indicators for which targets set # of indicators that met or exceeded targets % of indicators on which targets were met or exceeded shares knowledge, promotes learning, and builds mutual capacity Overall 34 27 74% In order to assess the strength of performance, the percent above or below target was calculated for each indicator, as illustrated in Table 1.2 below. On average across all indicators for all years, HESN Labs surpassed targets by 66 percent.20 (See Annex VI for the full set of performance analysis results.) The strongest performance has occurred in the area of improving data quality, access, and analytics to advance evidence-based development decision making (Objective 1). This reflects the fact that a major expertise of HESN Labs – rigorous data collection and analysis – has been emphasized in HESN’s initial stages of relationship building with USAID operating units and other development organizations. Rigorous data collection and analysis also tends to take less time to develop and implement,21 and is supportive of identifying needs for piloting and adoption of transformative innovations and approaches. An analysis of performance for each of the objectives is detailed below. Table 1.2 Average percent above target for goal-level and objective indicators Description Average percent above target for goal-level and objective indicators Goal: Create a global interdisciplinary network of Development Labs to solve distinct development challenges +30% Objective 1: Improve data quality, access, and analytics to advance evidence-based development decision making +167% Objective 2: Accelerate the creation, testing, and scaling of transformative innovations, technologies, and approaches +35% Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity +65% Overall +66%22 The areas in which targets were surpassed by the greatest margin included not only datasets, data analysis, and training in data tools, but also in beneficiaries reached and communities participating in any 20 This analysis aggregated all years of data available for each indicator in order to assess performance to date. These results do not indicate whether targets were adjusted upward when previous year targets were exceeded. As demonstrated in the annual reports, HESN Labs were in general attentive to whether or not targets were met each year. 21 Development of analytic tools/platforms and geocoded datasets may be the exception because of the longer time needed to develop them. 22 Calculated by averaging percent above/below target across all 34 indicators for which targets were set. 20 kind of activity23 as well as participation in ecosystem24-building activities such as competitions (e.g., Berkeley’s Big Ideas) and summits (e.g., IDIN International Development Design Summits). The 2015 budget cuts due to resource constraints25 were a factor in reduced travel for student field experiences, research fellowships, and other areas. Improved data quality, access, and analytics (Objective 1) As mentioned, a high density of activity occurred in the area of improved data quality, access, and analytics. HESN Labs provided or made accessible a total of 265 new data sets and 478 data-related analyses, mapping activities, and expert consultations to USAID operating units and programs, HESN partners, and the broader development community. In addition, 1,645 development professionals were supported by HESN Labs to become proficient in data management and use. This activity was significantly higher than targeted for five of the six indicators (see Annex VI). In addition to performing well against targets in this area, HESN Labs have made continued gains each year on 71 percent of the standard indicators. Annex VIII provides graphs of year-by-year progress on each indicator reflecting stages of program development: startup (Year 1), expansion (Years 2-3), and stabilization (Year 3 for number of data-related analyses, mapping activities, and expert consultations provided; number of development professionals proficient in data management and use). The only area in which HESN Labs fell short of targets for Objective 1 was the number of USAID operating units using geographic analysis to monitor and evaluate development projects (12 vs. 22 targeted). Accelerated creation, testing, and scaling of transformative innovations (Objective 2) As of Year 3, HESN Labs have engaged in a high level of activity related to preparing for developing, piloting, and testing innovations on the ground. A total of 694 communities participated in assessment, analysis, and evaluation of innovations, technologies, and approaches supported by HESN Labs (vs. 266 targeted). In addition, 1,205 stakeholders engaged in problem solving with an HESN Lab (slightly higher than the 1,010 targeted) and 222 Memoranda of Understanding (MOUs) or other agreements were signed between public sector, private sector, local community partners, and an HESN Lab (vs. 152 targeted). All of the HESN Labs reported valuing the opportunity to engage with local partners during the development stage of innovation and said that HESN funding has been a critical factor in making this possible. As a likely result of the engagement activities cited above, 210 transformative innovations, technologies, or approaches were developed by HESN Labs (compared to 148 targeted), 71 were piloted (vs. 94 targeted); and 18 were adopted (vs. 24 targeted). Twelve innovations, technologies, or approaches achieved wide-scale adoption (vs. 6 targeted) and 31 were evaluated (vs. 26 targeted).26 These deviations from targets may reflect the challenging nature of setting targets for innovations, particularly those requiring partners and implementation sites in the developing world. 23 The dataset did not disaggregate beneficiaries reached by objective or type of activity. 24 In the context of HESN, an ecosystem is a global interdisciplinary network of individuals and institutions interacting through a range of activities and involved in the generation, diffusion, and utilization of technology and approaches aimed at addressing international development challenges. 25 As noted under Evaluation Question 6, 64% of total Cooperative Agreement funding was awarded across the HESN Labs. 26 In most cases, the overall totals for these indicators was higher than the total for HESN Labs with targets, due to the fact that not all HESN Labs reporting on these indicators had set targets. A total of 80 innovations/approaches were piloted, 21 were adopted, 23 achieved wide-scale adoption, and 41 were evaluated. 21 Catalyze a global interdisciplinary ecosystem (Objective 3) Beginning with the university campuses at which HESN Labs are housed27 and extending beyond, HESN Labs have accomplished an extensive amount of ecosystem development activity. HESN Labs created 47 new development related classes or disciplines (vs. 36 targeted), and supported 120 others (vs. 88 targeted). A total of 571 students participated in short term practica28 or other field experiences (no targets were set for this indicator). Sixty collaborative platforms have been created by HESN Labs (compared to 42 targeted) and almost a million visitors had been logged for HESN knowledge-sharing platforms (well above the target of 672,300). Over 30 innovation hubs29 have been created by HESN Labs and there have been almost 350,000 participants in hubs, summits, and other problem-solving institutions (vs. 142,380 targeted). A total 4,200 participants have been engaged in crowd-sourcing or other open challenges created by HESN Development Labs (vs. 1,340 targeted). Four MOUs or other agreements have been signed with public sector, private sector, local community partners, and more than one HESN Lab, and 56 programs or projects have been undertaken collaboratively by HESN members. In addition to performing well against targets in this area, HESN Labs have made continued gains each year on 71 percent of the standard indicators. Annex VIII provides graphs of year-by-year progress on each indicator reflecting stages of program development: startup (Year 1), expansion (Years 2-3), and stabilization (Year 3 for number of data-related analyses, mapping activities, and expert consultations provided; number of development professionals proficient in data management and use). Evaluation question 1.b.: To what extent has HESN achieved the outcomes expected by key stakeholders30? The primary data source for answering this evaluation question was interviews with 68 key informants identified as “beneficiaries or users”31 through the BMC tool. These interviews were aimed at validating whether or not the opportunity or service provided by the HESN Lab was needed and beneficial to the recipient. Ninety percent of those interviewed validated that the intended benefit provided by the HESN Lab was, in fact, beneficial to them in their work. These key informants included representatives from USAID Missions and operating units (12), developing country innovators (16), NGOs (4), developing 27 Referred to as “HESN lead campuses” elsewhere in this report. 28 A practicum is a graduate level class, often in a specialized field of study, which is designed to give students supervised practical application of concepts and theories. 29 An innovation hub is a collaborative environment or work space that provides innovators access to subject-matter expertise, resources, and other support needed for innovation development. Innovation hubs enable active knowledge transfer between individuals with different skill sets, backgrounds, and focus areas. Hubs that provide tools, materials, and workshop space for building tangible innovations are sometimes referred to as “maker spaces.” 30 For the purposes of this analysis, “stakeholders” refers to HESN Lab beneficiaries/users identified through the BMC tool. 31 A beneficiary or user is someone to whom the HESN Lab seeks to deliver value, either through providing some kind of benefit or gain (e.g., providing students and faculty opportunities for interdisciplinary development-focused research), or by relieving a problem (e.g., lack of data needed to inform action). Beneficiaries/users for each HESN Lab were identified by the HESN Lab through the use of the Business Model Canvas tool adapted for HESN. SEAD held a global health innovation symposium in March 2015. Photo Credit: SEAD. 22 country local or private sector organizations (2), multilateral organizations (1), impact investors (2), developing country universities (2), graduate students (19), and researchers (10). The high rate at which HESN Labs have been able to deliver benefit to their target beneficiaries reflects the fact that they have been able to engage directly with these groups during the first three years of the HESN award and/or have been able to adjust their services/activities to better fit the needs of the beneficiary groups. Benefits compiled from interviews with HESN Lab beneficiaries include:  The opportunity for development actors to engage in dialogue with leading edge university researchers about problems and hypotheses related to development challenges, resulting in knowledge exchange and the design of research aimed more effectively at informing decisions and solutions.  Opportunities for developing country researchers to advance research skills, build networks and increase the recognition and legitimacy within their own institutions through joint, ongoing projects with U.S. university faculty and students.  Preparation for students to engage in effective development innovation: design thinking, appropriate technology, cultural awareness, lean experimentation, etc.  Providing researchers access to data and other kinds of engagement needed for delivering value to development actors (through partnerships as described above).  Ability for students to spend time in country and in target communities, which deepens collaborations and extends the data available for their research.  The ability of information, previously unavailable, to inform development decisions, such as consumer product evaluation and geocoded aid data.  Support for scaling innovations in ways that are appropriate for a given innovation, including community-level scaling, replication in new contexts, and using scaling parameters to design new products.  Opportunities for students in US and developing countries to take classes and participate in research and innovation projects focusing on development challenges. However, HESN Lab beneficiaries also reported ways in which their engagement with HESN Labs could be improved. This included timely delivery of reports, better communication, fewer bureaucratic delays (both universities and USAID), and uncertainty about future funding. Evaluation question 1.c.: To what extent have the outcomes achieved by HESN been consistent with the evaluators’ understanding of what may be expected of universities working on development innovation? Higher education and development innovation experts32 see HESN as successful in creating a set of reward structures, namely funding, institutional recognition, infrastructure for interdisciplinary collaboration, opportunities for publishable research, and partnerships with USAID and other prominent development institutions, that faculty, post doctorate, and graduate students need to support the expansion of development innovation research. 32 A total of 12 individuals were interviewed for this research question (7 USAID and 5 non-USAID informants). The list of interviewees is provided in Annex IV. 23 According to one informant, HESN has achieved outcomes comparable to those observed for the NSF Innovation Corps initiative (I-Corps), which translates results from NSF-funded, basic research projects into technologies with potential for impact in the commercial world. Two informants stated that HESN has expanded universities’ ability to interact with the international development world, especially through the channels of USAID Missions and Bureaus. It has helped attract faculty and deans with an international aspiration or helped to shape departments to focus on international development. Several informants believe that HESN has been able to achieve strong results to date because of its structural partnership with the Global Development Lab, USAID Missions, and other USAID operating units. Without this basis of relationship for identifying needs for datasets, analytic tools, and solutions, USAID-university researcher collaboration is limited to individual connections and standalone engagements outside of a cohesive and ongoing set of objectives. Transformative, sustainable solutions are less likely as a result. However, two innovation experts stated that more results can be accomplished when there is a more efficient way for university researchers to become aware of the most important challenges faced by development organizations. Interviews with higher education and development innovation informants identified three areas in which HESN has not met expectations for university-based development innovation:  Overall dissemination of research and solutions produced by HESN Labs is lower than expected by both USAID and non-USAID audiences. Both audiences would like a centralized source of dissemination.  There is a perceived lack of opportunity for researchers, students, and development professionals outside of HESN-affiliated universities to become involved in the development innovation ecosystem being cultivated by HESN Labs, whether through becoming an affiliated researcher, the ability to enter students into HESN Lab-sponsored competitions, or participating in a community of practice related to an area of work related to an HESN Lab. One informant had the perception that the HESN Labs are busy trying to meet their obligations to USAID and not able to invest time in expanding their networks.  Capacity building for developing country researchers is not being tracked and should be in order to connect it to other USAID higher education initiatives (See Recommendations). SUMMARY of Evaluation Question 1: To what extent has HESN been successful in achieving the outcomes of which it may be expected? Over the first three years of the project, HESN Labs performed well on the three objectives of the HESN Results Framework, surpassing targets for 79 percent of the standard indicators. Through HESN funding and in partnership with the key development actors (public sector, private sector, local community), HESN Labs have provided data and analytic tools, developed transformative innovations and approaches, and built the capacity of the ecosystem for development innovation. HESN engages Dr. Chris Baryomunsi, State Minister for Health General Duties, officially launching the First State of African Resilience Report at Makerere University. Photo Credit: RAN. 24 stakeholders who vastly (90 percent) reported a direct benefit from their engagement with the HESN Lab. Additionally, HESN appears successful in creating effective university reward structures that facilitate expansion of development innovation research, including funding, institutional recognition, and structural partnership with the Global Development Lab, USAID Missions, and other USAID operating units. However, HESN has been less successful in disseminating research products, offering opportunities to stakeholders outside of the network, and monitoring capacity building for developing country researchers. 25 EVALUATION QUESTION 2. WHAT HAVE BEEN THE COSTS AND BENEFITS OF HESN’S MODEL OF CONCENTRATING MULTIPLE OBJECTIVES AND ACTIVITIES WITHIN EACH HESN LAB? a. Have HESN Labs found synergies across the teams and disciplines that have led to more effective or efficient implementation? b. In comparison to other Activities (identified by the Evaluation Team in consultation with USAID/HESN), how effective have HESN Labs been in managing costs and staff? As the work of the HESN Labs has developed over the first three years, they have had to create and revise strategies and structures for achieving multiple objectives, carrying out a range of different types of activities, addressing the challenges of working across disciplines, and implementing projects on the ground. This was a major emphasis of the startup phase of the work and continues into Year 4. What have been costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? The primary data source for this evaluation question was interviews with 68 HESN Lab team members, particularly those involved operationally with implementation strategies and challenges. To date, the benefits of concentrating multiple objectives and activities within each HESN Lab appear to have outweighed the costs. Table 2.1 provides a summary of the benefits and costs. Table 2.1 Benefits and Costs of Concentrating Multiple Objectives within each HESN Lab Benefits Costs HESN Labs more broadly integrated into university mission Administrative burden of reporting on more M&E indicators Greater breadth and richness generated for each HESN objective More value propositions means greater management load HESN Labs have explored and experimented with many types of activities and beneficiary types, leading to useful pilots and learning More activities requires more coordination of linkages across activities A primary benefit of allowing the HESN Labs to address multiple objectives is that the work of each can be more fully integrated into the mission of the university compared to a standalone research institute or training grant. The three objectives align with what universities are best at: developing datasets, creating new tools and approaches, and training students and researchers. Furthermore, the fact that multiple HESN Labs are addressing each HESN objective has led to a collective breadth and richness of activity under each objective. Even when addressing more than one objective, some HESN Labs have made one objective primary, structuring activities related to other objectives to function in service of the primary objective. This supports coherence across activities. For example, AidData’s primary objective is producing geocoded datasets but AidData also works on innovative analytic tools for using geocoded data. Some HESN Labs have found specific ways of marrying different kinds of activities, such as DIL, which conducts impact evaluations in the context of the development of engineering innovations. The ability to address multiple objectives has allowed HESN Labs to be more agile in addressing needs and limitations of different development actors and contexts. If evidence or research capacity building is needed, they are able to provide that; if new solutions are needed, they can provide that; if student interns are needed, they can 26 also provide that. HESN Labs have explored and experimented with many types of activities and beneficiary types, leading to useful pilots and learning. However, the costs of addressing more objectives are that there are more value propositions to deliver on and a greater number and variety of activities to manage and coordinate. HESN Labs that address a greater number of objectives tend to report on more M&E indicators, which adds to their administrative load. As the end of the initial funding period approaches, it would benefit HESN Labs to develop logic models of the interrelationships among their objectives, to assess tradeoffs between prioritizing more vs. fewer objectives, and evaluate which activities are the most important in delivering the highest￾impact value propositions. Evaluation Question 2.a.: Have HESN Labs found synergies across the teams and disciplines that have led to more effective or efficient implementation? Analysis of the HESN Lab team member interview data as well as with 27 HESN Lab partners pointed to three kinds of synergies that have broadly enhanced the work of HESN Labs. Research studies help establish partnerships with development actors and reveal needs for new solutions. As described under Evaluation Question 1, the strongest performance has occurred in the area of improving data quality, access, and analytics to advance evidence-based development decision making (Objective 1). In fact, this expertise is supportive of identifying needs for piloting and scaling innovations and approaches (Objective 2). Conversely, HESN Labs reported taking advantage of an existing intervention effort to carry out new evaluation and learning activities. Universities excel at both evaluation research and R&D. The interdisciplinary focus of HESN has promoted synergy between these areas. Creating partnerships with development actors based on research into new solutions requires relationship building and time to discuss the nature of the problem and the need for new solutions. A number of HESN Labs have been carrying out impact evaluations and other forms of commissioned research that address a need by a development organization (e.g., AidData has done impact evaluations for USAID Missions; DIL conducted research for Kenya Power & Lighting Company to inform rural electrification policy). These focused research engagements, discussions, and relationship building can reveal needs for new solutions. Students are a key resource in delivering on research and innovation objectives. There are natural synergies between the development of innovative approaches and technologies (Objective 2) and preparation of students as development innovation researchers (Objective 3). Relationship building between HESN Labs and developing country partners is often supported through student involvement – students can travel more often and for longer than faculty staff, and develop relationships throughout an organization or community, versus only with top leadership. Beneficiaries of HESN Lab services become valuable partners. Key informant interviews with HESN Lab team members and HESN Lab partners surfaced the interesting finding that organizations identified as partners for HESN Labs often viewed themselves as beneficiaries. The HESN Lab needed the assistance of the partner, whether to provide in-country contacts, carry out data collection, extend reach, or provide funding. Through the partnership activity, the partner would receive some kind of capacity building from the HESN Lab, which provided an unanticipated benefit. This kind of synergy means that HESN Labs cultivate and maintain relationships that provide partnership as well as impact. Examples of partners that are also beneficiaries include:  IDIN partnered with Singapore Polytechnic to extend IDIN’s reach in Southeast Asia. IDIN provided faculty training and curriculum development support to Singapore Polytechnic faculty and staff. Singapore Polytechnic adapted IDIN’s innovation design approach to its Learning Express program for undergraduates and expands IDIN’s reach by 500 students per year. 27  SEAD partnership with Investors’ Circle has catalyzed discussions among an unusually broad range of investors and provided unanticipated value to participating investors. Investors’ Circle also partners with SEAD’s social entrepreneurs to support them to refine their investor pitches and to provide investment.  CITE is partnering with Mercy Corps to gain in-country access to communities appropriate for testing solar water pumps. CITE researchers will train Mercy Corps staff to collect data needed for the evaluation, which Mercy Corps anticipates will create valuable capacity that will allow them to conduct their own field-testing in the future. Mercy Corps anticipates that their in-country operations will also be informed by the product report produced by CITE. Evaluation Question 2.b.: In comparison to other Activities (identified by the Evaluation Team in consultation with USAID/HESN), how effective have HESN Labs been in managing costs and staff? In order to put the HESN Project into a broader context of USAID’s higher education and development innovation initiatives, the evaluation team carried out approximately one hour interviews with lead staff from six USAID initiatives33 related to higher education and/or development innovation. Interviewees were asked about the goals, objectives, and timeline of the initiative, staffing and cost structure, challenges of managing staff and costs, and factors slowing and supporting the success of the initiative. When interviewees had adequate knowledge of HESN, they were asked to compare their initiative to HESN. The evaluation team was not able to directly answer the question of how effective HESN Labs have been in managing costs and staff. As described in the Evaluation Purpose and Methodology section, the data required to assess the effectiveness of the individual HESN Labs in managing costs and staff was not feasible to collect in the evaluation timeframe. In addition, the evaluation team did not believe that it was appropriate to compare the management of costs and staffing at the individual HESN Lab level to the management of costs and staffing at the USAID program level. Our solution was to focus the analysis for Evaluation Question 2b on comparing the challenges of managing costs and staffing to achieve initiative goals and the strategies for addressing those challenges that were described in the interviews with lead staff from other USAID initiatives to HESN at the program management level. Compared to other USAID initiatives, HESN program staff have been effective in supporting HESN Labs to understand how to fit what they are offering to USAID Missions and operating units, in facilitating relationship building between university partners and USAID operating units, in helping HESN Labs develop a comparable number of projects and partnerships relative to more established initiatives, in placing a large number of graduate researchers into fellowships with developing country partners, and in encouraging HESN Lab faculty researchers to partner with developing country researchers. The comparison of HESN to other USAID initiatives points to the following strategies for related to staffing and cost allocation that could increase HESN’s effectiveness:  USAID HESN program staff focus considerable time effort on helping US researchers understand how to fit what they are offering to USAID Mission requirements. The development of a more robust structure that would invite USAID Missions to articulate upcoming challenges and needs and allow HESN Labs to submit applications for meeting those needs would allow the HESN program staff to focus more time and attention on the relationship-building activities that have played an important role in the success of HESN Lab-Mission partnerships. 33 A list of 12 initiatives was provided to the evaluation team along with email introductions from USAID HESN leadership. Contacts from initiatives who responded within two rounds of email follow-up by the evaluation team participated in a one hour interview. One respondent (University Support and Workforce Development, Afghanistan) did not feel this initiative was relevant to HESN, so the evaluation team substituted the more general interview protocol used with higher education experts. 28  HESN convenes HESN Lab Directors annually and sponsors HESN Labs to bring together USAID, HESN Labs, and other development innovation stakeholders at periodic Technical Convenings (TechCons) held at one of the HESN lead universities. Allocating funds for regional gatherings held in developing countries and attended by HESN Labs working in that region and regional ecosystem actors would provide a forum for knowledge sharing, discussion of pressing local challenges, and relationship building that has been successful for other innovation￾development initiatives.  Several US-based HESN Labs (e.g., ConDev, SEAD, DIL) have established long-term teams in countries where they work to mitigate the costs of travel and enhance the continuity of project management and relationship building. HESN goals would likely benefit from further development of in-country teams and/or hubs to cut down on travel costs and strengthen the role of developing country partners.  In order to better support USAID Missions, HESN could align the delivery of semi-annual reports produced by university partners with the timing of Missions’ portfolio reviews. This strategy would likely benefit the success of partnerships between HESN Labs and USAID Missions but would add an additional dimension to the role of HESN program managers by requiring them to monitor timelines for all Missions engaged with HESN Labs.  PEER34 asked HESN to help recruit developing country researchers to apply to PEER because many HESN-affiliated researchers are funded by US federal science agencies. However, there is no tracking at this point of developing country researchers involved with HESN Labs who are funded by PEER. Better monitoring of the developing country researchers involved with HESN Labs would allow PEER and HESN to compare the types of capacity building and other benefits to developing country researchers and HEI that occur within and outside of the HESN Lab structure. Comparisons of HESN to each of the six other USAID initiatives that participated in interviews for the HESN midterm evaluation are described below. Strategies and structures shared with higher education capacity building initiatives. The Higher Education Partnerships for Innovation and Impact (HEPII) is a new initiative that seeks to expand USAID's reach into academic and technical sectors to solve development challenges while engaging the private sector, foundations and other industry partners. HEPII is similar to HESN in that it seeks to introduce innovation into programming by USAID operating units. At the time of the interview, no awards had been made. HEPII is currently staffed by a program manager and a higher education advisor. HEPII sees its key challenges as educating USAID Missions to “uptake” innovations developed at US universities and helping university researchers to understand how to fit what they are offering into USAID Mission requirements. To address the first challenge of educating USAID Missions to “uptake” innovations, the HEPII program manager uses the strategy of providing technical assistance to USAID Missions in evaluation, program design, and alignment with higher education policies and practices. This strategy of preparing USAID Missions to receive and uptake innovation is not available to HESN program managers due to the fact that HESN is focused on a much broader array of program areas including agriculture, energy, economic development, health, and many others. Staffing for HESN requires both understanding how developing country systems operate and background in the domains of innovation related to each HESN Lab. To address the second challenge of helping university researchers understand how to fit what they are offering into USAID Mission requirements, HEPII has structures for both “demand-driven” and “supply￾side” strategies. USAID Missions are invited to articulate a need and university researchers can submit an application for meeting that need – a demand-driven strategy. Conversely, researchers can submit a 34 Partnerships for Enhanced Engagement in Research 29 concept paper based on an innovation or technology to a Mission or other operating unit without responding to a stated need. This supply-side relationship-building approach is similar to the strategy used by HESN, but HESN program managers supplement the efforts of HESN Labs with a considerable amount of prior relationship development and research to identify opportunities for a good fit. USAID HESN program managers also focus a great deal of effort on helping US researchers understand how to fit what they are offering to USAID Mission requirements. HESN program managers currently do use an informal “demand-driven” strategy35 to support the development of partnerships between researchers and USAID Missions. However, developing a more robust structure that would invite USAID Missions to articulate upcoming challenges and needs and allow HESN Labs to submit applications for meeting those needs would allow the HESN program staff to focus more time and attention on the relationship￾building activities that have played an important role in the success of HESN Lab-Mission partnerships. Like HEPII, HESN seeks to influence decision-making by USAID operating units, but HESN program managers also spend time laying the groundwork for HESN Labs and USAID operating units engage in collaborative problem solving and knowledge sharing. Higher Engineering Education Alliance Program (HEEAP) is an initiative managed by USAID/Vietnam in partnership with Arizona State University (ASU) to enhance traditional theory-based engineering and technical vocational programs by advocating the addition of applied and hands-on instructional approaches. HEEAP 236 aims to produce graduates who possess the applied and technical communication skills required to excel in multinational corporations. The initiative and its programs are designed by ASU and implemented primarily by a seven-person Vietnam-based staff with short-term visits from four ASU faculty. The staff time is allocated toward implementing training and engaging industry partners. HEEAP’s in-country management strategy reduces travel costs while supporting the goal of adapting innovative instructional approaches to the Vietnam context. Several US-based HESN Labs (e.g., ConDev, SEAD, DIL) have established long-term teams in countries where they work to mitigate this challenge. However, HESN Lab faculty and students require travel funds to work on projects in multiple countries. HESN goals would likely benefit from further development of in-country teams and/or hubs to cut down on travel costs and strengthen the role of developing country partners. In-country teams and hubs from US universities need to maintain coordination and communication with the USAID Mission, which requires ongoing time commitments from HESN program staff. Strategies and structures shared with university-developing country innovation partnership initiatives. The Higher Education for Development (HED) initiative operated for 10 years (2005-2015) to create and implement partnerships between researchers from US universities and host country HEI focused on development issues in a country or region. HED resulted in a total of 443 partnerships in 75 countries. In comparison, after three years, HESN Lab activities were taking place in more than 50 countries covered by a USAID Mission and there were 55 collaborations with USAID operating units in development, in operation, or completed, as well as numerous37 collaborations with other types of development actors including developing country HEIs. HED program staff spent the first three years helping Missions become aware of the initiative and supporting them to develop RFPs based on research needs that would support their Country Development Cooperative Strategies (CDCS). As noted above for HEPII, HESN does not currently include a demand-driven strategy to developing partnerships with HESN Labs and USAID operating units. 35 Potentially interested USAID operating unit staff present ideas for collaboration to the HESN program officer assigned to a specific HESN Lab. 36 HEEAP 1 was funded from 2010-2013. HEEAP 2 is funded from 2013-2018. 37 The exact number of non-USAID collaborations is not readily available through HESN M&E data but an estimate will be provided for the final version of this report. 30 HED eventually scaled up to 24 program staff who coordinated proposal reviews, ensured university partners submitted technical reports, performed regional field evaluations, monitored the performance of partnerships, and maintained relationships with US university partners With only 14 program staff members, HESN is producing a comparable number of collaborations between university researchers and development institutions compared to HED.38 Like HESN, HED faced the challenge of year-to-year fluctuations in budget allocations ($1-10 million), cited as a factor impacting the success of HED’s relationships with Missions. In order to better support USAID Missions, HED made an effort to time semi-annual reports produced by university partners with the timing of Missions’ portfolio reviews. This strategy would likely benefit the success of partnerships between HESN Labs and USAID Missions but would add an additional dimension to the role of HESN program managers by requiring them to monitor timelines for all Missions engaged with HESN Labs. Feed the Future Innovation Labs (FTFIL) funds 24 Innovation Labs (formerly Collaborative Research Support Programs, CRSP). Innovation Labs fall under six different research programs and are staffed by eight AORs and eight activity managers to manage the programs and Innovation Lab relationships. FTFIL focuses only on food-related issues and defines the topics for the Innovation Labs, while HESN focuses on a wide array of development issues and does not define the topics for the HESN Labs. Before awarding a new Innovation Lab, the FTFIL lead contacts a number of Missions with the most interest in the topic or that could benefit the most and sends them a questionnaire about the issues, constraints, and needs related to that Innovation Lab’s work. Unlike HESN, the FTFIL program staff pick the set of countries each Innovation Lab will work in. However, both FTFIL and HESN program staff work with Missions to adjust the scope of work proposed by HESN Labs before approving the workplan. Key costs for the FTFIL initiative are sub-awards, research, travel, staff salaries, lab director meetings, and regional meetings. Regional meetings are a new strategy used by FTFIL to provide the opportunity for exchange between the Innovation Labs working in a particular region and USAID Mission staff and other stakeholders. HESN does not currently fund regional stakeholder meetings in developing countries where HESN Labs work but allocates resources for annual HESN Lab Director meetings and sponsors HESN Labs to bring together USAID, HESN Labs, and other development innovation stakeholders at periodic Technical Convenings (TechCons) that are held at one of the HESN lead universities. Allocating funds for regional gatherings held in developing countries and attended by HESN Labs working in that region and regional ecosystem actors would provide a forum for knowledge sharing, discussion of pressing local challenges, and relationship building. The Research Innovation Fellowship (RI Fellows) initiative connects graduate researchers from six US universities to developing country organizations that wish to apply science, technology, and innovation to their work. The RI Fellows program will end in 2017 after three years of operation. The fellowships pay for graduate students who are interested in connecting their scientific experience to international development and staying in country for 2-12 months. The initiative uses an online platform for organizations to communicate their needs (a demand-driven approach) and for students to submit proposals for how their skills could meet those needs. RI Fellows program staff carried out review of applications for alignment with USAID goals and scientific merit in partnership with AAAS fellows throughout USAID. The program grew from 57 projects in the first year to 100 in the second year (the third year was in progress at the time of the interview). In comparison, 386 students have served as fellows in developing countries for more than one month through HESN Labs to date. RI Fellows program staff did planning, outreach, and relationship management with US university partners and host country organizations. Seventy-five percent of fellowship funds came from NSF but were disbursed through the six universities under cooperative agreements. The primary challenges have been ensuring that NSF sufficiently carried out outreach to ensure that NSF graduate research fellows could pursue 38 Based on comparing HED’s figure of 443 partnerships in ten years to HESN’s 55 USAID-HESN Lab collaborations plus at least 77 other collaborations related to HESN objectives. This will be assessed and modified for the final version of this report. 31 this opportunity and matching researchers to developing country organizations and opportunities. Similar to HESN, each of the RI Fellows university partners have different strengths and approaches, but unlike HESN, this has made it more difficult to match graduate researchers to opportunities. Because HESN Labs do that matching themselves, it is not a challenge or time allocation for HESN program managers. Within HESN, RAN uses an online platform for communicating opportunities available for students from other HESN Labs. A consequence of structuring the RI Fellows program around a demand-driven approach was that host organizations that do not get a student quickly enough lose interest. In addition, it has been difficult for RI Fellows to attract graduate researchers without a means for faculty researchers to be involved – unlike HESN Labs, which staff projects with teams of students and HESN Lab-affiliated faculty. Partnerships for Enhanced Engagement in Research (PEER) funds developing country researchers to conduct research that relates to the problems on which USAID Missions are focused. It was launched in 2011 as a way to leverage federal science agencies for international development. PEER provides support for research projects while building in-country research capacity. In comparison, HESN supports developing country research capacity indirectly through the partnerships between HESN Labs and developing country HEI. PEER funds graduate students and research scientists in partnership with a scientist or engineer funded by a US federal science agency. There are five full-time program managers to follow grants within each region, a part time partnership advisor, and staff from National Academies who manage sub-awards. Most of the cost goes to sub-grants, National Academies, and staff salaries. Buy-ins from USAID Missions generate half of PEER’s project budget. HESN Labs routinely partner with researchers from developing country HEI to collaborate on activities related to all three HESN objectives: data and research, innovative technologies and approaches, and opportunities for interdisciplinary learning. Many HESN Lab-affiliated faculty are funded by US federal science agencies and thus are qualified to partner with developing country researchers eligible for PEER funding. PEER asked HESN to help recruit developing country researchers to apply to PEER because many HESN-affiliated researchers are funded by US federal science agencies. However, there is no tracking at this point of the number of developing country researchers involved with HESN Labs who are funded by PEER. To do this, PEER would need to capture the university affiliation of the US science agency-funded researcher and whether or not the US researcher is affiliated with an HESN Lab. HESN Labs would also need to report the number of PEER-funded developing country researchers working with HESN-affiliated faculty. Better monitoring of the developing country researchers involved with HESN Labs would allow PEER and HESN to compare the types of capacity building and other benefits to developing country researchers and HEI that occur within and outside of the HESN Lab structure. Leveraging PEER funding would eliminate the need for HESN Labs to fund eligible developing country researchers. SUMMARY of Evaluation Question 2: What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? During the first three years of the HESN project, HESN Labs have become multidisciplinary structures that carry out a variety of activities in different settings. Some HESN Labs ensure coherence across activities by deeming one objective primary and the rest supporting activities. Allowing HESN Labs to address multiple objectives has led to broader integration into the mission their respective universities, it also allows them to be flexible and respond to different development actors and contexts. However, this multidisciplinary approach also creates an increased administrative burden and a more significant management challenge. EVALUATION QUESTION 3. TO WHAT EXTENT HAS HESN LED TO CHANGES AT HEIS THAT MAY INCREASE THEIR IMPACT ON INTERNATIONAL DEVELOPMENT? 32 a. Have the awards made to the eight HESN Labs had wider effects on the seven HEI leads’ campuses? If so, how? b. To what extent has HESN led to greater capacity of HEIs in developing countries? c. What is the likelihood that each HESN Lab will continue to exist once funding from USAID/HESN ceases? Evaluation Question 3.a.: Have the awards made to the eight HESN Labs had wider effects on the seven HEI leads’ campuses? If so, how? The primary data source for this evaluation question was interviews with 16 campus leaders on each lead HESN campus. The awards to the eight HESN Labs have had significant wider effects on the seven lead HESN campuses. The specific wider campus effects documented through the midterm evaluation include increases in the number of academic and research activities for undergraduate and graduate students related to international development; a greater number of development-focused collaborations across academic disciplines, research institutes, and student service units; increased hiring of faculty with interests in development challenges; and greater awareness and visibility of the role of science and engineering in improving conditions for people living in poverty. According to campus leaders and HESN Lab staff, this has not been an easy process. Departmental divisions are difficult to bridge, new classes take a great deal of time and effort to develop, and institutional procedures can be rigid and hard to change. Progress to date for each type of campus effect is summarized below. Increased number of academic and research activities for undergraduate and graduate students related to international development. In less than four years, HESN funding has led to large increases in new classes, research fellowships, and field-based internships related to international development. HESN award funding led to a total of 132 new classes offered by HESN Lab-affiliated and other faculty, impacting over 2,000 students.39 Given that the majority40 of these classes are an ongoing part of the curriculum at these universities and that HESN Labs continue to develop new classes related to their work, the number of students taking HESN-related courses is expected to continue to rise. HESN Labs also worked with faculty who teach classes in related fields to integrate examples from HESN Lab projects. In addition, HESN-affiliated and other faculty offer field-based classes in conjunction with developing country partners in order to expose students to on-the-ground development innovation work. Ongoing interdisciplinary development-focused classes created through HESN funding that are now offered at HESN lead campuses include:41  Duke University: Social Innovation and Entrepreneurship classes at both graduate and undergraduate levels; Design for the Developing World (focused on design of devices to meet the needs of developing 39 The number of students taking these classes was not tracked through the M&E indicators but some HESN Labs provided notes on class enrollment. Using a conservative figure of 15 students per class (the average for classes with notes on enrollment was 24), at least 2000 students took HESN-focused classes by the end of FY2015. 40 The exact number of ongoing classes was difficult to assess from the available data. 41 Makerere University did not add new individual classes but did develop new programs, listed below. CellScope is a mobile-based device that captures images of blood, sputum, or other patient samples and wirelessly transmit the data to clinical centers, allowing the patient to be evaluated remotely. Photo Credit: DIL. 33 world hospitals); Fundamentals of Global Health (which requires all teams to incorporate learnings from the SEAD Summit into their solutions, and uses a SEAD team member as a judge in the finals).  Michigan State University: Frugal Innovation Practicum (collaborative problem definition and solution/innovation proposition focused on urban 'wet' markets).  MIT: Technology and Development (based on teaching cases written by students funded through CITE to do field research for developing the cases); Technology Evaluation to Reach Scale (co-listed in Urban Studies & Planning and Mechanical Engineering); D-Lab: Design for Scale (students partner with IDIN network members to optimize the design of new technologies).  Texas A&M University: Economics of Foreign Intervention, Conflict and Development; three Capstone42 field-based classes in the School of Government and Public Service: Information and Voter Evaluation of Legislators in a Clientelistic Democracy (Benin), Development Projects in Child Nutrition and Youth Education (Democratic Republic of Congo), Youth Employment in Ghana: Conditions and Determinants (Ghana).  UC Berkeley: Ethics, Methods, and Pragmatics of Global Practice; Design, Evaluate, & Scale Development Technologies (a core course for PhD students with a designated emphasis in Development Engineering); Innovation OnRamp; Development Engineering Research and Practice Seminar; Design for Sustainable Communities; Designing Innovative Public Health Solutions; Poverty, Technology, and Development  William & Mary: Economic Development Policy; Empirical Microeconometrics; Intro to ArcGIS Online (explores how to leverage AidData’s geocoded data using ArcGIS Online and other ESRI tools to effectively illustrate the visual story of aid); Field Experiments for International Development (experimental methodologies and their applications in international development); Policy Oriented Research and Analysis Using Aid and Development Data (use of data products produced by AidData and the applications of quantitative data and analysis to development policy-making); Introduction to Development Mentored Research (focused mentored research project related to AidData’s work) In addition to classes, new student opportunities related to international development on the lead HESN campuses also included field-based practica,43 fellowships,44 and internship45 opportunities. A total of 386 US students have served as fellows in developing countries for more than one month via HESN Labs and 571 students have participated in short term practica or other field experiences through human, financial, or institutional resources contributed by HESN Labs.46 Field-based experiences for students through HESN funding have had a strong impact on students’ career paths. Students interviewed about these experiences described them as critical to setting the direction for their future work. Students who had never thought of an international career prior to doing an internship with HESN Labs have shifted their focus as a result of the experience. Fellowships through HESN Labs are an important mechanism for staffing HESN projects and promoting graduate research on development innovation. HESN funding also led to establishing and strengthening of academic programs focused on international development. Through DIL, UC Berkeley established the Development Engineering (DevEng) Designated Emphasis47 at UC Berkeley. DevEng is a cross-disciplinary program for PhD students in engineering, natural sciences, and quantitative social sciences that allows them to add a focus on development engineering to their research in their home department. DevEng “prepares students to develop, pilot, 42 A capstone course serves as the culminating and usually integrative experience of an educational program. 43 A practicum is a graduate level class, often in a specialized field of study, which is designed to give students supervised practical application of concepts and theories. 44 Fellowships support students to carry out research projects, usually supervised by a faculty member knowledgeable in the research topic. 45 An internship is a field experience not directly tied to an academic program. 46 Based on HESN standard indicators for FY2015-FY2015. 47 A “designated emphasis” is an optional certification in a specialty outside of a doctoral student's home discipline. It is the equivalent of a minor for PhD students. It is sometimes referred to as a “PhD concentration.” 34 and evaluate technological interventions designed to improve human and economic development within complex, low-resource settings.”48 All students must apply and be accepted to DevEng at least one semester before their qualifying examination. At least one faculty member of the Graduate Group in Development Engineering participates in the student’s home department qualifying examination committee.49 IDIN and CITE have contributed to further strengthening and visibility of D-Lab at MIT, a program aimed at building a global network of innovators to design and disseminate technologies that meaningfully improve the lives of people living in poverty, which existed for a number of years prior to HESN. RAN developed a Master of Science Degree in Community Disaster Resilience Studies at Makerere University, currently waiting for approval of the National Council for Tertiary Education and being piloted off campus at the University of Development Studies in Ghana. In addition, RAN’s East Africa Resilience Innovation Lab (EA RILab) worked with Tulane University’s Disaster Resilience Leadership Academy (DRLA) team to develop a RAN Resilience Course50 to be offered both on campus at Makerere University and online. In Year 3, ConDev began the process of developing an interdisciplinary certificate program in Conflict and Development at Texas A&M combining classes across the College of Agriculture and Life Sciences, College of Engineering, and College of Liberal Arts. The multiple levels of institutional review and approval are expected to be completed by the end of Year 5 of the HESN award. Hiring of faculty interested in development challenges. HESN Labs have also created wider campus effects by attracting faculty interested in interdisciplinary research and teaching in international development. William & Mary hired three new tenure-track faculty members who were attracted by the opportunity to be affiliated with AidData. At Texas A&M, a dean interviewed during the site visit stated that ConDev’s work has led to the recruitment of faculty who are interested in conflict and the development of a network of faculty, staff, and students interested in conflict-related issues. One recent hire at Texas A&M, the new dean of the School of Public Health, reported being motivated to take the position because of ConDev’s work in conflict-affected countries and its working relationships with USAID and its Missions, seeing this as an opportunity to strengthen the focus on global health at Texas A&M. At Duke, SEAD’s success at the intersection of global health and entrepreneurship has led to funding for a new faculty position in global health innovation. Interdisciplinary collaboration. Campus leaders at lead HESN institutions credit HESN for increasing connections across disciplines through the partnership structure and range of activities being conducted under the HESN award. Several HESN campuses have had NSF Integrative Graduated Education and Research Traineeship (IGERT) awards and commented that HESN Labs are more supportive of interdisciplinary research because they incentivize cross-department collaboration among faculty as well as graduate students and because they are focused on a specific set of challenges and research opportunities. Campus leaders at larger universities (e.g., Texas A&M, Michigan State University) credit HESN with bringing a stronger international perspective to participating departments. At a smaller university like William & Mary, the HESN award has led to greater attention to and visibility of international development across campus. According to the Chair of the Department of Urban Studies and Planning at MIT, CITE has created a multidisciplinary network of MIT faculty, staff, and students focused on the impact of technology in development, which the Vice Provost of International Relations stated is “promoting critical reflection on the interaction of technologies and the institutions and organizations that take them up.“ At Michigan State University, GCFSI has created collaboration among researchers in plant science, agriculture, and information and communication technologies (ICT) called ICT for Development, which is now available as an undergraduate or graduate specialization. Makerere 48 Retrieved from http://deveng.berkeley.edu/ on March 25, 2016. 49 This requirement creates student-driven outreach to encourage faculty who are not yet members of the Graduate Group in Development Engineering to apply. 50 This is a new program, not an individual university class. 35 University leaders report that the collaboration initiated by RAN between the School of Public Health and the Center for Engineering, Design, Architecture, and Technology has led to a broader definition of innovation that integrates new technologies, best practices, and approaches. At UC Berkeley, the DIL partnership structure created a strong collaboration between the Center for Effective Global Action and the School of Engineering, which did not exist prior to HESN funding. Finally, at Duke University, the HESN award to SEAD brought together Innovations in Healthcare, a nonprofit organization housed within the Global Health Institute at the School of Medicine, and the Center for Advancement of Social Entrepreneurship (CASE) located at the Fuqua School of Business. According to the Director of the Global Health Institute, SEAD has furthered the ability of the university to work in a unified way and continues to build Duke’s capacity for interdisciplinary collaboration. Awareness and visibility of the role of science and engineering in improving conditions for people in the developing world. Broader campus impacts catalyzed by the work of the HESN Labs also include raising awareness and understanding of the role that science and engineering can play in improving conditions for people living in poverty, through campus-wide competitions (e.g., UC Berkeley’s Big Ideas), student membership organizations (e.g., the Conflict and Development Academy at Texas A&M, aimed at promoting understanding of conflict and development, including for students who are cadets on a career path toward becoming US military officers after graduation), conferences (such as DIL’s upcoming State of the Science Conference on the Science of Scaling), and other campus outreach activities (e.g., CITE’s participation in MIT Water Days; ConDev’s and AidData’s Shark Tank activities at Texas A&M and William & Mary). Evaluation Question 3.b.: To what extent has HESN led to greater capacity of HEIs in developing countries? “USAID support of [developing country] government institutions in dollars only is not sustainable. When you support universities, you teach people to fish. You are creating capacity that is far beyond what you ever imagined. The [HESN] intervention is addressing the real issues.” (Campus leader) To date, the eight HESN Labs have developed an extensive network of partnerships with faculty and students at developing country higher education institutions (HEI). A total of 72 developing country higher education institutions and research institutes were working with HESN Labs as of FY2015 (see Annex X for full list).51 As shown in Table 3.1, HESN Labs varied in the number of partnerships conducted with developing country HEI. Some of these partnerships were driven by the design of the HESN Lab (e.g., RAN, IDIN) while others were developed as the work unfolded. Table 3.1 Number of developing country HEI and research institute partners engaged with HESN Labs52 Lab High Engagement Medium Engagement Total AidData 3 1 4 ConDev 4 0 4 CITE 2 3 5 DIL 13 3 16 GCFSI 6 1 7 IDIN 3 10 13 51 HESN Labs levels of engagement for each partner were determined by the HESN Labs. 52 Obtained from Dev Results Partner tables. Definition of level of engagement not defined by USAID; determined by individual HESN Lab. 36 RAN 13 10 23 SEAD 0 0 0 Total 44 28 7253 As a network of 20 African universities in 16 countries, RAN plays a unique role in building the capacity and ecosystem for innovation throughout Africa. Each RAN-affiliated university is connected to one of four regional Resilience Innovation Labs (RILabs). As the lead campus for RAN, Makerere University is both a developing country HEI partner for US-based HESN Labs as well as a capacity-building catalyst for RAN-affiliated and other African universities. RAN is the only HESN Lab to establish a deliberate partnership strategy for strengthening research and innovation capacity of a strategically-located system of developing country HEI. To date, RAN has developed training and engagement opportunities for faculty and students including training in resilience assessments and deliberative polling, travel to regional and international workshops, and participation in knowledge sharing platforms. Faculty are also involved in RAN's Intervention Strategy Workshops to analyze resilience information and generate priority intervention pathways. Although it was outside the scope of the midterm evaluation to assess the quality of these partnerships or the institutional impacts of HESN Lab activities on developing country HEI partners, the evaluation team used data from Dev Results tables, key informant interviews with developing country HESN Lab partners, and HESN Lab annual reports to identify the types of capacity building taking place through these partnerships. The primary pathways through which capacity building has taken place are: strengthening research capability, promoting development innovation through curricula and educational materials, partnership on HESN Lab projects, and connecting to wider research and development networks. Examples of each are provided below. Strengthening research capability.  Four Makerere University students were trained to interview farmers and other actors as part of CITE’s World Food Program project to bring crop storage technologies to scale in Uganda.  GCFSI trained faculty and students at Lilongwe University of Agriculture and Natural Resources (LUANAR) University on how to utilize statistical analysis to incorporate climate model data into their research. As a result, the scholars will be able to provide their own solutions for addressing food system challenges, learn new and innovative processes for facilitating student learning, produce high-quality research, improve LUANAR engagement with the private and public sector, and facilitate organizational change within LUANAR. Promoting development innovation through curricula and educational materials  Kathmandu University hosts summer fellows from AidData, who train students and faculty in GIS so they can use this tool in their teaching as well as research. As a consequence, Kathmandu University introduced a GIS course in its undergraduate curriculum. 53 There were a total of 72 engagements with developing country higher education institutions and research institutes. Five of these institutions had engagements with more than one HESN Lab. Group of students from universities in the US and Uganda that are part of the HESN internship program. Photo Credit: RAN. 37  RAN has recently supported Gulu University to introduce resilience and innovation through training and interactions with students and faculty/staff. Gulu University managed a resilience and innovation challenge. Collaboration and shared learning with HESN researchers and partnership on HESN Lab projects  Through CITE’s partnership with the Indian Institute of Management-Ahmedabad (IIM-A), Prof Sarin and his students collaborated fully as research partners in the Water Test Kit evaluation and the Educational Technology project.  Young researchers at Kathmandu University have had the opportunity to work on AidData projects in Nepal.  DIL’s partnership with Jadavpur University is focused on implementing India-compatible standards for handling arsenic sludge from arsenic remediation, a major issue that has not received adequate research attention, and to collaborate with community members to work out sustainability challenges, which has led to the failure of prior arsenic remediation efforts. Indian researchers have experience working on development problems and understand the context and complexities of the arsenic issues but wanted to collaborate with more advanced researchers on advancing the solutions. DIL researchers go to Jadavpur and work in the field to gain a different exposure to the problem. Jadavpur students spend time in UC Berkeley labs, which has changed how they look at the issues.  Gulu University faculty feel they have gained credibility from working with RAN researchers and have been introduced to the mindset of innovation and the importance of faculty-students interactions. Participation of developing country researchers in broader in-country and international academic and development partner networks  AidData fellows organized national level workshops and discussion on the role, use, and impact of open data portals for the government, NGO, and academic institutions in Nepal. The AidData partnership has supported Kathmandu University to connect with opportunities to work with national and international teams of experts.  A number of spinoffs have come out of Jadavpur University’s collaboration with DIL (e.g., a new project proposal on energy issues). Evaluation Question 3.c.: What is the likelihood that each HESN Lab will continue to exist once funding from USAID/HESN ceases? The primary data source for this evaluation question was interviews with 19 HESN Lab leaders, who have a primary role in the HESN Lab’s future sustainability, and 16 campus leaders from each lead HESN campus, who bring an institutional perspective on the sustainability of the HESN Lab. HESN Labs are actively seeking funding sources outside of USAID and creating sustainability plans. HESN Lab leaders do not believe their work can be maintained in its current form without some level of USAID funding. A primary challenge is the lack of US science funding sources for early stage development innovation research. HESN Lab leaders expressed concern about the sustainability for innovations that will not have “taken off” before USAID funding ends and felt the initial HESN investment would be at risk if USAID did not extend its funding beyond the first five years. HESN Lab leaders also pointed to the need to keep a buy-in mechanism in order to sustain engagement with USAID Missions Without USAID funding, few if any HESN Labs believed they would be able to remain a cohesive Lab or Center, instead retaining particular activities that might be possible to fund in some other way. Factors affecting the kinds of activities that would be sustained following the end of USAID funding include the level of the HESN Lab’s integration into the university structure, whether the HESN Lab existed in some form prior to HESN, and the level of commitment on the part of university leadership. All HESN Labs confirm that they would like to sustain at least some of their activities. 38 Competing needs for funding for HESN Labs include “buying out” faculty members’ time on the project to ensure an ongoing level of dedicated time for carrying out the HESN Lab’s work; funding for graduate students to carrying out core research functions; funding for operating staff functions such as project management, M&E, student engagement, communications, etc.; and travel for field research, community partnerships, and knowledge exchange with developing country partners. Sustainability in the case of innovation requires maintaining a structure to which the university is committed and donors (private or governmental, university or institution) are interested in contributing. Unanimously, HESN Labs agreed that the buy-in capacity of the HESN mechanism is by far the most critical and important component that they hope stays in place long term. During interviews, HESN Lab stakeholders cited a desire to have a more structured and defined process for connecting HESN Labs to Bureaus and Missions to utilize research coming out of the HESN Labs and/or to tailor the research coming out of universities to needs identified on the ground. Missions and the RAN network and approach to innovation design and development could facilitate this context-based and participatory approach to development. SUMMARY of Evaluation Question 3: To what extent has HESN led to changes at HEIs that may increase their impact on international development? The awards to the eight HESN Labs have had significant wider effects on the seven lead HESN campuses. HESN has allowed for an increased focus on international development within the respective HEIs, through classes, academic and research activities as well as field-based opportunities. The interdisciplinary structures created by HESN Labs have elevated the quantity and quality of research and innovation related to international development beyond models that previously existed, such as NSF IGERT. In developing country HEIs, HESN has led to an increased capacity for research and innovation, greater collaboration between US universities and developing country HEI, and wider engagement with in-country development actors and institutions. HESN Labs are actively seeking funding sources outside of USAID and creating sustainability plans. HESN Lab leaders do not believe their work can be maintained in its current form without some level of USAID funding. Without USAID funding, few if any HESN Labs believed they would be able to remain in their current form, and would instead retain a more narrow set of fundable activities. EVALUATION QUESTION 4. TO WHAT EXTENT HAS HESN INFLUENCED OR ASSISTED USAID OPERATING UNITS OTHER THAN THE U.S. GLOBAL DEVELOPMENT LAB? a. To what extent have HESN Labs affected operating units’ decision-making or other operations through data-driven methodologies, tools, or analytics? b. To what extent have USAID operating units leveraged the HESN Labs to accelerate their creation, testing, and scaling up of transformative innovations? c. To what extent have HESN Labs and USAID operating units engaged in collaborative problem-solving, knowledge sharing, and learning? The primary data sources for addressing this evaluation question were interviews with 12 USAID operating unit staff and interviews with 10 USAID HESN program staff. HESN Labs and USAID HESN staff have worked to establish and develop relationships between HESN Labs and USAID operating units. The interviews documented that partnerships with USAID Missions were not an initial focus of HESN. Once it became a priority for HESN Labs to influence decision making and provide other kinds of assistance to Missions, USAID HESN program staff played a critical role in establishing relationships with Missions, helping them become aware of the work of the eight HESN Labs, and seeking out opportunities for the HESN Labs and Mission staff to become familiar with one another through visits, 39 providing feedback, meeting at regional events, etc. This high level of “customer service” to Missions by USAID HESN staff was informed by a deliberate relationship-building strategy:54  Co-creation and Collaboration: HESN Labs and USAID Missions co-design activities to meet shared objectives; Missions buy-in to HESN award; HESN Labs contribute to Mission activities through activities, research, analysis, and/or participation in TDYs; USAID Activity Managers work closely with HESN Labs on activity design and provide technical input; USAID staff participate in HESN Working Groups.  Exchanges: HESN Labs host development practitioners in residence at their institutions; USAID hosts summer interns from HESN Labs.  Learning: HESN Labs review country CDCS prior to work in a country; HESN Labs provide pre- and post-briefings to Missions for in-country activities, through in-person meetings or conference calls; HESN Labs present work at brownbag presentations; Mission and Bureau staff participate in TechCon; HESN Labs share bi-annual reports and other deliverables with Missions and Bureaus.  Administration: HESN Lab activities in country presented to Missions for concurrence; Missions and Bureaus invited to review HESN Lab workplans, sub-awards, and other deliverables; Missions informed of any HESN travel in country. As a result of this strategy, hundreds of USAID stakeholders in Washington and in the field have provided input into the annual work plans developed by HESN Labs, helping to ensure that HESN activities are aligned with, and supportive of, USAID's objectives. As of December 2015, HESN Lab activities were taking place in more than 50 countries covered by USAID Missions and there were 55 collaborations in development, in operation, or completed between the eight HESN Labs and 34 USAID operating units (six Bureaus and 28 Missions) of the following types55:  HESN provides input to USAID activity (i.e. analysis, evaluation)  USAID provides input to HESN activity (activity development; partnership needs)  Buy-In by USAID operating unit to HESN Lab  Co-Designed Activity  Other (USAID provides periodic direction; HESN Lab provides updates on activities in country) Despite the challenges of establishing and maintaining these working relationships, key informants within USAID operating unit staff made it apparent that there is considerable enthusiasm for the work of HESN Labs. USAID staff who have worked with HESN Labs believe working with the HESN Labs is helping operating units advance their thinking and build up the base of research used for decision making. Despite occasional problems (e.g., late reports, travel visas, inexperienced graduate student researchers), all of the USAID staff who were interviewed56 saw significant value in the contributions of the HESN Labs and believe there is a great deal of future potential in continuing these partnerships. As one Mission advisor said: 54 Summary of Agency and Bureau Engagement (see Annex XII: Bibliography) 55 The HESN Obligation Tracker detailed buy-ins to date, while the April 2015 Agency Engagement spreadsheet described planned, in progress, and completed engagements with HESN Labs. 56 A total of 15 USAID Mission and operating unit staff were interviewed. 40 “Everyone at the Mission wants to continue to build relationships with the universities. They're forward thinking, bringing in new insights and keeping us from getting stuck in the approaches. They will help us make those mental shifts that we need as the field evolves. When they make a delivery of a product, we're getting educated along with having our questions answered.” This quote was not an exception but instead summarizes the benefits that other USAID Mission staff cited in interviews about their engagement with HESN Labs. In particular, the opportunity for direct engagement with high-caliber faculty and PhD researchers, exposure to leading edge research methods, and extensive technical knowledge were cited as the motivations for Mission staff to seek partnerships with HESN Labs. Factors cited by USAID interviewees that support successful collaboration between Missions and HESN Labs include: opportunities for Mission and HESN Lab researchers to meet and become familiar with each other’s work; HESN Labs’ familiarity with the country strategies (CDCSs57); the opportunity for further conversations about the potential gains from working with HESN Labs on specific development challenges; the timing of the HESN Labs work relative to the Mission’s planning cycle; and a commitment by the Mission director to evidence-based decision making. For their part, HESN Lab leader are positive about how HESN has engaged universities in a collegial, rather than authoritarian, manner. Contributions of HESN Labs to USAID operating units were particularly strong in the area of data￾driven methodologies, tools, or analytics. As of the end of FY2015, twelve USAID operating units were using geographic analysis to prepare strategies and design, implement, monitor, and evaluate development projects and ten evaluation projects aimed at generating analytic data for key decisions by USAID operating units were in progress or under development. Highlights of HESN Lab projects aimed at improving programs, operations, and decision making through data-driven methodologies include: o DIL: DIL’s Rural Electric Power Project presented its data and analyses on the demand for and impacts of on-grid electrification to USAID’s Financial Inclusion for Rural Microenterprises team. o ConDev: The Africa Bureau's education office requested a research analysis of educational outcomes related to school violence in three Sub-Saharan African countries. ConDev cooperatively developed the research methodology and worked closely with Africa/SD/ED to produce an analysis. The Africa Bureau team was highly supportive of ConDev's approach and impressed with the level of engagement. o AidData; A summer GIS intern expanded the Uganda Mission’s GIS Specialist capability to respond to data and visualization needs of the Mission as well as its implementing partners. USAID/Indonesia hosted an AidData Summer Fellow in Summer 2015 to help the Mission with its GIS porfolio. o ConDev: USAID/DRC asked ConDev to provide results from its Best Practices in Coffee and Cocoa study to inform their new Feed the Future country strategy and a paper on the DRC Cocoa Sector being written by a Mission staff member. o DIL: The USAID Millennium Water Alliance contracted with DIL for the Mezuri team to apply their expertise in sharing and analysis of data from field interventions to water provisioning programs in northern Kenya. 57 Country Development Cooperation Strategy AidData event. Photo Credit: AidData. 41 Upcoming projects include: o AidData: Impact evaluations for the Colombia, Georgia, Niger, and Ghana Missions o ConDev: ConDev is partnering with USAID Afghanistan to develop options for strategic grain reserves that reduce the likelihood of conflict. The Afghanistan Mission will engage ConDev to provide expert advice and consulting to help the Mission in its discussions with the government about grain storage. o ConDev: The DRC Mission is planning to align more with the work ConDev is doing in the eastern part of the country because of the approach to building evidence and research capacity that ConDev is taking in its work with local institutions there. The Mission believes it will benefit from drawing on a stronger basis of evidence to guide its work and sees the approach ConDev is taking to build local research capacity as critical to the success of peace-building and economy-strengthening efforts. o ConDev: The DRC provided a buy-in for ConDev to partner with the Texas A&M School of Public Health to conduct an evaluation of the Kinshasa School of Public Health. The extent to which USAID operating units leveraged the HESN Labs to accelerate their creation, testing, and scaling up of transformative innovations has been slow to develop but is now beginning to grow. HESN Labs produced 200 white papers, articles, assessments, analyses, and evaluations on development challenges, innovations, technologies, approaches, and contexts, an unknown number of which were provided to USAID operating units.58 Examples of projects in this area include:  CITE conducted an evaluation of post-harvest food storage technologies in Uganda, in partnership with the USAID Mission and the World Food Program.  CITE and USAID’s All Children Reading (ACR) Grand Challenge team are collaborating on an evaluation of educational technology. CITE is developing a framework to evaluate educational technologies and will apply it to awardees of the ACR "Enabling Writers" prize. E3/Education and World Vision (an ACR partner) are helping to shape the evaluation and may seek a buy-in to collaborate further.  DIL is successfully leveraging additional resources for the M-Pasandaaz pilot project that is part of its ICT-A portfolio curated by UC San Diego and the Policy Design and Evaluation Lab (PDEL). The project is studying the extent to which a mobile phone-based defined contribution savings account can improve the financial capabilities and welfare outcomes of salaried employees at Roshan, Afghanistan’s leading mobile communication provider.  GCFSI collaborates closely with the Lab and BFS on the climate resilient maize problem set. Upcoming:  A significant example of high-impact partnership between a USAID operating unit and an HESN Lab is the research CITE is doing for Food for Peace to inform their decision to implement a new technology for food storage. The manufacturer of the new technology (a food storage bag) had only tested the technology under warehouse conditions. Food for Peace lacked the research capacity to test the technology at a global scale, across the supply chain conditions required to move large quantities of food around the world over long periods of time under challenging climactic and travel conditions. CITE’s research will allow the program director to document the decision to use the new technology, which will have a significant impact on the need for fumigating food in transit. Finally, HESN Labs and USAID operating units have also begun to engage in collaborative problem￾solving, knowledge sharing, and learning through exchanges, competitions, and other forms of 58 There are no specific indicators for dissemination of innovations to USAID operating units. 42 learning-focused engagement. A total of 22 interns from HESN Labs were placed in USAID Bureaus and Missions.59 Two USAID Mission staff have served in "residencies" at UC Berkeley, providing insight and technical input into the Development Impact Lab. A total of 40 USAID operating unit and Mission staff participated in the first two TechCon gatherings. Highlights of knowledge sharing and learning exchanges between HESN Labs and USAID operation units are:  Mobiles for Reading was one of the categories for Big Ideas@Berkeley. The Berkeley team worked in close collaboration with the USAID Senior Education Technology Specialist for the creation of the category.  SEAD program and the USAID East Africa Regional Mission have conducted a two-year collaboration. SEAD works with stakeholders in the region to develop a more cohesive and supportive ecosystem to support global health innovation. This new program in East Africa prioritizes women and girls, both through support of female entrepreneurs and also through support of innovations that improve the lives and health of girls and women. Colleagues from the East Africa Regional Mission and/or Kenya Mission participate in semi-monthly planning calls with the HESN/SEAD Award Management team and also review and comment on relevant materials. They approve the SEAD East Africa Regional workplan.  The Bureau of Global Health has used SEAD’s research findings on innovation scaling to inform the USAID Saving Lives at Birth Grand Challenge. SEAD is unique in documenting lessons learned from its global health innovation accelerator program, which provides an important long-term impact on the field of global health innovation. SUMMARY of Evaluation Question 4: To what extent has HESN influenced or assisted USAID operating units other than the U.S. Global Development Lab? Despite not being an original goal of the project, HESN Labs have begun providing data to inform USAID operating unit decision making, collaborating to develop and test new technologies and innovative approaches, and engage in knowledge sharing and learning. As a result, HESN Labs have been working successfully with USAID on more than 55 assignments and activities in the field (including buy-ins by USAID operating units to HESN Labs). USAID HESN program staff played a critical role in establishing relationships with Missions, HESN Labs have influenced USAID operating units positively, especially in the area of data and analytics and through exchanges and learning-focused engagements. EVALUATION QUESTION 5. HOW CAN HESN MODIFY ITS STRATEGY AND STRUCTURE TO IMPROVE ITS EFFICIENCY AND EFFECTIVENESS? a. Given the findings on HESN’s success (Question 1), are there particular areas to which the HESN Project should devote more or less attention to? b. What characteristics of HESN Labs appear to have been most critical in achieving HESN’s objectives? c. What changes to the current program and the design of a follow on program may be made to better align HESN with current understandings of the success factors of university innovation ecosystems? Evaluation Question 5.a: Given the findings on HESN’s success (Question 1), are there particular areas to which the HESN Project should devote more or less attention to? The data sources used to address this evaluation question were interviews with HESN Lab team members and leaders and interviews with outside higher education and innovation experts. The analysis of the interview results point to four areas to which the HESN program should devote more attention: support for early stage innovation, ensuring consistent multi-year funding for graduate student researchers, deliberate capacity building of developing country researchers (both HEI-based and 59 2014: HESN placed 15 summer interns in BFS, GH, E3, E&E, and USAID/RDMA (through virtual arrangement). 2015: 7 interns were placed in BFS, E3, GH, and ASIA. 43 development institution-based), and advancing the evidence base for HESN Lab approaches. Existing efforts in all of these areas should be shared across the network for other HESN Labs to learn from and adapt. Support for early stage innovation. Innovation practices that are working well to accelerate the creation and development of innovations and approaches include innovation design, lean thinking, design thinking, human-centered design, co-creation, crowd-sourcing, rapid prototyping, small-scale pilots, and community participation. The evaluation data show that these approaches have been received well by partners, researchers, and other stakeholders working with HESN Labs, and that important capacity and results are being generated through these interactions. HESN Lab teams reported that there is no other source of funding for early stage field investigation and work with communities and practitioners close to the problems. Support for open-ended research (e.g. exploratory work to determine “unknown unknowns”) has allowed HESN Labs to identify promising new solutions that can then be prototyped and further developed. Ensuring consistent multi-year funding for graduate student researchers. Ph.D. researchers are critical to continuity and capacity building within HESN Lab projects, as discussed under Evaluation Question 2a. However, year-to-year funding uncertainties prevent HESN Labs from being able to secure multi-year commitments for doctorate students. This would ensure that HESN Labs are able to attract top Ph.D. students who are interested in developing their research and careers in the field of development innovation and that these students providing continuity in relationships with in country partners, support capacity building needs for developing, implementing and sustaining innovation, and maintain the momentum of HESN Lab projects. Deliberate strategy for building capacity of developing country researchers (both HEI-based and development organization-based). All of the HESN Labs have engaged with researchers from developing country HEI and several (ConDev, CITE, DIL, AidData) have conducted research capacity building for staff from development organizations partnering with HESN Labs to carry out data collection and other research￾related activities. The evaluation team heard accounts of successful capacity building and the enthusiasm US university researchers have for doing this work. However, HESN lacks a deliberate strategy for catalyzing and documenting local or regional research ecosystems. This should be linked to the PEER initiative’s funding for researchers from developing country HEI.60 Jadavpur University’s partnership with DIL on the long-term sustainability of the arsenic remediation technology is just one example of how local research capacity is critical for the success of many types of development innovation. Advancing the evidence base for HESN Lab approaches. More research is needed on the approaches that the HESN Labs are using to impact international development through research and innovation. The eight HESN Labs each have a different “theory of change” (i.e., set of hypotheses) about how their work will lead to improved policy, more effective programs, uptake in innovation, and alleviation of problems for people living in poverty. Each Lab is thus a learning laboratory with the opportunity and need to build an evidence base for how its innovative approach to development innovation leads to impact. Although several Labs (e.g., IDIN, SEAD, ConDev) have allocated staff and resources to study their approaches, this has not been prioritized by the HESN program and therefore is at risk of getting squeezed out. Given that an evidence base for how to effectively support and propagate development innovation of many different kinds is lacking, HESN represents an important knowledge sharing and field building opportunity. USAID and university-based innovation experts interviewed for the evaluation stressed the importance of the learning that comes out of studying how to support innovation, gain uptake, build capacity, etc. as equally or more important in the long run than the solutions themselves. Rather than grow the number of activities further for each Lab, there should be more attention to support for the HESN Labs to study their activities and approaches and to share their learning with 60 See the Recommendations section for a specific description of the linkage proposed between HESN and PEER. 44 HESN and the field of innovation and international development more broadly. One way of doing this would be through an HESN community of practice. The three areas identified by the midterm evaluation that should be de-emphasized were: a narrow definition of scaling, maximizing the number of objectives and activities each Lab engages in, and exclusively performance-focused M&E. Address business model issues earlier in the innovation process and expand the definition of scaling. Experts on innovation agree that it is difficult to predict which innovations will be able to reach broad scalability and that propagation of smaller scale solutions is equally or more supportive of lasting impact. Furthermore, university researchers and development institutions do not have the capacity and skill set for selecting and taking innovations to scale. HESN Lab beneficiaries and outside innovation experts believe HESN Labs should introduce business model issues earlier in the innovation process. Some HESN Labs have conducted research or developed a technology that has led to a policy decision to implement a solution at a broad scale, such as DIL’s research in partnership with the Kenya Power and Lighting Company that led to the government’s decision to extend the electrical grid to 600,000 low-income customers. However, there are also other pathways to and levels of scale that are emerging from HESN, which are equally important to document and disseminate. These include IDIN’s community-level scaling and CITE’s work to inform scaling during the design of innovations. Streamline the number of objectives and activities each Lab engages in. HESN Labs have gained significant experience in trying to deliver on the range of value propositions and objectives outlined by HESN and in the HESN Business Model Canvases. At this point, it would be valuable for HESN Labs to develop logic models showing interrelationships among objectives, value propositions, activities, and outcomes as a means of analyzing what they have learned about synergies and the costs of maintaining breadth. The objective would be streamline activities, adjust resource allocations, and increase synergies. Shifting the focus of M&E toward utilization and support of learning. The current focus of the M&E requirements is to assess performance but the number of indicators is large and the data systems do not support real time synthesis and sharing, even for routine performance assessment. As HESN matures, M&E should support meaningful utilization of the data for specific purposes, such as adaptation of activities, cross-Lab comparisons of results for similar stakeholder groups, or other real time purposes. Each Lab should use a focused set of indicators and connect those to its learning agenda. At minimum, data quality should be monitored and areas in which targets are challenging or counter-productive to set should be flagged and explored. Evaluation Question 5.b.: What characteristics of HESN Labs appear to have been most critical in achieving HESN’s objectives? Through interviews with HESN Lab leaders, team members, partners and beneficiaries, the following factors appear to have been the most critical in achieving HESN’s objectives to date: Faculty interest and excitement in working on development challenges and on interdisciplinary teams. HESN has attracted science and engineering faculty who have an interest in applying their work to the public domain, i.e., developing innovations that are commercially viable but are aimed at benefitting people living in poverty. This is a critical resource in the success of Objectives 1 and 2 as well as cultivating students and junior researchers to build their careers related to international development challenges. Lab leaders and faculty with knowledge and experience of USAID and developing country ecosystems. Five out of eight HESN Labs cited the importance of leaders and team members with USAID experience. This translates into concrete benefits, such as contacts in Washington and Missions and familiarity with operating procedures and planning cycles. It has also has had intangible benefits guiding the style of engagement, understanding of capacity building needs, and adapting to the culture of communication and interaction in USAID contexts. 45 Commitment to capacity building in the field. All three objectives rely on ongoing capacity building in organizations and communities. HESN Labs recognize the importance of capacity building to ensure the success of research and innovation activities but also see it as a key factor in long-term sustainability. Strong engagement from key university champions. All of the HESN Labs cited the importance of high-level support and engagement by deans, provosts, department chairs (particularly of faculty involved in interdisciplinary collaborations), vice presidents of research and other campus champions. This is particularly important for interdisciplinary research collaborations, which are often viewed as time￾consuming and problematic for compliance with university operating procedures. Ability to grow the core lab management team and adapt roles in response to the development of the work. Every HESN Lab team has undergone significant changes since the beginning of the HESN award. Team member roles that appear to be the most critical are strong project management, monitoring and evaluation (both internal and external for USAID), network outreach and engagement (both on and off campus), financial management, and communications. Relationship building emphasis of USAID HESN program staff. HESN Labs that have developed supportive, respectful, and collegial relationships with HESN program staff feel that their work is better understood and championed within USAID. The importance of intellectual exchange with HESN Labs was cited by 75 percent of USAID operating unit staff as critical for the success of innovation in international development work. Mechanism allowing Missions or Bureaus to easily enter into working relationships with HESN Labs. The buy-in structure has allowed for the research and innovations being worked on by the HESN Labs to be tied concretely to a problem or challenge the Mission is facing in a particular country. These working relationships have helped HESN Labs and Missions get to know one another and define additional opportunities for mutually beneficial partnership. Evaluation Question 5.c.: What changes to the current program and the design of a follow on program may be made to better align HESN with current understandings of the success factors of university innovation ecosystems? Changes to the current HESN program and to the design of a follow-on program that would better align with factors supporting the success of university innovation ecosystems fall into four areas: partnerships with development actors, operational improvements, enhancement of university environments, and leverage of existing network potential. Partnerships with Development Actors Emphasize learning about and sharing effective models of capacity building and engagement with local and regional systems of actors. Interviewees see the resources that university partners bring in terms of student and faculty researchers and cutting edge tools and creative thinking as critical to bring to local and regional systems of development actors, and that these collaborative efforts are key to building impactful capacity that can be sustained over time. Creating a better understanding of how these Second Annual GIS Hackathon in Partnership with AidData. Photo Credit: RAN. 46 collaborations work and how HESN projects convene and catalyze them is an important area for HESN to focus on in the next stage. Promising examples are seen in ConDev’s work in the DRC and AidData’s work in its focal countries. Create a mechanism for Missions to identify challenges and needs for research/data. HESN Labs feel they would benefit from knowing more about local and regional challenges from the perspective of USAID Missions and operating units so that HESN does not function exclusively as a “supply side” initiative. Although HESN Labs appreciate the relationship building approach that the USAID HESN program staff have emphasized and Missions highlight the value of interacting directly with university faculty and graduate student researchers, both sides believe there would be a gain from having a mechanism by which operating unit challenges and needs could be shared with the HESN community. This is corroborated by recommendations from innovation experts both outside (e.g., iCorps) and inside USAID (e.g., Feed the Future Innovation Labs). Create a stronger emphasis on the need for long-term HESN Lab partnerships with developing country higher education institutions. Each Lab should partner with one or more developing country HEIs to access in￾country researchers, communities, and innovators who can identify critical challenges and contextual needs. DIL’s partnership with Jadavpur University researchers to study the implementation and sustainability of arsenic remediation is an important example, as are RAN’s partnerships with a network of universities affiliated with each RI Lab. ConDev also establishes long-term partnerships with HEIs and builds capacity for research in-country to increase the evidence available to developmental actors and USAID Missions. GCFSI partners with LUANAR to build research capacity for climate change research. Increase awareness of HESN Labs as key actors in international development, not only sources of innovations to be used by USAID Missions. Regardless of their level of familiarity with HESN, interviewees emphasized the critical role that HESN Labs have as partners to USAID in promoting transformative solutions and the models of engagement needed for advancing international development. Although USAID Missions are key actors and important partners, HESN Labs should not be limited to producing outputs that have immediate uptake by Missions and other USAID operating units. In addition to developing tangible solutions, HESN Labs bring a critical role in building up research on development challenges, supporting capacity for innovation and research, and engaging civil society, government, and local institution actors in those efforts. Operational improvements Continue to develop mutual understanding of USAID and university systems. Both USAID and HESN lead campuses have learned a great deal about logistical and cultural differences in each other’s organizational systems. HESN Labs appreciate the efforts by USAID to adapt its requirements for university partners, and the HESN Labs have built up greater capacity for complying with USAID regulations. This mutual adaptation is an important part of development ecosystem evolution and should continue in the next iteration of HESN, especially if a larger number of university partners are involved. Streamline the M&E system. HESN Lab management teams unanimously agree that the M&E system as it currently stands is burdensome and does not provide enough value-added to internal management of IDIN’s local Zambia chapter visits the IDIN￾supported Kafue Innovation Center, where “technology meets community." Photo Credit: IDIN. 47 the project or to tracking of progress towards meaningful objectives. HESN Labs also are not aware of how USAID uses the information internally. The evaluation team suggests that HESN Labs should provide USAID with information about the indicators that are most relevant to their work, and USAID should analyze the extent to which individual indicators are comparable across HESN Labs. In addition, best practices in compiling and managing M&E datasets should be implemented to allow the investment by HESN Labs in collecting and reporting on M&E indicators to feed into semi-annual dashboards and be more feasible for evaluators to use. Enhancement of university environments to support development innovation Support expansion of curriculum models to draw more US and developing country students into the field. Several interviewees described the benefit for the development field from expanding the interdisciplinary, global, problem-oriented curriculum model that has been built up through HESN. Through drawing more people into the field, there will be a needle-in-a-haystack effect. The expansion of HESN-developed curricula creates a set of rewards (new degree programs, internship opportunities, fellowships) for students who want to be involved in academic research. Higher education plays a role in producing cultural norms and interest – for example, engineers who get involved in development engineering could be doing something else but instead are working on solutions to development challenges with social scientists. Support institutionalization of interdisciplinary degree programs and peer-reviewed journals. DevEng is an important example of creating a home and raising awareness of international development challenges as worthy of rigorous academic research and training. An interdisciplinary program structure like this also gives faculty a concrete basis on which to allocate time and publications outside of their home department, which is critical to ensuring their ongoing involvement in development innovation. Although developing programs and journals takes a significant amount of time and attention away from routine faculty responsibilities and HESN activities, the impacts are far-reaching. Leverage HESN’s existing network potential Increase and centralize the dissemination of scholarly articles and policy briefs coming out of HESN Labs. Inside USAID, awareness of the solutions and knowledge being generated by HESN Labs is low, while outside USAID, it is unknown even by stakeholders engaged in international science and technology innovation and research. All interviewees expressed a high level of interest in knowing more about the specific projects and interdisciplinary research efforts carried out through HESN Labs and developing country partners. Researchers doing related work expressed the desire to learn from and extend the research being done by HESN Labs. Strengthen HESN’s network potential. All HESN Labs cited disappointment during interviews in the level of collaboration among HESN Labs. Aside from TechCon and the Lab Directors’ Convening, no funding was allocated for Lab collaboration. Each Lab had varying degrees of collaboration with other HESN Labs, from relationships that are mutually beneficial, one-sided, or ad hoc. The HESN Labs that were more consistently collaborating with other HESN Labs often cited it as a supporting factor in implementing the award. SUMMARY of Evaluation Question 5: How can HESN modify its strategy and structure to improve its efficiency and effectiveness? The primary ways in which HESN should modify its strategy and structure to improve efficiency and effectiveness are supporting early stage innovations, ensuring multi-year funding for graduate student researchers, building the capacity of developing country researchers, and advancing the evidence base for HESN Lab approaches. HESN should put less emphasis on aiming for widely scaled innovations, maximizing the number of objectives and activities each Lab engages in, and exclusively performance￾focused M&E. Four main factors have been the most critical for achieving HESN’s objectives to date: enthusiasm for development challenges and interdisciplinary collaboration; HESN Lab staff with USAID 48 experience; strong engagement from key university champions; and commitment to capacity building in the field. In future programming, HESN should emphasize effective models of local capacity building and engagement, creating a mechanism for USAID Missions to identify challenges and needs for research and innovation that HESN Labs can fill, partnerships between HESN Labs and developing countries higher education institutions, and the role of HESN Labs as key actors in international development. The management of the project should also continue developing relationships between USAID and the university system and streamline the M&E system for easier and more targeted reporting of indicators. EVALUATION QUESTION 6. WHAT, IF ANY, UNANTICIPATED POSITIVE AND NEGATIVE CONSEQUENCES HAVE OCCURRED AS A RESULT OF THE HESN PROJECT? The primary data sources used to identify unanticipated positive and negative consequences were interviews with HESN Lab team members and leaders (87), HESN Lab beneficiaries (68), and HESN Lab partners (27). Positive Consequences Additional funding. Unanticipated funding from sources other than HESN award funding and initial cost sharing has grown quickly in the past two years as the work of the HESN Labs has gained visibility and momentum. During site visits to the HESN Labs, the evaluation team took note when staff and team members mentioned unanticipated additional funding from a variety of governmental and non￾governmental sources that had been attracted because of HESN-funded activity. Unanticipated funding commitments indicate that the expertise and services provided by HESN Labs are seen as needed and valuable in the development ecosystem, and that organizations are willing to invest their funding to access the HESN Labs. One source of additional funding has come from buy-ins to HESN Labs from USAID operating units. While the HESN program was not initially designed to accept buy-ins, award ceilings for each HESN Lab were raised in 2014, from between $2,000,000 to $9,000,000 per Lab, to allow USAID Missions and Washington Operating Units to support specific projects performed by the HESN Labs. Since this time, 13 buy-ins to HESN Labs totaling $9,105,232, have been processed to support activities such as impact evaluations and focused research projects. There are also a variety of additional buy-ins in various stages of development. To date, five of the eight HESN Labs have received buy-ins. In addition to USAID buy-ins, HESN Labs have attracted additional funding from competitions, other funders, developing country government agencies, and other sources. In order to assess the amount of unanticipated funding in light of HESN award funding, Table 6.1 provides a summary of award funding,61 buy-ins from USAID operating units as of February 2016, buy-ins under negotiation reported by HESN Labs as of February 2016, and estimated additional funding62 from other sources as reported by HESN Labs. Based on current estimates, HESN Labs have collectively generated over $18 million of unanticipated funding, which represents 20 percent of total HESN award funding. 61 HESN Obligation Tracker was used as a source of HESN award and buy-in funding 62 Data self-reported by HESN Labs and do not include cost-share commitments in the signed cooperative agreements. Some amounts are estimates based on commitments that are not yet fully disbursed. 49 Table 6.1 Summary of HESN Lab Award and Unanticipated Funding HESN Lab HESN award funding through FY 201663 Signed Buy-ins Awarded to date64 Buy-ins in negotiation as reported by HESN Labs65 Estimated additional funding self￾reported by HESN Labs66 Total AidData $16,768,998 $1,653,385 $865,136 $067 $19,287,519 CITE $6,548,776 $3,500,000 $236,889 $0 $10,285,665 ConDev $3,886,213 $427,765 $0 $068 $4,313,978 DIL, SDS $19,020,370 $1,395,454 $0 $2,000,00069 $22,415,824 GCFSI $13,879,518 $0 $0 $2,000,000 $15,879,518 IDIN $9,093,246 $0 $0 $3,305,300 $12,398,546 RAN $18,124,278 $0 $0 $836,967 $18,961,245 SEAD $6,105,563 $2,128,628 $0 $330,000 $8,564,191 Total $93,426,962 $9,105,232 $1,102,025 $8,472,267 $112,106,486 Summary of award and additional funding $93,426,962 $18,679,524 20% Levels of interdisciplinary collaboration. Although interdisciplinary collaboration was an explicit goal of HESN, six out of the eight HESN Labs reported that the extent of interdisciplinary collaboration that has occurred to date was higher than anticipated. Several HESN Labs cited funding from USAID as a key factor in moving through the challenges of doing cross-disciplinary work. Specific examples of this include DIL’s establishment of the Development Engineering graduate minor at UC Berkeley as well as the DevEng journal, CITE’s collaboration between Urban Studies & Planning, Mechanical Engineering, and two research centers at MIT, and GCFSI’s ICT for Development collaboration at Michigan State. At the same time, the difficulty of interdisciplinary collaboration was greater than any of the HESN Labs expected. As one HESN Lab staff member said “we didn’t understand how challenging it would be to bring different players and disciplines to the table and the enormous investment of time that would be involved in doing that.” 63 These values represent complete funding through FY 2016, including funding that is committed to the HESN Lab but not yet transferred for use. 64 Signed buy-ins includes only fully executed award modifications, and values represent the full award term. 65 Buy-ins in negotiation includes buy-ins in various stages of discussion (as of February 2016) and financial values may be estimated. 66 Data self-reported by HESN Labs (as of February 2016) and do not include cost-share commitments in the signed cooperative agreements. Some amounts are estimates based on commitments that are not yet fully disbursed. 67 Does not include $621,000 additional funding received from the US Dept. of Defense Air Force Office of Scientific Research – Minerva Initiative for advancing AidData’s objectives under HESN. 68 Does not include $1,270,483 pending through an agreement with FHI360 for work by ConDev. 69 Does not include $596,000 additional US federal funding committed to DIL projects. 50 Women in Ranoda, a village outside Ahmedabad, India, demonstrate how to use the Orlab water quality test kit. Testing water is one of the responsibilities as members of the Community Water Committee. Photo credit: CITE. Capacity for working with USAID. All of the HESN Labs cited an increased understanding of USAID, its regulations, and how to work with them as a positive unexpected outcome. Although five HESN Labs reported having staff who understand USAID as a major factor supporting success, most HESN-affiliated researchers were not familiar with the practical reality of development funding and institutions in general, and USAID in particular. In addition, HESN Labs that partner with small developing country organizations (e.g., IDIN) have educated those partners about how to comply with USAID reporting requirements, thus increasing their capacity for working within larger development institutions. IDIN’s finance staff have also advocated within MIT to build the university’s institutional impact for working globally with smaller entities (e.g., wiring funds to unbanked grantees). Working with other HESN Labs and their students. Five of eight HESN Labs named the opportunity to work with other HESN members as a key unanticipated outcome. In particular, RAN did not expect to have such a large number of HESN partners requesting to collaborate with them. The HESN Data Working Group had monthly conference calls, providing information on generating data, and created a strategy for aligning open data policies with the realities of university research. Students from HESN-affiliated universities participated in other HESN Labs’ competitions, such as Berkeley’s Big Ideas, and grant programs, such as ConDev’s Student Media Grant Program. Students affiliated with HESN Labs were particularly enthusiastic about the opportunities for interaction with like-minded students from other universities who were working across disciplinary boundaries on development challenges. Through TechCon and Lab Directors’ Convening, HESN Lab staff sparked ideas for collaboration and exchange through attending each other’s workshops and visiting each other’s projects in the same country. Influence on policy and practice. Several HESN Labs related examples of having unanticipated influence on policy or practice. RAN reported unexpectedly affecting policy change at the district level in resilience and disaster preparedness. Through Deliberative Polling, the Government of Uganda realized the importance of a bottom-up approach to policy formulation. Following RAN’s development of a resilience course for its disaster preparedness officers, UNDP unexpectedly asked them to train all district members. SEAD convened a wide range of investors as a support for its cohort global health entrepreneurs, but this convening has led to a better understanding of global health among impact investors and most important, an effort to create new solutions within the ecosystem of funding for global health innovation that address current gaps. Level of student interest and enthusiasm for research tied to international development. All of the HESN Labs mentioned that the level of student interest in opportunities for research tied to innovation in developing countries was larger than they expected. Approximately 80% of the HESN-affiliated students interviewed as part of the evaluation said they did not expect to find opportunities for working directly on international development problems in the context of their graduate program; all of them reported that the experience of working with an HESN Lab shaped their career direction and opportunities. Planned activities leading to new unanticipated activities. The evaluation repeatedly surfaced examples of new possibilities emerging from planned activities. For example, students doing theses related to IDIN’s work created a research methods class for D-Lab. CITE thought that the Suitability component of its product evaluation methodology would be the core, but have found that the Scalability component is 51 resonating strongly with development partners (such as Food for Peace), and identified a need for context-specific product evaluations instead of one general evaluation of a product family. Furthermore, uptake of the results of CITE’s product evaluations by designers and innovators could eventually lead to greater levels of scale for product innovations. The interest in the evaluation methodology by designers has been a significant unanticipated outcome for CITE. Negative Consequences Unanticipated reduction in funding. Although not a consequence of HESN activity, a major unanticipated impact on the HESN program was a significant reduction in USAID funding in FY15, when only 64% of total Cooperative Agreement funding levels was awarded across the portfolio. The actual reduction in funding varied for individual HESN Labs. Original Cooperative Agreement amounts totaled $140.5M over five years across the eight HESN Labs. However, due to external resource constraints, USAID has not been able to fully fund the originally requested Cooperative Agreements over the first four years of the program. Starting in Year 3 as part of the annual workplanning process each summer, the USAID HESN team has sent a “funding letter” to each the HESN Labs to indicate what obligation amount they should plan for in the upcoming year. Recoveries and other cost savings allowed for small increases in funding for some of the HESN Labs to be provided at the end of various fiscal years (beyond the originally noted amounts in the funding letters). However, these funds could not be considered for the HESN Labs’ planning purposes since they were generally received after workplans were complete for the following year. Table 6.2 summarizes the Cooperative Agreement and obligated amounts by year. Table 6.2 Summary of Cooperative Agreement and Obligated Amounts by Year Year FY Amount Obligated Total of Cooperative Agreement Amounts Percent Funded 1&2 FY11 FY12 FY13 $52.1 Million $55.7 Million 93.5% 3 FY14 $26.1 Million70 $28.8 Million 90.6% 4 FY15 $18.5 Million $28.7 Million 64.5% The unanticipated reduction in funding in Year 4 resulted in major cuts in activities across the HESN Labs and uncertainty about funding for Year 5. Funding cuts were cited as a major factor slowing progress on outcomes by all of the HESN Labs. USAID reporting and compliance requirements. All of the HESN Lab teams described the burden of USAID and university regulations, reporting, and approvals as a major negative unanticipated aspect of their HESN award. Half of the HESN Labs report that some of their faculty have decided the costs of working with USAID outweigh the benefits. Although there is recognition that USAID significantly alleviated some of the initial reporting requirements, the amount of administrative time needed has been greater than anticipated. 70 The HESN Labs could not initially plan to program to $2.7M of this as it was received as cost savings at the end of Year 3. 52 Difficulty meeting USAID expectations and timelines. Five out of eight of the HESN Lab teams said that a negative consequence of the HESN program has been the mismatch between USAID expectations and the timeline and adaptability needed for innovation. As key informant put it: "There is always a balance between agency expectations and ability to implement. The disconnect is largely in the timeline for innovation. We could build another solar oven easily, but real innovation takes time." HESN Lab growth has led to increase in organizational hierarchy. As HESN Labs have grown and more students have become involved, there have been an unintentional consequence of less direct interaction between students and lab leadership and/or faculty. For example, AidData has shifted from a smaller, flatter organization to a larger, more hierarchical one. This has meant that students are less directly involved in shaping AidData’s strategic direction, a concern that has led it to creating “shark tank” and other kinds of activities aimed at generating new student ideas. Students involved with two other HESN Labs also report having less direct contact with HESN Lab leadership, a shortage of participating faculty relative to the number of interested students, and fewer opportunities to work with HESN-affiliated faculty. Shift in emphasis on scaling. Five of the eight HESN Lab teams believed that the movement of HESN into the Global Development Lab and the change in leadership created an abrupt and unexpected emphasis on implementation and scaling. This shift meant that those HESN Labs that were better positioned to scale and closer along the spectrum were better equipped to succeed than those universities that were further away. Changes in M&E reporting requirements. Half of the HESN Lab teams cited changes in M&E reporting requirements as an unanticipated challenge. None of the HESN Labs felt the M&E system made sense or provided them with information useful to their own programs. Adapting to the changes was time￾consuming and HESN Labs lack confidence that the data is being used. SUMMARY of Evaluation Question 6: What, if any, unanticipated positive and negative consequences have occurred as a result of the HESN project? Both positive and negative consequences have occurred as a result of the HESN project. Positive unanticipated consequences include funding from other sources than the HESN award (such as Mission buy-ins), high levels of interdisciplinary collaboration, increased capacity for complying with USAID requirements, influence on policy and practice, student enthusiasm for international development, and new activities developing from planned activities. Negative unanticipated consequences included USAID’s funding cuts (due to external circumstances), difficulties with USAID expectations (M&E reporting, compliance requirements, and difficulty meeting expectations and timelines), and changes in the structure of the HESN Labs due to rapid expansion. 53 V. RECOMMENDATIONS In framing a set of recommendations for future HESN programming, the evaluation team has made an effort to balance two competing realities. On one hand, the investment made by USAID over the first five years is difficult to sustain. On the other hand, the HESN Labs are still evolving as complex systems of innovation and capacity building and, although no longer in startup mode, have not fully stabilized around a coherent set of high-impact activities and target beneficiaries. After five years, HESN will be positioned to leverage its biggest achievements: building an internal and external infrastructure for impacting development challenges; implementing an effective mode of direct “thinking and designing with” engagement with development actors; generating a body of data and knowledge seen as valuable to practitioners and policymakers; generating early stage innovations (both approaches and solutions); laying a groundwork of networks and partnerships for systemic change; and aligning HESN Lab services with priority development objectives and stakeholder needs. The recommendations outlined below are aimed at building on these achievements while also addressing the shortcomings identified through the midterm evaluation.71 Requirements and supports for scalability and replicability are not reliably introduced at the level of individual HESN Lab projects and/or broader HESN Lab activities. Except for RAN, HESN Labs lack a strategy for building research capacity in developing country higher education institutions. Uptake by USAID Missions and other operating units has been slow. There is a wide audience for learning and results from HESN Labs within USAID, other US Government innovation initiatives, and university innovation researchers but dissemination has not reached that audience. HESN Labs are different from technology companies and social innovation startups that design scalable innovations, but their role in generating data, science, and human resources for systemic change is poorly understood. 1. HESN Labs should streamline activities, adjust resource allocations, and increase synergies based on the insights gained through the first five years.  HESN Labs should identify the subset of activities, pilots, approaches, and partnerships with the greatest potential for generating sustainable impact, including likely high-potential activities currently under development. They should articulate a set of criteria based on this assessment that can be used to evaluate opportunities going forward. The utility of the activities, pilots, approaches, and partnerships should be assessed alongside Mission priorities and the existing evidence base (and number of other players) should be taken into consideration as well to avoid unnecessary duplication of efforts.  HESN Labs should identify the core critical operating functions and costs that will be needed to support this subset of activities, pilots, approaches, and partnerships. They should also identify the practices and structures that have created synergy and coherence among different value propositions and activities so as to maximize impact and resources.  HESN Labs should update or replace the Business Model Canvases created during the midterm evaluation to correspond to the streamlined set of activities and use these as the basis for monitoring, evaluation, and learning (see Recommendation 6). 2. USAID should extend funding to HESN Labs needed for core critical operating functions and costs, and should maintain the capacity for Missions and other operating units to use buy-ins to facilitate 71 It should be noted that during the contracting and execution of this evaluation, there were shifts to the strategic priorities of the U.S. Global Development Lab, the USAID Operating Unit where HESN is housed. This evaluation is not intended to make definitive recommendations related to those shifts, and only speaks to the overall program of HESN. 54 partnerships with HESN Labs. In addition, partnerships with USAID Bureaus, such as SEAD’s partnership with the Global Health Bureau, should be cultivated, potentially including funding.  One option that USAID can consider is a phased system for HESN Lab activities and innovations (technologies or approaches).72 Once an HESN Lab develops and validates an innovation or approach as beneficial to target beneficiaries/users, the next phase would involve working with implementation partners to build capacity, conduct further testing, and identify funding sources.  Once an approach developed by an HESN Lab is funded, support from USAID would be shifted to a “buy-in only” model. Additional university development innovation partners would be brought into HESN and given seed funding for a designated period of time with which to prototype, pilot, and evaluate the viability of an approach or solution for use by a development partner. 3. USAID and HESN Labs should work together to hone an effective strategy for creating partnerships with USAID Missions, including a mechanism for USAID Missions and operating units to articulate challenges and needs to HESN Labs. At the same time, HESN should retain the relationship-building, “supply side” strategy it has used to date. USAID HESN program staff have spent considerable time effort helping US researchers understand how to fit what they are offering to USAID Mission requirements. The development of a more robust structure that would invite USAID Missions to articulate upcoming challenges and needs and allow HESN Labs to submit applications for meeting those needs would allow the HESN program staff to focus more time and attention on the relationship-building activities that have played an important role in the success of HESN Lab￾Mission partnerships. 4. USAID should centralize (or contract to centralize) the dissemination of research and policy briefs, academic publications, and other knowledge products produced by HESN Labs for better access by the broader academic and development community. A model suggested for this by one university￾based innovation expert is that used by Abdul Latif Jameel Poverty Action Lab (J-PAL). A USAID￾HESN Lab working group on dissemination could meet at the next TechCon to develop a strategy for centralized knowledge sharing. Centralized knowledge management and dissemination would also support Missions to become familiar with HESN Labs. 5. Each Lab should create an ongoing partnership with one or more strategically-located developing country HEIs to access in-country researchers, communities, and innovators who can identify critical challenges and contextual needs and support capacity for research and innovation in HEI partners. These collaborations could also lead to the development of regional hubs, based on RAN’s framework and learning. To the extent possible, eligible developing country researchers involved in these partnerships should be facilitated to apply for PEER funding under the mentorship of an HESN￾affiliated faculty member who is funded by a US government science agency. Leveraging PEER funding will eliminate the need for HESN Labs to fund developing country researchers. In addition, developing country researchers involved with HESN Labs who are funded by PEER should be tracked to increase the effectiveness of both programs. Better monitoring of the developing country researchers involved with HESN Labs would make it possible for PEER and HESN to compare the types of capacity building and other benefits to developing country researchers and HEI that occur within and outside of the HESN Lab structure.  PEER could capture the university affiliation of the US science agency-funded researcher and whether or not the US researcher is affiliated with an HESN Lab. 72 This recommendation is based on the 2012 report by Monitor Group, From Blueprint to Scale (http://acumen.org/content/uploads/2013/03/From-Blueprint-to-Scale-Case-for-Philanthropy-in-Impact-Investing_Full-report.pdf) 55  HESN Labs could report the number of PEER-funded developing country researchers working with HESN-affiliated faculty. 6. USAID should reduce the number of indicators required for reporting and allow individual HESN Labs to use a focused set of indicators that correspond to a core set of value propositions (based on updated HESN Lab Business Model Canvases as described above). HESN Labs should provide USAID with information about the indicators that are most relevant to their work, and USAID should analyze the extent to which individual indicators are comparable across Labs. In addition, best practices in compiling and managing M&E datasets should be implemented to allow the investment by HESN Labs in collecting and reporting on M&E indicators to feed into semi-annual dashboards and be more feasible for evaluators to use. 7. HESN should explore linkages with related USG efforts to share knowledge and increase understanding of the overlaps between university research and technology entrepreneurship, such as the State Department’s Global Innovation through Science and Technology (GIST) initiative73 (State Department) and NSF’s Innovation Corps (iCorps). Creating these linkages would connect HESN to other USG efforts to foster innovation capacity building and measuring its impact. 8. USAID and HESN Labs should develop a strategy, such as conferences and/or communities of practice, to engage development innovation stakeholders outside of HESN in directions both synergistic and complementary with the current set of research areas and value propositions. This strategy should be aimed at identifying gaps and opportunities for collaboration and partnership beyond what can be resourced through USAID and HESN Labs. 73 GIST consists of two programs that foster innovation and technology entrepreneurship through social networking, skill development, and financing. 1 ANNEXES Table of Contents Annex I: Evaluation Statement of Work ........................................................................ 3 Annex II: Evaluation Methods By Question .................................................................... 8 Annex III: Data Collection Instruments ........................................................................11 HESN Lab Business Model Canvas ......................................................................................... 11 Key Informant Interview Protocols ....................................................................................... 14 Interview Protocol for Users/Beneficiaries of HESN Lab Innovations and Core Activities .....................................14 Interview Protocol for Key Partners Needed to Deliver Innovations and Core Activities .....................................16 Interview Protocol for Higher Education Institution (HEI) leaders at lead HESN campuses and at developing country HEI campuses partnering with HESN labs .............................................................................................................17 Interview Protocol for HESN Lab Teams..............................................................................................................................18 Interview Protocol for USAID/HESN Award Management Team Members ...............................................................19 Interview Protocol for Staff of Similar USAID Activities ...................................................................................................20 Interview Protocol for Individuals Outside of HESN .........................................................................................................21 Interview Protocol for HESN Lab Leadership......................................................................................................................22 Focus groups ............................................................................................................................. 23 Focus Groups with HESN Lab leadership and team members ........................................................................................23 Annex IV: List of Interviews Completed .......................................................................25 Annex V: Disclosure of Any Conflicts of Interest.........................................................39 Annex VI: Performance on Indicators Relative to Targets ........................................43 Annex VII: Percent of HESN Labs Meeting Target By Year ......................................47 Annex VIII: Graphs of HESN Indicator Results by Year .............................................49 Goal: Create a global interdisciplinary network of Development Labs to solve distinct development challenges .......................................................................................................... 49 Objective 1: Improve data quality, access, and analytics to advance evidence-based development decision making ................................................................................................ 52 Objective 2: Accelerate the creation, testing, and scaling of transformative innovations, technologies, and approaches ................................................................................................. 54 Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity ............................ 55 Table of Contents 2 Annex IX: HESN Lab Canvases .....................................................................................59 AidData ..................................................................................................................................... 59 CITE .......................................................................................................................................... 66 ConDev ..................................................................................................................................... 73 DIL ............................................................................................................................................. 80 GCFSI ........................................................................................................................................ 84 IDIN ........................................................................................................................................... 90 RAN ........................................................................................................................................... 93 SEAD ....................................................................................................................................... 103 Annex X: List of Developing Country HEI and Research Institute Partners by HESN Lab .......................................................................................................................110 Annex XI: HESN Lab Collaboration ............................................................................113 Annex XII: Bibliography ................................................................................................114 Evaluation Statement of Work 3 ANNEX I: EVALUATION STATEMENT OF WORK I. STATEMENT OF WORK ABBREVIATIONS HEI Higher Education Institution HESN Higher Education Solutions Network RFA Request for Applications USAID United States Agency for International Development USAID/HESN The Higher Education Solutions Network staff at USAID DESCRIPTION OF PROJECT TO BE EVALUATED Table 1: Project Summary Title/Field Project Information Higher Education Solutions Network Higher Education Solutions Network Start-End Dates November 2012 – September 2017 Life of Project1 Budget $140,559,741 Obligations through April, 2015 $70,805,129 Prime Implementers College of William & Mary, Duke University, Makerere University, Massachusetts Institute of Technology, Michigan State University, University of California – Berkeley, University of Texas A&M I.(a)Background The Higher Education Solutions Network (HESN) was formed in November, 2012 to harness the ingenuity and passion of scientists, students, faculty, and entrepreneurs to solve some of the world’s most pressing development challenges. HESN consist of eight Labs, each of which is led by a higher education institution (HEI), and a core coordinating body at USAID. The goal of HESN is to create an interdisciplinary network of Development Labs (renamed “HESN Labs” after the creation of the U.S. Global Development Lab) to solve distinct development challenges. HESN defines a “Development Lab” or “HESN Lab” as an “entity that conducts research and experimentation on issues related to international development.” HESN’s goal is supported by three objectives (Figure 1):  Objective 1: Improve data quality, access and analytics to advance evidence-based development decision making  Objective 2: Accelerate the creation, testing, and scaling up of transformative innovations2, technologies, and approaches 1 The specialized sense of “Project” from USAID’s ADS glossary is used throughout this Statement of Work 2 In USAID’s M&E plan, “innovations” are understood to be comprised of “technologies” and “approaches” Evaluation Statement of Work 4 Figure 1: HESN Results Framework Evaluation Statement of Work 5  Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge promotes learning and builds mutual capacity.  HESN Labs are intended to be virtual centers of knowledge that will help USAID and the larger development community better understand problems, and evaluate and develop new solutions to development challenges (see HESN Request for Applications; Appendix 1). HESN Labs and their partners seek to foster innovations in science and technology, and engage and inspire a new and broader community of scholars and students involved in the complex issues of international development planning, execution, and assessment. The HESN Labs support USAID and other development organizations to improve their analytical capabilities in understanding development challenges and core barriers to addressing them, catalogue solutions to different challenges, catalyze and bring forward novel approaches to addressing development problems, and encourage the development and application of new tools within science, technology, and engineering to improve the efficacy and decrease the cost of development interventions. I.(b) Results Framework and Data Collected In consultation with the HESN Labs, USAID/HESN created a results framework to align the HESN labs activities with USAID’s strategic objectives. In addition to the aforementioned goal and three objectives, intermediate results (IRs) were selected that tied into each objective. Thirty-five (35) standard indicators have been tracked to monitor HESN Labs’ progress (Attachment 2). Because each HESN Lab is working in different areas, not all of the indicators are applicable to every HESN Lab. Most HESN Labs report on about 10-20 of the standard indicators. The HESN Labs also monitor performance against indicators that are custom to their labs. The HESN Labs provide monitoring data to USAID on a semi-annual basis. Data on all of these indicators will be provided to the evaluation Contractor at the start of the contract. Specific results that are tracked include:  Innovations developed or advanced  Resources leveraged from non-USAID sources (including cash and in-kind resources)  Data tools and sets made available  Evaluations conducted  Student fellows and new classes I.(c) Description of Individual HESN Labs The HESN RFA asked for novelty and a diversity of approaches and as a result the HESN Labs that were selected operate in different areas with distinct approaches. The eight labs, based out of seven HEIs, are described below.  University of California, Berkeley’s Development Impact Lab focuses on shepherding innovative technological breakthroughs in energy, health, and information and communication technology.  Duke University’s Social Entrepreneurship Accelerator scales up scientific, technical, and business innovations for healthcare delivery and preventive services addressing issues such as AIDS and limited access to medicine and health education in rural areas. Evaluation Statement of Work 6  Massachusetts Institute of Technology’s International Development Innovation Network, comprised of village-based innovators as well as leading research institutions, is establishing a new paradigm for international development that places at its center locally-generated solutions to issues faced by people living in poverty.  Massachusetts Institute of Technology’s Comprehensive Initiative on Technology Evaluation is developing a rigorous methodology for evaluating technological solutions to challenges in the developing world to help donors and policy-makers identify and invest in the best of these solutions.  The College of William and Mary’s AidData Center for Development Policy concentrates on high resolution geospatial data, conducts analysis and applies imagery as decision support tools that enable the global development community to more effectively target, coordinate, deliver, and evaluate their aid investments.  Conflict and Development at Texas A&M University aims to increase the efficiency, efficacy, and impact of USAID development policies and programs in fragile and conflict-affected societies through multidisciplinary applied research and engagement activities by harnessing the innovative power of Texas A&M University and partner research institutions, universities, and organizations in host countries.  Makerere University’s ResilientAfrica Network innovates and accelerates science and tech￾based development tools in concert with USAID and a diverse set of stakeholders to strengthen African resilience to its greatest challenges, such as environmental variability, the effects of urbanization, government transparency, chronic civil conflict, and disease.  Michigan State University’s Global Center for Food Systems Innovation employs tested solutions that help bend the global trends toward equitable development of the global food system and are relevant to countries’ local conditions. II. EVALUATION QUESTIONS Through the Mid-Term Evaluation, USAID seeks to answer the following questions: 1) To what extent has HESN been successful in achieving the outcomes of which it may be expected? a. To what extent has HESN achieved the outcomes outlined in its Results Framework? b. To what extent has HESN achieved the outcomes expected by key stakeholders (e.g. leadership at universities and HESN labs, education officers at USAID and other donors) c. To what extent have the outcomes achieved by HESN been consistent with the evaluators’ understanding of what may be expected by universities working on development innovation? 2) What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? a. Have HESN Labs found synergies across the teams and disciplines that have led to more effective or efficient implementation? b. In comparison to other Activities (identified by the Evaluation Team in consultation with USAID/HESN), how effective have HESN Labs been in managing costs and staff? Evaluation Statement of Work 7 3) To what extent has HESN led to changes at HEIs that may increase their impact on international development? a. Have the awards made to the eight HESN Labs had wider effects on the seven HEI leads’ campuses? If so, how? b. To what extent has HESN led to greater capacity of HEIs in developing countries? c. What is the likelihood that each HESN Lab will continue to exist once funding from USAID/HESN ceases? 4) To what extent has HESN influenced or assisted USAID operating units other than the U.S. Global Development Lab? (The U.S. Global Development Lab should be excluded in answering sub-questions 4a-4c.) a. To what extent have HESN Labs affected operating units’ decision-making or other operations through data-driven methodologies, tools, or analytics? b. To what extent have USAID operating units leveraged the HESN Labs to accelerate their creation, testing, and scaling up of transformative innovations? c. To what extent have HESN Labs and USAID operating units engaged in collaborative problem-solving, knowledge sharing, and learning? 5) How can HESN modify its strategy and structure to improve its efficiency and effectiveness? a. Given the findings on HESN’s success (Question 1), are there particular areas to which the HESN Project should devote more or less attention to? b. What characteristics of HESN Labs appear to have been most critical in achieving HESN’s objectives? c. What changes to the current program and the design of a follow on program may be made to better align HESN with current understandings of the success factors of university innovation ecosystems? 6) What, if any, unanticipated positive and negative consequences have occurred as a result of the HESN Project? Evaluation Methods by Question 8 ANNEX II: EVALUATION METHODS BY QUESTION 1. To what extent has HESN been successful in achieving the outcomes of which it may be expected? Sub-questions Evaluation Methods a) To what extent has HESN achieved outcomes outlines in its Results Framework?  Analyze HESN indicator data  Dev Results tables  Review of HESN lab reports b) To what extent has HESN achieved the outcomes expected by key stakeholders?  Beneficiary key informant interviews  Partner key informant interviews  HESN HEI campus leader interviews  Higher education and innovation expert key informant interviews c) To what extent have the outcomes achieved by HESN been consistent with the evaluators’ understanding of what may be expected by universities working on development innovation?  Higher education and innovation experts key informant interviews 2. What have been the costs and benefits of HESN’s model of concentrating multiple objectives and activities within each HESN Lab? Sub-questions Evaluation Methods a) Have HESN Labs found synergies across the teams and disciplines that have led to more effective or efficient implementation?  Interviews with core HESN Lab Team members  HESN Lab team focus groups b) In comparison to similar USAID activities, how effective has HESN Labs been in managing costs and staff?  Interviews with core HESN Lab Team members  Interviews with staff of similar USAID activities 3. To what extent has HESN led to changes at HEIs that may increase their impact on international development? Sub-questions Evaluation Methods a) Have the awards made to the 8 HESN Labs had wider effects on the 7 HEI leads’ campuses? If so, how?  HESN HEI campus leader interviews  Interviews with HESN Lab staff and leaders b) To what extent has HESN led to greater capacity of HEI’s in developing countries?  Analysis of HESN Lab reports  Interviews with HEI stakeholders c) What is the likelihood that each HESN Lab will continue to exist once funding from USAID/HESN ceases?  HESN HEI campus leader interviews  Interviews with HESN Lab leaders 4. To what extent has HESN influenced or assisted USAID operating unites other than the U.S. Global Development Lab? Sub-questions Evaluation Methods a) To what extent have HESN Labs affected operating units’ decision-making or other operations through data-driven methodologies?  Review of HESN Lab reports  Interviews with USAID operating unit staff Evaluation Methods by Question 9 b) To what extent have USAID operating units leveraged the HESN Labs to accelerate their creation, testing, and scaling up of transformative innovations?  Interviews with core HESN Lab Team members  Analysis of indicator data c) To what extent have HESN Labs and USAID operating units engaged in collaborated problem-solving, knowledge-sharing, and learning? 5. How can HESN modify its strategy and structure to improve its efficiency and effectiveness? Sub-questions Evaluation Methods a) Given the findings on HESN’s success, are there particular areas to which the HESN Project should devote more or less attention?  Synthesis of findings from key informant interviews  Focus groups with HESN Lab staff b) What characteristics of HESN Labs appear to have been most critical in achieving HESN’s objectives? c) What changes to the current program and the design of a follow-up program may be made to better align HESN with current understandings of success factors of university innovation ecosystems? 6. What if any unanticipated positive and negative consequences have occurred as a result of the HESN project? Sub-questions Evaluation Methods a) None  Analysis of HESN Lab reports  Synthesis of findings from key informant interviews  Focus group with HESN Lab staff Data Collection Instruments 11 ANNEX III: DATA COLLECTION INSTRUMENTS In order to generate the body of data needed to address the six evaluation questions and sub￾questions of the HESN mid-term evaluation, the evaluation team designed a series of draft data collection instruments. Revised versions of each instrument are provided in the following sections of this document. Each of the sections below also provides information about participants/sampling and the practical approach to administering the instrument during fieldwork. The data collection strategy is anchored by a widely used tool in the field of innovation, the Nonprofit Business Model Canvas (BMC).3 This tool allows diverse kinds of transformative innovations to be assessed on nine common areas4 critical to scalable impact. It is applicable to any kind of product, service, technology, tool, or approach at any stage of development or implementation. In the HESN Midterm Evaluation, the Nonprofit Business Model Canvas provides the framework for creating a detailed, aggregate picture of the actual and potential value generated by the HESN Project to date. HESN Lab Business Model Canvas Overview: The HESN Lab Business Model Canvas (BMC) captures comparable data on the diverse mix of objectives, solutions, approaches, beneficiaries, and partners central to the work of each HESN Lab. It was administered to the eight HESN Labs through a training and support process conducted by the Dexis Evaluation Team with the core HESN Lab team.5 The BMC identified the stakeholder subgroups, both partners and users/beneficiaries, who were targeted for key informant interviews. It also provided a qualitative analysis of the value propositions and delivery model for each HESN Lab’s analytic, innovation, and ecosystem activities. The HESN Lab BMC is an adaptation of the Nonprofit BMC, a tool used widely for planning, documenting, comparing, and assessing diverse kinds of transformative innovations. The Nonprofit Business Model Canvas was adapted from the original Business Model Canvas6 to provide a focus on impact as the primary outcome rather than revenue. The HESN Lab BMC consisted of questions organized into the following areas: ● Beneficiary/User Subgroups (primary and secondary, as relevant ● Value Propositions (products, services, other activities) ● Delivery Channels ● Impacts ● Key Partners ● Key Activities Sampling/Participants: Leadership and key staff from each of the eight HESN Labs. Administration: The Senior Technical Advisor conducted consultations with each HESN Lab to familiarize them with the BMC and answer any questions the HESN Labs might have. Other 3 Retrieved from http://www.innovativenonprofit.com/2015/08/the-nonprofit-business-model-canvas￾2/#.VkuXXXudn9Q 4 Beneficiary/User Segments, Value Proposition (product, service, other benefit), Distribution Channels, Beneficiary/User Relationships, Impact Metrics, Key Partners, Key Activities, Key Resources, Cost Structure. 5 Leadership and key staff who have a primary role in the HESN Lab’s central activities. 6 Osterwalder, Alexander, and Yves Pigneur. "Business model canvas." Self published. Data Collection Instruments 12 members of the evaluation team provided additional support as needed using their knowledge of each HESN Lab gained from the document review process. HESN Labs were asked to spend 1-2 hours to complete an initial draft of the BMC,7 which the evaluation team will in turn review and send back with feedback for revisions as needed. HESN Labs initially used an online version of the tool.8 Questions in the nine areas were answered through short bullet-point descriptions. Ideally, the BMC is completed by a small team of those responsible for the overall program, but it can also be completed by one person and circulated to other core team members for review. HESN Lab Team Instructions: Based on the session provided by the evaluation team,9 complete a separate BMC for each key beneficiary/user subgroup you believe is especially valuable to understanding your HESN Lab’s impact. At least one BMC should focus on USAID Missions or operating units.. A key beneficiary/user is the actual person who will be using the product/service/approach or is the decision maker about using the product/ service/ approach. For example, the beneficiary/user would not be the “USAID Mission” but might be the “GIS Manager” or “Operations Manager.” The goal is to understand exactly who is using or will use the product/service/approach developed by the HESN Lab and what job or task is being helped either through pain reduction or gain creation. 1. Beneficiary/User Subgroup: The specific users, decision makers, and other kinds of beneficiaries who have gained and are hypothesized to gain from the core activities and output of your HESN Lab. a. What jobs are you helping them do? b. Pains you solve? c. Gains you create? 2. Value Propositions: The products, services, approaches, or other offering for each type of user/beneficiary identified above.10 a. Products, services, approaches, or other offering by your HESN Lab b. Pain relievers c. Gain creators 3. Delivery Channels: The touch points through which your HESN Lab is interacting with each core set of users/beneficiaries. a. How are you raising (or will you raise) awareness of the product/service/ approach developed by your HESN Lab? In other words, how do/will you reach and deliver it to the right users and decision makers? b. How are you enabling evaluation of your product/service/approach? c. How are you providing delivery? d. How are you continuing support? 4. Impacts: Impact and influence can be measured by the revenue coming into the HESN Lab from sponsors based on the value propositions that the lab is delivering to their users, by specific gains for beneficiaries/users (e.g., impacts of decisions, programs), by the value 7 HESN Labs will be given approximately two weeks to complete the initial draft and can request more time as needed. 8 https://canvanizer.com/new/business-model-canvas 9 The Dexis evaluation team plans to offer one hour webinars, tentatively scheduled for December 21 and January 4, to HESN Labs to explain how to complete the BMC and answer questions. The team will provide additional guidance by phone or video conference as requested by each HESN Lab. As each HESN Lab completes its BMC, we will ask if they are willing to share their completed BMCs with other HESN Labs. 10 The Value Proposition is any activity being offered for someone’s benefit. This can include an analytic tool, a dataset, a methodology, a service or product (whether or not for profit), some kind of assistance or support, an opportunity (such as an innovation challenge contest), a research grant, a training, etc. Data Collection Instruments 13 placed by beneficiaries/users on the usefulness of the product/service, or other impacts resulting from the HESN Lab’s work that can be identified and assessed. a. What are expected impacts on this beneficiary subgroup? b. What additional revenue has been generated by/for this work? 5. Key Partners: The key partners needed to deliver each type of user/beneficiary impact, through added connections, resources or activities, and the reasons each are needed. Examples of reasons partners may be needed: a. Optimization & cost effectiveness b. Risk/uncertainty reduction c. Key resource/activity acquisition 6. Key Activities: The most important activities required to deliver each type of user/beneficiary impact. a. Production? b. Problem Solving? c. Platform/Network? Figure 1 shows a the HESN Canvas template adapted from the Nonprofit BMC. The canvas provides a one-page visual representation of the model for generating value for a particular beneficiary/user segment. The BMC is not a process flow diagram. It shows the relationships between the different elements of the HESN Lab’s program model. Figure 1. Adaptation of Nonprofit Business Model Canvas Used for HESN Midterm Evaluation Value Propositions: Primary beneficiary/user of services/activities: Secondary beneficiary: Key activities/products/services: Key partners: Delivery channels: Impacts: Data Collection Instruments 14 Key Informant Interview Protocols Interview Protocol for Users/Beneficiaries of HESN Lab Innovations and Core Activities Overview: The purpose of these interviews will be to confirm and measure how well the HESN Labs’ intended impacts in each area of the HESN results framework match the perception or experience of each corresponding type of user/beneficiary, whether the distribution channels and relationships are effective and appropriate, and how the users/beneficiaries themselves evaluate the results of the HESN Labs’ intended impact or value. Anticipated user/beneficiary subgroups include students and faculty participating in core HESN Lab activities, USAID mission decision-makers, local government decision-makers, participants in HESN Lab trainings, etc. Data from beneficiary/user interviews will be used to address evaluation questions 1, 4, and 6. Sampling/Participants: Interviews will be done with a subgroup of key informants11 within each set of users/beneficiaries identified through the BMC process completed by each HESN Lab. Input from USAID/HESN Program Managers will be used to inform the selection of interview participants. Administration: Interviewees will be contacted in advance by email or phone to explain the purpose and nature of the interview and to arrange a time for a 30 minute interview via phone or Skype. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and whether she/he gives consent to begin the interview. Interview Questions: The questions below will be tailored to the type of beneficiary/user and whether they have already received the product/service/benefit offered by the HESN Lab or are an anticipated future recipient. 1. Name 2. Email 3. Date 4. Could you give an overview of the background and nature of your relationship with [name of lab]? 5. Impact: What is the benefit to you from [activity/product/service of the HESN Lab]? What job does this help you complete? What gains has it created or what pains has it relieved? 6. Delivery channels: How did you find out about this product or service [or what would be the best way for this to be made available to you]? How were you able to evaluate it? How was/should it be delivered to you? What kind of ongoing support do/would you need? 11 The number of beneficiary/user interviews for each lab will be determined based on the diversity of the beneficiary/user segments identified by each HESN Lab. The sample size will be as large as possible given the constraints of time, budget, and logistics. We project between 10-20 beneficiary interviews per HESN Lab. Data Collection Instruments 15 7. Relationship: What is the nature of your relationship with the provider of the product/service? Is there an ongoing relationship? How does/would the level of service need to be maintained/increased over time? 8. Impacts: What are the impacts of this product/service? For example, what decisions would/has it impacted and what difference has that made? How would/has it impacted policy/practice/funding? What other specific kinds of impacts has it had? 9. What unintended consequences occurred as a result of your involvement with [name of lab], either negative or positive? Data Collection Instruments 16 Interview Protocol for Key Partners Needed to Deliver Innovations and Core Activities Overview: The purpose of these interviews will be to determine why these individuals or organizations, known as key partners, are partnering with the HESN Lab, and whether there are unexpected positive or negative consequences from the partnership. Data from key partner interviews will be used to address evaluation questions 1, 2, 5, and 6. Sampling/Participants: Interviews will be done with informants12 from each key partner identified through the BMC process completed by each HESN Lab. Input from USAID/HESN Program Managers will be used to inform the selection of interview participants. Administration: Interviews will be conducted primarily via Skype or phone. Each interview will last approximately 30 minutes. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and whether she/he gives consent to begin the interview. Interview Questions: 1. Name 2. Email 3. Date 4. Could you give an overview of the background and nature of your relationship with [name of lab]? 5. What purpose does this partnership serve? How is that purpose being fulfilled? 6. What are your motivations for partnering with [name of lab]? 7. What is your relationship with the beneficiary/users of the innovation/activity offered by [name of lab]? 8. Have you realized or do you anticipate any benefits from this partnership that you had not expected upon start-up? If so, what are they? 9. Have there been or do you anticipate any negative effects for you from this partnership? If so, what are they? 12 The sample will include informants from current or past key partners. Data Collection Instruments 17 Interview Protocol for Higher Education Institution (HEI) leaders at lead HESN campuses and at developing country HEI campuses partnering with HESN labs13 Overview: The purpose of these interviews will be to determine whether the HESN award and activities have had wider effects on the seven HEI leads’ campuses, and the extent to which the HESN awards and activities have led to greater capacity at developing country HEIs partnering with the HESN Labs. Data from these interviews will be used to address evaluation question 3. Sampling/Participants: Interviews will be done with 2-3 campus leaders identified by each HESN Lab team on their lead campus who can speak to the wider effects of the HESN award and activities. Anticipated types of campus leaders include provosts, vice provosts, deans, department heads, and program directors. In addition, informants from developing country HEIs identified as key partners by the HESN Labs on the BMC Assessment will be targeted for interviews. In the likely event that there are too many developing country HEI key partners to interview informants from all of them, the evaluation team will ask USAID/HESN Program Managers and the HESN Labs to select 2-4 priority informants per HESN Lab. Contacts at the remaining developing country HEI partner campuses will be included in a survey of developing country HEI informants. Possible informants include faculty members, provosts, deans, department heads, and institute/center directors. Administration: Interviews will be conducted via Skype or phone or during site visits. Each interview will last approximately 30 minutes. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and whether she/he gives consent to begin the interview. Interview Questions: 1. What’s the nature of your involvement with [name of lab]? How long have you been involved with them? 2. What value does [name of lab] have or represent more broadly within your domain and/or the university? 3. Have there been wider effects on campus of the HESN award made to [name of lab]? Are any of these impacts related to the potential for the university to have an impact on USAID or international development more broadly? 4. How does your unit/the university view its role in [name of lab]’s long-term sustainability? What ongoing investments, financial or otherwise, has the university made in ensuring the likelihood that [name of lab] will continue beyond the HESN award? 13 Developing country HEI campuses will be identified through the Key Partners section of the BMC Assessment completed by each HESN Lab. The HESN Lab will be asked to identify one or more individuals who can speak to whether the partnership with the HESN Lab has led to greater capacity for the HEI. In the case of the Resilient Africa Network, informants from each of RAN’s Resilience Innovation Lab partner campuses will be selected for this interview. Data Collection Instruments 18 Interview Protocol for HESN Lab Teams Overview: The purpose of these interviews will be to gain information from HESN Lab leadership and key staff related to evaluation questions 1, 2, and 5. Sampling/Participants: Interviews will be done with key HESN Lab team members as determined by the HESN Lab and with input from the USAID/HESN Program Managers. Administration: Interviews will be conducted in person or via Skype or phone. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and whether she/he gives consent to begin the interview. Interview Questions: 1. What is your role with [name of lab] and how long have you been involved? 2. What core activities of [name of lab] are you responsible for or primarily engaged with? For each activity: 3. What supports you in successfully carrying out this activity (structures, input/assistance from --, contacts, networks, other resources)? 4. What makes this activity challenging or has slowed its progress? 5. How would you describe the impacts or benefits of the HESN award on this activity – what has it made possible that would not have been possible without it? 6. Who are your primary colleagues and contacts for this activity within the [name of lab] network? 7. Who are your primary contacts and collaborators at other HESN Labs? What is the purpose of these connections? Repeat for each activity this team member is engaged with. Data Collection Instruments 19 Interview Protocol for USAID/HESN Award Management Team Members Overview: The purpose of these interviews will be to gain information from USAID/HESN staff on their perspective and experiences of engaging with their assigned HESN Lab over the past several years, to capture the extent to which this HESN Lab has influenced or assisted USAID operating units other than the U.S. Global Development Lab, and to describe the characteristics of the HESN Lab that have been most critical in achieving HESN’s objectives. The primary use of this data is for preparing for site visits. Sampling/Participants: Interviews will be done with the USAID/HESN Award Management team members assigned to each HESN Lab. Administration: Interviews will be conducted in person or via Skype or phone. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and gives consent to begin the interview. Interview Questions: 1. What is your understanding of your assigned HESN Lab’s primary beneficiary/user segments and the current or future benefit delivered to each? 2. In reviewing the HESN Lab’s BMC, are there any areas that you see differently? If so, how would you represent those areas? Is there anything that has been left out from your perspective? 3. What characteristics or factors have facilitated or slowed this HESN Lab in achieving success on the HESN objectives? In creating value for each primary beneficiary/user segment? 4. Have there been any unintended positive or negative consequences that have occurred as a result of this HESN Lab’s award? 5. To what extent has this HESN Lab affected decision-making or other operations within USAID operating units? 6. To what extent have USAID operating units leveraged this HESN Lab to accelerate its creation, testing, and scaling up of transformative innovations? 7. To what extent has this HESN Lab and USAID operating units engaged in collaborative problem-solving, knowledge-sharing, and learning? Data Collection Instruments 20 Interview Protocol for Staff of Similar USAID Activities Overview: The purpose of these interviews will be to gain information from similar USAID initiatives on cost and staff management to provide a comparison against which to assess the effectiveness of HESN Labs. The data from these interviews will be used to address evaluation question 2 (see Annex B). Sampling/Participants: Approximately 10-15 interviews will be done with the staff of similar USAID activities identified by USAID/HESN. Administration: Interviews will be conducted via Skype or phone and will last approximately one hour each. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and gives consent to begin the interview. Interview Questions: 1. Are you familiar with HESN? If so, what is your understanding of it? 2. What are/were the intended goals, objectives, and timeline of this initiative? 3. What is the staffing structure of this initiative? (FTEs by years; part time) 4. What is the cost structure of this initiative? (most important costs; most expensive) 5. What have been the challenges of managing staff and costs? 6. What has facilitated effective staff and cost management? 7. What characteristics or factors have facilitated or slowed this initiative in achieving success on its objectives? Data Collection Instruments 21 Interview Protocol for Individuals Outside of HESN Overview: The purpose of these interviews will be to gain information about the approach and value￾added of the HESN Project from the perspective of thought leaders (both within and outside of USAID) in international development, science/technology, and innovation. The data from these interviews will be used to address evaluation questions 1 and 5. Sampling/Participants: Interviews will be done with 5-8 thought leaders identified by HESN Labs, USAID/HESN, and the Dexis evaluation team who are able to speak about the strengths and weaknesses of HESN’s approach. Administration: Interviews will be conducted via Skype or phone. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and gives consent to begin the interview. Interview Questions: 1. What do you see as the need for the HESN approach of creating an interdisciplinary network of development labs to solve distinct development challenges? 2. What are the challenges of introducing transformative solutions into the context of international development, given the manner in which USAID Missions and other development institutions plan, implement, and manage strategies and solutions? 3. In what ways do HESN Labs appear to be adding value to policy and practice in international development, whether through improved data/analytics, development of transformative innovations/approaches, or developing an ecosystem of interdisciplinary R&D/problem-solving? 4. What is the value of universities related to tangible scalable innovations? 5. What is the value of universities related to research that can be used to advance development solutions? 6. What is the value of the HESN network for partnerships with developing world actors? Data Collection Instruments 22 Interview Protocol for HESN Lab Leadership Overview: The purpose of these interviews will be to gain information from HESN Lab leadership and key staff related to evaluation questions 1, 3, and 6. Sampling/Participants: Interviews will be done with the HESN Lab director(s) and others in leadership roles. Administration: Interviews will be conducted in person or via Skype or phone. The interviewer will explain that the interviewee’s responses will not appear individually in any report or other form but will instead be combined with responses from other similar interviewees into an aggregate analysis. Personal identifying information will be removed from the interview notes prior to the analysis. Before using any direct quotes from the interview, the interviewee’s permission would be requested, and any quotes would not be incorporated without the interviewee’s consent. The interviewee may decide to end the interview at any point. Before moving on to the interview questions, the interviewee will be asked if she/he has any questions and whether she/he gives consent to begin the interview. Interview Questions: 1. Name/title 1. (pivots, turning points) What have been the most critical decisions or events that have led to significant changes in [name of lab]’s work? What brought about these changes and how they have impacted the work? 2. What is your view of the gain for USAID in having [name of lab] in the HESN network? How has [name of lab] work been affected by receiving the HESN award? 3. What have been the most significant unintended consequences (positive or negative) for [name of lab] as a result of the HESN award? 4. Who outside HESN is best able to speak about the strengths and weaknesses of [name of lab]’s approach/products/programs? 5. Has the HESN award impacted [name of lab]’s funding from other sources? If so, how? What other sources of funding are in place or anticipated following the initial HESN award? 6. What type and level of relationship do you have with each donor? How much time and other resources is spent on these relationships? What activities does each donor require? Data Collection Instruments 23 Focus groups Focus Groups with HESN Lab leadership and team members Overview: There are two sets of purposes for focus groups with leadership and key staff at each HESN Lab.  Review the BMC canvases developed by the lab and update as needed; discuss the HESN lab’s work as seen through this framework (what does it help make more visible; what does it leave out about value of the lab’s work); discuss learning and impacts related to each beneficiary group (using beneficiary interview findings to date); evolution of value propositions of lab as related to international development; what are key resources and partnerships needed and how are those supported/maintained; discuss what would enhance the lab’s ability to fulfill its goals and the goals of the HESN award.  Understand the success factors and obstacles in the HESN Lab’s success on the HESN objectives, unintended consequences of the HESN award (positive/negative), the effectiveness of the HESN Lab’s own ecosystem, and the effectiveness of the HESN Lab’s relationships with USAID Mission staff or other USAID/Washington staff. The data collected from these focus groups will address evaluation questions 1, 2, 5, and 6. Participants: HESN Lab leadership and team members who have a primary role in the HESN Lab’s central activities. The size of the focus group will vary depending on the HESN Lab but is anticipated to be approximately 5-10 people. Session 1 (2 hours): Clarify the areas of the BMC and hone the value propositions and impact metrics 1. Briefing on canvas; why we're using; what sections mean in terms of HESN labs  It helps us look at the different things going on inside each lab and across labs using a common framework.  The value proposition states what the deeper purpose of the lab’s activities are.  The beneficiary is better stated as the type of person with whom the lab engages in carrying out its core activities. Sometimes there is a related beneficiary shown alongside the main beneficiary, which can be someone who has uptake of the beneficiary’s work.  The impacts (shown as revenue streams on the Canvanizer) are the quantifiable impacts expected as a result of successfully achieving the value proposition. This can be additional grants/revenue as well as community-level impacts, outcomes of better policy, etc. 2. Review the BMC canvases developed by the lab  Ask questions about the value propositions  What does this format capture? What does it leave out?  Switch over to table of value propositions, etc. by area of work  Sometimes talking about activities leads to clarity of value proposition  Make sure beneficiaries are those with whom the lab engages in carrying out its core activities, not people receiving “benefits” such as grants. If they are receiving financial support through grant program, how does their activity contribute to the value proposition? 3. Discuss the HESN lab’s work as seen through this framework  What does it help make more visible?  What does it leave out about value of the lab’s work? Data Collection Instruments 24 Session 2 (1-2 hours): 1. Record names of attendees 2. What about HESN could be improved to facilitate your lab in doing its work and HESN overall? 3. What to date have been the main factors supporting [name of lab] in achieving its objectives? What to date have been the main challenges to [name of lab] in achieving its objectives? 4. Have there been unintended positive or negative consequences for [name of lab] as a result of the HESN award? 5. What value or benefit of [name of lab]'s work is hard to capture or is not well understood by USAID? List of Interviews Completed 25 ANNEX IV: LIST OF INTERVIEWS COMPLETED Lab: Interview Type: Name: Date: Title: AidData Beneficiary Dan Maliniak 12/15 International Relations Professor AidData Beneficiary David Saldivar 3/2/16 Policy and Advocacy Manager, Aid Effectiveness, Oxfam AidData Beneficiary Deborah Naatujuna Via email Engagement Manager, Resilient Africa Network AidData Beneficiary Douglas Johnson 2/29/16 Senior Impact Assessment Advisor, USAID MERLIN (EIA) AidData Beneficiary Paula Pickering 12/15 Government Professor AidData Beneficiary Philip Roessler 12/15 Government Professor AidData Beneficiary Richard Okello via email GIS Specialist, USAID/Uganda AidData Beneficiary Sagar Sharma Via email Director, Department of Development Studies, Kathmandu University AidData Beneficiary Tania Alfonso 3/10/16 Senior Evaluation Specialist, USAID/PPL- Office of Learning, Evaluation and Research AidData Campus Leader Dennis Manos 12/14 Vice President for Research AidData Campus Leader Michael Halleran 12/15 Provost AidData Campus Leader Sarah Stafford 12/14 Director, Public Policy Program AidData Campus Leader Steven Hanson 12/15 Vice Provost for International Affairs, Director of the Reves Center) AidData Lab Leadership Brad Parks 12/14 Co-Executive Director AidData Lab Leadership David Trichler 12/14 Director of Operations AidData Lab Leadership Nancy Choi 12/14 Co-Executive Director AidData Lab Team Member Alena Stern 12/14 Project Manager AidData Lab Team Member Ariel BenYishay 12/15 AidData, Chief Economist AidData Lab Team Member Dan Runfola 12/15 AidData Senior Geospatial Scientist List of Interviews Completed 26 Lab: Interview Type: Name: Date: Title: AidData Lab Team Member Jessica Wells 12/15 Research/Evaluation Senior Program Manager AidData Lab Team Member Sam Custer 12/15 AidData Director of Policy Analysis AidData Partner Michael Crino 3/1/16 Deputy Director, USAID/GeoCenter CITE Beneficiary Brian King 2/28/16 Myanmar Office, Mercy Corps CITE Beneficiary Greg Olsen 2/11/16 Program Operations Division Director, Office of Food For Peace, Bureau for Democracy, Conflict and Humanitarian Assistance, USAID CITE Beneficiary Jon Pearlman 2/11/16 Director, International Society of Wheelchair Professionals Associate Director, Human Engineering Research Laboratories, University of Pittsburgh. CITE Beneficiary Maarten Vrouenraets Via email PhD Student, Delft University of Technology, Netherlands CITE Beneficiary Martin Fowler 2/18/16 Agriculture and Livelihoods Advisor, USAID Mission Uganda CITE Beneficiary Phillip Greene 2/10/16 Economist, USAID Mission Uganda CITE Beneficiary Shanti Kleinman 2/11/16 Energy and Environment Advisor, Mercy Corps CITE Campus Leader Richard Lester 02/19 Associate Provost of International Affairs CITE Lab Leadership Bish Sanyal 02/18 CITE Director CITE Lab Leadership Joanne Mathias 02/19 Associate Director CITE Lab Team Member Amit Gandhi 02/18 PhD Student, Research Assistant CITE Lab Team Member Brittany Montgomery 02/18 Course DevelopmentPhD Student, Research Assistant CITE Lab Team Member Cauam Cardoso 02/18 Course DevelopmentPhD Student, Research Assistant CITE Lab Team Member Gaurav Kewlani 02/18 Post Doc Associate, Course Developerpment List of Interviews Completed 27 Lab: Interview Type: Name: Date: Title: CITE Lab Team Member Jennifer Green 02/18 Research S3 Lead (Sustainability) CITE Lab Team Member Jesse Kaminsky 02/19 Financial and Program Assistant CITE Lab Team Member Kendra Leith 02/18 Monitoring and Evaluation Manager CITE Lab Team Member Oli L De Weck 02/19 Faculty S3 Lead (Sustainability) CITE Lab Team Member Stephen Graves 02/18 Scalability Faculty Lead CITE Partner Allison Hynd 02/17 Director of Programs at the Public Service Center CITE Partner Brennan Lake 2/17/16 Programs Director, Technology Exchange Lab CITE Partner Chintan Vaishnav 02/18 Tata Center Director CITE Partner Chintan Vaishnav 2/17/16 Academic Director, Tata Center Senior Lecturer, Sloan School of Management, MIT ConDev Beneficiary Apryl Williams 01/21 Graduate PhD Student, College of Liberal Arts ConDev Beneficiary Deepthi Shekar 1/11/16 PhD Student ConDev Beneficiary Grant Gordon 1/11/16 PhD Student ConDev Beneficiary Jaehyan Ahn 01/22 PhD Student, School of Public Health ConDev Beneficiary Jeff Tilton 1/25/16 USAID Mission, Democratic Republic of the Congo ConDev Beneficiary Junyi Chen 1/11/16 PhD Student ConDev Beneficiary Katharina Anton￾Exleben 1/11/16 AAAS Fellow, Education Division, Africa Bureau, USAID ConDev Beneficiary Kofi Boa 1/25/16 Director, Center for No-Till Agriculture in Ghana ConDev Beneficiary Michael Martin 1/26/16 Director of the Office of Agriculture, USAID Mission in Afghanistan List of Interviews Completed 28 Lab: Interview Type: Name: Date: Title: ConDev Beneficiary Michael Petriello 1/11/16 PhD Student ConDev Campus Leader Alan Sams 01/21 Executive Associate Dean, College of Agriculture and Life Sciences ConDev Campus Leader Fred Broadu 01/21 Professor and Assistant Department Head, Agricultural Economics ConDev Campus Leader Mark Hussey 01/22 Vice Chancellor and Dean, College of Agriculture and Life Sciences ConDev Lab Leadership Ed Price 01/21 ConDev Director ConDev Lab Leadership Shahriar Kibriya 01/21 ConDev Associate Director ConDev Lab Team Member Chris Woodruff 01/22 ConDev Program Coordinator for M&E ConDev Lab Team Member Emily Baker 01/22 ConDev DRC Program Coordinator ConDev Lab Team Member Gavin Finnegan 01/22 ConDev DRC Program Coordinator ConDev Lab Team Member Graham Savio 01/22 ConDev DRC Program Coordinator ConDev Lab Team Member Jessica Gottlieb 01/21 Assistant Professor, Bush School of Government and Public Service ConDev Lab Team Member Kelly Prendergast 01/22 ConDev Communications Manager ConDev Lab Team Member Leslie Ruyle 01/21 ConDev Network Coordinator and Student Engagement Manager (past M&E Coordinator) ConDev Lab Team Member Natalia Valdez￾Gonzalez 01/22 ConDev Research Associate ConDev Partner Ann Kelly 1/11/16 President, Howard G. Buffett Foundation ConDev Partner Jay Maddock 01/21 Dean, School of Public Health ConDev Partner Joe Hewitt 1/25/16 Office of Conflict Management and Mitigation, USAID List of Interviews Completed 29 Lab: Interview Type: Name: Date: Title: Former ConDev Activity Manager ConDev Partner Johanna Roman 2/8/16 Program Manager, Conflict & Development Foundation ConDev Partner Larry Garber 1/25/16 Member, ConDev Advisory Board Visiting Faculty, National Defense University Former Deputy Assistant Administrator, Bureau of Policy, Planning and Learning, USAID DIL Beneficiary Brandie Nonnecke 02/01 Research and Development Manager, CITRIS Connected Communities Initiative and Health Initiative DIL Beneficiary Daniel Fletcher 02/01 Bioengineering Faculty DIL Beneficiary Daniel Handel 2/10/16 Mission Economist at USAID/Rwanda DIL Beneficiary Ha Phan 2/3/16 Vietnam In-Country Representative, University of California San Francisco DIL Beneficiary Kail Padgitt 2/9/16 Former Economist, Economic Growth & Infrastructure, USAID Afghanistan Mission DIL Beneficiary Kurtis Heimmerl 1/29/16 Software Engineer, Facebook Former PhD student, UC Berkeley DIL Beneficiary Kweku Opoku￾Agyemang 2/4/16 Postdoctoral Fellow, UC Berkeley DIL Beneficiary Pierce Gordon 2/5/16 PhD Student, UC Berkeley DIL Beneficiary Samson Ondiek 2/4/16 Chief Officer, Corporate Planning, Business Strategy, Kenya Power & Lighting Company DIL Lab Leadership Ashok Gadgil 01/01 Lab Co-Director (Principal Investigator – DIL) Professor, Civil and Environmental Engineering List of Interviews Completed 30 Lab: Interview Type: Name: Date: Title: DIL Lab Leadership Shankar Sastry 02/01 Lab Co-Director (Principal Investigator SDS) Dean of Engineering DIL Lab Leadership Temina Madon 02/02 Managing Director Executive Director, Center for Effective Global Action DIL Lab Team Member Alice Agonino 02/01 Lead for Dev Eng designated emphasis Professor, DIL Lab Team Member Catherine Wolfram Faculty Director, Ecosystem for Development Engineering Professor of Business Administration DIL Lab Team Member Eric Brewer 02/01 Faculty Director, Development Data Analytics Professor, Computer Science DIL Lab Team Member Heather Lofthouse 02/02 Associate Director, Partnerships DIL Lab Team Member Joyashree Roy 02/04 Director for International Partnerships Professor of Economics at Jadavpur University DIL Lab Team Member Ted Miguel 02/01 Faculty Director, Innovation Incentives Faculty Director, Center for Effective Global Action Professor, Environment and Resource Economics DIL Partner Craig McIntosh 02/02 Co-Director, Policy Design & Evaluation Lab Professor of Economics, School of Global Policy and Strategy, UCSD DIL Partner Phillip Denny 02/02 Program Director, Big Ideas at Berkeley List of Interviews Completed 31 Lab: Interview Type: Name: Date: Title: DIL Partner Wallied Shirzoi 02/02 Project Manager, Policy Design and Evaluation Lab, UCSD GCFSI Beneficiary Craig McIntosh 2/1/16 Professor of Economics, School of Global Economy and Strategy, UC San Diego GCFSI Beneficiary David Kramer 1/26 Professor, Plant Science Lab GCFSI Beneficiary Jason Snyder 1/26 PhD Student GCFSI Beneficiary Linlin Liang 1/25 PhD Student GCFSI Beneficiary Tian Cai 1/26 PhD Student GCFSI Beneficiary Vivian Hoffman 1/27 Research Fellow, International Food Policy Research Institute GCFSI Campus Leader David Poulson 01/25 Associate Provost and Dean for International Studies and Programs GCFSI Campus Leader DeAndra Beck 01/26 Associate Dean for Research for International Students and Programs GCFSI Campus Leader Doug Gage 01/26 Assistant Vice President, Office of Interdisciplinary Research and Internal Grants GCFSI Campus Leader Steven Hanson 01/26 Associate Provost and Dean for International Studies and Programs GCFSI Lab Leadership Eric Crawford 01/25 Director, GCFSI GCFSI Lab Leadership Kurt Richter 01/25 Assistant Director, GCFSI GCFSI Lab Team Member David Tschirley 01/26 Professor, International Development, Agriculture, Food, and Resource Economics GCFSI Lab Team Member Gretchen Neisler 01/26 Director, Global Connections in Food, Agriculture, and Natural Resources GCFSI Lab Team Member Joe Messina 01/25 Professor of Geography, College of Social Science; Associate Dean for Research GCFSI Lab Team Member John Bonnell 01/26 Capacity Development and Outreach Specialist List of Interviews Completed 32 Lab: Interview Type: Name: Date: Title: GCFSI Lab Team Member John Medendorp 01/26 Project Manager GCFSI Lab Team Member Michael Hamm 01/26 C.S. Mott Professor of Sustainable Agriculture GCFSI Lab Team Member Pouyan Nejdhashemi 01/26 Associate Professor of Biosystems and Agricultural Engineering GCFSI Partner Cynthia Donavan 01/26 Deputy Director of the FTF Legume Innovation Lab IDIN Beneficiary Betty Ikaleny 1/27/16 IDIN Network Member, Uganda IDIN Beneficiary Carl Jensen 2/10/16 IDIN Network Member, Zambia IDIN Beneficiary David Saleh 2/5/16 IDIN Network Member, Colombia IDIN Beneficiary Heewon Lee 2/8/16 IDIN Network Member, South Korea IDIN Beneficiary Roy Ombatti Via email IDIN Network Member, Kenya IDIN Lab Leadership Amy Smith 02/17 Director and Principal Investigator Founder and Co-Director, DLab IDIN Lab Leadership Kofi Taha 02/17 Associate Director IDIN Lab Team Member Asif Obaidee 02/17 Finance and Program Administrator IDIN Lab Team Member Elizabeth Hoffecker 02/16 Research Coordinator IDIN Lab Team Member Jona Repishti 02/16 Network Coordinator IDIN Lab Team Member Laura Budzyna 02/16 MEL Coordinator IDIN Lab Team Member Lauren Mckown 02/16 Communications Manager IDIN Lab Team Member Molly Rubenstein 02/17 Innovation Center Coordinator IDIN Lab Team Member Nai Kalema 02/17 Finance and Program Assistant IDIN Lab Team Member Sher Vogel 02/16 Summits Coordinator IDIN Partner Ben Linder, Oscar Mur￾Miranda 02/18 Faculty, Olin College of Engineering List of Interviews Completed 33 Lab: Interview Type: Name: Date: Title: IDIN Partner Erwin Kinsey 2/11/16 ECHO, Tanzania IDIN Partner Harold Msanya 2/11/16 ECHO, Tanzania IDIN Partner Jorge Espinosa 2/12/16 Instructor, Program for International Energy Technologies and the UC Davis D-Lab IDIN Partner Keely Swan 02/17 Public Service Center, Ideas Global Challenge Manager IDIN Partner Kurt Kornbluth 2/12/16 Associate Director, Blum Center for Developing Economies, UC Davis Founder and Director of Program for International Energy Technologies and the UC Davis D-Lab IDIN Partner Min Ng 2/10/16 Singapore Polytechnic IDIN Partner Shelby Sack 2/4/16 Global Social and Sustainable Enterprise MBA program, Colorado State University RAN Beneficiary Innovators 02/11 Innovation Consortium RAN Beneficiary Non-grantee beneficiary 02/10 No touch water tap RAN Beneficiary Non-grantee beneficiary 02/10 Digital Grain Moisture RAN Beneficiary Non-grantee beneficiary 02/10 Bacterial Vaginosis Kit RAN Beneficiary RAN grantee 02/09 EDAD RAN Beneficiary RAN grantee 02/09 Matibabu RAN Beneficiary RAN grantee 02/09 Rootio RAN Beneficiary Steven Goldfinch 02/12 UNDP Representative RAN Beneficiary Students 02/10 Student beneficiaries without innovation projects RAN Beneficiary Students 02/11 Non-university student beneficiaries RAN Campus Leader Professor Lynn Atuyambe 02/11 College of Health Sciences Faculty List of Interviews Completed 34 Lab: Interview Type: Name: Date: Title: RAN Campus Leader Professor Nawagwe 02/10 Deputy Vice Chancellor, Finance and Administration, Makerere University RAN Lab Leadership Professor Bazeyo 02/08 RAN Chief of Party and Dean of Makerere School of Public Health RAN Lab Leadership Professor Serwadda 02/08 RAN Senior Technical Advisor RAN Lab Team Member Christine Muhumuza 02/09 Resilience and Research Team RAN Lab Team Member Debbie Namirembe 02/12 Administrative Officer RAN Lab Team Member Deborah Naatujuna Nkwanga 02/09 Engagement Manager RAN Lab Team Member Dr. Dorothy Okello 02/08 Director of Innovation RAN Lab Team Member Dr. Julius Ssentongo 02/10 Program Coordinator, East Africa RI La RAN Lab Team Member Dr. Nathan Tumuhamye Kipande 02/09 Director, East Africa RI Lab RAN Lab Team Member Dr. Roy Mayega 02/08 Deputy Chief of Party RAN Lab Team Member Grace Mongo Bua 02/10 Community Liaison Officer RAN Lab Team Member Harriet Adong 02/09 Communication Manager RAN Lab Team Member Harriet Namta 02/12 Monitoring and Evaluation Manager RAN Lab Team Member Herbet Ampeire 02/12 Accountant RAN Lab Team Member Joseph Mukaawa 02/09 MKITS Team RAN Lab Team Member Ronald Kayiwa 02/10 Innovations Officer RAN Lab Team Member Sheila Agabe 02/11 Lab Technical Officer RAN Partner Daniel Komakech 02/12 Gulu University Focal Person SEAD Beneficiary Dorothy Mangale 1/27 Former student SEAD Beneficiary Elizabeth Bailey 1/26 Member of Global Health Advisory Board; Director of Consortium for Affordable Medical Technologies at Massachusetts General Hospital SEAD Beneficiary Evelyn Powery 2/10 Former student List of Interviews Completed 35 Lab: Interview Type: Name: Date: Title: SEAD Beneficiary Jennifer Fluder 1/27 Former student, now employed by USAID SEAD Beneficiary Karen Clune 1/28 Senior Innovation Advisor, Center for Accelerating Innovation and Impact, Bureau of Global Health, USAID SEAD Beneficiary Katie Kirsch 2/4 Social entrepreneur, Sisu Global Health SEAD Beneficiary Libby King MacFarlane 01/29 MBA Student SEAD Beneficiary Megan Mukuria 2/3 SEAD Social Entrepreneur, Zana Africa SEAD Beneficiary Moka Lantum 1/26 SEAD Social Entrepreneur, MicroClinic Technologies SEAD Beneficiary Shelley Saxena 1/25 SEAD Social Entrepreneur, SevaMob SEAD Beneficiary Stephanie Wilson 1/27 Angel Investor, Member of Investors’ Circle SEAD Beneficiary Trey Sinyard 01/29 MBA and MD Student SEAD Campus Leader Matt Nash 01/29 Managing Director, Social Innovation and Entrepreneurship SEAD Campus Leader Mike Merson 02/09 Director, Duke Global Health Institute SEAD Lab Leadership Cathy Clark 01/28 Director of CASE i3 SEAD Lab Leadership Erin Worsham 01/28 Executive Director, CASE SEAD Lab Leadership Krishna Udayakumar 01/28 Executive Director, Innovations in Healthcare SEAD Lab Team Member Anna Katharine Wales 01/28 Senior Research Manager SEAD Lab Team Member Carrie Gonnella 01/28 Program Director, Impact Investing and Innovator Support, SEAD and CASE i3 SEAD Lab Team Member Dennis Clements 01/29 Professor, Pediatrics, Community and Family Medicine; Senior Advisor, Duke Global Health Institute SEAD Lab Team Member Jenny Cook 01/28 Communications Manager List of Interviews Completed 36 Lab: Interview Type: Name: Date: Title: SEAD Lab Team Member Joe Egger 01/29 Research Scholar, Duke Global Health Institute; Co￾Chair, SEAD Research Committee SEAD Lab Team Member Kim Langsam 01/28 Program Director, Research Evaluation and Student Engagement SEAD Lab Team Member Logan Couce 01/28 Project Associate SEAD Lab Team Member Patricia Odero 01/28 East Africa Regional Director SEAD Lab Team Member Sylvia Sable 01/28 East Africa Project Associate SEAD Partner Agiso Odhuno 2/4 Activity Manager, East Africa Regional Mission, USAID SEAD Partner Bonny Moellenbrock 01/29 Executive Director, Investors’ Circle SEAD Partner Jennifer Headley 01/29 Evaluation Research Specialist, Duke Global Health Evidence Lab SEAD Partner Joy Noel Baumgartner 01/29 Associate Director, Duke Global Health Institute Evidence Lab SEAD Partner Rachele Haber￾Thomson 01/29 Director of Operations, Investors’ Circle SEAD Partner Rae Jean Proeschold￾Bell 01/29 Director, Duke Global Health Institute Evidence Lab Organization: Interview Type: Name: Date: Title: USAID Other USAID Higher Education and Innovation Initiatives Aaron Miles 1/19 Higher Education Partnerships for Innovation and Impact (HEPII), Higher Education Advisor USAID Other USAID Higher Education and Innovation Initiatives Annica Wayman 02/26 Partnerships for Enhanced Engagement in Research (PEER), Division Chief for the Research Partnerships for Development Team USAID Other USAID Higher Education and Innovation Initiatives Carole Levin 01/15 Feed the Future Innovation Labs, Program Analyst List of Interviews Completed 37 Organization: Interview Type: Name: Date: Title: USAID Other USAID Higher Education and Innovation Initiatives Ezra Simon 01/26 Higher Engineering Education Alliance Program (HEEAP), Education Officer, USAID/Vietnam USAID Other USAID Higher Education and Innovation Initiatives Gary Bittner 01/13 Higher Education for Development (HED), Division Chief for Higher Education, Workforce Development, and Training USAID Other USAID Higher Education and Innovation Initiatives Nicholas Bassey 02/09 Research Innovation Fellowship (RI Fellows), Team Lead Azusa Pacific University Higher Education and Innovation Experts Chris Collins 02/04 Assistant Professor, Department of Doctoral Higher Education Engineering for Change ASME Foundation Higher Education and Innovation Experts Noha El-Ghobashy 03/14 President Executive Director George Washington University Higher Education and Innovation Experts Jong-On Hahm 03/11 Special Advisor for International Research Pennsylvania State University Higher Education and Innovation Experts Khanjan Mehta 02/29 Humanitarian Engineering and Social Entrepreneurship (HESE) Program, Founding Director Assistant Professor of Engineering Design U.S. Department of State Higher Education and Innovation Experts Sara Klucking 03/04 Global Engagement Lead, GIST, Office of Science and Technology Cooperation USAID Higher Education and Innovation Experts Barbara Schneeman 02/24 E3, Higher Education Coordinator USAID Higher Education and Innovation Experts Cameron Bess 03/04 Partnerships for Enhanced Engagement in Research (PEER), Senior Research Advisor USAID Higher Education and Innovation Experts Christie Vilsack 03/03 E3, Senior Advisor for International Education USAID Higher Education and Innovation Experts Karen Clune 03/07 Center for Accelerating Innovation and Impact, Bureau for Global Health List of Interviews Completed 38 Organization: Interview Type: Name: Date: Title: USAID Higher Education and Innovation Experts Robert Schneider 03/11 Partnership to Accelerate Entrepreneurship (PACE), Center for Transformational Partnerships USAID Higher Education and Innovation Experts Susan Owens 03/01 Bureau for Food Security/BIFAD, Former Activity Manager for GCFSI USAID Higher Education and Innovation Experts Wendy Taylor 03/14 Center for Accelerating Innovation and Impact, Bureau for Global Health Disclosure of Any Conflicts of Interest 39 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST Disclosure of Any Conflicts of Interest 40 Disclosure of Any Conflicts of Interest 41 Disclosure of Any Conflicts of Interest 42 Performance on Indicators Relative to Targets 43 ANNEX VI: PERFORMANCE ON INDICATORS RELATIVE TO TARGETS HESN Indicators14 An * denotes no reported values for Year 1 (2013), assumption that the indicator was not tracked in Year 1 Overall Total15 Total for Labs with Targets16 Target17 Goal: Create a global interdisciplinary network of Development Labs to solve distinct development challenges 0in1: Total dollar value of outside (non-USAID) resources utilized $46,880,902 $44,554,295 $55,416,741 0in2: # transformative innovations, technologies, or approaches that were developed with human, financial, or institutional resources contributed by HESN Development Labs 210 210 148 0in3: # of transformative innovations, technologies, or approaches that were piloted with human, financial, or institutional resources contributed by HESN Development Labs 80 71 94 0in4: # of transformative innovations, technologies, or approaches that were adopted with human, financial, or institutional resources contributed by HESN Development Labs 21 18 24 0in5: # of transformative innovations, technologies, or approaches that achieved wide-scale adoption with human, financial, or institutional resources contributed by HESN Development Labs 23 12 6 0in6: # of transformative innovations, technologies, or approaches evaluated with human, financial, or institutional resources contributed by HESN Development Labs * 41 31 26 0in7: # of US students via HESN partners serving as fellows in developing countries (for more than one month) * 386 355 237 0in8: # of innovations, technologies, or approaches in the innovation pipeline 518 293 149 0in9: # of innovations, technologies, or approaches that completed at least one of the five stages in the innovation pipeline 320 3918 45 14 Only indicators reported in the PITO Report were included in the analysis of the HESN Mid-term Performance Evaluation indicator analysis 15 Total of actuals for all Labs reporting any given year, not only those with targets, aggregated across all three years: 2013 – 2015. The values reported annually are assumed to be new counts, not cumulative total to date. 16 Total of actuals for Labs reporting both actuals and targets any given year, aggregated across all three years: 2013 – 2015. The values reported annually are assumed to be new counts, not cumulative total to date. 17 Total of targets for Labs reporting both actuals and targets any given year, aggregated across all three years: 2013 – 2015. The values reported annually are assumed to be new counts, not cumulative total to date. 18 The large discrepancy in total actuals vs. actuals for Labs with targets is primarily due to IDIN and DIL not reporting targets Performance on Indicators Relative to Targets 44 HESN Indicators14 An * denotes no reported values for Year 1 (2013), assumption that the indicator was not tracked in Year 1 Overall Total15 Total for Labs with Targets16 Target17 0in10: # of beneficiaries reached 4,177,440 825,66619 358,350 0in11: # of innovations, technologies, or approaches that have reached more than 1 million people * 5 2 2 0in12: # of innovations, technologies, or approaches that have reached more than 5 million people * 2 1 1 Objective 1: Improve data quality, access, and analytics to advance evidence-based development decision making 1.0in1: # of new data-related technologies, tools, approaches, and best practices supported or applied with human, financial, or institutional resources contributed by HESN Development Labs 125 81 67 1.0in2: # of data sets provided to or made accessible to USAID operating units and programs, HESN partners, and the broader development community with human, financial, or institutional resources contributed by HESN Development Labs 265 264 51 1.0in2_new: # of new data sets provided to or made accessible to USAID operating units and programs, HESN partners, and the broader development community with human, financial, or institutional resources contributed by HESN Development Labs * 22 N/A20 N/A 1.0in3: # of data-related analyses, mapping activities, and expert consultations provided for USAID operating units and programs, HESN partners, and the broader development community with human, financial, or institutional resources contributed by HESN Development Labs 478 470 126 1.1 Expand the availability and improve the quality of development data 1.1in1: # of citations in targeted fora/ publications/ projects of data collected or made available through human, financial, or institutional resources contributed by HESN Development Labs N/A21 N/A N/A 1.2 Create and improve data-driven methodologies, tools, and analytics 1.2in1: # of users who access data and tools made available with support from human, financial, or institutional resources contributed by HESN Development Labs 801,588 798,947 492,945 1.3 Build a development ecosystem that applies to data, analytics, and evidence to drive solutions and improve decision making 19 The large discrepancy in total actuals vs. actuals for Labs with targets is primarily due to DIL and IDIN not reporting targets 20 No Labs reported both actuals and targets for any given year for this indicator 21 No actuals or targets were reported for any year for this indicator Performance on Indicators Relative to Targets 45 HESN Indicators14 An * denotes no reported values for Year 1 (2013), assumption that the indicator was not tracked in Year 1 Overall Total15 Total for Labs with Targets16 Target17 1.3in1: # of USAID operating units using geographic analysis to prepare strategies and design, implement, monitor, and evaluate development projects * 12 12 22 1.3in2: # of development professionals proficient in data management and use due to human, financial, or institutional resources contributed by HESN Development Labs 1,645 1,645 440 Objective 2: Accelerate the creation, testing, and scaling of transformative innovations, technologies, and approaches 2.0in1: # months required for developing, piloting, adopting, scaling, and evaluating transformative innovations, technologies, and approaches receiving human, financial, or institutional resources contributed by HESN Development Labs * N/A22 N/A N/A 2.1 Expand the research, identification, and design of transformative innovations, technologies, and approaches No indicators reported - - - 2.2 Increase assessment, analysis, and evaluation of innovations, technologies, and approaches 2.2in1: # of white papers, articles, assessments, analyses, and evaluations on development challenges, innovations, technologies, approaches, and contexts (drafted with human, financial, or institutional resources contributed by HESN Developments Labs) published in targeted fora and publications OR provided to USAID operating units, HESN partners, and the broader development community 200 200 213 2.2in2: # of citations of white papers, articles, assessments, analyses, and evaluations (drafted with human, financial, or institutional resources contributed by HESN Developments Labs) on development challenges, innovations, technologies, approaches, and contexts in targeted fora/publications/projects * 17 17 32 2.2in3: # of targeted communities who participated in assessment, analysis, and evaluation of innovations, technologies, and approaches supported with HESN Development Lab 694 694 266 2.3 Foster and expand collaborations among private and public sector actors and local communities 2.3in1: # of MOUs or other agreements signed with public sector, private sector, local community partners, and one HESN Development Lab 259 222 152 2.3in2: # of stakeholders engaged in problem solving with one HESN Development Lab 1,205 1,205 1,010 22 This indicator does not lend itself to being aggregated across Labs and years Performance on Indicators Relative to Targets 46 HESN Indicators14 An * denotes no reported values for Year 1 (2013), assumption that the indicator was not tracked in Year 1 Overall Total15 Total for Labs with Targets16 Target17 2.4 Build network members’ mutual capacity for high-risk development, testing and implementation of solutions No indicators reported - - - Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity 3.0in2: # of MOUs or other agreements signed with public sector, private sector, and local community partners and more than one HESN Development Lab * 9 4 4 3.0in3: # new development related classes or disciplines created by university departments with human, financial, or institutional resources contributed by HESN Development Labs 60 47 36 3.1 Build and support an infrastructure for collaborative problem-solving among HESN Labs and USAID 3.1in1: # of development programs/projects/efforts undertaken collaboratively by Network members 56 56 30 3.2 Catalyze ongoing learning and knowledge sharing among HESN Labs and USAID 3.2in1: # visitors to Network knowledge-sharing platforms 964,741 957,149 672,300 3.2in3: # successes and failures circulated on Network knowledge-sharing platforms * 11 11 4 3.3 Create new disciplines, collaborative platforms, and learning opportunities that train students, staff, and faculty to solve development challenges 3.3in1: # of classes supported by HESN Development Labs with human, financial, or institutional resources contributed by HESN Development Labs 132 120 88 3.3in2: # of collaborative platforms created by the HESN or with human, financial, or institutional resources contributed by HESN Development Labs 111 60 42 3.4 Engage students, staff, and faculty in solving distinct development challenges 3.4in1: # of students participating in short term practica or other field experiences through human, financial, or institutional resources contributed by HESN Development Labs * 571 479 3,502 3.4in2: # of Hubs created with human, financial, or institutional resources contributed by HESN Development Labs 36 33 26 3.4in3: # of participants in Hubs, summits, and other problem-solving institutions created with human, financial, or institutional resources contributed by HESN Development Labs 347,732 347,544 142,380 3.4in4: # of participants in crowd-sourcing or other open challenges created with human, financial, or institutional resources contributed by HESN Development Labs 4,211 4,200 1,340 Percent of HESN Labs Meeting Indicator Targets By Year 47 ANNEX VII: PERCENT OF HESN LABS MEETING TARGET BY YEAR 2013 2014 2015 Indicator # Labs with actuals and targets # Labs met targets % Labs meeting targets # Labs with actuals and targets # Labs met targets % Labs meeting targets # Labs with actuals and targets # Labs met targets % Labs meeting targets Goal – level indicators 0in01 5 2 40 8 4 50 8 5 63 0in02 3 2 67 5 2 40 4 2 50 0in03 1 1 100 3 1 33 4 1 25 0in04 1 1 100 1 0 0 1 0 0 0in05 0 0 1 0 0 1 1 100 0in06 0 0 1 1 100 3 2 67 0in07 4 4 100 8 5 63 7 4 57 0in08 2 1 50 2 2 100 6 5 83 0in09 2 1 50 2 1 50 4 1 25 0in10 1 0 0 1 0 0 4 2 50 0in11 0 0 0 0 1 1 100 0in12 0 0 0 0 1 1 100 Objective 1 indicators 1.0in1 3 3 100 4 3 75 2 1 50 1.0in2 2 2 100 4 4 100 5 3 60 1.0in2 new 0 0 0 0 0 0 1.0in3 1 1 100 4 4 100 4 4 100 1.2in1 2 2 100 4 2 50 4 4 100 1.3in1 0 0 1 0 0 1 0 0 1.3in2 1 1 100 3 3 100 3 3 100 Objective 2 indicators 2.0in1 0 0 3 1 33 4 2 50 2.2in1 1 1 100 4 1 25 7 5 71 2.2in2 0 0 2 1 50 2 1 50 2.2in3 1 1 100 3 2 67 3 3 100 2.3in1 2 0 0 5 2 40 6 6 100 2.3in2 5 4 80 7 2 29 7 5 71 Objective 3 indicators 3.0in2 0 0 0 0 1 1 100 3.0in3 2 1 50 4 3 75 5 5 100 3.1in1 1 1 100 3 3 100 3 2 67 3.2in1 2 2 100 6 4 67 6 6 100 Percent of HESN Labs Meeting Indicator Targets By Year 48 2013 2014 2015 Indicator # Labs with actuals and targets # Labs met targets % Labs meeting targets # Labs with actuals and targets # Labs met targets % Labs meeting targets # Labs with actuals and targets # Labs met targets % Labs meeting targets 3.2in3 0 0 1 1 100 1 1 100 3.3in1 2 2 100 4 4 100 6 4 67 3.3in2 2 2 100 5 4 80 1 1 100 3.4in1 3 2 67 7 3 43 7 3 43 3.4in2 1 1 100 3 2 66.67 2 2 100 3.4in3 1 1 100 7 6 85.71 7 7 100 3.4in4 1 1 100 3 3 100 3 3 100 Avg % target met 81 60 73 Graphs of HESN Indicator Results by Year 49 ANNEX VIII: GRAPHS OF HESN INDICATOR RESULTS BY YEAR Goal: Create a global interdisciplinary network of Development Labs to solve distinct development challenges 0 5000000 10000000 15000000 20000000 25000000 30000000 HESN_0in01: $ Total dollar value outside (non-USAID) resourced utilized 2012 2013 2014 2015 2016 0 1 2 3 4 5 6 7 8 9 10 2012 2013 2014 2015 2016 HESN_0in04: # transformative innovations, technologies, or approaches that achieved early adoption 0 10 20 30 40 50 60 70 80 90 2012 2013 2014 2015 2016 HESN_0in02: # transformative innovations, technologies, or approaches that were developed 100 110 120 0 5 10 15 20 25 30 35 40 2012 2013 2014 2015 2016 HESN_0in03: # transformative innovations, technologies, or approaches that were initially piloted Graphs of HESN Indicator Results by Year 50 -5 0 5 10 15 20 25 2012 2013 2014 2015 2016 HESN_0in05: # transformative innovations, technologies, or approaches that transitioned to scale 0 5 10 15 20 25 2012 2013 2014 2015 2016 HESN_0in06: # transformative innovations, technologies, or approaches evaluated 0 50 100 150 200 250 2012 2013 2014 2015 2016 HESN_0in07(Aggregated): # US students serving as fellows 0 20 40 60 80 2012 2013 2014 2015 2016 HESN_0in08: # innovations, technologies or approaches in the innovation pipeline 100 120 140 160 180 200 220 240 260 280 300 320 340 Graphs of HESN Indicator Results by Year 51 0 20 40 60 80 100 120 140 160 180 200 2012 2013 2014 2015 2016 HESN_0in09: # innovations, technologies or approaches that compoleted at least one of the five stages in the innovation pipeline 0 250000 500000 750000 1000000 1250000 1500000 1750000 2000000 2250000 2500000 2750000 3000000 2012.5 2013 2013.5 2014 2014.5 2015 2015.5 HESN_0in10: # beneficiaries reached 0 0.5 1 1.5 2 2.5 3 3.5 2012 2013 2014 2015 2016 HESN_0in11: # innovations, technologies, or approaches that have reached more than 1 million people 0 0.2 0.4 0.6 0.8 1 1.2 2012 2013 2014 2015 2016 HESN_0in12: # innovations, technologies, or approaches that have reached more than 5 million people Graphs of HESN Indicator Results by Year 52 Objective 1: Improve data quality, access, and analytics to advance evidence-based development decision making 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 2012 2013 2014 2015 2016 HESN_1.0in1: # new data-related technologies, tools, approaches, and best practices 0 20 40 60 80 100 120 140 160 180 200 2012 2013 2014 2015 2016 HESN_1.0in2: # data sets provided to or made accessible -5 0 5 10 15 20 25 2012 2013 2014 2015 2016 HESN_1.0in2_new: # new data sets provided to or made accessible 0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 2012 2013 2014 2015 2016 HESN_1.0in3: # data-related analyses, mapping activities, and expert consultations provided Graphs of HESN Indicator Results by Year 53 0 50000 100000 150000 200000 250000 300000 350000 400000 450000 500000 2012 2013 2014 2015 2016 HESN_1.2in1: # users who access data and tools made available -2 0 2 4 6 8 10 12 2012 2013 2014 2015 2016 HESN_1.3in1: # USAID operating units using geographic analysis to prepare strategies and design, implement, monitor, and evaluate development projects 0 2012 2013 2014 2015 2016 HESN_1.3in2: # development professionals proficient in data management and use 100 200 300 400 500 600 700 Graphs of HESN Indicator Results by Year 54 Objective 2: Accelerate the creation, testing, and scaling of transformative innovations, technologies, and approaches 0 5 10 15 20 25 30 35 40 45 2012 2013 2014 2015 2016 HESN_2.0in1: # months required for developing, piloting, scaling, and evaluating transformative innovations, technologies, and approaches - average 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 2012 2013 2014 2015 2016 HESN_2.2in1: # white papers, articles, assessments, analyses, and evaluations published or provided 0 2 4 6 8 10 12 2012 2013 2014 2015 2016 HESN_2.2in2: # citations of white papers, articles, assessments, analyses and evalutions 0 50 2012 2013 2014 2015 2016 HESN_2.2in3: # targeted communities who participated in assessment, analysis, and evaluation 100 150 200 250 300 350 400 450 500 Graphs of HESN Indicator Results by Year 55 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 2012 2013 2014 2015 2016 HESN_2.3in1: #MOUs or other agreements signed with public sector, private sector, local community partners and one HESN Development Lab 0 50 100 150 200 250 300 350 400 450 500 550 600 650 2012 2013 2014 2015 2016 HESN_2.3in2: # stakeholders engaged in problem solving with one HESN Development Lab Objective 3: Catalyze a global interdisciplinary ecosystem of individuals and institutions that shares knowledge, promotes learning, and builds mutual capacity 0 1 2 3 4 5 6 7 8 9 2012 2013 2014 2015 2016 HESN_3.0in2: # MOUs or other agreements signed with public sector, private sector, and local community partners and more than one HESN Lab 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 2012 2013 2014 2015 2016 HESN_3.0in3: # of new development related classes/disciplines created by university departments Graphs of HESN Indicator Results by Year 56 0 5 10 15 20 25 30 35 40 2012 2013 2014 2015 2016 HESN_3.1in1: #development programs/projects/efforts undertaken collaboratively by Network members 0 2012 2013 2014 2015 2016 HESN_3.2in1: # visitors to Network knowledge-sharing platforms 100000 200000 300000 400000 500000 600000 700000 0 1 2 3 4 5 6 7 2012 2013 2014 2015 2016 HESN_3.2in3: # successes and failures circulated on Network knowledge-sharing platforms 0 10 20 30 40 50 60 70 80 2012 2013 2014 2015 2016 HESN_3.3in1: # classes supported by HESN Development Labs Graphs of HESN Indicator Results by Year 57 0 5 10 15 20 25 30 35 40 45 50 55 60 2012 2013 2014 2015 2016 HESN_3.3in2: # collaborative platforms created by the HESN 0 50 100 150 200 250 300 350 2012 2013 2014 2015 2016 HESN_3.4in1(Aggregated): # students participating in short term practica or other field experiences 0 2 4 6 8 10 12 14 16 18 20 2012 2013 2014 2015 2016 HESN_3.4in2: # of hubs created 0 25000 50000 75000 100000 125000 150000 175000 200000 225000 250000 275000 300000 325000 350000 2012 2013 2014 2015 2016 HESN_3.4in3: # participants in hubs, summits, other problem-solving institutions Graphs of HESN Indicator Results by Year 58 0 500 1000 1500 2000 2500 3000 3500 2012 2013 2014 2015 2016 HESN_3.4in4: # participants in crowd-sourcing or other open challenges HESN Lab Canvases 59 ANNEX IX: HESN LAB CANVASES AidData Beneficiary Segments:  USAID Program Officers  USAID GIS Specialists  Government Aid Management Unit Staff  Civil Society  Research AidData: USAID Program Officers Value Proposition:  Geocoded Aid Information: AidData provides rigorous, relevant, and spatially precise data on aid programs; Gives better sense of where a donor’s development projects are located vis-à-vis subnational measures of need and opportunity and projects funded by other donors; Enables new forms of analysis (e.g. targeting efficiency analysis that can inform program design) that would otherwise not be possible to undertake.  Introduction to new methods of analysis and evaluation: e.g. Geospatial Impact Evaluation and Geospatial Value-for-Money Analysis. Enables USAID program officers to evaluate programs that otherwise may not be subject to a rigorous evaluation or to do so at a lower time or financial cost than a traditional, randomized control trial.  Data Analytic Support: Capacity support from Data Analytics team (remote) and fellows (in office); Enables USAID program officers to undertake analysis that they otherwise wouldn't have time to complete.  Collaboration with World-Class Researchers from University Sector: Connect program officers with the best scientists in their respective fields through a process with far lower transaction costs; Enables USAID program officers to access researchers who would otherwise not be accessible (because of the high barriers to entry and opportunity costs associated with pursuing collaboration through traditional USAID contracting vehicles). Primary recipient of services/activities:  USAID Program Officer: Manage a program or multiple programs at USAID. o Want to know if their programs are effectively allocated and achieving their desired impact. o Need evidence of programmatic impact. o Want to design and target new programs effectively. o Need analysis to make evidence-based decisions regarding how to target aid. o Want to coordinate with other donors to use limited resources effectively. Secondary beneficiary:  Citizens of aid recipient country: target beneficiaries of programs. o Want aid programs that meet their needs. o Want their community to receive an equitable distribution of resources (eg. no funding gaps) o Want transparency of donor activities to hold accountable for service delivery.  Government Line Ministries and Other Donors: Implement Development Activities. o Use this information to inform own resource planning to identify complementarities and reduce duplication of efforts.  US Government Stakeholders / Policy Makers HESN Lab Canvases 60 o Want to ensure that taxpayer or government resources are being used by implementation agency to best effect. Key activities/products/services:  Geocoded Aid Information: Provides comprehensive and spatially precise information on donor-funded programs, and the programs of other donors working in similar sectors/ areas.  Data Analysis Response Team  AidData Summer Fellows  Aid Management Fellows: Support provision of geocoded aid information and training of USAID staff on data reporting and use.  Research Collaboration  Strategic Response Innovations: Create custom technology to enable officers to access and use relevant geospatial data. Partners:  WM UT-Austin DG BYU Esri: Geocoded Data Production  Partner Governments: Geocoded Data Production  HESN Staff: Help broker connections with USAID Missions  Other USAID Units: To co-create research  Researcher Universities: (eg. Duke, Yale, LSE, Pennsylvania State University, UT-Austin, American, BYU, etc.) Delivery channels:  HESN Technical Convening  HESN staff  AidData Website: Access geocoded aid data  AidData Staff: Provide analysis, new methodologies, connect with researchers  Webinars  Working paper series/published research  AidData Summer Fellows and Aid Management Fellows Impacts:  USAID Buy-Ins Cost Amount - ~2.5 M  HESN Funding  Cost Share  Uptake of new methodologies for wider adoption AidData: USAID GIS Specialists Value Proposition:  Geocoded Aid Data: AidData provides rigorous, relevant, and spatially precise data on aid programs; Gives better sense of where a donor’s development projects are located vis-à-vis subnational measures of need and opportunity and projects funded by other donors; Enables new forms of analysis (e.g. targeting efficiency analysis that can inform program design) that would otherwise not be possible to undertake.; Previously, may not have had as spatially precise data on own activities or any data on the activities of other donors; Key informational input for data analysis (have to spend less time finding data); Enables more precise analysis.  AidData Summer Fellow: Expands bandwidth of GIS Specialist; Helps raise awareness of GIS Specialist work within the Mission and reduce feeling of isolation.  Spatial Analysis Support: Capacity support from Data Analytics team (remote) and fellows (in office); Enables them to undertake analysis that they otherwise wouldn't have time to complete.  New Evaluation Methods: Geospatial Impact Evaluation (GIE) and Geospatial Value-for-Money methodologies (Geo-VfM) enable GIS specialist to evaluate the impacts and cost effectiveness of spatially distributed programs.  Data Reporting Support: Enable GIS Specialist to more easily report data to government aid management system; Help improve quality of data reported through support from Aid Management Fellows. Primary recipient of services/activities: Secondary beneficiary: HESN Lab Canvases 61  USAID Missions – GIS Specialist: o Manage Mission geospatial data systems. o Collect data on Mission programs and report to government aid management system. o Liaise with government and other actors to collect geospatial indicator data. o Produce analysis at the request of program officers and technical units to inform Mission programs and strategies.  USAID program officers and technical units: design Mission programs and strategies. o Receive improved spatial analysis outputs to inform project/ strategy design. o More effectively identify impact of programs through GIEs.  Citizens of aid recipient country: target beneficiaries of programs. o Want aid programs that meet their needs. o Want community to receive equitable distribution of resources o Want transparency of donor activities to hold accountable for service delivery. Key activities/products/services:  Geocoding production and quality assurance  Provide technical assistance and training  Preparing evaluations  Data Analysis  Software Development  AidData Summer Fellows  Aid Management Fellows Partners:  Researcher Universities: (eg. Duke, Yale, LSE, Pennsylvania State University, UT-Austin, American, BYU, etc.)WM UT-Austin DG BYU Esri: Geocoded Data Production  Partner Governments: Geocoded Data Production  HESN Staff: Help broker connections with USAID Missions Delivery channels:  HESN Technical Convening  HESN staff  AidData Website: Access geocoded aid data  AidData Staff: Provide analysis, new methodologies, connect with researchers  Working Paper Series  Evaluation Studies  Webinars  GeoCenter GIS Specialists Training Impacts:  USAID Buy-In: Cost Amount - ~2.5 M  Cost Sharing  HESN Funding AidData: Government Aid Management Unit Staff Value Proposition:  Geocoded Aid Information: AidData provides rigorous, relevant, and spatially precise data on aid programs; Gives better sense of where a donor’s development projects are located vis-à-vis subnational measures of need and opportunity and projects funded by other donors; Enables new forms of analysis (e.g. targeting efficiency analysis that can inform program design) that would otherwise not be possible to undertake  Mentorship and Training: Work side-by-side with government counterparts to collect, manage and analyze geocoded aid information, and integrate into existing systems and processes. Provide extensive HESN Lab Canvases 62 systems and geospatial data collection and analysis training. Integrate geospatial data into ministry’s existing data management plan.  Data Analytic Support: Capacity support from Data Analytics team (remote) and fellows (in office); Enables them to undertake analysis that they otherwise wouldn't have time to complete.  Data Visualization: Training in data visualization helps analyst make more compelling analysis, get attention of boss and policy-makers. Communicate value of aid information.  Public Aid Information Management System (AIMS) Portal: Enables the public to access aid information. Gains public attention for their work and generates demand for aid information.  AIMS GIS Module: GIS Module enables easy visualization of geocoded aid information in the AIMS; Enables the government to more easily share data with others and communicate the value of data; Facilitates analysis of geocoded data by government. Primary recipient of services/activities:  Aid Management Unit: Analyst and management-level staff in the local government's Aid Management Unit (typically within a Ministry of Finance or Planning). o Mandated to collect and manage information on all donor-funded aid activities. o Want to use information for planning and coordination with line ministries and donors o Expect donors to provide precise and comprehensive information. o Want to provide high-quality aid information to stakeholders. o Want to reduce transaction costs associated with data collection and sharing, and stimulate public demand for aid information. Secondary beneficiary:  Donor Partner Program Office, Technical, and GIS Staff: Design strategies and programs. o Gain access to more precise aid information on own activities and those of other donors.  Civil Society Staff: Analyze aid allocation and outcomes. o Gain access to precise aid information that often was not previously publicly available. Key activities/products/services:  Geocoded aid information: Informs understanding of geographical spread and concentration of donor programs in various sectors, compared to need, and compared to government investments. Perform quality assurance on data.  Aid Management Fellows: Provide training and technical assistance to government counterparts to strengthen data management and analysis skills.  AIMS GIS Module and Public Portals: Provide public access to aid information. Facilitate analysis of data. Partners:  Development Gateway: Help with geocoded data production, outreach. Existing Aid Management Program facilitates connections with partners.  Esri: Provide GIS software  WM, UT, BYU: Support geocoding work, students serve as AidData summer fellows, lead outreach and training, provide cost share. Delivery channels:  AIMS GIS Module and Public Portal: Allow for public access to data  AidData Website: Access geocoded aid data  AidData Staff: Provide analysis, new methodologies  Aid Management Fellows: Provide training, technical assistance, outreach Impacts:  HESN funding  Cost Share  Government funding  External Funding for AMPs AidData: Civil Society Value Proposition:  Geocoded Aid Information: Provide public access to rigorous, relevant, and spatially precise information on aid projects in their country; Often information was not previously public; Also, provide more detailed information on project locations that enable analysts to undertake new forms of analysis HESN Lab Canvases 63 (e.g. targeting efficiency analysis that can inform program design) that would otherwise not be possible to undertake  Training: Provide training in collection, management and analysis of geocoded aid information.  AidData Summer Fellows: Provide training and technical assistance to enable use of geocoded aid data by CSOs. Expand organizational bandwidth.  GIS Software: Provide free GIS software from Esri and open source software tools (such as QGIS or Tabula).  Dissemination: AidData uses blog, working paper series, and social media to disseminate the work of our CSO partners.  Data Visualization: Training in data visualization helps analyst make more compelling analysis, get attention of boss and policy-makers. Primary recipient of services/activities:  Civil Society Analyst: Analyst at a civil society organization (university, think tank, CSO) that produces analysis that aims to influence development policy. Job may include conducting research on development issues, analyzing data, preparing reports, disseminating to policy makers. o Want to be able to leverage open data to undertake compelling, evidence-based advocacy activities. o Want to use geocoded aid information to hold donors and government accountable for service delivery. Secondary beneficiary:  Citizens of aid recipient country: target beneficiaries of aid programs. o Want to be able to effectively advocate for aid programs that meet their needs and equitable allocation of resources. o Want to hold donors and government accountable for service delivery. Key activities/products/services:  Geocoded data production and quality assurance.  AidData Summer Fellows: Embed with CSOs for 10 weeks during summers  Training in geocoded data. management, collection, and analysis  Produce Research/Analysis: Work collaboratively with CSO partners to produce research and analysis using geocoded data.  Outreach Events: Hackathons and data working groups support outreach around data use.  Communications and Dissemination: Provide platform for our partners’ work through working paper series, blog social media. Partners:  Development Gateway: Help with geocoded data production, outreach.  Esri: Provide GIS software.  CSO partners: Help AidData build network of CSO partners in country.  WM, UT, BYU: Support geocoding work, students serve as AidData summer fellows, lead outreach and training, provide cost share.  Other Universities: Students serve as summer fellows and provide cost share. Delivery channels:  AidData Summer Fellows: Provide Trainings  AidData Website: Provide access to geocoded data; disseminate collaborative work  AIMS GIS Module and Public AIMS Portal: Allow for public access to data  AidData Staff: Provide analysis, new methodologies  Social Media: Put out call for summer fellow applications; disseminate collaborative work  Professional Events: Share data, trainings, collaborative research findings Impacts: HESN Lab Canvases 64  University Cost Share  Private Donations  HESN Funding AidData: Research Value Proposition:  Geocoded data sets provide researchers with quality information on aid flows at the sub-national level.  AidData’s spatial data repository provides a key public good to the research community through processing vast amounts of spatial data with global or near-global coverage to facilitate easy analysis.  Policy Impact: Broker relationships between researchers and USAID units to co-create projects. Disseminate findings to policy community through events and workshops.  Funding: Provide direct funding through research RFA competition and workshop funding application and indirectly by brokering relationships with USAID units that buy-into award to fund research  Dissemination: Through working paper series, research workshops and conferences supported by AidData, and policy-events hosted by AidData  AidData supports the development of innovative research methodologies using spatial data- including geospatial impact evaluations and geospatial value for money analysis. Primary recipient of services/activities:  Researcher (university, think tank, CSO) that produces analysis that aims to understand development impact. Job may include conducting research on development issues, analyzing data, preparing reports, disseminating to policy makers. o Want to be able to leverage open data to undertake compelling, evidence-based advocacy activities. o Want to use geocoded aid information to understand effectiveness of donors and government accountable for service delivery. o Want research to influence aid policies and outcomes. Secondary beneficiary:  USAID Program Officer: Manage a program or multiple programs at USAID. o Want to know if their programs are effectively allocated and achieving their desired impact. o Need evidence of programmatic impact. o Want to design and target new programs effectively. o Need analysis to make evidence-based decisions regarding how to target aid. o Want to coordinate with other donors to use limited resources effectively.  Government Stakeholders / Policymakers o Want to ensure that taxpayer or government resources are being used by implementation agency to best effect. Rigorous evaluation helps to assess impact and improve future investments. Key activities/products/services:  Data Provision: Geocoded data production and quality assurance.  Research Funding: Research competition, workshop funding, and brokered relationships with USAID units  Research Conferences and Workshops  Communications and Dissemination: Provide platform for our partners’ work through working paper series, blog social media.  Spatial Data Repository: Powered by W&M’s high computing cluster  AidData Staff: or outreach to researchers, administration of research awards, brokering relationships between Partners:  AidData Research Consortium  DG: Support production of geocoded data sets.  WM, UT, BYU: Support geocoding work, students serve as AidData summer fellows, lead outreach and training, provide cost share.  USAID Missions/Units: To co-create research, provide research funding HESN Lab Canvases 65 researchers and USAID units, and disseminating findings Delivery channels:  AidData Website: Provide access to geocoded data; disseminate collaborative work  AidData Staff: Provide analysis, new methodologies, dissemination, relationship building  HESN Staff: Help broker relationships with USAID units  Workshops, Conferences, Policy Events: Share data, trainings, collaborative research findings Impacts:  USAID Buy-In: Cost Amount - ~2.5 M  University Cost Share  Private Donations  HESN Funding HESN Lab Canvases 66 CITE Beneficiary Segments:  Consumers at bottom of the pyramid (Anticipated)  Faculty and Students  Policy Makers (Anticipated)  Development Practitioners  Manufacturers  Distributers  Retailers  Entrepreneurs/Product Designers CITE: Consumers at the bottom of the pyramid (Anticipated) Value Proposition  Provide data and information to enable people living in poverty to make better informed decisions regarding which technologies to purchase, so that they can select products that best meet their needs and improve their livelihoods.  Train people living in poverty on the importance of evaluation and the use of tools and methodologies/approaches to enable people living in poverty to carry out evaluations and potentially generate income. Primary recipient of services/activities:  Anticipated beneficiaries: People living in poverty in emerging markets such as the Self Employed Women's Association members in India Secondary beneficiary: Downstream user/beneficiaries Activities  Implement product evaluations and provide information in the style of a consumer report  Develop and share training materials, evaluation tools and methodologies  Lead trainings  Develop an effective communication and dissemination strategy so that information reaches people living in poverty  Develop a business model and implementation plan for the development of evaluation centers to train people living in poverty and carry out evaluations Partners: Who helps deliver/inform/etc. on this Value Prop  International NGOs; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies;  In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries:  Networks and direct personal contacts of CITE faculty, staff, and students  Focus groups and interviews in the field  Indirectly through communication with NGO’s, donor agencies, and other development practitioners via: o Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more. o Social media – CITE has continued to grow its Twitter following by targeting key NGOs, USAID missions, universities, and other USAID Global Development Labs with its follows and tagged tweets. o CITE ‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more. o CITE’s request for proposals which is distributed via CITE’s email list and also by USAID’s GDL Impacts: HESN Lab Canvases 67 Funding/revenue/reach/influence/specific gains:  Informed choice of technologies from those in the market leading to (i) more cost-effective spending by the consumer and (ii) improved standard of living  Potential increase in revenue generation through (i) increased productivity due to use of appropriate technology for work and (ii) access to skills that can be used to work CITE: Faculty and Students Value Proposition  Provide data, insights, methodologies/approaches and tools to guide further research and generate questions surrounding evaluation of technology products in the developing world context.  Collaborate on technology evaluation research in emerging markets to generate and publish data and information for relevant stakeholders  Educate the next generation of development practitioners through course development and experiential learning opportunities to develop their skills, knowledge and attitudes and work on relevant research projects Primary recipient of services/activities:  Current Beneficiaries: MIT undergraduates and graduate students; students at Makerere University and the Indian Institute of Management – Ahmedabad (IIM-A); Prof Ankur Sarin (IIM-A), D-Lab and the Tata Center at MIT.  Anticipated beneficiaries: Faculty, staff at academic institutions and students Secondary beneficiary: Downstream user/beneficiaries Activities  Publish research, data, insights, methodologies/approaches and tools  Create and maintain knowledge sharing platform  Collaborate on implementation of evaluations and development of methodologies/approach  Develop a network of collaborators and seek opportunities for future funding  Develop courses and supervise students Partners: Who helps deliver/inform/etc. on this Value Prop:  International NGOs; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries:  Networks and clubs within MIT and HESN and partnerships with international universities  CITE seminar and courses  Field opportunities as a CITE RA or intern  Presence at academic conferences and events  Academic publications  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more.  Social media – CITE has continued to grow its Twitter following including universities, and other academic institutions  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more.  CITE’s request for proposals which is distributed via CITE’s email list and also by USAID’s GDL  CITE reports HESN Lab Canvases 68 Impacts: Funding/revenue/reach/influence/specific gains:  Acquisition of data and new knowledge  Funding for further work in this area  Next generation of appropriately skilled global development practitioners, thought leaders and researchers CITE: Policy makers (Anticipated) Value Proposition  Provide access to user-centered data and insights to help policy makers to educate consumers and develop informed policy around products designed to reduce poverty Primary recipient of services/activities:  Anticipated beneficiaries: National and International Government such as a state government in India Secondary beneficiary: Downstream user/beneficiaries Activities  Implement product evaluations and publish reports and the results of CITE’s work  Develop an effective communications and dissemination strategy so that information reaches policy makers (includes creation and maintenance of a knowledge sharing platform) Partners: Who helps deliver/inform/etc. on this Value Prop  International NGOs; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  CITE reports  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more. CITE seeks to identify appropriate individuals who influence policy within government to add to its email list.  Social media – CITE has continued to grow its Twitter following and aims to include policy makers  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more.  CITE’s request for proposals which is distributed to USAID Missions via CITE’s email list and also by USAID’s GDL  Academic publications Impacts: Funding/revenue/reach/influence/specific gains:  Influence and education around products designed to reduce poverty CITE: Development Practitioners - Donor Agencies and Procurement Professionals Value Propositions  Provide them with information and data on product characteristics to make appropriate choices when procuring large quantities of technologies to deploy in the developing world  Provide expertise and the opportunity for collaboration to solve “real-world” development problems associated with appropriate choice of technology and scaling HESN Lab Canvases 69  Train practitioners on the importance of evaluation, tools and methodologies/approaches to enable them to carry out evaluations Primary recipient of services/activities:  Current beneficiaries: Organizations like Mercy Corps  Anticipated beneficiaries: Donor agencies such as the World Food Program, International Society of Wheelchair Providers Secondary beneficiary: Activities  Implement product evaluations and provide information in the style of consumer reports, models and insights  Develop and share methodologies and tools to facilitate evaluation of relevant products  Collaborate with development organizations – engagement as innovation  Develop an effective communications and dissemination strategy to ensure information reaches the development practitioners (includes creation and maintenance of a knowledge sharing platform)  Develop and share training materials, evaluation tools and methodologies  Lead trainings Partners:  International NGO's; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries:  CITE reports  Presence at conferences and events  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more.  Social media – CITE has continued to grow its Twitter following and aims to include development practitioners specifically  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more.  CITE’s request for proposals which is distributed via CITE’s email list and also by USAID’s GDL Impacts: Funding/revenue/reach/influence/specific gains Funding/revenue/reach/influence/specific gains:  Informed choice of technologies from those in the market leading to (i) more efficient use of capital and (ii) sustainable use of technologies deployed in the developing world  New knowledge and data  Practitioners equipped with the appropriate skills and knowledge to carry out evaluations and an appreciation of the importance and relevance of evaluation CITE: Manufacturers Value Proposition  Provide access to critical information about gaps in the market, user needs and scaling which would allow manufacturers to produce a product that better meets the needs of the user and reaches scale  Provide access to tools for evaluation to be used by the manufacturers from the earliest design stage, such that they can evaluate their own products Primary recipient of services/activities: Secondary beneficiary: Downstream user/beneficiaries HESN Lab Canvases 70  Anticipated beneficiaries: Broad range of manufacturers such as Greenlight Planet, Scale-Ups fellows, or Practical Impact Alliance members interested in emerging markets Activities  Implement product evaluations and provide user centered data to drive informed product design, business model development and operational decisions  Develop and share methodology and tools to carry out evaluation of their own products  Develop an effective communication and dissemination strategy so that information reaches manufacturers (including creation and maintenance of a knowledge sharing platform) Partners:  International NGO's; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries:  CITE reports  Interaction with networks such as the Practical Impact Alliance at D-Lab and presence at events such as Scaling Development Ventures  Interviews and surveys  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more.  Social media – CITE has continued to grow its Twitter following and aims to include manufacturers as part of its audience  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more.  CITE’s request for proposals which is distributed via CITE’s email list and also by USAID’s GDL Impacts: Funding/revenue/reach/influence/specific gains:  Products better designed to meet the needs of the poor from a suitability, scalability and sustainability perspective CITE: Distributors Value Proposition  Provide access to critical information about gaps in the market and barriers to scale which would allow them to distribute products that better meet the needs of the user Primary recipient of services/activities:  Current beneficiaries: Solar Sister; World Food Program  Anticipated: broad range of organizations such as the World Food Program and companies interested in entering new emerging markets Secondary beneficiary: Downstream user/beneficiaries Activities  Implement product evaluations and provide information on products and supply chains  Develop an effective communications and dissemination strategy so that information Partners:  International NGO's; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, HESN Lab Canvases 71 reaches distributors (including creation and maintenance of a knowledge sharing platform) access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries:  CITE reports  Interaction with networks such as the Practical Impact Alliance at D-Lab and presence at events such as Scaling Development Ventures  Interviews and surveys  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more.  Social media – CITE has continued to grow its Twitter following and aims to include distributors as part of its audience  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more. Impacts: Funding/revenue/reach/influence/specific gains:  Products better designed to meet the needs of the poor from a suitability, scalability and sustainability perspective CITE: Retailers (Anticipated) Value Proposition  Provide access to information about appropriate products for consumers in emerging markets, so that consumers have access to products that better meet their needs Primary recipient of services/activities:  Anticipated: social entrepreneurs Secondary beneficiary: Downstream user/beneficiaries Activities  Implement product evaluations and provide user centered data to drive informed product selection, business model development and operational decisions  Develop an effective communication and dissemination strategy (including creation and maintenance of a knowledge sharing platform) Partners:  International NGO's; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels:  CITE reports  Interviews and surveys  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more. Impacts:  Products better designed to meet the needs of the poor from a suitability, scalability and sustainability perspective CITE: Entrepreneurs/Product Designers Value Proposition HESN Lab Canvases 72  Provide access to critical information about gaps in the market to improve upon or create new products for emerging markets. Deepen impact of products designed to reduce poverty  Provide access to tools for evaluation to be used by the entrepreneurs from the earliest design stage, such that they can evaluate their own products Primary recipient of services/activities:  Current: D-Lab Scale-up fellow (solar lighting)?; Tata team that used water quality report  Anticipated: Broad range of individuals including Scale-up fellows and Tata Fellows from MIT Secondary beneficiary: Downstream user/beneficiaries Activities  Implement produce evaluations and provide user centered data to drive informed product design, business model development and operational decisions,  Develop and share methodology and tools to carry out evaluation of their products Partners:  International NGO's; Academic Institutions; Government Organizations, Multi-lateral Organizations; Private Companies; Individual manufacturers, suppliers, retailers and consumers: In-field support with logistics, access to stakeholders, understanding context, shaping the research, capacity building, project specific expertise, data gathering Delivery channels:  CITE reports  Interaction with networks such as the Practical Impact Alliance at D-Lab and presence at events such as Scaling Development Ventures  Email - CITE has built an email list from its existing contacts and a tailored newsletter is sent once a month. The newsletter includes the latest blogs and/or news, a “by the numbers” section, and the latest information on evaluations. CITE uses the newsletter to inform its audience of the latest project work in succinctly-written snapshots, while driving traffic to the website for those who wish to learn more.  Social media – CITE has continued to grow its Twitter following and aims to include entrepreneurs and product designers  CITE‘s website. CITE drives traffic to the website using a regular newsletter, tailored emails, social media, and more.  Networks and clubs at MIT  Academic publications Impacts:  Products better designed to meet the needs of the poor from a suitability, scalability and sustainability perspective HESN Lab Canvases 73 ConDev Beneficiary Segments:  Researchers  Local Institutions  International Development Practitioners  Students ConDev: Researchers Value Proposition:  ConDev conducts its own and supports others' research, data-gathering, and analysis to produce new knowledge and guide policy-makers in drawing conclusions related to conflict and development issues around the world. o This is accomplished through a diverse set of strategies and tools, including:  ConDev's original research and publications  ConDev's financial and institutional support for the research of others  ConDev's partnerships with key philanthropic resources  ConDev's relationships with key local institutions and in-country logistical support systems, etc. Primary recipient of services/activities:  Faculty/Researchers at TAMU (College of Ag and Life Sciences, School of Public Health, Bush School, etc.): Those who receive financial support from ConDev, participate in ConDev-sponsored student capstone projects, etc.  Faculty/Researchers at other Higher Education Institutions, NGOs, US federal agencies, foreign government entities, etc. Secondary beneficiary: Key activities/products/services:  Designing and managing competitive processes for awarding research funding or identifying key researchers for support.  Building and facilitating cross-departmental and interdisciplinary research teams and collaborations, including collaborations between multiple universities/institutions.  Setting research priorities and key questions/hypotheses for investigation in line with ConDev's mission.  Collecting Data, Reports, Conducting Analyses, and Producing Informative/Actionable Materials, including assisting with in-country logistics through USAID partnerships or partnerships with local institutions.  Major Events and Platforms for Raising Awareness: Participating in academic conferences, publishing articles, promoting student competitions, offering competitive research grant programs, teaching academic courses and trainings, etc. Partners:  TAMU: College of Ag and Life Sciences, School of Public Health, Bush School, College of Engineering, College of Liberal Arts: ConDev gets the specialized expertise of faculty/researchers, access to highly-trained and interested students, financial and institutional resources, etc. They get ConDev's expertise in agricultural development in conflict situations, connections to USAID and project partners, financial resources, opportunities to expand their reputation and research experience, etc.  Other Higher Education Institutions: ConDev gets access to their expansive knowledge, partners, established programs, research specialties, etc. They get ConDev's specialized expertise in agricultural development and conflict issues, connections with USAID, partners, and established programs.  USAID Bureaus and Missions in Beneficiary Countries: ConDev gets financial resources, access to data and key partners/people, and HESN Lab Canvases 74 assistance with in-country logistics, etc. They get ConDev's expertise, agility, research experience, access to university resources and partners, etc.  Other US Federal Government Agencies/Departments: ConDev gets financial resources and influence with future "boots on the ground" in conflict situations. They get ConDev's specialized knowledge, institutional reputation of Texas A&M, and partner relationships.  Conflict and Development Foundation and Howard G. Buffett Foundation: ConDev gets financial resources, specialized expertise of experienced in-country project implementation/data collection staff, assistance with local logistics/connections, and valuable branding/publicity opportunities, etc. They get ConDev's expertise in agricultural development in conflict situations, connections to USAID and project partners, financial resources, opportunities to expand their reputation and research experience, access to TAMU's institutional resources and partners, etc. Delivery channels:  Direct communications with researchers: At meetings, in the field, during study design, at presentations/conferences, etc.  Written/Indirect communications with researchers and the broader academic/scholarly community: Through published reports, articles, journals, website and social media posts, emails, advertisements, etc.  Utilizing the HESN program for communications and to generate collaborations with other labs/institutions/individuals.  Delivery of research products that have been specifically requested or "ordered" by partners: e.g. Education Division of USAID’s Bureau for Africa’s Office of Sustainable Development. Impacts:  Production of Policy Briefs, Academic Papers, Conference Presentations, etc.: Resulting from research and data analysis from researchers supported by ConDev.  Increasing the number of researchers/faculty working on conflict and development issues: Including teaching courses, producing knowledge, gaining access to conflict-prone regions of the world, becoming noted experts and influencing policy and practice in those regions, etc.  Private investment in ConDev leveraged: $1,500,000 from Howard G. Buffett Foundation  Training of students, local institution personnel, etc. to work in conflict scenarios: In research, data gathering, monitoring and evaluation, methods and data analysis techniques.  Encouragement, Training, and Providing Funding Support for local researchers/capacity: Local universities are increasingly capable and involved in studying conflict and development issues in their own countries/regions.  Value of not having conflict in countries (less needs to be spent to achieve the same development goals by USAID and others) ConDev: Local Institutions Value Proposition: HESN Lab Canvases 75  ConDev brings specialized expertise, financial/institutional resources, and partnerships/connections to improve or expand the mission and impact of local institutions in promoting peace in conflict-prone regions; Expertise in agricultural development, research and data analysis related to conflict, running development programs in conflict-prone regions, and other TAMU partner expertise in public health, government, etc.; Financial resources from HESN USAID, HGBF, CDF, TAMU, and other partners; Institutional resources from TAMU; Partnerships and Connections through TAMU, USAID, CDF, other partners from local level to global multilateral organizations.  ConDev pulls out lessons learned from studies and projects funded and implemented by others to determine what is effective and what isn't.  Connects local institutions to larger groups like USAID who can scale up and multiply the impact of their work by showcasing it to other bigger entities, empowering those connections, and thus increasing impact.  Setting up credible institutions, helping them gain popularity and looked at as trusted authorities who can really play important roles when conflict does flare up.  Helping to highlight the role of women in conflict mitigation and prevention. Primary recipient of services/activities:  Directors and Project Managers at Local Institutions: Those who set the vision and provide overall leadership of local institutions, including those who formalize partnerships with ConDev, enter into funding agreements, etc.  Project / field staff who implement projects with communities and households: Those conducting interviews, facilitating community dialogues, collecting survey data, monitoring local conditions, training workers in new technologies/methods, distributing information, etc.  Buyers of organic coffee and cacao in DR Congo and DR Congo government policy-makers: International buyers are offering a premium market for the local produce, but they are imposing on the farmers to grow "organically." The buyers recommend using no soil inputs at all. This has very detrimental impacts on the fertility of the soil over the long term because the farmers aren't using the correct prescription of inputs, which can be done "organically." The farmers then have to move, increase land use, etc. because their soil is degraded, causing conflict. ConDev has tried to tell the buyers, growers, and DR Congo authorities that the soil fertility is going down due to lack of use of inputs, and we are providing the guidance about inputs, etc. Improved farming techniques and land use will reduce conflicts that arise over land ownership, registration, misuse of natural resources, poaching, etc.  DR Congo (and other countries) federal government officials overseeing land registration processes: ConDev is working to counter the general view in countries like DR Congo that land registration is something that has to go through a very lengthy and tedious process, whereby people have to go to the capital for multiple days, etc. in order to properly/legally register land. This plays into the mentality that only the federal government can do distribution of land rights correctly (rather than local authorities). ConDev is helping change attitudes toward getting laws changed through increasing transparency. Secondary beneficiary: HESN Lab Canvases 76 ConDev’s partners at UCBC are showing that local groups with GIS tools, etc. can adjudicate land claim disputes in a fair and objective way without federal level control. Documentation and mapping of lands pre￾dispute is very powerful and important for preventing conflict.  Local Institutions involved in or facilitating "Peace Dialogues": Through dialogues we can deal with issues before they result in conflict. Key activities/products/services:  Collaborative projects: ConDev staff working alongside staff from the local institutions to carry out projects in conflict-prone regions, including ConDev hiring local personnel to support the institutions.  Build new and maintain existing partnerships for ConDev and for the local institutions.  Fundraising and other types of support/awareness raising to promote the work of the local institutions and propel it toward success. Partners:  TAMU: College of Ag and Life Sciences, School of Public Health, Bush School, College of Engineering, College of Liberal Arts: ConDev gets the specialized expertise of faculty/researchers, access to highly-trained and interested students, financial and institutional resources, etc. They get ConDev's expertise in agricultural development in conflict situations, connections to USAID and project partners, financial resources, opportunities to expand their reputation and operational experience, etc.  Conflict and Development Foundation and Howard G. Buffett Foundation: ConDev gets financial resources, specialized expertise of experienced in-country project implementation staff, assistance with local logistics/connections, and valuable branding/publicity opportunities, etc. They get ConDev's expertise in agricultural development in conflict situations, connections to USAID and project partners, financial resources, opportunities to expand their reputation and operational experience, access to TAMU's institutional resources and partners, etc.  USAID Missions: ConDev gets financial resources, access and influence with high￾level policy makers, and assistance with in￾country logistics, etc. They get ConDev's expertise, agility, experience, access to university resources and partners, other relationships with local institutions and NGOs, etc. Delivery channels:  Direct communications with local institutions: At meetings, in the field during project implementation, training events, official visits to local headquarters, etc.  Written/Indirect communications with local institutions or benefitting the broader community of local institutions: Through published reports, articles, ConDev website and social media posts, emails, advertisements, etc.  Providing financial support: Through competitive grants or fee-for-service payments Impacts:  Conflict / Local elite competition mitigated through peace building processes: e.g. community dialogues and resolution processes carried out at the Congo Peace Center. HESN Lab Canvases 77  Successful testing and implementation of transformative solutions / innovative strategies to increase food security and youth employment: e.g. ConDev's support for the partnership between a student team and a local institution in Guatemala to design a more efficient way for women to wash and package vegetables for sale/export.  Increased capacity levels for local institutions to carry out their work more effectively.  Private investment in ConDev leveraged by our programs: $1,500,000 from Howard G. Buffett Foundation  Value of not having conflict in countries (less needs to be spent to achieve the same development goals by USAID and others) ConDev: International Dev Practitioners Value Proposition:  ConDev creates data-driven products to influence and inform policy: ConDev staff and partners utilize sophisticated statistical models for collection and analysis of data to assess development strategies and responses/results.  ConDev creates and tests innovative approaches and technologies for reducing and mitigating conflict and facilitating recovery, through: promoting food security; engaging vulnerable youth populations; conducting natural-resource management as a peace-building strategy; empowering local institutions to shift competition among elites toward cooperation and conflict resolution. Primary recipient of services/activities:  High-Level Policy Makers in US Federal Government Agencies/Departments: o USAID bureaus and in-country mission personnel who desire: data collection and analysis related to development strategies in regions impacted by conflict; production of policy briefs; testing of development strategies; creation and testing of innovative approaches and technologies; local partnerships and connections; project implementation experience; and specialized expertise in agriculture, conflict, and development issues. o Other US federal agency department personnel and units in need of: special training related to conflict and development, innovative approaches to solve development challenges, connections to university students and researchers, etc.  Faculty, Labs, Centers, etc. at Higher Education Institutions: Those desiring specialized knowledge, research facilities, agricultural development experience, relationships and partner connections in certain parts of the world, access to specialized demographics of students, etc.  Program officers, philanthropists, and NGO project personnel: Those desiring specialized data collection and analysis, experience working in conflict areas, expertise in agricultural development, TAMU institutional resources and partners, connections to USAID and federal policy makers, etc. Secondary beneficiary: HESN Lab Canvases 78  Program and project officers with foreign government agencies: Those interested in ConDev's specialized expertise in agricultural development in conflict-prone areas, institutional resources and connections around the world, relationship with USAID and other US federal agencies, etc.  Other Business and Development Actors: Those interested in ConDev's specialized expertise in agricultural development in conflict-prone areas, institutional resources and connections around the world, relationship with USAID and other US federal agencies, etc. Key activities/products/services:  Building and Managing Teams of Professional Development Personnel  Training and Overseeing Student Teams: For capstone projects, field schools, site visits, internships, research and data collection, etc.  Major Events and Platforms for Raising Awareness: Participating in academic conferences, publishing articles, promoting student competitions, offering competitive research grant programs, teaching academic courses and trainings, etc.  Creation/Testing of Models, Materials, and Methods: For distribution and capacity building with key partners. Partners:  USAID Bureaus and Missions in Beneficiary Countries: o Bureaus: Africa Bureau's Education Division, Europe and Eurasia Bureau, Bureau for Food Security/Feed the Future, Asia Bureau/East Asian Affairs, etc. o Missions: DR Congo, Uganda, Ghana, Afghanistan, El Salvador, etc. o ConDev gets financial resources, access and influence with high￾level policy makers, and assistance with in-country logistics, etc. They get ConDev's expertise, agility, experience, access to university resources and partners, etc. Delivery channels:  Direct communications with policy makers and project personnel: At meetings, in the field during project implementation, official visits to partner headquarters, etc.  Written/Indirect communications with policy makers and the broader community of international development practitioners: Through published reports, articles, website and social media posts, emails, advertisements, etc.  Utilizing the HESN program for communications with partner institutions.  Delivery of policy briefs and analyses that have been specifically requested and or "ordered" by partners: e.g. USAID Africa Bureau for Education Impacts:  Influence into governmental and multi-lateral policies dealing with Conflict and Development in vulnerable places.  Investment in Transformative Solutions projects.  Private investment in ConDev leveraged by our programs: i.e. $1,500,000 from Howard G. Buffett Foundation  Creation and Testing of Innovative Approaches and Technologies for implementation by partners/beneficiaries to solve conflict and development challenges  Value of not having conflict in countries (less needs to be spent to achieve the same development goals by USAID and others) ConDev: Students HESN Lab Canvases 79 Value Proposition:  ConDev provides students with valuable opportunities to access existing and generate their own expertise, knowledge, and research experience: Students supported by ConDev are learning and reporting extremely valuable information for their own edification and to contribute to the field of conflict and development. E.g. students have found connections between climate change (temperatures, rainfall, etc.) and incidences of conflict. These are new discoveries, and with further analysis and application, these data can literally save lives through their policy and implementation potential.  Financial support: For interns, graduate research assistants, recipients of Transformative Solutions grants, recipients of Student Media Program grants, etc.  Partnerships, credibility, institutional backing, and other connections: These are the types of intangible resources that are enjoyed by students connected to ConDev who want to take part in research or development activities in regions impacted by conflict. Primary recipient of services/activities:  Students at TAMU: Those involved in Bush School capstone projects, interning/working with ConDev as research assistants, taking courses focused on conflict and development, working on transformative solutions projects through coursework, etc. Secondary beneficiary:  Students at other universities (including TAMU): Those awarded Student Media grants, Transformative Solutions grants, engaged through field projects, etc. Key activities/products/services:  Advising and Mentoring  Teaching courses and leading capstone projects  Offering competitive grant programs: e.g. Student Media Grants program  Hosting and sponsoring events: e.g. Virunga documentary showing, Aggies Invent challenge. Partners:  Howard G. Buffett Foundation  Conflict and Development Foundation  TAMU Bush School  TAMU School of Public Health  TAMU College of Engineering  Christian Bilingual University of the Congo (UCBC)  USAID Missions Delivery channels:  ConDev communications via: Website, social media, blog posts, and student listserv  Direct communications with students working with ConDev or taking ConDev courses Impacts:  Training a new generation of conflict and development professionals/researchers.  Graduate and Undergraduate courses taught and fellowships offered.  Transformative Solutions and Innovative Technologies/Approaches Designed for future implementation: The results of student projects to gather data, produce policy briefs for USAID missions, publish articles in scholarly journals.  Private investment in ConDev leveraged by our programs: >$1,500,000 from Howard G. Buffett Foundation.  Value of not having conflict in countries (less needs to be spent to achieve the same development goals by USAID and others). HESN Lab Canvases 80 DIL Beneficiary Segments:  University Researchers  Direct Beneficiaries of DIL Innovations (with examples from Cellscope)  Implementing Partners and Decision-Makers DIL: University Researchers Value Proposition:  Development Engineering: DIL’s approach, Development Engineering, represents a new way of innovating for international development. We build scale into the design of technologies for the poor, by combining engineering advances with innovations in the behavioral and social sciences.  Fill Gaps in Research Funding: Support for early-stage, interdisciplinary research including international travel grants to obtain in-country feedback early in the design process.  Enable & Accelerate Research: Through research administration and project management support, IRB training, harmonization of lessons across research teams, access to experts and visiting mentors.  Translate Technical Research and Facilitate Research Dissemination: Through co-development of policy briefs, collateral, OpEds and other digestible outlets which make important (but often complex work) accessible to wide audiences.  Pipeline Development: Complement existing accelerators, investors, programs while sparking new technologies and approaches for development.  Academic Recognition: Traditionally, faculty are incentivized to seek publication in highly-cited peer￾reviewed journals. Given career pressures in academic environment, this can take priority than product development, translation, or even social impact. To make development innovation more compatible with existing academic incentive structures, DIL has created a journal for publication of applied research.  Training & Field Opportunities for Students: Graduate students are pivotal to much of the research and field work supported by DIL. Given the narrow focus of university degree programs, they may lack access to colleagues, approaches, and expertise in other disciplines (even when this is required for the success of their research). A suite of interdisciplinary training activities has been designed to improve the outputs of PhD students working on faculty-led innovations.  Promotion of Latest Field Tools and Shared Knowledge: Through webinars focused on mobile data collection tools, DIL Scientists Meetings. Primary recipient of services/activities:  University Faculty Researchers: o University researchers affiliated with DIL receive grant funding, trainings, publication outlets, media coverage, convenings, networking, and other support from DIL's administrative staff. These 'customers' are looking for support for innovation, evaluation and scale-up. However, measuring demand for "soft" inputs (non￾monetary) has proven difficult. o DIL creates academic and institutional recognition (incentives) for Faculty conducting interdisciplinary research for global development. o DIL's research grants fill a funding gap for early-stage, interdisciplinary research which allows Faculty to pursue projects they might not otherwise be able to work on.  PhD candidates / students participating in DIL research projects: Access to interdisciplinary faculty & peers within research teams including training and student mentorship to shift career trajectories and advance research. Secondary beneficiary:  PhD candidates participating or enrolled in DIL's Dev Eng program and supporting activities. HESN Lab Canvases 81  University Staff Researchers: DIL research funding allows for continued staff scientist support which ensures efficient and sustainable support for DIL research projects. This sustained drives projects through the pipeline more quickly. Key activities/products/services:  Catalytic funding for applied and exploratory research: Research Grant Competitions (Outreach management, peer review, feedback)  Capacity Building & Innovator Support Systems: Through targeted DIL workshops,trainings, mentorship/feedback from relevant mentors accessible through DIL programming.  Matchmaking across disciplines and with field partners: Through workshops, adhoc introductions, and creating incentives (e.g. research competitions). Faculty often lack relationships with research, implementation, and policy partners in developing countries. Affiliation with DIL and HESN consortia allow for new introductions and facilitation of new partnerships. Partners:  Partners institutionalizing the DIL Approach: Development Engineering: (e.g. Elsevier, EPFL)  Universities & Researchers: Researchers across campuses and disciplines contribute to DIL research projects (e.g. University of California, University of Washington, Portland State University, Jadavpur University, IIT Bombay, Makerere University, LIGTT).  Ideas Competition Partners: Design, fund, or participate in Big Ideas Competitions (e.g. World Vision, USAID, Makerere).  Innovator Support Partners: Work with the DIL Management staff to contribute to the network as mentors, workshop facilitators, and visiting fellows (e.g. Microsoft Education, Sanergy, Better Ventures, USAID).  Research Implementing Partners: Work with research teams in-country supporting field knowledge and implementation of research design (e.g. IPA). Delivery channels:  Research Competitions  Journal of Development Engineering  Conferences, Trainings, Workshops, Tech Salons  Polished Communications: Usually targeting potential donors or decision-makers. 1-pagers, brochures, reports, concept notes/proposals, event invites.  DevEng Curriculum  Personal Communications: Frequent personalized emails, skype, phone calls, in-person visits, and other directed communications are essential for implementation and advisory relationships.  Mass Communications: Emailed newsletters, websites, social media Impacts:  Increased significant early-stage, interdisciplinary research: Impacting low-income populations.  Training and capacity building for the next generation of “development engineers” including both academics and development practiotners.  Shift in career trajectories of technical experts to work on international development or other social challenges. DIL: Direct Beneficiaries of DIL Innovations (with examples from the Cellscope) Value Proposition:  Novel Tech Innovations for Improving Livelihoods: (e.g. Cellscope – a disease diagnosis tool and algorithm for tuberculosis screening). o Boomerang technology which is applied in US: Often innovations designed for developing country settings have benefits in domestic settings (e.g. Cellscope has spun out as a private company in the US selling Cellscope otoscopes for diagnosis of ear infections in the home)  Continued R&D for new iterations and applications of innovation: a characteristic seen in many of DIL’s most successful innovations is their applicability to multiple target populations or differing challenges/regions (e.g. The Cellscope team continues to refine and iterate their innovation including developing the same concept for new developing country contexts and ailments) HESN Lab Canvases 82 Primary recipient of services/activities:  Direct Beneficiaries of DIL Innovations (on-the ground users of DIL-sponsored products or services): In the case of Cellscope this is patients at risk for tuberculosis, LoaLoa, and eye diseases.  Indirect Beneficiaries of DIL Innovations: In the case of Cellscope, this includes the Vietnam Ministry of Health, Community Health Workers, Lab technicians, children/family members of direct beneficiaries. Secondary beneficiary: Key activities/products/services:  Develop the technology (hardware)  Develop the hardware (software)  Collaborate with Physicians re Disease Burdens / Population Needs  Test Prototype In Country  Pilot In Country  Design Prototype Partners:  Patients/individuals  Local Government: Vietnam MOH  In-Country Partner: Community Health Workers, Physicians, and Lab Technicians Delivery channels:  Public Partnerships: In-Country Government Buy-In  Private Spin-Out Company for Domestic Markets CellScope Impacts (to date):  Improved health outcomes due to Cellscope’s low-cost, rapid disease diagnosis of tens of thousands of patients at risk for tuberculosis, Loa Loa, as well as eye diseases. DIL: Implementing Partners and Decision-Makers Value Proposition:  Access to novel technologies/approaches, data and analysis, including rigorous evidence of what works: Through on-the-ground interactions with researchers, embedding of scientists in-house, joint data collection and analysis, etc.  Development Engineering: DIL represents a new way of innovating for international development. We build scale into the design of technologies for the poor, by combining engineering advances with innovations in the behavioral and social sciences.  News, Media,public communications, one pagers, interviews, etc.  Knowledge pieces & digestible summaries of evidence Primary recipient of services/activities:  Implementing Partners and Decision￾Makers: Government agencies (Kenya Rural Electrification Authority, State of West Bengal, Vietnam Ministry of Health); Start￾ups (GramPower, Endaga, Premise); on￾the-ground NGOs providing a service to researchers (NextDrop); Research organizations (Innovations for Poverty Action); Private companies (Facebook, Google, Kenya Power & Lighting Company) Secondary beneficiary:  USAID (including HESN, Bureaus and Missions): Example: A Science Technology (or STIP) Advisor in a Mission, or a chief economist in a mission, who might need a new innovation for a specific problem, or who might need a team of researchers for an evaluation or other project.  Spin-Out Companies or NGOs (DIL-linked): Start-ups (GramPower, Endaga, Premise); on￾the-ground NGOs (NextDrop)  Decision-makers at a Distance (not directly involved): Multilaterals/Donors (World Bank); Foundations/NGOs in T4D (Hewlett, IDEO, BRAC); Private firms involved in development (Bechtel, Elsevier)  Private Sector Partners Key activities/products/services:  Learning through collaboration (Research Grants): Fund research that involves both Partners:  University Researchers (Faculty & Students): Example: faculty member with a novel tech4dev HESN Lab Canvases 83 DIL innovators and field partners-- where the partners have access to new tools, data, or analysis as a result of the collaboration.  Host Knowledge-Sharing Convenings: (e.g., DIL State of the Science Conference in 2014 and planned 2016 Science of Scaling event)  Strategic Communications: Newsletters, targeted campaigns (i.e., DIL mission engagement outreach), using social media, emails and websites to share peer-reviewed and grey literature with development actors. innovation or a student with an idea looking to gain context before prototyping a novel technology or approach. Both would inform DIL's approach which would be conveyed to development actors. Delivery channels:  Mass Communications: Newsletters, websites, social media  Topical Convenings: (e.g., DIL 2014 State of the Science conference on "revealing demand" open to USAID, international NGOs, and government contractors; or the Science of Scaling event planned for Fall 2016)  Personal Communications/Outreach: Frequent personalized emails, skype, phone calls, in-person visits, and other directed communications are essential for implementation and advisory relationships.  Polished Communications & Knowledge Pieces: Usually targeting potential donors or decision-makers. 1-pagers, brochures, reports, concept notes/proposals, event invites.  Technical Evidence: Peer-Reviewed Publications, Dev Eng Journal, Research Presentations. Impacts:  Policy and/or program decisions informed by DIL evidence (e.g. Kenya Rural Electrification Authority)  Financial buy-in from external partners: Cost Share/Leverage HESN Lab Canvases 84 GCFSI Beneficiary Segments:  Innovation End User  Innovator  National-International Researcher  Research End-User  Student  Student Innovator GCFSI: Innovation End User Value Proposition:  Productivity-increasing innovations: Innovations developed with full or partial support from GCFSI allow adopters to improve productivity through increases in output or quality/value of output, or reductions in costs per unit of output. o Examples include (1) mobile-phone-based information systems to improve farmers' decisions about what to plant, where, when, and with what inputs package, (2) crop processing that improves quality and reduces waste, (3) construction of low cost storage with evaporative cooling features that help preserve perishable crops for longer periods, (4) development of incentives for farmers and millers that reduce aflatoxin in maize, (5) use of video and TV programming starring local farmers to boost credibility of extension messages, etc. Primary recipient of services/activities:  Innovation User: Someone who adopts an innovation developed with entire or partial support from a GCFSI innovation grant. Adopters may be actors at any level of the food system including producers as well as actors that supply inputs to producers or who market, process, and sell the producers' outputs. Secondary beneficiary: Key activities/products/services:  Ensuring adequate supply of the innovation: Includes supply of products or services or organization of approaches that incorporate the innovation in question.  Stimulating demand for the innovation: Publicizing and demonstrating the innovation to potential users, and addressing constraints that might hinder adoption. Partners:  Partners that support creation of the innovation: GCFSI, its partners (CRDF Global, Wageningen University, LUANAR) and funders (USAID plus those providing cost share, leverage, and buy-in funds). Wageningen University (WU) was one of the initial consortium partners for GCFSI. WU receives funding from GCFSI for faculty and staff salaries, field research costs, travel, and administrative overhead. It was expected to contribute to all project objectives. Its contributions to date include input to the formulation of the RFAs for innovation grants, and the design and implementation of research and capacity-building activities. A WU faculty member sits on the GCFSI Core Technical Team and serves as the primary liaison with other WU units. In Year 2 of the project, LUANAR was selected as the host institution the GCFSI's Innovation Hub serving East Africa. LUANAR receives funding from GCFSI for faculty and staff salaries, field research costs, travel, and administrative overhead. LUANAR faculty and students contributed to GCFSI research conducted in Malawi in 2014, and will contribute to research, innovation grant management (within Malawi), and design and implementation of capacity-building activities during 2015 and HESN Lab Canvases 85 beyond. A senior faculty member from the Vice Chancellor's office serves as our primary point of contact.  Partners that support dissemination of the innovation: Dissemination includes commercial or public programs to market and scale-up use of the innovation, and to advise adopters on how to use it effectively. Most GCFSI-supported innovation R and D activities are still at the pre￾dissemination stage.  Partners that help address constraints on adoption of innovation: Partners whose resources and expertise can help break down barriers to adoption of the innovation, through public and private investment or public policy change. Mediae and Shamba Shape-up in Kenya are examples of organizations partially funded by GCFSI who are collaborating with GCFSI to deliver participatory video extension activities. Delivery channels:  Programs to improve innovation supply  Programs to enhance innovation demand: By potential end users of the innovation  Programs to obtain user input on innovation effectiveness: As input for activities that refine and improve the innovation. Impacts:  Increased net revenues earned by the innovator: As a result of adoption of the innovation.  Increased revenues earned by actors in the "innovation system": Research and development institutions who may receive a share of the value created by widespread adoption of the innovation.  Increased net revenues earned by other food system actors: As an indirect effect of increased net revenues earned by the innovation adopter, particularly for food system actors who are "upstream" (input suppliers) or "downstream" (processors, traders) of the innovation adopter. GCFSI: Innovator Value Proposition:  Funding and mentoring: GCFSI programs help innovators to secure funding and mentoring to support their innovation-oriented research. In general, the demand for funding to support innovative research on global food systems greatly exceeds the supply of funding. GCFSI funding helps to reduce that excess demand. GCFSI programs also provide mentoring that is not always a part of research grants. Lastly, some grant funds have been reserved for Malawi faculty and students, which increases their chances of success in obtaining grants, relative to what it might be in an open competition.  Capacity building of innovators: By building the capacity of innovators to improve the innovation design/development/scale process, you will accelerate innovation (amplify scale, shorten scaling timeframes, strengthen innovation, give innovators confidence to innovate) Primary recipient of services/activities:  University researcher: University professors and post-docs Secondary beneficiary:  Grantees(innovators): improving their innovation process; providing them with a space to share ideas; time; network of like￾minded folks; creative space to design/innovate; bounce ideas Key activities/products/services:  Innovation grants and mentoring: Provision of grants and mentoring to support Partners: HESN Lab Canvases 86 innovation research; Grant provision entails formulation of RFA, collection and review of proposals (often involving recruitment of external review team), selection of grantees, preparation and management of subawards to grantees, mentoring during the grant period to help "accelerate" grant outcomes; Grant mentoring involves selection of mentors and design and implementation of any workshops or other complementary activities.  CRDF Global: Manages GCFSI's open competitive programs for major innovation grants  Researcher's home institution: University where the researcher is affiliated. Provides administrative and financial support.  USAID: Global Development Lab and Bureau for Food Security LUANAR: Lilongwe University of Agriculture and Natural Resources. Hosts the GCFSI Innovation Hub for Eastern and Southern Africa. Delivery channels:  Grant programs: Grants provided through open competition or targeted programs  Mentoring & training: Provided through contacts with individual mentors or training workshops  Potential long-run collaboration Impacts:  Buy-ins from USAID units: Funds from USAID missions or central bureaus to support expansion of GCFSI activity beyond that supported by the initial USAID grant.  Cost-share funds: From MSU, MSU partners, or innovation grantees and their institutions.  Leveraged funds: Funds from other projects whose activities support GCFSI grants indirectly  Funds from other donors: Private sector firms or other private or public organizations, including foundations. GCFSI: National-International Researcher Value Proposition:  Funding and mentoring: GCFSI programs help innovators to secure funding and mentoring to support their innovation-oriented research. Primary recipient of services/activities:  Center/Institute researcher: Researchers at national or international research centers or institutes. Secondary beneficiary: Key activities/products/services:  Innovation grants and mentoring: Provision of grants and mentoring to support innovation research. Partners:  CRDF Global: Manages GCFSI's open competitive programs for major innovation grants.  Researcher's home institution: Organization (center, institute, etc.) where the researcher is affiliated; Provides administrative and financial support.  USAID: Global Development Lab and Bureau for Food Security.  LUANAR: Lilongwe University of Agriculture and Natural Resources. Hosts the GCFSI Innovation Hub for Eastern and Southern Africa. Delivery channels:  Grant programs: Grants provided through open competition or targeted programs.  Mentoring & training: Provided through contacts with individual mentors or training workshops.  Potential long-run collaboration: Among researchers affiliated with GCFSI, MSU, and the researcher's home institution. Impacts:  Cost-share funds: From MSU, MSU partners, or innovation grantees and their institutions.  Leveraged funds: Funds from other projects whose activities support GCFSI grants indirectly. HESN Lab Canvases 87  Funds from other donors: Private sector firms or other private or public organizations, including foundations.  Buy-ins from USAID units: Funds from USAID Missions or central bureaus to support expansion of GCFSI activity beyond that supported by the initial USAID grant. GCFSI: Research End User Value Proposition:  Creation of new knowledge relevant to the end user: Research results that help improve productivity or effectiveness of policies or programs.  Communication of key research results to relevant end users: Research outreach and communications materials prepared jointly by researchers and communications specialists  Climate resilient maize (CRM): By scaling up adoption of improved CRM varieties by farmers, increase ag yields and farm household incomes. Supply side: Increase production/dissemination of improved CRM seed. Demand side: Address knowledge, risk, and storage and output marketing barriers that limit demand; help farmers to realize it's worth buying these seeds. Primary recipient of services/activities:  Research User: Someone--at the local, national, or international level--who uses the results of research conducted with entire or partial support from a GCFSI innovation grant. Research end users may include other researchers, actors within the food system, or staff of other public, private, and civil society organizations. Secondary beneficiary (CRM varieties):  USAID/BFS and Lab: Both interested. Lab wants to scale new varieties; BFS works with international ag research centers that recently developed CRM varieties. Pain: helping USAID to scale CRM varieties off the shelf and into farmers’ fields. Key activities/products/services:  Implementation of research activities that focus on identifying ways to improve food systems performance: Research by GCGSI and its partners.  Communication of research results: Through print or online publications or formal or informal presentations, live or recorded.  Research (in Malawi and at MSU): -look at where effect is climate related vs social; Survey fertilizer application on maize; collect soil samples to better understand the responsive maize production to fertilizer. Variation in soil type, geography.  Frugal Innovation Practicum  Decision Support and Informatics Resource: One-stop access to major development datasets for analysis and mapping. Collaboration with AidData, Geo-Center, FTFMS. Partners:  GCFSI research team: GCFSI core faculty research team, representing six colleges.  Researchers at GCFSI core partners: (Wageningen University, LUANAR)  GCFSI and partner communications personnel: Communications personnel at GCFSI, at MSU, at USAID (Lab, BFS), and at other partner organizations (Wageningen University, LUANAR). Delivery channels:  Print publications  Online publications  Live and recorded live presentations: Presentations, briefings, workshops, webinars, podcasts, videos, etc. Impacts:  Increased net revenues earned by the research end user: As a result of incorporation of research results into end user programs, policies, etc. HESN Lab Canvases 88  Increased net revenues earned by other food system actors: As an indirect effect of increased net revenues earned by incorporating research results into programs, particularly those affecting global food systems.  Increased revenues earned by actors in the "research system": Researchers and research and development institutions who may receive a share of the value created by widespread adoption of the research results. GCFSI: Student Value Proposition:  Graduate students receive financial support for their degree programs: Graduate assistantships cover costs of degree programs (tuition and fees) and living expenses.  Graduate students receive financial support for their thesis research: Graduate assistants working on project research teams benefit from funding of research activities that relate directly or indirectly to the focus of their thesis research.  Graduate students receive research experience and skills: Graduate assistants learn valuable research skills "on the job" and gain ideas for good research topics through their contacts with faculty.  Study abroad and internship participants: Benefit from experiential learning opportunities, whether domestic or international.  All students: All students benefit from expanding their professional networks and contacts through their work with GCFSI. Primary recipient of services/activities:  Graduate Student: Graduate students pursuing degree programs, and working as research assistants on research or implementation teams with GCFSI or its partner organizations.  Other graduate or undergraduate students: Benefit from the project when employed as hourly workers, or as participants in study abroad or internship programs. Secondary beneficiary: Key activities/products/services  Appoint graduate students as research assistants (GRAs): Primary role of GRAs is to contribute to project objectives for research, outreach, and capacity-building. Ideally, the GRA's research should align with the student's degree research interests, and thus contribute to his/her MS or PhD thesis/dissertation.  Provide financial or other support that contributes to degree objectives: Funds to support field research, participation in conferences or in other professional activities. Partners:  GCFSI research team: GCFSI core faculty research team, representing six colleges.  Researchers at GCFSI core partners: Wageningen University - worked with students (grad student) to implement the Malawai research project; LUANAR. Delivery channels:  Graduate research assistantships: GRAs provide financial support (tuition, fees, living expenses); GRAs are generally selected and supervised by GCFSI core faculty who play key roles in major work plan activities; RA assignments give students research, publication, and presentation experience, through which they acquire professional skills and contacts; As enrolled students, GRAs are also eligible for other university financial support on a competitive basis, e.g., travel to professional meetings, predissertation field visits, etc.  Faculty: Largely recruited by faculty- based on existing relationships and/or mutual areas of interest Impacts: HESN Lab Canvases 89  Grad Research Assistants: Funding to support their degree programs, research experience and faculty mentoring, publications, professional networking opportunities, possible time savings in degree completion due to reduced need to take jobs unrelated to degree focus, and increased research experience and opportunity to use GRA research results as basis for thesis.  Students doing study abroad/internships: greater likelihood of participating in such programs, reduced out-of-pocket costs, greater opportunity for overseas exposure and experiential learning. GCFSI: Student Innovator Value Proposition:  Funding and mentoring: GCFSI programs help innovators to secure funding and mentoring to support their innovation-oriented research. Primary recipient of services/activities:  Student innovation grantee: Students supported by GCFSI student innovation grants through open or targeted grant programs Secondary beneficiary: Key activities/products/services:  Innovation grants and mentoring: Provision of grants & mentoring to support innovation research. Partners:  CRDF Global: Manages GCFSI's open competitive programs for major innovation grants.  Researcher's home institution: Organization (center, institute, etc.) where the researcher is affiliated. Provides administrative and financial support.  USAID: Global Development Lab and Bureau for Food Security.  LUANAR: Lilongwe University of Agriculture and Natural Resources. Hosts the GCFSI Innovation Hub for Eastern and Southern Africa. Delivery channels:  Grant programs: Grants provided through open competition or targeted programs.  Mentoring & training: Provided through contacts with individual mentors or training workshops.  Potential long-run collaboration: Among students and researchers affiliated with GCFSI, MSU, and the student's home institution. Impacts:  Buy-ins from USAID units: Funds from USAID Missions or central bureaus to support expansion of GCFSI activity beyond that supported by the initial USAID grant.  Cost-share funds: From MSU, MSU partners, or innovation grantees and their institutions.  Leveraged funds: Funds from other projects whose activities support GCFSI grants indirectly.  Funds from other donors: Private sector firms or other private or public organizations, including foundations. HESN Lab Canvases 90 IDIN Beneficiary Segments:  Network Members  Local Innovators  University Students IDIN Direct Beneficiaries: Network Members Value Proposition:  International Development Design Summits: 2-4 week intensive design experience  Project Grants: Scale-up Fellowships, Microgrants, Picogrants  Mentorship/Advising  Tools/Workshop Space: Innovation Centers  Local Chapters: IDDS alumni and local innovators  Collaborative Platforms  Access to Student Teams  Connections to External Opportunities Primary recipient of services/activities:  IDIN Network Members: Participants/alumni of IDIN's 2-4 week International Development Design Summits - from 62 different countries, ranging in age from 19-80, all walks of life Secondary beneficiary:  Communities where IDIN Network members work (i.e. users of Network member-produced technologies, participants of Network member-led trainings, etc.) Key activities/products/services:  Summits/Design Trainings: International Development Design Summits (IDDS); Creative Capacity Building Trainings (CCB); Other Local Trainings/Workshops (Skill Builders, Build￾Its/Unbuild-Its, etc.)  Network Support: Microgrants; Picogrants; Scale-Ups Fellowships; Training Grants (new); Mentorship/Advising; Local Chapters; Collaborative Platforms; External Opportunities Partners:  Academic Partners: Develop and deliver curriculum at summits and courses, conduct research, advance technologies in classes. o MIT D-Lab o Olin College o UC Davis o Colorado State University o Kwame Nkrumah University of Science and Technology o Singapore Polytechnic  Institutional Partners: Support local Network members through small grants, trainings and advising to enable technology development and dissemination. o NTBC (Zambia) o ECHO (Tanzania)  Innovation Center Partners: Make spaces that connect local Network members to tools, workshop space, and technical advice. o Twende (Tanzania) o Caos Focado (Brazil) o Tet Center (Uganda) o +9 others Delivery channels:  Summits/Design Trainings: In-person encounters  Grant/Mentorship Programs: Year-round engagement from IDIN headquarters  Local Partners: Year-round engagement from local partners HESN Lab Canvases 91  Online Platforms: Virtual communication and collaboration among Network members Impacts: 1. Co-Create Effective Solutions a. Ex. Progress of projects (development, pilot, adoption/commercialization, scale) b. Reach of products 2. Build Local Capacity for Innovation and Design a. Ex. Reach of trainings b. Ex. Engagement/ innovation activity of participants after trainings 3. Generate Knowledge and Spread the Approach a. Ex. Resources produced and shared b. Ex. Network members/partners starting their own initiatives to support local innovation IDIN Direct Beneficiaries: Local Innovators Value Proposition:  Creative Capacity Building Trainings  Skill Building Workshops  Mentorship/Advising  Tools/Workshop Space: Innovation Centers  Project Grants: Picogrants Primary recipient of services/activities:  Local Innovators: Participants in year-round field-based programming on design and innovation led by IDIN's local partners in Tanzania, Uganda, Ghana, Zambia, Brazil and others. Includes user of innovation centers and participants in 3-5 day Creative-Capacity Building trainings. Secondary beneficiary: Key activities/products/services:  Innovation Centers: IC Grants, Tools and Workshop Space, Advising/Capacity Building Partners:  Academic Partners: Support local innovators through trainings and student collaboration o KNUST (Ghana)  Institutional Partners: Support local innovators through small grants, trainings and advising to enable technology development and dissemination. o NTBC (Zambia) o ECHO (Tanzania)  Innovation Center Partners: Make spaces that connect local innovators to tools, workshop space, and technical advice. o Twende (Tanzania) o Caos Focado (Brazil) o Tet Center (Uganda) o +9 others Delivery channels:  Design Trainings In-person encounters  Local Partners: Year-round engagement Impacts: 1. Co-Create Effective Solutions a. Ex. Progress of projects (development, pilot, adoption/commercialization, scale) b. Reach of products 2. Build Local Capacity for Innovation and Design a. Ex. Reach of trainings HESN Lab Canvases 92 Ex. Engagement/ innovation activity of participants after trainings IDIN Direct Beneficiaries: University Students Value Proposition:  University Courses  Fieldwork funding  Fieldwork Projects/Connections Primary recipient of services/activities:  University Students: Students in US, Ghana, Tanzania, etc. who engage in coursework, research, and fieldwork in collaborative design. Secondary beneficiary:  Partner institutions and Network members in the field who benefit from student collaboration Key activities/products/services:  Research: Ongoing studies in Local Innovation Processes and Ecosystems; Development Impacts of Local Innovation; Enabling and Scaling Local Innovation  Student Engagement: University Courses, Student Projects and Fieldwork, Internships and Fellowships Partners:  MIT D-Lab, Olin College, UC Davis, Colorado State University, Kwame Nkrumah University of Science and Technology, Singapore Polytechnic Delivery channels:  IDIN Opportunities Page and fund matching, IDIN Research Fellow program, University courses at Academic partner institutions Impacts:  Project advancement (innovations, ventures, research, etc.), Research produced and disseminated, Student career shifts/advancement HESN Lab Canvases 93 RAN Beneficiary Segments:  Students  Non-RAN Grantee Innovators  Development Actors  Target Communities  Ugandan Policy-Makers  Faculty RAN: Students Value Proposition:  RAN aims to build substantial capacity for students to become resilience innovators so that they develop solutions to their regions' priority resilience challenges.  RAN targets engaging 20 African partner Universities from 16 countries. The students from thesepartner Universities, coordinated in 4 regional lead universities that host Resilience Innovation Labs (RILabs), learn ideation and prototyping skills which helps them in their course projects to think of and develop innovative projects.  Students from other Universities neighboring Makerere University in Uganda that are not part of the partnership: engage in RAN Internship Program which exposes them to the real-life innovation process and also enables them to earn academic credits. Student interns are exposed to a working environment that exposes them to the innovation process at RAN. They also gain professional skills such as research, communication, ICT and innovation among others.  Students participate in innovation outreach events, in the process developing skills to change their attitudes about innovation.  Equip girls with technical and entrepreneurial skills, with a goal of creating applications that can be launched to market. Primary recipient of services/activities:  Students from 16RAN partner Universities coordinated in 4 regional leaduniversities. Many traditional university programs do not teach about innovation. Students therefore lack skills in 'how to innovate' and 'how to undertake human-centered design'.  Students from other Universities neighboring Makerere University in Uganda that are not part of the partnership: Universities mainly concentrate on academic programs andextra-curricular projects do not receive adequate attention.  Female students: Several institutional barriers prevent female students from being at the frontiers in innovation skills acquisition (e.g. students perceive that innovation is limited to the science programs which seem to be male dominated). Secondary beneficiary: Key Activities:  RAN Lab organizes student-friendly capacity building opportunities to students in form of physical courses as well as through short learning videos called MKITS.  Design Thinking Course: Students are equipped with knowledge and skills on the Human Centered Design approach of designing solutions based on user needs, brainstorming techniques, rapid prototyping and testing and iteration of innovation projects.  The Resilience course introduces critical aspects of resilience analysisand a better understanding of risk, vulnerability and adaptationof communities so as to identify entry-points for resilience innovation. Partners:  Faculty: assist in identifying students that participate in these courses  UNDP: Sponsors students who take these courses, particularly resilience courses.  Internship Supervisors, Faculty: Identify students to participate in the Internship programs as well supervising them in the field  Faculty: Publicizing the student opportunity to student mailing lists HESN Lab Canvases 94  Internship Opportunities: Student interns are exposed to a working environment that exposes them to the innovation process at RAN. They also gain professional skills.  Research methodology: Students participate in RAN qualitative and quantitative resilience surveys as research assistants, gaining skills in resilience assessment. Some students are supported to conduct resilience data analysis and scientific writing.  Student Movie Night (Students watch a motivational movie related to innovation): SMN is an opportunity for students to engage with RAN in an informal way but learn from inspirational stories on what it takes to innovate.  Bar Camp Sessions: Bar camps are designed to engage students,particularly females,and to motivate them to work on innovation projects so as to increase percentage of female in innovation activities.  The 'Technovation Challenge': This is a technology entrepreneurship competition for young women. It teaches young women to build mobile applications to solve community challenges. Professional women mentor teams of girls and together they go through the online curriculum over a period of three months. The program equips the girls with technical and entrepreneurial skills, with a goal of creating applications that can be launched to market.  Supporting students to apply for grants across the HESN and other funding sources.  Female Experts, Innovation Hubs: participate in bar camp debates as well as motivate female students to innovate  Global Technovation: To give us content and a platform to run the challenge Delivery channels:  Students from 16RAN partner Universities coordinated in 4 regional lead universities called Resilience Innovation Labs (RILabs): (Makerere, Uganda – EARILab); Jimma, Ethiopia (HoARILab); UDS Tamale, Ghana (WARILab) and Pretoria, South Africa (SARILab).  Students engage in RANInternship Program which exposes them to the real-life innovation process and also enables them earn academic credits.  Students engage in outreach activities conducted by RAN at their Universities Impacts: Impacts of these activities  Some students who trained in design thinking skills were able to organizeand train other students  Some students applied the skills gained to prototype their ideas which they pitched and have won awards  Students report that they have gained confidence in prototyping their ideas  Students have disseminated their work to a wider community including conferences and some have submitted their work for publication.  Some students have co-founded start-ups RAN: Non-RAN Grantee Innovators Value Propositions:  RAN Lab helps non-grantee student and non-student innovators to develop their ideas to a level that are substantially innovative. RANprovides platforms where they can present their ideas to get feedback on idea refinement, both from technical experts and from people trained in the human-centered design process. RAN’sjob is to help them clearly define the 'innovation' within the idea, identify solution options and converge on the best solution options as well as key failure modes. HESN Lab Canvases 95  RAN Lab provides an opportunity for the Innovators to get their ideas 'to the next level'. Through the weekly pitching sessions, ideas are critiqued to provide positive feedback on how to move the idea forward, as well as guiding the innovators through their journey of innovation. Some of them get connected to Mentors and funding opportunities for their innovations where possible. They also get opportunities to show-case their ideas to stakeholders through exhibitions at conferences and seminars. Student Innovators in this category are also engaged in Design Thinking Workshops to get knowledge and skills on how to design forhumans.  Non-grantee student innovators are also connected to possible funding opportunities and some of them have won funding from outside the RAN to develop their ideas. RAN helps them to develop their concepts so that they have a good pitch in such competitions. These could be HESN opportunities or other university and institutional funding opportunities for innovations. Non-grantee innovators gain constructive feedback on their ideas so that they can fine-tune their ideas to make them more innovative and human-centered.  Student Innovators in this category also get attached to mentors through the RAN Lab and the Mentors guide them on how to make their projects better. Primary recipient of services/activities: Student Innovators: Different from the general students' body in the target universities, this category of students already have an innovative idea at different stages of development from a concept to a pilot project that they are working on at the time RAN engages them or they come to RAN. Because university environments are not structured to support informal learning opportunities, many students have innovative ideas that fall into 'the valley of death' due to lack of support.  Examples of projects that RAN is supporting under this category include: Pneumonia Diagnosis Vest, Bacterial Vaginosis kit, No Touch Water Tap, Ceramic Water Filtering Projects, Musawo drugs, Digital Foetal Scope, Breast IT, Sickle Cell Diagnosis Kit, Agro Market day, Ultra-Sound Tower Malaria Control System, Low cost Grain Moisture Meter, E-Liiso (for Diagnosis of Trachoma), MACOTUBA (A TB Diagnosis App) Various non-student innovators walk into RAN Lab seeking support to develop their ideas further. Examples of these include: The Electronic TB Information Management System, Hydroponic Fodder Systemand Solar Powered Egg Incubator. Secondary beneficiary: Key Activities:  Pitch Sessions: held on a weekly basis. They help innovators to get feedback on their ideas. Feedback is either technical, business viability or on alignment with human-centered design. Some of the students get connected to other regional exhibition opportunities and innovation conferences because their projects are registered in the RAN Innovation database.  Design Thinking Course: equips students with skills on human-centered design including rapid prototyping, need-finding, and on how to do an elevator pitch of their ideas. Partners:  Faculty, Practicing Experts from different disciplines: To sit on the panel and give feedback in addition to the RAN team, some of them get interested inmentoring the teams.  Innovation Consortium Engineers: The Engineers attend Garages to share practical experiences and also support student and RAN innovators to refine their projects. These students also have access to the Innovation Consortium workshop to quickly HESN Lab Canvases 96  Monthly Innovation Garage: a monthly innovation clinic developed through a partnership leveraged with a private association of Engineers/Architects that are interested in innovative solutions. The association is called the 'Innovation Consortium Ltd'. The garage provides a platform for innovators with ideas that involve engineering prototypes to get solutions to design challenges. Ideas are critiqued and different solution options obtained. Innovators are then helped to converge on a few solutions. The engineers have also provided their private lab as a free space where innovators can develop and refinetheir prototypes.  Monthly Social Design Clinic: Realizing that innovation activities have been dominated by technology and IT based disciplines and that humanities related disciplines have not been adequately involved in innovation activities, the Social Design Clinic was created to spur innovative ideas involving the humanities. It engages multi-disciplinary teams from the Arts, Law, Behavioral and Social Scientist in brainstorming to develop innovative ideas on different social problems presented as themes. prototype and weld components of their projects where there is need.  Faculty, Organizations like Straight Talk foundations, Social Innovation Academy (SINA), Teenage Centers: Partners like SINA are hosted to moderate the discussions or present defined themes for the design clinic. Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  RAN provides weekly pitching sessions and monthly innovation garages where non- grantee innovators pitch their ideas and they are advised on how to improve them  During workshops/exhibitions/events organizedor co-organized by RAN these innovators are sourced to show-case or exhibit their ideas  RAN conducts outreach sessions where non-grantee innovators are supported to pitch and refine their ideas Impacts: Impacts of these activities  Using the feedback given to innovators, they have improved their prototypes and have pitched their refined prototypes to RAN  Some of these innovators have participated in summits/competitions where they have pitched their ideas and some have won prizes.  Some of the innovators have proceeded to pilot their prototypes where they have received feedback from the community  The non-grantees have further publicized RAN’s work in the communities  There has been an increase in the females that are engaged in innovation  Increased participationof multidisciplinary teams in the innovation process RAN: Grantee Innovators Value Proposition:  Take grantees through 'the Innovation Pipeline': Refine their ideas, develop, pilot and scale their solutions.  Build capacity of innovators in successfully navigating the innovation pipeline plus in-building the potential for scale in their projects. This opportunity is made available to funded projects as well as to projects/ideas/concepts that do not receive funding from RAN. HESN Lab Canvases 97  Capacity building in key areas to promote impact potential e.g. Design Thinking, needs-finding, etc.  Access to target communities provides a platform through which communities’ actual needs are matched with the innovators products; innovators iterate their products to suit community needs. Primary recipient of services/activities: Innovators in need/search for capacity building & innovation support: Strategic guidance on navigating the innovations pipeline; access to key resources including funding, mentorship, access to communities, etc. Access to resources to enable them develop their idea to the next level. Secondary beneficiary: Key Activities:  Grants to fund development of the idea: Provides seed funding necessary for them to develop and refine their idea to the next level.  Innovation Management: Support in developing an M&E Plan, setting milestones and workplan development; Impact Potential tracking: A phased process to guide innovators.  Needs-finding Training, implementation and linkage to RAN Communities: All RAN's innovators are taken through a needs-finding process in the early stages of incubation.  Mentorship: All RAN-funded innovators are helped to find appropriate technical mentors.  Design Thinking Training: All RAN Innovators undergo training in design-thinking based co￾creation  Support in Pilot Design Partners:  Innovators sourced through RIAP (EA and HoA), RIC4A.  CE, RIC4FIG and those anticipated through CRID4RED, CRID4FAL and YSiG.  Innovations Officers and M&E Officers in all RILabs are charged with helping innovators to set M&E parameters and milestones and to track their progress in relation with the pipeline; Provide the day-to-day innovator tracking and support activities.  Technical and Business Development experts  Scaling partners  Community Leaders  Experts in training and in utilization of needs￾finding techniques.  Stanford's ChangeLabs  Experts in training and in utilization of Design Thinking Techniques. Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  RAN guides innovators on how to map the change they want to bring about, how to interact with the end￾users, developing business models, linking them to key stakeholders they need and to think of scaling at an early stage.  The innovators are provided working space and as such they receive support during implementation of their innovations  RAN supports teams to access target communities for Needfinding, piloting, testing, scaling and community co-creation.  RAN organizes bi-monthly meetings with grantees to get status updates and discuss progress of innovations. Impacts: Impacts of these activities  Grantee innovators can illustrate the systematic procedures that successfully support the innovation process  Some of the grantee innovators have secured additional funding for their innovations  Grantee innovators are not working in silos but synergizewith the other innovator teams  The grantee innovators can tell their innovation story by text, video and pitching  The Grantees are able to refine their prototypes incorporating the community’s actual needs (community￾centric innovations)  All the Grantees have been supported to refine and pilot their prototypes in target communities  Some of the projects have been able to develop business models to turn their ideas to self-sustaining HESN Lab Canvases 98  Some communities have adopted RAN innovations even while they are still at pilot phase and these innovations have addressed their most pressing challenges and changed their lives. RAN: Development Actors Value Proposition:  Supporting in linking this organization to a university environment to harness Universities' participation in discussions regarding innovations in humanitarian work. RAN links development agencies to students and faculty in Universities.  Creating a critical mass of operational level staff who understand resilience.  The IGAD lab will be able to obtain a methodology with which to run the innovation activities in its labs; it will also access a large base of university students and youth.  UNDP's advocacy role enhanced through the involvement of skilled moderators from the academic environment.  Fostering a better understanding of how big data can be useful to understanding social problems.  Improve access to girl participants from partner universities and local schools.  Improve access to university students/youth to participate.  RAN is partnering with these US based agencies to develop new approaches and tools to aid understanding of complex systems under the USAID supported Monitoring, Evaluation, Research and Learning (MERL) Program.  RAN provides a platform on which Mozilla Uganda can reach more youth in institutions of higher learning.  RAN is developing a curriculum and materials for training local innovators in business plan development and another course on community based innovation.  (HESN Labs) Wide dissemination of HESN Lab innovation grant challenges to RAN's partner universities, hence widening African participation in their calls. A platform on which international students conduct internships in Uganda, pair-up with Ugandan students to conduct innovation related research in RAN partner communities. International students and faculty also helped to train local students in areas like GIS mapping and Ethnography for innovations. Primary recipient of services/activities:  ICRC (International Committee of RedCross) - Kampala, Uganda  United Nations Development Program (UNDP) East Africa -and the Intergovernmental Authority on Development (IGAD) under UNDP11/01/2016  UNDP Uganda  UN Global Pulse Lab: UN Global Pulse Lab was set up to harness big data safely and responsibly as a public good by accelerating discovery, development and scaled adoption of big data innovation for sustainable development and humanitarian action. Global Pulse is working to promote awareness of the opportunities RAN has engaged with Pulse Lab in organizing workshops on big data analytics.  Global Entrepreneurship Program for Girls - Technovation Challenge: The Technovation Challenge is a technology entrepreneurship program and competition for young women. This program teaches young women to build mobile applications to solve community challenges.  United Nations Population Fund Secondary beneficiary: Key Activities:  Partnership to organize ICRC's annual Humanitarian Event which this year focused on innovations for humanitarian service delivery. Comprised of a panel discussion and a humanitarian Innovations Exhibition. Partners:  Student mobilizers; facilitators: Mobilization of students; facilitation of the hackathon (UNFPA).  Makerere Faculty: Conducting the trainings, moderated dialogues; HESN Lab Canvases 99  RAN and UNDP-IGAD have written a joint proposal to establish an 'Innovation Hub' that will incubate Innovations targeting Climate Resilience in East and Horn of Africa hosted by IGAD but run using RAN's innovation methodology.  RAN developed a Post Graduate Certificate Course in Disaster and climate resilience.  RAN contributed to discussions on how universities can use big-data to address key development challenges.  RAN moderated a UNDP hosted national resilience dialogue that brought together a range of policy makers from international NGOs, development partners, government, civil society etc. to discuss priorities for operationalization of the Sendai Framework and Climate related SDGs.  Technovation challenge with the Global Entrepreneurship Program to encourage female innovators to step out with their ideas.  RAN partnered with UNFPA to organize a youth Design Challenge and Mobile Hackathon on Women Reproductive Health (Hack for Youth).  Monthly clinics for innovation projects in the area of engineering and architecture.  Wide dissemination of HESN Lab innovation grant challenges to RAN's partner universities. Created a platform on which international students conduct internships in Uganda, pair-up with Ugandan students and to conduct innovation related research in RAN partner communities. (AIDDATA, MIT-IDIN, MIT-CITE, UC Berkeley, DIL) – only lab with other HESN labs as beneficiaries. Brainstorming on the utility of big data to social research.  School teachers (girls initiative): Mobilization of girl teams to participate in the competitions.  Faculty from Nairobi University (IGAD).  Ampion Venture Bus to connect mentors, entrepreneurs and incubators from other regions Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  Held meetings where possible areas of collaboration for each party are delineated  Co-organizing the actual eventse,g GIS Hackathon, community co-creation,  RAN offered UNDP staff a course in disaster and climate resilience  Engagements between Makerere University students and US students from other HESN Labs(AIDDATA, MIT-IDIN, MIT-CITE, UC Berkeley, DIL) Impacts:  Additional funding: Rockefeller (through Bridgespan), Global Technovation and UNDP  Linkage to university faculty; linkage to RAN innovators as a test platform.  Access provided to university students/youth to participate.  Access provided to girl participants from partner universities and local schools.  # of operational level staff trained to understand resilience.  Trained 20 Disaster Management Officers in Uganda. 4 fellows were funded to conduct short studies on resilience to climate variability. 3 Fellows have submitted their manuscripts to peer reviewed journals for publication RAN: Target Communities Value Proposition:  Communities will be actively involved in the innovation process, right from needs-finding to solution refinement and piloting. HESN Lab Canvases 100  Access to innovations that address the actual needs of the communities. Through needs-finding, the end￾users are regularly consulted to provide iterations on the products and their insights help to refine the product in such a way that will address their actual needs. Primary recipient of services/activities:  Target communities selected because the Resilience problem and its magnitude is well manifested in that community. RAN has 18 partner Communities spread in the 4 resilience innovation hubs (EARILab-7; SARILab-4; WARILab 4; HoARILab 3);End-users-Farmers, traders,local leadership,women, youths, district leadership, etc. Key Activities:  Communities were actively consulted in the background formative qualitative resilience assessments to understand key drivers of resilience.  Quantitative surveys conducted in 15 of RAN's partner communities to quantify resilience factors and develop indices for measurement of resilience in these communities.  A new method of representative community consultation known as Deliberative Polling piloted (Uganda and Ghana), resulting in clear informed community recommendations regarding key policies affecting them.  Target communities involved in larger pilots and preparation for scale.  Target communities involved in testing of innovation concepts for RAN Grantees. Partners:  RAN Champion Agents: critical because they exercisealot of influence in the grassroots communities to enhance uptake of innovation products among the potential beneficiaries especially women.  RAN District Focal Persons: The District Leadership through the RAN Focal persons and structures at the grassroots e.g. sub-county chiefs exercise their authority and influence to enable innovators to access a variety of community groups for iterationand ethnography activities.  Funding partners: provide resources.  Business modelling experts; scaling partners: guide and accelerate the innovators' products.  Local Community members; Leaders Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  Community leaders and RAN Focal persons are contacted as entry points into the community  RAN identified community focal persons to link RAN to the target communities  The innovation teams have RAN champion agents in target communities  Community Co-creation workshop on innovation with the target communities  Community Consultations in the target communities through key informant interview, Focus group discussions, surveys  Stakeholder dissemination events are conducted in target communities to showcase progress of innovations Impacts: Impacts of these activities  Community participation increased through RAN’s Deliberative Polling approach to have people’s voicescontribute to policy reforms  Communities associate with RAN’s innovations since their views were solicited at the design stage (community-centric innovations)  Communities have supported RAN innovators to refine and pilot their prototypes in the target communities  Some communities have adopted RAN innovations even while they are still at pilot phase and these innovations have addressed their most pressing challenges and changed their lives.  The communities are appreciative of the innovations in addressing their most pressing needs e,g offered a free radio talkshow program to sensitize more communities  Communities have embraced partnerships with RAN innovations e.g signed MoUs to host some projects in the target communities for free RAN: Ugandan Policymakers Value Proposition: HESN Lab Canvases 101  Provide evidence on priority intervention pathways that will build resilience in communities affected by priority shocks and stressed in the target countries. There is a major gap in the involvement of universities in development programs and generating evidence for policy. Provide more evidence to inform policy on priority intervention pathways for resilience building.  Develop relevant and innovative solutions to priority resilience challenges. There is a major gap in how to measure resilience and what constitutes resilience in the local contexts.  Provide evidence on key policies, how communities perceive them, and how they can be improved to foster community acceptability. Primary recipient of services/activities:  Office of the Prime Minister (OPM); District local governments  Government ministries with a stake in RAN's thematic areas of focus (Water and Environment; Gender, labor and Social Welfare; ICT)  Parliament of Uganda (Committee on Physical Infrastructure, Gender, Labor & Social Development, Public Service & Local Government, Science & Technology, Natural Resources and ICT) Secondary beneficiary: Key Activities:  1st Annual State of Resilience Report for Africa  Intervention Strategy Workshops (ISWs): Through them, information from resilience assessments is processed by intervention experts to develop priority project pathways for resilience building.  Policy briefs on key findings on community opinions regarding key development issues from community Deliberative Polls.  Intervention Pathways identified in the ISWs resulted into innovation grant calls that have led to promising innovations.  Provide a set of national resilience indicators for their programming, response and recovery activities. Partners:  Tulane University's DRLA and CSIS'; IGAD: Technical support to data collection, analysis and report writing; Support in regional dissemination of the report.  Stanford University's ChangeLabs: Development of the ISW Tools and process.  Stanford University's Center for Deliberative Democracy; RILab teams; RAN Innovators; Tulane's DRLA: Development of the Deliberative Polling approach; technical support to implementation; Management of the innovation portfolio; development of the ideas; Technical support to the resilience surveys. Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  Launch of the 1stState of Resilience Report for Africa  Breakfast meeting where Deliberative Polling findings were disseminated  Intervention Strategy Workshops  Community Consultations  Innovation Advisory Boards  Expert panel reviewers during competitions for Grant Calls  Resilience Dialogue with key stakeholders Impacts: Impacts of these activities  RAN presented to parliamentarians and ministers about resilience to key stakeholders  RAN’s Deliberative Polling findings were presentedat a Breakfast meeting where MPs, ministers, policy makers and other development partners were in attendance.  Some Development partners have reached out to RAN to partner and support our innovators e.g Ministry of Gender, Labor and Social Development supported 4 of our innovators to attend a summit in Nairobi  RAN has leveraged support from line ministries e.g Issuing of licenses to pilot by Uganda Communications Commission HESN Lab Canvases 102 RAN: Faculty Value Proposition:  RAN provides opportunities to Faculty Innovators to get their ideas critiqued and to get positive feedback on how to improve their ideas. RAN also provides faculty who participate in resilience assessments with opportunities to publish the work from the assessments.  Acquiring skills in resilience assessments and conducting Deliberative polls, networking opportunities with other Faculty members in other Universities, consultancy opportunities to conduct the assessments, Publications and authorship for the studies conducted.  Offering them opportunities to participate in the development of Intervention strategy pathways, Funding opportunities for the faculty with innovations, pitching innovations, linking faculty innovators to scaling partners and Mentorship. Primary recipient of services/activities:  Faculty that are involved in Resilience Activities  Faculty involved in innovation activities for Faculty Secondary beneficiary: Key Activities:  Faculty are given training in resilience assessments and how to conduct Deliberative polls, Networking through regional and international travels for workshops, Knowledge sharing platforms.  Faculty are involved in RAN's Intervention Strategy Workshops to process resilience information and generate priority intervention pathways.  Faculty are involved in brainstorming ideation to develop concepts that are submitted for Grant opportunities e.g Global Resilience PartnershipFaculty innovators receive support in a range of innovation related capacity building activities including: Training in needs-finding, community co-creation, Funding for innovative projects, lab space in which to develop their concepts, support in the development of business models for their projects and mentorship. Partners:  Community  Government  HESN Universities  USAID  Scholars  Mentors  private sector  UN Agencies  Banks Delivery channels: The touch points through which your HESN Lab is interacting with this set of users/beneficiaries.  Intervention Strategy Workshops  Collaborative meetings for grant writing  Launch of the Resilience Innovation Challenge for Adverse Climate Effects (RIC4ACE) and the 1st state of resilience report  Consultation on qualitative and quantitative studiesconducted  Involve them as judges for Resilience innovation challenges and other competitions in the Lab  RAN uses faculty as entry points into University departments  RAN also uses faculty as mentors and coaches in supporting innovation projects  Involved faculty as experts in the recruitment process Impacts: Impacts of these activities  Faculty participate in innovation development  They appreciate and workwith the communities in the innovation process  Faculty engage Master students to find solutions to thecommunity challenges identified in RAN as a part of their Master theses  Faculty have leveraged additional funding for their projects  Faculty have gained skills and knowledge to innovate  RAN has shared the resilience methodologies with faculty to enhance knowledge generation and sharing HESN Lab Canvases 103 SEAD Beneficiary Segments:  Social Entrepreneurs  Other Social Entrepreneurs  Intermediaries  Impact Investors  Duke Students  Duke Faculty SEAD: Social Entrepreneurs Value Proposition:  Capacity building support: SEAD helps social entrepreneurs to scale their social impact by developing and strengthening skills to design effective business models, develop and implement scaling strategies, and attract sufficient resources. Also includes support to measure and articulate impact.  Connections, networking opportunities: SEAD facilitates connections and networking opportunities for SEs in support of their scaling strategies, including funders, potential partners, and mentors. Also includes opportunities for peer learning within SEAD cohort, and access to Duke Faculty.  Credibility in being associated with Duke brand; Use twitter and other media to highlight their successes. Primary recipient of services/activities:  Founders & Executive Management of Global Health Social Ventures in the SEAD Program: SEAD supports these beneficiaries to address common challenges faced by growth-stage social ventures in their effort to scale their impact. Challenges include: Strategic Planning; Access to Funding and Investment; Performance Management; Organizational Leadership and Talent; Leveraging the Ecosystem; Product/Service Development. Secondary beneficiary: Key activities/products/services:  SEAD Summit: Annual 3 day event bringing together SEAD SEs and relevant experts for learning, networking.  Peer-learning discussions: In-person or phone￾based conversations with all or a subset of innovators sharing best practices, discussing a specific subject, etc. (e.g. innovator exchange at SEAD summit).  Expert-led discussions: In person or phone￾based conversations with experts (e.g., Duke faculty, USAID, corporate partners, other leaders) on a pre-defined question or specific issue (e.g. patient data privacy, research strategies).  Fundraising support: Support thinking through funding strategy, preparing pitch materials or grant proposals, and practicing pitches for equity, debt or grant funding (help either from students or SEAD staff). Support can take the form of in-person or phone-based conversations as well as document review/feedback via email. Partners:  Innovations in Healthcare: Key SEAD Implementing partner; Leverage existing network and connections to provide support to SEAD SEs; Regular engagement with SEAD SEs to provide support in key challenge areas; Overall program management and strategic direction.  Center for the Advancement of Social Entrepreneurship (CASE): Improve SEAD SEs' ability to attract capital (CASE Initiative on Impact Investing); Overall program management and strategic direction.  Investors' Circle (IC): IC is an impact investing network, and provides support to SEAD SEs to: 1. Improve their ability to attract capital (coaching, mentoring) 2. Create a more engaged investor community, and therefore more investment opportunities 3. Provide opportunities for SEAD SEs (who are ready) to pitch to and engage with investors. HESN Lab Canvases 104  Facilitated directed connections/networking: Targeted introductions by SEAD team to strategic contacts (e.g. Innovations in Healthcare corporate partners) to explore business development opportunities and/or share challenges/solutions. Invitations by SEAD team to participate in SEAD-hosted or otherwise-hosted regional/global events to network and showcase organizational success (e.g. WISH summit).  Faculty/Student Projects: In-depth support on specific challenges (i.e. market research, business plan development) performed either by students (including summer internships) or by Duke faculty (including research collaborations).  Check-ins and conversations with SEAD team: Individualized support provided by key members of the SEAD team related to specific issues (i.e. strategic planning, performance metrics). This can also include regular check￾ins with engagement managers.  Tools/resources part of the SEAD knowledge base: Online learning, worksheets, case studies or other materials providing targeted information about key scaling challenges (e.g. fundraising strategies paper, innovator profiles).  SEAD Social Entrepreneurs: Provide peer learning opportunities for each other.  Duke Global Health Institute Evidence Lab: Impact evidence tool development; piloting and technical support to SEAD SEs.  Excelsior Group: Sub-contracted to provide direct support on specified project for select SEAD SEs in East Africa (co-funded with USAID/EA buy-in).  Open Capital Advisors: Sub-contracted to provide direct support on specified project for select SEAD SEs in East Africa (co￾funded with USAID/EA buy-in).  On Frontiers: Sub-contracted to provide direct support on specified project for select SEAD SEs in East Africa (co-funded with USAID/EA buy-in).  USAID/HESN  USAID/GH/CII  USAID/East Africa  Duke Global Health Institute: DGHI Evidence Lab Delivery channels:  Email  Phone calls/Skype  Webinars  Online modules  Site visits  Conference attendance  Workshops  SEAD Summit (3 day event)  Regional Meetings  Communication through partners Impacts:  Core HESN funding  Duke Cost-Share  Buy-in Funding from USAID/EA  Leveraged Innovations in Healthcare budget  Additional external funding achieved by an entrepreneur SEAD: Other Social Entrepreneurs Value Proposition:  Shared learning on factors that foster and inhibit scale of impact for SEs.  Tools and resources to support scale of impact.  Improved funder and accelerator environment for growth-stage SEs: Based on learnings from SEAD. Primary recipient of services/activities: Secondary beneficiary: HESN Lab Canvases 105  Founders and Management within social ventures: Across sectors (but with particular relevance to health); across geographies. Key activities/products/services:  Online Learning  Conferences  Research and publications  SEAD Symposium  IIH Program: IIH's greater program has been largely informed and adapted based on learnings gained from the SEAD program.  Knowledge products – papers, blogs, etc. Partners:  Innovations in Healthcare (IIH): Immediately applying learnings from SEAD to direct work with other SEs in their network; sharing learnings through publications, conferences, etc.  CASE: Sharing learnings through online modules, research.  Duke Global Health Institute Evidence Lab: Impact evidence tool development.  Investors' Circle: Increasing the number of impact investors interested in global health deals.  USAID/GH/CII: Adapting their program with SEs based on learning from SEAD.  IIH Program: IIH's greater program has been largely informed and adapted based on learnings gained from the SEAD program.  SEAD Social Entrepreneurs: Provide data regularly to SEAD through surveys and focus groups to help formalize and accelerate learnings to ultimately benefit other SEs. Delivery channels:  Conferences and events  Conversations with funders, accelerators  Online modules  Peer learning  Twitter and other social media  Knowledge products – papers, blogs, etc. Impacts:  Core HESN funding  Duke Cost-Share  Leveraged CASE budget  Leveraged Investors’ Circle budget  Buy-in from USAID/EA  Leveraged IIH budget SEAD: Intermediaries Value Proposition:  Improved understanding of the factors that foster and hinder the scale of impact of social entrepreneurs: To help improve and shape other programs in support of social entrepreneurs. Primary recipient of services/activities:  Program Directors and Coaches within accelerator programs for social entrepreneurs.  Grants managers and program strategy design staff within funding institutions. Secondary beneficiary: Key activities/products/services:  Conferences and events  Research and publications  SEAD Symposium Partners:  USAID/HESN, USAID/GH/CII, USAID GDL: Helping to spread out learning within and beyond USAID. HESN Lab Canvases 106  Online learning: Offering new ways to support social entrepreneurs.  CASE  IIH  Investors' Circle: Helping intermediaries to understand what impact investors expect and need. Delivery channels:  Conferences and events  Conversations with funders, accelerators  Online modules  Publications Impacts:  Core HESN funding  Duke Cost-share  Leveraged CASE budget  Buy-in from USAID/EA  Leveraged IIH budget  Accelerator: Increased understanding: understanding nuance of money you need, skills you need. Better matchmaking. Better advice given to innovators (same issues happening over and over again. innovators intermediaries are working with aren't ready for the money they're trying to pair them).  Use of products being developed by SEAD: online modules - use with innovators use of SEAD research for lessons learned; Be more evidence-based conference presentations, etc; Train the trainers model; capacity of intermediaries to be better intermediaries and help innovators.  Level of capacity of intermediaries  Quality control around accelerator inception SEAD: Impact Investors Value Proposition:  Increased number of strong candidates for global health deals: Better pipeline for potential investors.  Increased number of investors interested in investing in global health deal (shared risk).  Development and testing of innovative deal structures. Primary recipient of services/activities:  Impact Investors: Individual angel investors, impact investors within banks, foundation investors, fund managers, development finance institutions and other government funders, corporate funders. Secondary beneficiary: Key activities/products/services:  Global Health Advisory Board Meetings  IC Beyond the Pitch Events  Field Visits with Investors  Roundtable Discussions  Research & Publications: Identify the gaps, opportunities, and challenges for impact investors to provide appropriate capital for global health social ventures; share findings with funders and investors.  Coaching for global health social ventures: Efforts to better prepare global health social ventures to meet Partners:  Investors’ Circle  CASE/CASE i3  Investors’ Circle Global Health Advisory Board (GHAB)  Calvert Foundation  USAID & other grant-makers: Creating more effective hand-offs between types of funders; the benefit for impact investors would be entrepreneurs that are more ready for impact capital. HESN Lab Canvases 107 with and pitch to investors helps to instill more confidence in impact investors that these ventures are well￾prepared and investable. Delivery channels:  Conference side-meetings  Conference presentations/panels  Pitch events  Field visits with global health social ventures  Investors’ Circle network communications  GHAB Meetings: Quarterly meetings of the GHAB  Ad hoc, informal expert conversations: Ad hoc conversations between IC, CASE, and SEAD experts with impact investors or potential impact investors.  Publications, blogs Impacts:  Core HESN funding  Leveraging Investors’ Circle’s budget  Buy-in from USAID/EA  Duke Cost-Share  Leveraging CASE i3 effort  Awareness of investable deals in the global health space: Understanding and awareness of the growing body of investable innovations, especially at mid-stage; awareness that there are other players they can co-invest with that there are people they can pair with.  Angel Investors: Ways to reduce risk by doing early stage investments in health care in these two regions. Address preconceptions about working with investors that are also receiving government support.  Level of investment in global health.  Amount of investment that comes back to investors on time.  Confidence in investment.  Value of successful deals.  Number of new investors in global health.  Proxy indicator they're collecting: number of investors at their events that have global health deals. SEAD: Duke Students Value Proposition:  Experiences and skill building in global health innovation & social entrepreneurship  Inspiration  Connecting with students across disciplines  Resume-building  Learning Primary recipient of services/activities:  Duke graduate and undergraduate students with an interest in global health, social entrepreneurship, social innovation. Secondary beneficiary: Key activities/products/services:  Direct Engagement with global health innovation & social entrepreneurs: Summer internships- Consulting projects through Duke courses- Case Competition- SEAD Symposium (public forum)- Research projects- guest speakers in classes and events. Partners:  USAID/HESN: Provides summer internship program, opportunity to attend TechCon, and connections to other HESN Lab student opportunities (e.g. Berkeley Big Ideas, AidData summer internships).  Innovations in Healthcare: Provides opportunities for student research, HESN Lab Canvases 108  Academic learning opportunities: class projects; guest speakers.  Problem-solving opportunities: workshops; innovation challenge; TechCon; hackathon (EA).  Events  Support student innovation: Support student work in Pratt School of Engineering's Developing World Healthcare Technologies Lab; Support to student entrepreneurs, and student innovation competitions. internships, and projects with global health social entrepreneurs.  CASE (including CASE i3): Provides opportunities for student projects, internships, education, and participation in case competitions and innovation challenges related to social entrepreneurship.  Duke Innovation & Entrepreneurship Initiative: co-hosting events and opportunities with SEAD  Duke Global Health Institute: co-hosting events and opportunities with SEAD Delivery channels:  Internships  Research projects  Short-term consulting projects  Events: speakers; SEAD Symposium, TechCon  SEAD SAC  Competitions: Case competition, innovation challenge, support for other on-campus competitions  Website  Newsletter  Classroom opportunities Impacts:  HESN Funding  Leverage from Duke students (travel, etc)  Students working in international development: Have framework or knowledge for how to use business principles and how to think about sustainability and scale than they would without SEAD.  Buy-in from USAID/EA  Duke cost-share  Duke Africa Initiative funding  Increased multi-disciplinarily of student thinking  Level of interest in social entrepreneurship  Capacity of students to be better innovators/social entrepreneurs to address development challenge: and do so with an approach that's more informed. Affect career path and skill set, even those that won't work with USAID. SEAD: Duke Faculty Value Proposition:  Funding for research  Exciting, relevant, and inspiring course content  Opportunity for collaboration with other faculty  Building up the surge at the intersection of global health and social enterprise: *Creating broader academic hub  Hiring more faculty  Health care delivery system at Duke embracing international vision.  Global Health Institute  Contributes to Duke’s positioning as leader in global health innovation. o Gates Foundation $20 mill grant o New Center for Health policy -$16m global innovation angle o University-wide Innovation and Entrepreneurship Initiative -- core program in social entrepreneurship (led by Matt Nash) raise by 15m+ in philanthropy in entrepreneurship for Duke. HESN Lab Canvases 109  Recruit SEAD social enterprise leader into it  Contributing to and strengthening faculty-led efforts globally. Duke Africa initiative, have contributed to/received $ fund. Soon-to-be India initiative.  Duke faculty working with USAID in ways they wouldn't have before. Primary recipient of services/activities:  Duke Faculty with an interest in social innovation, global health, and/or social entrepreneurship - along with international development: Particularly SEAD research grantees, DGHI Evidence Lab, course collaborators, former Research Working Group members. Secondary beneficiary: Key activities/products/services:  Research Working Group: Met during first two years of SEAD to help make connections and encourage collaboration.  Research Grants: Awarded 4 competitive grants to diverse group of faculty; that initial funding has led some faculty to then take deeper interest in innovation and international development. For example, well￾known Duke Professor Dan Ariely received a SEAD research grant to work with an innovative health micro-insurance company in EA; while that line of research itself did not pan out, the exposure led that professor to create a more formal research program in East Africa.  Course content contribution: class projects related to GH & SE, case studies, guest speakers. Partners:  CASE  IIH  DGHI Delivery channels:  Courses  Research Working Group Listserv  SEAD, CASE, IIH communications Impacts:  USAID/HESN Core funding  Buy-In from USAID/EA List of Developing Country HEI and Research Institute Partners by HESN Lab 110 ANNEX X: LIST OF DEVELOPING COUNTRY HEI AND RESEARCH INSTITUTE PARTNERS BY HESN LAB High Engagement23 Medium Engagement Lab Dev country HEI Country Dev country HEI Country AidData Makerere University Uganda Universidad Nacional Autonoma de Honduras Honduras AidData Instituto Mora Mexico AidData Kathmandu University Nepal ConDev Christian Bilingual University of Congo's Integrative Research Institute (IRI) DRC ConDev Université Catholique du Graben DRC ConDev University of San Carlos in Guatemala Guatemala ConDev Congolese Institute for Nature Conservation (Virunga National Park) DRC CITE India Institute of Technology￾Gandhinagar India Singapore University of Technology and Design Singapore CITE India Institute of Management￾Ahmedabad Universidad de Ingeniería y Tecnología Peru CITE Kwame Nkrumah University of Science and Technology Ghana DIL Chiang Mai University Thailand Uganda National Tuberculosis Reference Laboratory Uganda DIL Instituto Mexicano de la Juventud Mexico University of the Witwatersrand South Africa DIL Makerere University Uganda Center for Research on Filariasis and other Tropical Diseases Cameroo n DIL Maseno University in Kisumu Kenya DIL Rwanda School of Public Health Rwanda DIL The Energy & Resources Group (TERI), India India DIL De La Salle University Philippines DIL Indian Institute of Technology - Bombay India DIL Iowa State University - Uganda Program: Establish & Grow/Nutrition Education Centers (formerly VEDCO) Uganda 23 Obtained from Dev Results Partner tables. Definition of level of engagement not defined by USAID; determined by individual HESN Lab. List of Developing Country HEI and Research Institute Partners by HESN Lab 111 High Engagement23 Medium Engagement Lab Dev country HEI Country Dev country HEI Country DIL National University, Philippines Philippines DIL Tecnologico de Monterrey Mexico DIL Jaduvpur University India DIL University of the Philippines, Diliman Philippines GCFSI Lilongwe University Of Agriculture And Natural Resources Malawi Alliance for a Green Revolution in Africa (AGRA) Kenya GCFSI The Energy Resources Institute (TERI) India GCFSI Biosciences Eastern and Central Africa - International Livestock Research Institute Kenya GCFSI Kudu Uganda GCFSI Innovation for Poverty Action Uganda GCFSI Centro Internacional de Agricultural Tropical Colombia IDIN United States International University (USIU) Kenya Arusha Technical College Tanzania IDIN Kwame Nkrumah University of Science and Technology (KNUST) Ghana Copperbelt University Zambia IDIN Singapore Polytechnic Singapore Nelson Mandela Institute of Science and Technology Tanzania IDIN Universidad del Valle Colombia IDIN Universidad Nacional de Bogotá Colombia IDIN B.S. Abdur Rahman University India IDIN University of Botswana Botswana IDIN Innovations for Poverty Alleviation Lab (I-PAL) at Information Technology University (ITU) Pakistan IDIN Centre for Research in Energy and Energy Conservation (CREEC) Uganda IDIN Center for Integrated Research and Community Development Uganda (CIRCODU) Uganda RAN Gulu University Uganda University of Ghana Ghana RAN University of Limpopo South Africa Ashesi University Ghana List of Developing Country HEI and Research Institute Partners by HESN Lab 112 High Engagement23 Medium Engagement Lab Dev country HEI Country Dev country HEI Country RAN Addis Ababa University Ethiopia Ghana Technology University College Ghana RAN Jimma University Ethiopia Mbarara University of Science and Technology Uganda RAN University of Pretoria South Africa Nkumba University Uganda RAN Benadir University Somalia University of Bamako Mali RAN University of Zimbabwe Zimbabwe Medical University of South Africa (MEDUNSA) South Africa RAN Lilongwe University of Agriculture and Natural Resources Malawi Kampabits Digital Design School Uganda RAN National University of Rwanda Rwanda University of Nairobi Kenya RAN University of Kinshasa Dem. Rep. Congo Muhimbili University of Health and Allied Sciences Tanzania RAN University for Development Studies (UDS) Ghana RAN Africa University Zimbabwe RAN Bule Hora University Ethiopia SEAD none none HESN Lab Collaboration 113 ANNEX XI: HESN LAB COLLABORATION AidData CITE ConDev DIL GCFSI IDIN RAN SEAD AidData (Black) (Blank) (Blank) (Blank) (Blank) (Blank) (Blank) (Blank) CITE (Blank) (Black) (Blank) (Blank) (Blank) (Blank) (Blank) (Blank) ConDev (Blue) (Blank) (Black) (Blank) (Blank) (Blank) (Blank) (Blank) DIL (Blue) (Blank) (Blue) (Black) (Blank) (Blank) (Blank) (Blank) GCFSI (Blue) (Blank) (Blue) (Blue) (Black) (Blank) (Blank) (Blank) IDIN (Blank) (Blank) (Blue) (Blank) (Blue) (Black) (Blank) (Blank) RAN (Blue) (Blue) (Blue) (Blue) (Blue) (Blue) (Black) (Blank) SEAD (Blank) (Blank) (Blank) (Blank) (Blank) (Blank) (Blank) (Black) (Blue): Collaboration occurred between Labs (Black): No lab partnership Bibliography 114 ANNEX XII: BIBLIOGRAPHY “Addendum (Evaluation Guide): RAN Final Pitch and Grant Selection Criteria.” ResilientAfrica Network. Makerere University, 2014. “An Agricultural Innovation Pipeline for Cambodia: Climate Change and Pressures on Resources.” Global Center for Food Systems Innovation. Michigan State University, n.d. “Annex A: Preparing Composite Maps of Pigeon Pea Suitability in Malawi.” Global Center for Food Systems Innovation. Michigan State University, 2015. “Annex B: Main Zones of Legume and Maize Production, Consumption, and Sales.” Global Center for Food Systems Innovation. Michigan State University, 2015. “Appendix A: East Africa Higher Education Handbook.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Assessing the Role of Information and Communication Technologies to Enhance Food Systems in Development Countries.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Blind Spots in International Development: A Big Ideas Essay Competition.” Development Impact Lab. University of California, Berkeley, Sept. 2014. Callen, Michael, Joshua Blumenstock, and Tarek Ghani. “Mobile-izing Savings: Evaluating a Phone-Based Defined-Contribution Account in Afghanistan.” Development Impact Lab. University of California, Berkeley, Jan. 2015. Carland, Corinne, Jarrod Goentzel, and Stan Finkelstein. “Evaluation Proposal: Malaria Rapid Diagnostic Tests.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute for Technology, Dec. 23, 2014. “Comments on Megatrend 2 Supplemental Paper on Urban Agriculture.” U.S. Agency for International Development. Washington D.C., Apr. 30, 2014. “Concept Note for USAID/Caucasus’ in Georgia Signature Effort: Increasing Citizen Engagement through STIP.” AidData. College of William and Mary, Sep. 25, 2014. “Conflict & Development: Monitoring and Evaluation Plan.” Center on Conflict and Development. Texas A&M University, Jul. 18, 2013. “Conflict & Development Six-Month Workplan: June 1, 2013 – Sept. 30, 2013.” Center on Conflict and Development. Texas A&M University, 2013. “Conflict & Development Six-Month Work Plan: Nov. 9, 2012 – May 9, 2013.” Center on Conflict and Development. Texas A&M University, 2013. Davis, Taryn, Diego Dimunzio, and Vanessa Goas. “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in Bangladesh.” “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in the Philippines.” AidData. College of William and Mary, Sep. 11, 2015. Dentoni, Domenico, Felix Krussmann, Mohammed Degnet, and Ali Noor. “Institutional and Policy Constraints to Innovation in the Malawian Legume Value Chains: Current Status and Business Actors’ Coordination for Institutional Change.” Global Center for Food Systems Innovation. Michigan State University, 2015. “Development Impact Lab: Year 3 Work Plan.” Development Impact Lab. University of California, Berkeley, Aug. 1, 2014. Dirkx, John et al. “Workforce Development in East Africa: Fostering Transformative Change in the Agricultural Education and Training System.” Global Center for Food Systems Innovation. Michigan State University, 2013. Dirkx, John, Thomas Smith, Lucas Hill, Inese Berzina-Pitcher, Mary Vann, Cuong Nguyen, Keran Zhu, Taylor Bauchan, and Kyler Ransom. “East African Implementation: Malawi Research Report Megatrend Three Skills and Workforce Development Team.” Global Center for Food Systems Innovation. Michigan State University, 2015. Bibliography 115 “Draft Work Plan for Solar Pump Evaluation.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute for Technology, n.d. Dzanja, Joseph, Mirriam Matita, Henry Kankwamba, Michael Dolislager, and David Tschirley. “Mapping Market Prospects for Grain Legumes in Malawi.” Global Center for Food Systems Innovation. Michigan State University, May 2015. “Experimentation in Product Evaluation: The Case of Solar Lanterns in Uganda, Africa.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, Jan. 20, 2015. “Final Data PPT IDDS Zero Waster.” International Development Innovation Network. Massachusetts Institute of Technology, Jun. 2015. “First Year Workplan: Higher Education Solutions Network.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, Jul. 11, 2013. Goas, Vanessa, and Joshua Powell. “AidData Center for Development Policy – Indonesia Assessment.” “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in the Philippines.” AidData. College of William and Mary, Mar. 6, 2014. Goas, Vanessa, and Joshua Powell. “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in the Philippines.” AidData. College of William and Mary, May 8, 2015. Goentzel, Jarrod. “Evaluation Proposal: Food Aid Packaging.” Center on Conflict and Development. Texas A&M University, n.d. Goentzel, Jarrod, Timothy W. Breitbach, Mark E. Brennan, Emily J. Gooding, Stephan Graves, Jennifer L. Green, Jonars B. Spielberg, and Mike Veatch. “Post-Harvest Food Storage Refined Research Framework.” Center on Conflict and Development. Texas A&M University, n.d. “Higher Education Solutions Network – Annual Report.” Global Center for Food Systems Innovation. Michigan State University, Oct. 30, 2013. “Higher Education Solutions Network Annual Report.” Global Center for Food Systems Innovation. Michigan State University, Nov. 7, 2014. “Higher Education Solutions Network – FY2014 Work Plan (Year 2).” Center on Conflict and Development. Texas A&M University, Aug. 21, 2013. “Higher Education Solutions Network – FY2014 Work Plan (Year 2).” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, Jul. 30, 2013. “Higher Education Solutions Network – FY2014 Work Plan (Year 2).” ResilientAfrica Network. Makerere University, Mar. 14, 2014. “Higher Education Solutions Network – FY2014 Work Plan (Year 2).” International Development Innovation Network. Massachusetts Institute of Technology, Aug. 30, 2013. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” Center on Conflict and Development. Texas A&M University, Aug. 10, 2014. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, Sep. 26, 2014. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” Global Center for Food Systems Innovation. Michigan State University, Oct. 6, 2014. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” International Development Innovation Network. Massachusetts Institute of Technology, Aug. 17, 2014. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” ResilientAfrica Network. Makerere University, 2015. “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” Social Entrepreneurship Accelerator at Duke. Duke University, 2014. Bibliography 116 “Higher Education Solutions Network – FY2015 Work Plan (Year 3).” Social Entrepreneurship Accelerator at Duke: East Africa. Duke University, Oct. 13, 2014. “Higher Education Solutions Network – FY2016 Work Plan (Year 4).” Global Center for Food Systems Innovation. Michigan State University, Sep. 9, 2015. “Higher Education Solutions Network Monitoring & Evaluation Plan.” Development Impact Lab. University of California, Berkeley, Jan. 14, 2014. “Higher Education Solutions Network Monitoring & Evaluation Plan.” Global Center for Food Systems Innovation. Michigan State University, Jul. 1, 2013. “Higher Education Solutions Network – Year 1 Work Plan.” International Development Innovation Network. Massachusetts Institute of Technology, 2013. “Higher Education Solutions Network: ResilientAfrica Network: Proposed Sub-Awards Under the Innovation Accelerator/Eastern Africa Resilience Innovation Lab (EARI Lab).” ResilientAfrica Network. Makerere University, Aug. 2014. “IDDS Aarogyam Final Data Summary.” International Development Innovation Network. Massachusetts Institute of Technology, Aug. 2015. “IDDS D’kar Summit Slides.” International Development Innovation Network. Massachusetts Institute of Technology, Aug. 2015. “IDDS Presentation for USAID Tanzania-1.” International Development Innovation Network. Massachusetts Institute of Technology, 2014. “India ECAR Visit_June 2015 Compant.” Development Impact Lab. University of California, Berkeley, Jun. 2015. “Information and Communication Technology for Development.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Information and Communication Technologies in Agriculture in South and Southeast Asia: White Paper 2014 Series.” Global Center for Food Systems Innovation. Michigan State University, 2014. Leith, Kendra. “CITE Monitoring and Evaluation Plan for USAID.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, Mar. 25, 2014. “Making it Matter: Using AMP as a Tool for Change.” AidData. College of William and Mary, 2013. Me-Nsope, Nathalie and Michelle Larkins. “Gender Analysis of the Pigeon Pea Value Chain: Case Study of Malawi.” Global Center for Food Systems Innovation. Michigan State University, 2014. “Meeting the Challenge of Open Data: AMP Good Practices Workshop 2014.” AidData. College of William and Mary, 2014. “Megatrend One: Population Growth, Climate Change and Pressure on the Land.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Megatrend Three: Evolution in Skills Required by Food Systems Transformation.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Megatrend Two: Rapid Urbanization and Food System Transformation.” Global Center for Food Systems Innovation. Michigan State University, Aug. 2013. Messina, J, U. Adhikari, J. Carroll, R. Chikowo, M. DeVisser, L. Dodge, P. Fan, S. Langely, S. Lin, N. Me-nsope, N. Moore, S. Nawyn, A. Nejadhashemi, J. Olson, A. Smith, and S. Snapp. “Population Growth, Climate Change and Pressure on the Land: Eastern and Southern Africa.” Global Center for Food Systems Innovation. Michigan State University, 2014. “MIT-CITE Factsheet.pdf.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute of Technology, n.d. “MIT-IDIN-Fact Sheet-Sept112014.” International Development Innovation Network. Massachusetts Institute of Technology, Sep. 11, 2014. Bibliography 117 “MIT International Development Innovation Network (IDIN) Monitoring and Evaluation Plan.” International Development Innovation Network. Massachusetts Institute of Technology, Apr. 2, 2014. “Monitoring & Evaluation Plan.” Social Entrepreneurship Accelerator at Duke. Duke University, 2015. Moore, Nathan, Amirpouyan (Pouyan) Nejadhashemi, Joseph Messina, et al. “Malawi Final Report – Megatrend One.” Global Center for Food Systems Innovation. Michigan State University, 2015. Nash, Matthew T.A., Richard Bartlett, and Kimberly Langsam. “Higher Education Solutions Network – FY2014 Work Plan (Year 2).” Social Entrepreneurship Accelerator at Duke. Duke University, 2013. Pearlman, Jon and Matt McCambridge. “Proposal to Evaluate Wheelchairs for use in Less Resourced Settings.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute for Technology, n.d. “Population Growth, Climate Change and Pressure on the Land.” Global Center for Food Systems Innovation. Michigan State University, 2013. Powell, Josh. “Opening the Development Experience Clearinghouse (DEC): AidData Program Report.” AidData. College of William and Mary, Jul. 23, 2015. “Program Overview and Reflections.” Social Entrepreneurship Accelerator at Duke. Duke University, Apr. 2015. “Project 3: Educational Technology Work Plan.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute for Technology, Feb. 15, 2015. “Proposal for Supplementary Funding: A Randomized Evaluation of Household Grants in Rwanda.” Development Impact Lab. University of California, Berkeley, Aug. 5, 2015. Reardon, Thomas, David Tschirley, Michael Dolislager, Jason Snyder, Chaoran Hu, and Stephanie White. “Urbanization, Diet Change, and Transformation of Food Supply Chains in Asia.” Global Center for Food Systems Innovation. Michigan State University, May 2014. “ResilientAfrica Network: First Round of Projects Selected for Support Under the Resilience Innovation Challenges (RIC).” ResilientAfrica Network. Makerere University, 2014. “ResilientAfrica Network: Monitoring and Evaluation Plan.” ResilientAfrica Network. Makerere University, Sept. 2014. “ResilientAfrica Network – Year One Workplan Final_5_10_13.” ResilientAfrica Network. Makerere University, May 10, 2013. “Rethink Relief: Rethink Relief 2013 Toward Well-Being.” D-Lab, Makerere University, 2013. “Revised Research Plan for Water Test Kits, India 2015.” Comprehensive Initiative on Technology Evaluation. Massachusetts Institute for Technology, 2015. “SEAD Capacity Building Planning Process and Map: Summary of Year 3 Program Plan.” Social Entrepreneurship Accelerator at Duke. Duke University, 2015. “SEAD FY14 Annual Report: Lesson Learned.” Social Entrepreneurship Accelerator at Duke. Duke University, 2014. “SEAD Proposal for Collaboration with USAID East Africa Mission.” Social Entrepreneurship Accelerator at Duke. Duke University, Feb. 14, 2013. “SEAD Summit 2015.” Social Entrepreneurship Accelerator at Duke. Duke University, 2015. Snapp, Sieglinde S., Vicki L. Morrone, Wezi Mhango, and Leo C. Zulu. “Malawi 2014 Final Report – Malawi Agroecology Team.” Global Center for Food Systems Innovation. Michigan State University, 2014. “Social Entrepreneurship Accelerator at Duke (SEAD): Lessons from Work with ‘Pilot Cohort.’” Social Entrepreneurship Accelerator at Duke. Duke University, 2014. “Social Entrepreneurship Accelerator at Duke (SEAD) – USAID’s Higher Education Solutions Network: Year One Workplan.” Social Entrepreneurship Accelerator at Duke. Duke University, 2012. Bibliography 118 Steinfield, Charles, Susan Wyche, Hastings Chiwasa, Tian Cai, and Japhet Mchakulu. “Using Participatory Video for Smallholder Farmer Training in Malawi: A Report from the GCFSI and LUANAR ICTD Project Team.” Global Center for Food Systems Innovation. Michigan State University, Apr. 2015. Stern, Alena. “Uganda Civil Society Sustainability Assessment Results.” “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in the Philippines.” AidData. College of William and Mary, August, 2014. Tierney, Michael J. “2015-2016 Higher Education Solutions Network (HESN) Scope of Work Data Transparency/Junior Program Manager.” “Feasibility Study: Implementation of a Multi-Donor Geocoding Project in the Philippines.” AidData. College of William and Mary, March, 2015. Tierney, Michael J., Michael G. Findley, and Joseph K. Young. “USAID Regional Governance Activity (RGA) in Colombia: Impact Evaluation Project Description 2015-2019.” AidData. College of William and Mary, Mar. 27, 2015. Tschirely, David, Steven Haggblade, and Thomas Reardon. “Africa’s Emerging Food System Transformation: Urbanization, Diet Change, and the Midstream and Downstream Transformations.” Global Center for Food Systems Innovation. Michigan State University, 2013. “Updates from IDIN Academic Partners: April – June 2015.” International Development Innovation Network. Massachusetts Institute of Technology, Jun. 2015. “Updates from IDIN Innovation Center Partners: April 2015.” International Development Innovation Network. Massachusetts Institute of Technology, Apr. 2015. “Updates from IDIN Innovation Center Partners: Dec. 2014.” International Development Innovation Network. Massachusetts Institute of Technology, Dec. 2014. “Updates from IDIN Innovation Center Partners: February 2015.” International Development Innovation Network. Massachusetts Institute of Technology, Feb. 2015. “Updates from IDIN Innovation Center Partners: From the July 2015 Reports.” International Development Innovation Network. Massachusetts Institute of Technology, Aug. 2015. “Updates from IDIN Innovation Center Partners: October – November 2014.” International Development Innovation Network. Massachusetts Institute of Technology, Nov. 2014. “USAID Development Lab For Technology Impact: Year 1 Work Plan.” Development Impact Lab. University of California, Berkeley, n.d. “Where the Money Goes: Mapping Aid for Better Impact: Monitoring and Evaluation Plan (M&E Plan).” AidData. College of William and Mary, Mar. 20, 2014. White, Stephanie and Eric Crawford. “Draft: Synthesis Report on Malawi Research Projects, FY 2014.” Global Center for Food Systems Innovation. Michigan State University, Apr. 14, 2015. White, Stephanie and Michael W. Hamm. “Urban Agriculture and a Planning Approach to Urban Food Systems.” Global Center for Food Systems Innovation. Michigan State University, n.d. White, Stephanie A., Michael W. Hamm, Agnes Mbachi Mwangwela, Judith F.M. Kamoto, Jessice J.M. Kampanje-Phiri, Fanny C. Chigwa, and M. Thonodolo. “Traditional Urban Legume Exchange in Lilongwe, with a Focus on Pegeon Pea: Identifying Opportunities and Constraints to the Scaling of Multipurpose Legume Innovations in Maize-Based Farming Systems.” Global Center for Food Systems Innovation. Michigan State University, Apr. 2015. “Work Plan – Year 1: November 8, 2012 to September 30, 2013.” Global Center for Food Systems Innovation. Michigan State University, Apr. 4, 2013.