September 2014 AN EXTERNAL PERFORMANCE EVALUATION OF THE FEED THE FUTURE INNOVATION LABS FOR NUTRITION IN ASIA AND AFRICA By: Barbara J. Stoecker, Kathleen Musante, Jessica Fanzo Awards Numbers: AID-OAA-L-10-00005 (Asia); AID-OAA-L-10-00006 (Africa) Photo credit: MP/Kyu Lee TABLE OF CONTENTS Table of Contents............................................................................................................................................. 1 List of Acronyms............................................................................................................................................... 3 List of Figures and Tables................................................................................................................................ 5 Executive Summary .......................................................................................................................................... 6 Background ..................................................................................................................................................... 14 Program Management................................................................................................................................... 17 Findings ........................................................................................................................................................ 17 Conclusions................................................................................................................................................. 35 Lessons Learned ........................................................................................................................................ 37 Recommendations ..................................................................................................................................... 37 Research Program ......................................................................................................................................... 39 Findings ........................................................................................................................................................ 41 Conclusions................................................................................................................................................. 60 Lessons Learned ........................................................................................................................................ 60 Recommendations ..................................................................................................................................... 61 Training Program........................................................................................................................................... 62 Findings ........................................................................................................................................................ 62 Conclusions................................................................................................................................................. 71 Lessons Learned ........................................................................................................................................ 71 Recommendations ..................................................................................................................................... 71 Institutional Collaboration.......................................................................................................................... 73 Findings ........................................................................................................................................................ 73 Conclusions................................................................................................................................................. 82 Lessons Learned ........................................................................................................................................ 82 Recommendations ..................................................................................................................................... 83 Program Future.............................................................................................................................................. 84 Findings ........................................................................................................................................................ 84 Conclusions................................................................................................................................................. 86 1 Recommendations ..................................................................................................................................... 87 Recommendations for second phase .................................................................................................... 88 Appendix A: Scope of work ........................................................................................................................ 90 Appendix B: Evaluation plan........................................................................................................................ 96 Appendix C: Travel itinerary, locations & dates of visits.................................................................... 172 Appendix D: Methods................................................................................................................................. 173 Appendix E: List of persons contacted ................................................................................................... 179 Appendix F: List of materials reviewed................................................................................................... 185 Appendix G: Photographs.......................................................................................................................... 187 Appendix H: Management structure ....................................................................................................... 190 Appendix I: Summary of research projects............................................................................................ 193 Appendix J: Project publications and research briefs.......................................................................... 201 Appendix K: ME (Tufts University) comments...……………………………………………….203 2 LIST OF ACRONYMS A4NH Agriculture for Nutrition and Health AAAS American Association for the Advancement of Science ACBT Aquagenx Compartment Bag Test AOR Agreement Officer’s Representative BBNC Bangalore Boston Nutrition Collaborative BFS Bureau for Food Security BoD Board of Directors CAES College of Agricultural and Environmental Sciences CC Community Connector Project (In Uganda) CGIAR Consultative Group for International Agricultural Research CRSPs Collaborative Research Support Programs CTO Cognizant Technical Officer DAI Development Alternatives International DHS Demographic and Health Survey ECD Early Child Development EET External Evaluation Team FANTA Food and Nutrition Technical Assistance F&V Fruit and Vegetables HKI Helen Keller International HSPH Harvard School of Public Health IFPRI International Food Policy Research Institute IL/CRSPs Innovation Labs/Cooperative Research Support Programs IOM Institute of Medicine (Nepal) IRB Institutional Review Board for the Protection of Human Subjects JHU Johns Hopkins University LUANAR/Bunda Lilongwe University of Agriculture and Natural Resources/formerly Bunda College LWA Leader with Associates ME Management Entity MOAD Ministry of Agriculture and Development MOU Memorandum of Understanding NASA National Aeronautics and Space Administration N/CRSP Nutrition Collaborative Research Support Program NGOs Non-governmental Organizations NLSS Nepal Living Standards Survey NTAG Nepali Technical Assistance Group OPM Office of the Prime Minister PD Project Director PI Principal Investigator 3 POPM Policy and Operating Procedures Manual PoSHAN Policy and Science for Health, Agriculture and Nutrition RFAs Request for applications ROP Rural Outreach Programme SOW Statement of Work SPRING Strengthening Partnerships, Results, and Innovations in Nutrition Globally SUN Scaling Up Nutrition TAG Technical Advisory Group USAID United States Agency for International Development USG United States Government VDC Village Development Committee WASH Water, Sanitation and Hygiene 4 LIST OF FIGURES AND TABLES Figure 1: Organizational Structure of the Global Nutrition Innovation Lab................................... 18 Table 1: Summary of Research Projects................................................................................................... 50 Figure 2: Appropriateness of Training ...................................................................................................... 67 Figure 3: Quality of Training........................................................................................................................ 67 Figure 4: Training Relevance ....................................................................................................................... 68 Figure 5: Subject Areas of Future Workshops........................................................................................ 69 Figure 6: Quality of Mentoring ................................................................................................................... 70 Figure 7: Adequate Funding for Training.................................................................................................. 70 5 EXECUTIVE SUMMARY The External Evaluation Team (EET) concludes that both Nutrition Innovation Labs have met, and, in some cases, when challenges faced are taken into account, exceeded what the EET considers would be reasonable expectations for projects of this scale at the four-year mark. The EET recommends that both Nutrition Innovation Labs should continue into a second 5- year funding phase. However, the EET finds that both Innovation Labs need to address issues in communication among partners and with policy makers and the broader scientific community. Sharing of ideas and methods among the projects, seeking additional expertise and partners as needed and timely implementation of data collection phases will enhance utility of the work. At this writing, the Nutrition Innovation Lab/Asia is running more smoothly than the Nutrition Innovation Lab/Africa, and has had more useful interactions with policy makers. MAJOR FINDINGS AND RECOMMENDATIONS Management The EET finds the governance structure of the Nutrition Innovation Labs to be clearly laid out and that the governing entities – the Management Entity (ME), Board of Directors (BoD), and the Technical Advisory Committee (TAC) -- operate in the ways in which they are outlined in the Policy and Operating Procedure Manual (POPM), although the role of the TAC is not as large as the POPM suggests it should be. The ME sees the TAC as a more informal body of advisors than some TAC members would prefer. In general, overall fiscal management of both Nutrition Innovation Labs is effective and timely, although there is a specific issue in the Nutrition Innovation Lab/Africa with some confusion and a lack of consensus about the ways in which student research at Makerere University is to be funded and carried out. Issues in the fiscal management of the training program at Makerere University raise concerns about the effectiveness of communication among the several partners involved – the ME at the Tufts University Friedman School of Nutrition Science and Policy, Tuskegee University and Makerere University. Overall management of both of the Labs is acceptable, but the management of the Nutrition Innovation Lab/Asia is more effective than the management of the Nutrition Innovation Lab/Africa. The global ME and the Program Directors (PDs) for both Nutrition Innovation Labs have strong collegial relationships with most of the partners, both U.S. based and host country based. The ME and the Nutrition Innovation Labs currently have good relationships with the host country USAID Missions and the implementation projects with which they partner. Early tensions due to misunderstandings of the Nutrition Innovation Labs 6 objectives have been resolved in both countries (Nepal and Uganda). However, the EET concludes that the major sub-awardee for the research in Uganda, Harvard University, does not have a strong Nutrition Innovation Lab based presence in Uganda. This, along with changes in personnel and delayed roll-out of the Uganda Community Connector Project has resulted in delays in initiating the cohort study and carrying out the second data collection time-point of the panel study. Miscommunication among the partners involved in training Masters’ students at Makerere University is not yet resolved and needs attention. This issue applies both to fiscal management and the impact of the program on the training and capacity building missions of the Nutrition Innovation Lab. Tuskegee University may not have sufficient fiscal management strengths or administrative skills to manage the budget, and provide adequate annual reports. There also needs to be some thought into who should manage the students and, moving forward, how student projects should be designed to support the objectives of the Nutrition Innovation Lab/Africa and how many students are manageable for each cohort. The EET also finds, and the ME staff agree, that the ME staff is overstretched. There is need for increased capacity of the ME staff as the projects attract more Associate Awards. The ME also needs to fill the position of Communications Director and devote attention to the website, communication with stakeholders, and publications. In terms of communications, the Nutrition Innovation Lab website is not currently an effective communication tool and the Global Nutrition Collaborative Research Support Program (NCRSP) Workspace is minimally used as a vehicle for communication among partners. However, Research Briefing Papers available on the website are informative, timely and readily available. More importantly, while it may be too early to be making strong policy statements on the basis of the data collected to date, policy makers in Nepal, Uganda and Malawi feel that they would like more feedback on policy implications of the projects. Overall, the EET concludes that the model of the Nutrition Innovation Labs under a single ME has provided important advantages, both in the efficient use of resources, but also as a way of addressing the overall goal of the projects to address in depth, and at scale, the critical pathways through which nutrition, health and agricultural interventions are connected. The management of the Labs is more efficient by sharing an ME, but the current staff is overstretched and the ME may need to add one or more staff positions. While ME members clearly have strong collegial relationships and share insights with each other, the PDs and sub- awardee Principal Investigators (PIs) do not share expertise as effectively as they could. Research activities between the two Innovation Labs, guided by similar objectives, are more comparable than they might have been if the Labs did not share an ME. However, while projects in Nepal and Uganda provide different cuts through the same set of objectives, they 7 do not use the same design or share all survey research instruments. Management of the projects under one ME, BoD and with the input of a single TAC has not resulted in a shared design or shared metrics for the projects. Research The primary research projects – the Policy and Science for Health, Agriculture and Nutrition (PoSHAN) community and policy projects in Nepal, and the panel study in Uganda -- are on track to strategically answer the three high level questions to varying degrees with the majority being able to address the Agriculture to Nutrition Pathways: seeking greater clarity on cause and effect pathways (agriculture to nutrition). For example, they should be able to address what ‘types’ of agriculture investment have greatest net impacts on nutrition in the countries in which they are being carried out. However, several of the projects would benefit from a greater level of integration of agriculture, climate and ecosystems research with the ongoing research on nutrition and health outcomes. Several researchers with the kinds of expertise that could be incorporated, especially colleagues at Purdue University, are already available within the consortia. More active incorporation of these colleagues would increase the likelihood that robust questions focused on agricultural options are incorporated in the research. Currently, most of the project design and data collection is led by public health experts. As currently designed, the research projects are largely addressing program effectiveness. However the EET believes that due to differences in design and metrics, it may prove difficult to put the findings from the different projects together into a cohesive story that can be effectively communicated. Although the current research projects are distinct, the projects should be able to fit together strategically into the higher level questions. At present, the EET finds that some of the projects are opportunistic and interesting, but not as strategically linked as they could be. The EET concludes that there needs to be a more cohesive thread of research that might rest on greater attention to shared methods and instruments, joint publications and policy statements for USAID’s programmatic and investment decision-making. There is a missed opportunity to develop a consistent set of metrics and comparable instruments both across the several research efforts and for use in future Associate Awards and by other Innovation Labs and international researchers. Policy analysis is critical to the mission of the Nutrition Innovation Lab projects, but the potential for impact has faced challenges in both Labs as a result of shifting government personnel in Nepal, the stability and consistency of governments in both countries where the labs work, and limitations imposed by U.S. policy with respect to corruption in the Office of the Prime Minister in Uganda. These issues present a challenge for the research regarding timing 8 and sharing of results. There is also a need to ensure that presenters are sensitive to the political nature of the data presented. Finally, at the time of the evaluation, several key stakeholders in both regions noted that they see the Nutrition Innovation Labs’ work as too much of an academic exercise, with data being made available for policy use very slowly. Policy makers in Nepal, Uganda and Malawi expressed concern that the data and analyses they needed were not yet available. However, since the EET visits to these countries, the ME has reported that they have made a series of policy presentations in Nepal. The EET knows of no similar series of presentation in Uganda. The EET concludes that much of the research being carried out in both Innovation Labs will address existing knowledge gaps and provide high quality data both on the fundamental relationships between agricultural production, nutrition and health, and on the effectiveness of some targeted interventions in the areas under study. They will contribute to an understanding of how to effectively deliver packages of agriculture and nutrition interventions in various settings. However, the data are being presented more slowly than policy makers and partners would like. The number of research publications currently published or in review is small. Training Overall, Nutrition Innovation Lab is seen as making an important contribution to strengthening the training and capacity building in nutrition for scientists working across sectors, particularly in Nepal, where there is a clear dearth of nutritionists in the country, and in Malawi, where there is, at this time, no dietetics curriculum in place. Both Nutrition Innovation Labs have met their targets for short and long term training opportunities. Participants in both long- and short-term training have a positive view of the quality of their training, and overwhelmingly see it as important to their own professional development and appropriate to the problems in their countries. However, to date, there have been few opportunities for host country collaborators to participate as authors or co-authors in data analysis and peer reviewed publications. The EET finds that long-term degree training is less advanced than short-term training. At Purdue University, two students are well along in their curriculum, and are producing useful analyses of data related to the project. A faculty member at Uganda’s Gulu University, completing his doctoral degree at Cornell University, has been supported for dissertation research. However, other degree students do not as yet seem to be producing data and analyses related to the project. 9 The EET finds that more can be done to integrate and build capacity for agriculturalists working in public health and nutrition. More training is recommended to continue to build strengths in research design and methods, and following that, training in data analysis and writing publications. The EET believes that the Nutrition Innovation Labs can play a more prominent role in ensuring that host country researchers are incorporated into data analysis and are afforded opportunities to be first authors on peer reviewed publications, present findings stemming from the research, and be seen as the “go to” knowledge keepers with those that make a difference in the country: policymakers and programmers. Building Institutional Collaborations Overall, the ME has done an effective job in communicating and establishing collaborations with host country governments, academic institutions, international non-governmental organizations (NGOs) and local NGOs within a very complex set of projects. However, this also means that there are many entry points for capacity development – some that have been harnessed and others that may still need to be developed. The capacity building and collaborations at the “global level” have been very successful in that the Labs two Program Directors (PDs) have presented findings and concepts in many fora, as well as, with other Feed the Future Innovation Labs. In the U.S. and within the international fora, the bulk of presentations have been made by ME members. The EET finds, however, that there is some concern among collaborators and policy makers interviewed that not many local collaborators have taken ownership of data analysis or writing of papers resulting from the research that has taken place thus far by the Nutrition Innovation Labs. There may be a need to ensure more time investment is made to build the research capacity of local persons in the areas of agriculture-health-nutrition linkages in the country, being that currently, very little exists. For the Nutrition Innovation Lab/Asia, the EET finds that there is significant effort to build the capabilities of researchers to enhance future collaborations in Nepal. The Lab is working with local institutions to develop a curriculum for a master’s degree in public health nutrition. Local collaborators and project staff have opportunities to engage in project planning and implementation. The scientific seminars organized under the Nutrition Innovation Lab by Johns Hopkins University are widely seen as critical fora for the exchange of information and ideas. However, in the case of the PoSHAN community project, the local PI and collaborating universities are less involved in data collection, management and analysis than in the research in Uganda. In Nepal the data collection is managed by JHU PIs and staff, and the data collection is carried out by data collection companies with whom the PIs contract for that purpose. 10 The Nutrition Innovation Lab/Africa efforts in Malawi to address a severe lack of expertise are notable and very important. In Uganda the EET sees less of a lack of human resources, but a distinct need for capacity building in existing resources for data analysis, especially for policy analysis. However, the nature of the collaboration with host country partners is rich, in that the PI and staff at Makerere University directly manage data collection. Presumably, in both regions, host-country officials understand the key objectives of the Nutrition Innovation Labs and of Feed the Future programs well enough to make strong presentations. In many settings, an outstanding presentation from a host-country official would be more powerful than from a visitor. Both Nutrition Innovation Labs could support an enabling environment for such activities by providing assistance for effective communication of project objectives and results. Program Future Overall the EET’s analysis of the Nutrition Innovation Labs is positive. The data being generated by the Labs fits directly into the Feed the Future Research Strategy as well as USAID’s Multi￾Sectoral Nutrition Strategy (2014-2025). However, the “wrap around” strategy, in which, in the vision of the original Agreement Officer’s Representative (AOR or USAID project manager), the Nutrition Innovation Labs would wrap around the production and value chain oriented Feed the Future projects in a way that would allow the Labs to address the pathways between agriculture and nutrition has not yet gained traction with other Feed the Future programs. Associate Awards, including several currently under development, have the potential to further enhance the degree to which the Nutrition Innovation Labs directly address operational concerns in the broader Feed the Future Research Strategy, and the current research programs are generating data that has a strong likelihood of impact. It would be of value to see how the findings from all the current and upcoming Associate Awards coalesce and what messages and evidence can be garnered from the entire body of work collectively. However, because of understandable delays in the initiation of data collection, neither Nutrition Innovation Lab has yet collected the data proposed to address operational research issues nor some of the key Feed the Future indicators. Work on a set of metrics that could be adopted by other Feed the Future projects has not yet been addressed, but could be in the future. As a result of research design, the impact of the research programs will be partially dependent on hypothesis driven data analyses that address the top-level questions posed in the proposals. A stronger incorporation of information from agricultural research and intervention would strengthen the likelihood that the research programs will provide the information necessary to fill critical gaps in knowledge. 11 While specific recommendations for each area of concern can be found in the body of this report, the EET recommends several broad areas in which the Nutrition Innovation Labs could enhance the programs in a second phase: Re-assess Collaborators and Partnerships In a second phase, the ME should take the opportunity to re-assess collaborations that are not working well in the first phase of funding. The EET has specifically identified relationships with Tuskegee University for both the Innovation Labs, and Harvard University in the Nutrition Innovation Lab/Africa as problematic. In each case it is unclear if the issues are with specific individuals, or if the issues are more systematic in the institutions. The ME should either work to fix the relationships by providing technical assistance, by developing a stronger supervisory relationship, or by seeking new partners. Re-assess ME Staffing Current ME staff are over-committed. The addition of Associate Awards should be accompanied by an increase in staffing. Currently the weakest areas are in timely analysis and release of data, in management of communication and in-country marketing of the project, and in maintenance of the website. The website is not an effective means of communication at this time, which is unfortunate, because it provides a window to the world on the important work happening in the Nutrition Innovation Labs. Broaden the Multi-disciplinary Base of the Research The EET was struck by the apparent lack of input by agricultural researchers in research design and data analysis; and also by the low number of agricultural staff and students included in long￾and short-term training and capacity building. Some of the expertise needed is already included in the Innovation Labs research team, but is not being incorporated across research projects. Develop an Integrated Set of Metrics The Nutrition Innovation Labs in collaboration with members of the TAC have conducted a workshop on metrics. However, the EET did not see evidence that a coordinated set of metrics was being used across the research programs. To be clear, the EET is not suggesting that the research programs use identical methods, but that they agree on drawing from a set of appropriate instruments and designs to enhance the degree to which research findings are comparable across research projects and regions. This seems to be a lost opportunity for the Labs to influence a broader research agenda with state of the art methods. Broaden the Scope of the Research Agenda The current approach to the research agenda invests in a small number of large-scale studies designed to generate critical data sets. The EET also sees an opportunity in a second phase, and through the mechanism of Associate Awards, to design and conduct research studies that take 12 a closer look at context, and the ways in which specific pathways operate in specific regions. The EET sees the Nutrition Innovation Labs moving in this direction at this time, but also see the increased opportunity to do so in a second phase of research. Broaden Capacity Building There is an opportunity to broaden the base of capacity within the U.S. as well as in the regions in which the Innovation Labs operate through increased inclusion of students, host country researchers and policy makers in data analysis, data presentation, and publication. The Nutrition Innovation Labs could increase the inclusion of other disciplines in training in nutrition and health, and develop a strategy for cross-disciplinary training. Move More Quickly to Provide Policy Relevant Analysis The most consistent concern expressed to the EET by policy makers was their perception that policy relevant data were being made available too slowly. This concern was phrased by several interviewed stakeholders as a concern that the researchers were more concerned with publishing in peer-reviewed journals, than impacting program design and delivery or intersecting with policy makers. The EET certainly understands that data collection and analysis take time, and that longitudinal or panel studies may need several rounds of data collection to have meaningful results. However, these data should be carefully presented to policy makers, and a second phase of funding should include explicit strategies to present and interpret project data for local policy makers. 13 BACKGROUND The Nutrition Innovation Lab/Asia and the Nutrition Innovation Lab/Africa are funded through Leader with Associates (LWA) Award Cooperative Agreements. Both Labs were awarded to Tufts University’s Friedman School of Nutrition Science and Policy as the Management Entity (ME) with a start date of October 4, 2010. Both projects are in the fourth year of their initial five-year awards, which end on October 3, 2015. Total funding received, as of September 2013, is $4,800,000 for Africa and $4,500,000 for Asia. The ceilings for the five-year LWA awards are $7,361,494.89 for Africa and $7,321,861.42 for Asia. The USAID/Malawi Associate Award level of funding is projected to be $450,000, but the award is not yet funded. Several other Associate Awards have been proposed or are in the negotiation phases. The organization of the Nutrition Innovation Labs represents a novel model for focusing U.S.- supported research on food and nutrition issues in developing countries. This includes: i) a geographic focus (following Feed the Future priority regions and countries); ii) an applied focus (operations research rather than discovery science); iii) a focus on country-ownership (supporting research that informs nationally-defined priorities in food and nutrition); iv) resources allocated to large grants to work at scale (rather than many small grants supporting studies of experimental or pilot activities); and v) building capacity for analysis and policy formulation through formal education activities as well as engagement of local partners at stages of the research agenda. Originally called Collaborative Research Support Programs (CRSPs), the Nutrition Innovation Labs were renamed to incorporate them into the Feed the Future Initiative under the Program for Research on Nutritious and Safe Foods in the Food Security Innovation Center created in early 2013. The goals and objectives of the Labs did not change at the time of the reorganization; they remain as they were in the original Nutrition CRSP technical proposals and the work plans for project years 1 and 2. Year 3 Annual Reports and Year 4 Work Plans reflect the reorganization of the CRSPs as Feed the Future Innovation Labs. While each Nutrition Innovation Lab has its own Project Director (PD), the two Innovation Labs share management and logistical staff, a single Co-Project Director, Associate Director, Program Manager, Communications Specialist (currently unfilled), a number of faculty researchers and a combined website. 14 At this time, the projects center their work in three Feed the Future countries: Nepal (Nutrition Innovation Lab/Asia) and Uganda and a small project in Malawi (Nutrition Innovation Lab/Africa) although several Associate Awards are planned, under negotiation, or awaiting implementation. The Tufts-based Management Entity (ME) coordinates a number of U.S. partner institutions – Tuskegee University, Purdue University, Johns Hopkins University, and Harvard University - and a number of host country collaborators. The original goal of the Nutrition Innovation Labs was to determine if investments in agriculture-based strategies, human and institutional capacity, agricultural and food system policies and human health and nutrition could have an impact on nutrition and, if so, what specific investments could help to achieve: • Large scale and sustainable improvements in nutrition outcomes in children less than two years of age and vulnerable adults, especially pregnant women. • Significant and sustained increase in the number of households with improved dietary quality and diversity. • Improved national nutrition policies and community capacities to combat malnutrition. • Significant and sustained increase in the number of households with improved infant and young child feeding practices. • Advancement of new tools to improve nutrition. With this goal in mind, the stated mission of the Nutrition Innovation Labs is to discover how policy and program interventions can most effectively achieve improvements in maternal and child nutrition by leveraging agriculture, at scale. The aims of the Nutrition Innovation Labs are to: (1) Discover how policy and program interventions can most effectively achieve large-scale improvements in maternal and child nutrition, particularly by leveraging agriculture; and (2) Build human and institutional capacity for applied policy analysis, research and program implementation. As conceived in the original Request for Application (RFA) and articulated by the initial AOR, the Nutrition Innovation Labs were to be a central resource on the impact of agricultural programs on nutrition and health around which other Feed the Future activities were to wrap. In the words of the initial AOR, the Nutrition Innovation Labs would “wrap around” the production and value chain oriented Feed the Future projects, which would provide high quality basic data at scale and operations research in Feed the Future countries to identify and assess the impact of both agricultural and health interventions. The research programs of the Nutrition Innovation Labs were to provide basic, interdisciplinary research data collected “at 15 scale” that is, with samples large enough to investigate questions remaining in the existing literature concerning the pathways through which agricultural programs have an impact on nutrition and health. In retrospect, such a strategy would have required detailed advance coordination between USAID/Washington and the Mission-funded projects to allow timely implementation. The EET was asked to evaluate (a) the program management by the Management Entity in Tufts University of Nutrition Science and Policy at Tufts University, (b) the research programs in Asia and Africa, (c) the training programs, and (d) the institutional capacity collaborations. In addition, the EET was asked to provide recommendations to inform the decision on program extension and, if appropriate, provide recommendations as to any necessary management adjustments and potential research focus changes during a second phase. The Scope of Work, Evaluation Plan, Travel Itinerary, Methods, Persons Contacted, Materials Reviewed and Photographs from Field Trips can be found in Appendices A – G respectively. 16 PROGRAM MANAGEMENT The Nutrition Innovation Labs, formerly called Collaborative Research Support Programs (CRSPs) were organized from the beginning as a consortium of institutions with Tufts University providing the Management Entity for the Nutrition Innovation Lab/Asia and the Nutrition Innovation Lab/Africa. This organization is distinct from the historical CRSPs and the other current Innovation Labs. The Scope of Work for the Nutrition Innovation Labs evaluation asked the External Evaluation Team (EET) to address the following questions with respect to management: 1. Have the Management Entities for the two Nutrition Innovation Labs effectively managed their respective research and training activities in Africa and Asia? How effectively have the MEs and their partners communicated, coordinated and engaged with the Missions? What have been specific challenges faced in terms of management, and how has each ME addressed them? 2. In the past, the CRSP model has been based on one lead university serving as the ME and managing a global program of multiple projects. Have the two Nutrition Innovation Lab awards created value added benefits? Have the two MEs built synergies between their regional programs to ensure comparability among findings? How have these synergies contributed to the Nutrition Innovation Labs’ objectives? How could the synergies between the two regional programs be strengthened? FINDINGS QUESTION 1: EFFECTIVENESS OF MANAGEMENT The effectiveness of the management of the Nutrition Innovation Labs in several areas was assessed by the EET. The EET reviewed overall governance of the Labs; examined the roles of the key management bodies, including the Management Entity (ME); examined the means and methods of communication between the ME and the various partners, among the partners, and between the Nutrition Innovation Labs and the wider community of scholars, implementation program staff and policy makers; and explored the degree to which ME staff, U.S. and host country partners and policy makers assess the effectiveness and impact of the Labs. 17 Governance EET members reviewed several documents relating to the governance of the Labs including the Global Nutrition CRSP Policy and Operating Procedures Manual (POPM), which outline the governance structure and the roles of the several entities involved in the governance of the Labs; available minutes from meetings of the several entities outlined in the POPM; and Annual Reports from each Nutrition Innovation Lab. The EET members also interviewed a number of key informants who serve on these bodies representing the ME, the U.S. core partners and individuals recruited as external advisors. Figure 1: Organizational Structure of the Global Nutrition Innovation Lab This overall structure of the ME (Figure 1) is outlined in the technical proposals for each of the Labs and is similar to the governance structure of the historic CRSPs. The ME has a core management team, with a single Board of Directors (BoD), a single set of core partners and the TAC. The two Nutrition Innovation Labs are organized under the ME, each with their own Project Director (PD) and team of partners. Governance procedures are outlined in detail in the POPM. The POPM was developed in collaboration with partners and minutes show that it was discussed in the first meeting of the BoD early in 2011. The current version was finalized and circulated in April 2011. The EET finds the POPM to be clearly written and sufficiently detailed. Nutrition Innovation Labs Management Entity The core management team consists of the two Innovation Lab PDs, the Global Program Director, the Nutrition Innovation Labs Associate Director, the Associate Director for Communications (position vacant as of this writing), and the Global Program Manager. Members of the core management team are shown in Appendix H. 18 The POPM outlines the ME’s responsibilities in three key areas: financial management, program management, and communications. The POPM states the responsibilities of the ME as follows: The Management Entity (ME) is an institution with the legal status of judicial body that administers the Cooperative Agreement from USAID and manages the Global Nutrition CRSP and its activities, including collaborative research, education and outreach programs. Tufts University’s management approach and ME structure will ensure efficient coordination and accountable management, with clear lines of communication (POPM 2011, pp. 6-7). The management performance of the ME will be addressed fully below. Board of Directors The governance of the Nutrition Innovation Labs also includes a Board of Directors (listed in Appendix H), which is made up of the members of the ME as well as representatives from the U.S. core partner institutions, representatives of USAID including the Agreement Officer’s Representative (AOR), and an external advisor. The two Nutrition Innovation Lab Project Directors serve as Co-chairs. The POPM describes the BoD as “…the key policy-setting entity … involved in defining CRSP policies, approving overall and annual budgets, and advising the ME on strategic programmatic issues.” The POPM also states the BoD “…advises/oversees the ME activities in areas associated with policy, technical and program management, collaborating host country coordination, budget management and review (POPM, 2011:2).” The POPM states that the BoD is to meet twice per year. However, it should be noted that the annual work plans state that the ME will call “at least one” meeting per year. Minutes were made available to the EET for BoD meetings on: January 6, 2011 (the inaugural meeting), March 18, 2011, June 2012, and April 26, 2013. A fifth meeting of the BoD, for which minutes were not available, was conducted in early 2014. The annual reports note that after the first year, the BoD met only once per year. However there were at least two “partners’ meetings” conducted by conference call between BoD meetings, for which the EET has the have minutes of. Although not full BoD meetings, some of the key issues usually discussed in the BoD meetings were also discussed in the interim partners’ conference calls. While the minutes of the BoD meetings understandably contain a clear record of expository presentations by members of the ME, they also support the stated role of the BoD. That is, the minutes show that all BoD members engaged in discussions of key issues. The record also shows comments and opinions made by academic partners and other members. Some members of the BoD expressed the concern to the EET in interviews that their comments were not given the 19 deserved weight. The EET cannot know how comments were received by colleagues at the time that they were made, but the minutes do show that comments by everyone were recorded. In some cases, the interview materials corroborate the materials contained in the minutes. For example, of the representative of Tuskegee University noted that the Tuskegee team did not support providing funds to the supervisors of Makerere students to oversee the students’ research. This comment is clearly recorded in the minutes. A Friedman School representative commented on the Nutrition Innovation Lab governance structure noting that in it is well within the range of variation of the historical CRSPs. This comment is also recorded in the minutes of the discussion of the POPM and governance structure. The overlap seen between interview comments and minutes support the EET’s assessment that the minutes are accurate. The minutes of the BoD meetings show a ME exerting leadership among the various groups represented, but also the clear opportunity for other BoD members to express opinions and influence final decisions. Interviewees who had participated in BoD meetings, including the current AOR, expressed satisfaction with the experience and the outcomes. Members who have participated in the BoD for other CRSPs are divided on the extent of these ME’s influence on their BoD. However, in interviews, several key informants report that there is little coordination with respect to research design and methods of data collection between the two Nutrition Innovation Labs, and even across projects within each Nutrition Innovation Lab. This will be addressed more fully in the section on Research, but the review of the meeting minutes supports the observation by key informants that, while issues of Nutrition Innovation Lab policy and overall project outcomes are discussed, there is little time devoted at BoD meetings to coordinating the two sets of projects with respect to methods and outcomes. Several key informants suggested that there is a feeling of disconnection between the two Nutrition Innovation Labs, and between the MEs and their sub-awardees. The EET notes that this was expressed as opinion and impression, without specific data, but it was expressed by several key informants who were members of the BoD, not members of the ME. Finally, several members of the BoD expressed concern, perhaps not at the level of complaint, about the timeliness of scheduling of BoD and TAC meetings. More than one member suggested that scheduling earlier would improve the ability of some members to physically attend the meetings. Technical Advisory Committee The governance of the Nutrition Innovation Labs also includes a Technical Advisory Committee (TAC) that serves both Labs, current members of which are listed in Appendix H. This group is made up of members of the ME and representatives of core partners as well as several external advisors recruited from among highly respected experts in the field. 20 The POPM describes the role of the TAC as providing the ME with “…technical advice concerning the scientific merit and development implications of program activities.” And lists its responsibilities as including, but not limited to: “Reviewing technical progress relative to annual work plans and proposing modifications therein. Reviewing research activity reports, annual activity work plans, and progress reports and presentations at the annual meeting. Providing input into coordination of U.S. and host country programs. Providing input and assisting the ME in preparing for reviews. Participate (voluntary) in expert panel to review applications submitted to issued RFAs and calls for applications. Monitoring the development of the Global Nutrition CRSP Workspace. Facilitating workshops, symposiums, site workshops, and other meetings promoting Nutrition program objectives” (POPM: 7). The EET is impressed with the caliber of the individuals who have been recruited to the TAC as external advisors. The TAC is quite diverse and consists of a range of disciplines that are central to the strategic vision of the Nutrition Innovation Labs. This is a group of highly respected professionals in their fields and are very appropriate as members of the TAC. The POPM states that the TAC is to meet “physically” once per year, and be available for consultation throughout the year. The TAC is to be the body from which the Ad-Hoc Committee to review proposals is to be drawn. Minutes were made available to the EET for TAC meetings on: January 7, 2011 (the inaugural meeting), June 2012 (held jointly with BoD), and April 26, 2013. Available minutes and the annual reports show that the TAC has met physically every year. However, the minutes show poor attendance by some external advisors over the years. Others have been quite active, having attended every meeting, although sometimes by phone. One TAC member organized a Nutrition Innovation Lab affiliated workshop on integrated metrics, and is a co-author on at least one report. Analysis of the minutes of meetings corroborate the observations of TAC members that the meetings are mostly expository with presentations of project activities made by Nutrition Innovation Lab ME members and PIs. Both Project Directors and the Global Co-director stated in interviews that, from their points of view, the most important inputs and advice from external advisors comes informally during the year, when members of the ME and partners call upon specific TAC members for assistance in making contacts and solving problems. When asked about the role of the TAC all of the individuals noted that the role of the TAC and its member is “more informal”. However, several members of the TAC suggested that the TAC does not have the level of involvement with the Nutrition Innovation Labs that the members of the TAC themselves would have liked to have. The TAC is not “kept in the loop” as much as some members would like; does not get contacted as “TAC members” as much as they are willing to be. While some TAC members have been more involved, at least one current member expressed the concern that the role of 21 the TAC “has been minor” in the Nutrition Innovation Labs. This member would have liked a more active role. Performance of the Management Entity Following the original Nutrition Innovation Lab RFA, the technical proposals and the POPM, the ME has the major responsibility for (1) all fiscal management of the Nutrition Innovation Labs; (2) coordination of all aspects of Lab activities, even when performed by sub-awardees and other partners; (3) monitoring and evaluation of all Lab activities; (4) managing all aspect of the relationship with USAID, including USAID Missions in the primary host countries, and for any Associate Awards developed; and (5) reporting and communication with partners, scholars and the wider community of practice (POPM pp. 6-7). To assess the overall effectiveness of the ME and to address the specific questions posed in the evaluation Statement of Work (Appendix A), the EET interviewed a wide range of key informants in Nepal, Malawi and Uganda, both in person and by telephone from the U.S. Those interviewed can be found in Appendix E. This report will comment on the effectiveness of the ME, and each of the Nutrition Innovation Labs. The EET found that Nutrition Innovation Lab PDs and other ME staff are well known in all of the three countries visited (Malawi, Nepal and Uganda). PDs and ME staff have visited the countries multiple times, and have met with key host country collaborators at appropriate intervals. Data on travel to the host countries for year three of the grant period show that there were 8 visits to Nepal by the Nutrition Innovation Lab/Asia PD and ME staff, and 7 visits by the Nutrition Innovation Lab/Africa to Uganda, five of which were taken by the Africa PD. In both Nutrition Innovation Labs, the ME is seen as responsive and collaborative. Nutrition Innovation Lab host country coordinators, and major collaborators in both Nepal and Uganda, report that the PDs and ME staff act in a collegial and collaborative way; the interactions represent true partnerships; and both Nutrition Innovation Labs’ PDs, the Associate Directors and the Program Manager are seen as responsive to the ideas and concerns of host country partners. The only concerns raised by interviewed stakeholders for either Nutrition Innovation Lab management team was the timeliness of access to data for collaborators, and more importantly, for policy makers. 22 U.S. partner PIs and staff are generally happy with the relationships with the ME. In Nepal, PoSHAN PIs interviewed reported very good relationships with the ME and a smooth working relationship. In an interview the Harvard University PIs on the Nutrition Innovation Lab/Africa suggested that they sometimes feel out of the loop, but in general, feel the relationships are collegial. The Nutrition Innovation Lab/Africa PD has stated that he had to intervene to facilitate and speed up the hiring process of an individual by Harvard for the Uganda research program. However, the area of major breakdown is the relationship between the ME and Tuskegee University. There are several issues. The impact on the training program is described in detail in the Training Section of this report, but the EET also sees several other areas of concern. Tuskegee seems to do a poor job of reporting. The Tuskegee annual reports are significantly less informative than any of the other partner reports. At the same time, the Tuskegee PI has expressed frustration with communications from the ME. Nutrition Innovation Lab/Asia All evidence suggests that the Nutrition Innovation Lab/Asia is very well managed. The Program Director (PD) and his staff visit Nepal on a regular basis and are well known in the country. The PD has traveled extensively to Nepal and is recognized as a key resource by the staff of collaborating Nepali Government Ministries, universities and NGOs. Indeed, in one interview a Nepali university official commented that he was not sure what is being done by Tufts University, as a separate entity, and the Nutrition Innovation Lab since ME staff work on non - Nutrition Innovation Lab activities. For example, the PD serves on the editorial board of the Journal of the Nepali Institute of Medicine. Some of the work in Nepal with curriculum development is carried out under a specific MOU with Tufts University. Interviewees also frequently mentioned the Nutrition Innovation Labs Associate Director as being very engaged and supportive in the projects. The EET interviewed over 20 people in Nepal including in country staff, academic partners, and policy makers (see Appendix E). In addition, interviews were conducted with the sub award PIs at Johns Hopkins University by phone. Collaborators, even minor collaborators, spontaneously mention ME staff, especially the Program Director, but also the Associate Director and the Program Manager, when the Nutrition Innovation Lab/Asia is mentioned. All reported good contact with members of the ME and with the in country coordinator. Host country Lab staff reported that the ME is very responsive to their inquiries, getting in contact and returning messages and calls promptly, despite busy travel schedules. Policy makers uniformly had praise for the members of the ME and felt they had good relationships with them. 23 Moreover, the PIs and staff of the major sub awardee, Johns Hopkins University (JHU), are also well known, accessible and seen as resources in Nepal. While there appear to have been some minor problems with roll-out of the research program at the outset, the filling of the position of project coordinator for the JHU activities has apparently solved early problems in coordination. The JHU PIs are well recognized in Nepal and the JHU project coordinator is in good contact with the JHU PIs, host country PIs, and with the Nutrition Innovation Lab/Asia host country coordinator. Data collection is reported to now be smooth and timely. In should be noted that the survey data collection was rolled out later than originally planned, but is timely with respect to the constraints imposed by the slowness in choosing sites for the collaborating USAID Nepal Mission funded Suaahara project. The second wave went into the field as planned during the EET visit to Nepal in May. One example of the involvement of the PIs is the way in which they detected data quality control issues in the sub-contractor carrying out the data collection in the field. In reviewing data collection procedures they detected a problem with the standardization of the anthropometric data collection and worked with the sub-contractor to improve the training of data collectors. The Harvard University managed project in Nepal is small in the overall Nutrition Innovation Lab/Asia program, but the research being conducted in Bhaktapur provides an opportunity for community based, long-term context specific research in a way not present in the PoSHAN study. Harvard staff are in good contact with host country investigators. A former Harvard researcher (now at WorldFish) has visited in country, and several papers from this research are published with co-authorship among the Harvard-based and host country investigators. Nutrition Innovation Lab/Africa The Nutrition Innovation Lab/Africa PD and ME staff are also well known in Uganda and Malawi. Ugandan host country staff at Makerere University report good contact with weekly/biweekly phone/SKYPE calls at least with the Associate Director and the key staff person at the major sub awardee, Harvard University. ME staff, including the Africa Program Director and the Associate Director, have traveled multiple times to Uganda. The Global Program Manager has traveled several times. Trip reports show that the PD has traveled to Uganda up to 6 times in one year. Even though Harvard reports that they are involved in both the panel and cohort studies in Uganda, at the time of the site visit to Uganda in May 2014, the EET could find no documentation showing that the Harvard University PIs had ever traveled to Uganda for this project. The EET could only find documentation for one visit to Uganda by the Harvard project coordinator, even though one round of data collection in the panel study has 24 already taken place. The Harvard- based coordinator does participate in the scheduled weekly calls, but informants both in Uganda and at other U.S. partners report that the PIs rarely participate in these calls. Harvard’s footprint on the ground in Uganda seems to be “light” at this point. Furthermore, data collection for the second round of the panel study had been originally projected for Fall of 2013, and was postponed to Fall 2014, and the cohort study was also behind schedule. The delays are in part, the result of changes in key research positions outlined below, and, the EET believes, in part the result in delays by the Harvard research group. The recent hiring of a Harvard supported project staff member at Makerere University has helped. However, the Lab PD states that the ME had to step in and manage the hiring process in order to make this hire happen. The EET found that the current Makerere University PI has responsibilities not only as PI but also as a faculty member and Dean of the College of Agricultural and Environmental Sciences and may be over-committed. The current Uganda host country Project Coordinator is continuing her full￾time faculty role as well as serving as the Project Coordinator. At the time of the site visit in May 2014, the Makerere University PI and Uganda Project Coordinator were expecting to visit the cohort project sites during the week of June 9th to work out logistics for the cohort study. Harvard University project staff and the Nutrition Innovation Lab Associate Director were expected to visit Uganda the week of June 16th. The EET understands that these visits did take place and there has been some movement on the planning of the data collection for both the cohort study and the second panel study. Launch of the research data collection for the cohort study is currently projected for August 2014. However, at the time of the EET visit in May , data collectors had not yet been hired and trained. A number of key informants spoke about the degree to which turnover in key administrative positions had challenged the management of the Nutrition Innovation Lab projects and slowed progress. There have been changes in the Nutrition Innovation Lab/Africa Project Director at Tufts University; a change of affiliation within Uganda’s Makerere University from a PI in the College of Health Sciences to PIs in the College of Agriculture and Natural Resources; a subsequent change of PI in the College of Agriculture and Natural Resources; and a change in the host country project coordinator when the first project coordinator left for graduate study at Tufts. This last shift left a gap of over five months in which the project did not have a coordinator, and is reported as having taken place without a written manual of operations or formal handover of responsibilities. Some key informants believe that these shifts have contributed to disconnects in communication between research collaborators as well as with country partners. In addition to the shifts in staffing, USAID has entered into a “strategic pause” with 25 one of the Nutrition Innovation Lab’s key government collaborating entities -- the Office of the Prime Minister -- further hampering the ability of the Lab to interact with policy makers. Finally, while they maintain collegial relationships, both the Nutrition Innovation Lab/Africa Project Director and the Harvard University sub award PIs report tension between the Harvard project and the Nutrition Innovation Lab/Africa management team. The Harvard PIs note that they were not invited to the meetings held when the EET met with the ME in Boston in February. Financial Management Records and the interview with the current AOR show that the Nutrition Innovation Lab/Asia has consistently met or anticipated deadlines for submission of work plans, budgets and annual reports. The Nutrition Innovation Lab/Africa has submitted all required documents, but often “at the last minute” and submission of the annual reports for the last two years has been delayed. All partners in Nepal reported satisfaction with the financial arrangements of the Nutrition Innovation Lab/Asia and the transfer of funds. Some early minor concerns having to do with the transfer of funds to at least one partner have been resolved, and financial transfers appear to move smoothly and in a timely fashion. Funds for the project flow through Helen Keller International (HKI) and the director at HKI has extensive experience with funds management. However, of special concern among the partners is the relationship between the ME, sub￾awardee Tuskegee University, and sub-sub awardee Makerere University with respect to training and capacity building. It is clear from talking with the Makerere contact person for the Tuskegee activities and the students supported in MSc programs at Makerere, that there has been substantial miscommunication among the partners. The Tuskegee PI reports that the ME did not provide the full support to Tuskegee that was expected in the original sub-award. The PD notes that some of the Nutrition Innovation Lab/Africa funds were sequestered, and this contributed to a delay in the availability of funds. Both the PD and the Tuskegee PI report that the ME Nutrition Innovation Lab unilaterally redirected support from the Tuskegee project directly to Makerere students rather than passing it through Tuskegee. The Management Entity notes that it has been very difficult to get a “clean” invoice from the Makerere focal person for the student scholarships. The EET finds credible support for the development of a cascade of communication breakdowns in this process, and some clear differences of opinions among the actors. Specifically, the Tuskegee PI is reluctant to provide direct funding to the faculty supervisors of the students. The ME sees no problem in providing some compensation to supervisors working with students. An honest difference of opinion may have contributed even more confusion to what the Makerere PI sees as conflicting messages to the students. What is 26 clear is that the students are confused, frustrated and unsure of what their next steps should be, and they feel that their progress in their programs has been slowed as a result. The ME is moving to resolve these issues by providing training to their Ugandan partners. ME project staff were expected at Makerere University in the weeks following the EET’s visit to conduct training on project financial management and invoicing for Makerere staff. The ME has not moved to provide similar support to Tuskegee University at the time of this report. In Malawi, delays in funding appear to be the result of inexperience in the USAID Mission with respect to issuing an Associates Award. Tufts University has forward funded Lilongwe University of Agriculture and Natural Resources (LUANAR), formerly Bunda College, for work on the curriculum project in order to move the project forward. The PIs and the project coordinator at LUANAR are very positive with respect to the relationship with the ME. The Sub-award Process Review of documents and interviews with the major research sub awardees suggest that the sub-award process was fairly and transparently conducted. The PIs at both Johns Hopkins University and Harvard University report that the process of awarding the RFAs was smooth and from their point of view (as the winners), fair. Authors of unsuccessful applications were not available for interview. The PD of the Nutrition Innovation Lab/Africa described a process of negotiation in the awarding of the sub-award for Uganda in which the AOR at the time brokered a collaborative award among several of the original bidders. The Nutrition Innovation Lab/Africa PD felt that this was a very positive outcome. Relationships with Collaborating Implementation Projects Both Nutrition Innovation Labs began with the intention to collaborate with proposed intervention programs funded by the Missions. For Nepal, this was USAID’s Suaahara Project, and for Uganda, the Community Connector Project. In both cases there was an early expectation in the USAID Missions and with some of the implementation project staff that the Nutrition Innovation Lab activities would serve as monitoring and evaluation arms of the implementation projects. In both cases, the Lab activities have been essentially decoupled from the implementation projects although in both cases there is some geographical overlap with Nutrition Innovation Lab data collection and project implementation sites, and in both cases Nutrition Innovation Lab research has the potential of providing some data on the impacts of the implementation projects. However, the decoupling process in both countries has caused some confusion and even conflict with the implementation project and the USAID Missions. 27 The original RFA for the Nutrition Innovation Labs does not specifically state the Labs should work directly with implementation projects to provide monitoring and evaluation. The two specific paragraphs that address the research objectives of the proposed Nutrition Innovation Lab focus on operations research not monitoring and evaluation. The initial USAID AOR reports that this was clear from the beginning. However, the EET conclude that miscommunication about the Nutrition Innovation Lab roles, and, in some cases, overpromising by Nutrition Innovation Lab staff, have contributed to a relationship at the USAID Mission level, especially in Uganda, that was less than ideal. This is further elaborated in the Research section of this report. Relationships with USAID Missions in Nepal, Uganda and Malawi Early misunderstandings about the role of the Nutrition Innovation Labs in evaluation and monitoring of intervention projects contributed some friction between the Labs and USAID Missions in the early days of both projects. The Nutrition Innovation Lab/Asia resolved these conflicts fairly early, and USAID staff with whom the EET spoke with were very supportive of the project and its goals. The issues in Uganda are gradually being resolved. USAID Mission staff report that they have come to accept the role of the Nutrition Innovation Lab/Africa and see the data collection activities as useful and important in their own right. The Mission’s Feed the Future Coordinator who works with all of the Innovation Labs even expressed a positive view of the potential of the research. However, he reports that he has only recently come to that view. The description given by a Foreign Service National staff member would support a conclusion that early on Nutrition Innovation Lab project staff and the USAID AOR over-promised, and more likely “oversold” the potential of the project to provide data that directly addresses the monitoring and evaluation needs of the Community Connector Project. In Malawi, USAID Mission staff take responsibility for delays in funding the project which appears to be their lack of familiarity with issuing an Associate Award. Originally formulated as a three part project that included (1) the development of a curriculum for a dietetics program, (2) the creation of country specific food composition tables, and (3) the enhancement of nutrition training in public health, the project has only been able to address the first component, and this has been primarily on the basis of forward funding by Tufts. A series of emails between Malawi Mission and Bureau for Food Security staff show that the current AOR is working with the Mission to get this resolved. The EET understands that the Associate Award is now moving forward. 28 The EET did note that, even though ME staff have been to Malawi at least twice, the Office Chief for Sustainable Economic Growth in the USAID Mission had very little idea of what the Nutrition Innovation Lab was doing in Malawi. He stated that he had never met with the PD or other ME staff on any of the visits they had made to Malawi and that he has not been contacted by them. Relationships with USAID/BFS Staff and AORs The original AOR and the current AOR both report that the ME has been very responsive to USAID concerns. Both find the current PDs easy to work with. Both PDs also report good relationship with the AORs over the years. Organization of the ME The EET recognizes that there are increasing demands on the ME staff as Associate Awards are awarded to the Nutrition Innovation Labs. For example, Associate Awards are pending in several countries for each of the Nutrition Innovation Labs. While ME staff say that they are coping, they also report that they are at, or above, full capacity now. When the EET reviewed the activities managed by each staff member in the ME, including the PDs, the team came to the conclusion that the current staffing arrangement is very over-stretched. As noted below, some activities the EET considers important -- such as maintenance of the website -- have been relegated to a lower priority status, and it is not clear that the communications team specified in the original proposal is functioning. The ME will need to increase staffing as new Associate Awards are made. Communications The POPM (2011: 6) states the role of the ME in communication as follows: The ME will serve as the hub for a diversified communication structure. The PD will be the primary contact for the USAID CTO, consortium partners, and other collaborators. The ME will facilitate interaction across the CRSP research groups and the broader scientific audience by: • Maintaining a web-based knowledge base and interactive forum for information exchange. • Publishing CRSP policy and research brochures, reports, and other communications. • Facilitating publication of research results in refereed articles and books. • Circulating electronic newsletters on the nutrition CRSP research, conferences, publications, etc. • Requiring all CRSP research awards to have stakeholder communication plans. • Establishing relationships with the USAID mission. • Facilitating regional meetings and workshops to promote sharing of results and lessons learned. 29 The Nutrition Innovation Labs Website and the NCRSP Workspace The Nutrition Innovation Labs website has two sections. The first is an open access section that contains materials open to the broader public and community of scholars. A second space, accessed through the website, is the NCRSP Workspace. This part of the website is monitored and available only to registered users. It provides a space for commentary, exchange of ideas, sharing of data and drafts. It has four sub groups: Nutrition Innovation Lab/Uganda, Nutrition Innovation Lab/Nepal, BoD/TAC, and Suaahara. Anyone wishing to access this space completes a registration request and has to be approved for membership. The EET has reviewed the Nutrition Innovation Labs website including the availability of annual reports, research briefs, peer reviewed publications and workshop reports and has accessed the Labs Workspace. In addition, a number of key informants, especially those in the host countries of Nepal and Uganda, were asked about the frequency with which they consult the website for information and participate in the Nutrition Innovation Labs Workspace. At this writing, the website contains 15 Research Briefing Reports (RBR # 6 is curiously missing) and these are up-to-date. However, the EET has noted a number of issues with the website as a vehicle for communication. While the EET was given access to over 55 reports, papers, and PowerPoint presentations as part of the evaluation, few of these are posted as papers or even briefs on the website. Clearly, when several investigators are presenting research materials at different conferences to different audiences, it would not be appropriate to post all of the resulting PowerPoint presentations haphazardly. It could well be confusing and, in some cases, presentations could be contradictory, as data analysis evolves. However, presentations at conferences are public, and analyses, once presented, are in a sense public, and one might imagine that clean integrated copies of PowerPoint presentations on key topics, and research briefs based on draft/preliminary analyses would be of use to a community of scholars and practitioners. Posting of Annual Reports is very incomplete. Only the Annual Report for the Nutrition Innovation Lab/Africa Year 2 is available on the website. None of the Nutrition Innovation Lab/Asia Annual Reports are posted on the website. Only six publications in peer-reviewed journals are listed. Five are letters or review articles. The sixth is based on data from the Bhaktapur, Nepal project. Three were published in 2014 and three in 2013. The EET heard of several other publications that do not appear on the website. For the “in press” titles there is no indication of publication time lines. Most appear to be “in preparation” rather than “in press”. Some appear to just be ideas. None were available for review. The EET knows of at least one and possibly now two papers 30 that are available in draft form as working papers on the IFPRI website. They appear to be listed in the “in press” section of the Nutrition Innovation Labs website. It would be relatively easy to just add a link to the IFPRI working paper version. Finally, the latest “news” item on the website was posted on April 10, 2013. The “archives” have event postings from as late August 2013 – nothing later in either place. The Nutrition Innovation Labs Workspace is supposed to be a space in which project staff and partners can communicate with each other. Two members of the EET tested this by applying for and were granted membership in the Workspace. There are some on-line resources available on the site. However, the last comment posting on the main page of the workspace is from 2012. The last comment posted in any of the four specific working groups is from April of 2014, but three of the four have not had a posting since 2013. There are no events listed at this time. The total number of participants globally is 53, with one fourth of those representing the BoD and TAC members and two of the members of the EET. Very few of the key informants interviewed regularly visited the website. Only one of the key informants who was not a direct member of the Nutrition Innovation Labs, a Nepali government employee, reported using the website as a resource. Of those that regularly visited the website, only two reported visiting the Workspace. At least one of those was a student. When asked about impressions of the Nutrition Innovation Labs website and Workspace the most frequent response from the informants was a blank stare. Overall the Nutrition Innovation Labs website does not appear to be an effective vehicle for dissemination and communication. The website is not seen as a resource for project staff and partners. There appear to be at least two problems. One, and perhaps the basic one, is that the website is not well maintained. Postings are erratic and not timely. Key resources appear to be missing. The second is that the Nutrition Innovation Labs website has not been well promoted as a go-to space for topics of interest to practitioners in nutrition and agriculture. The website could be a strong vehicle and a “window to the outside world” for promoting the Nutrition Innovation Labs to key partners and more globally to those interested in progress of the research and its findings. However, the EET would not recommend promotion of the site until it has been improved. Publications, Research Briefing Papers and Presentations As noted above, there are 15 research briefing papers available on the website. These are informative and provide a description of the project and some of the results. Research Briefings are up-to-date. The latest Research Briefing Papers is dated July 2014. 31 See the Research section of this report for further discussion of research and the dissemination of research results. The bulk of the dissemination of project materials is through presentations and in Annual Reports. The EET counted well over 50 presentations with PowerPoint presentations by Nutrition Innovation Labs staff and researchers delivered to a wide variety of audiences including scientific conferences, policy bodies in several countries, other Innovation Labs, multilateral organizations, and current and potential partners. The staff and researchers including the Project Directors and researchers have taken advantage of many, perhaps every, opportunity(ies) to present the goals of the Nutrition Innovation Labs, the design of research, and preliminary data. Communication with Policy Makers in Nepal and Uganda Communication of data and policy analysis was the one area, in Nepal, Uganda and Malawi that the policy makers interviewed had negative comments. In each country one or more individuals in policy positions commented that they were disappointed in the timeliness of the availability of policy relevant data. This is in spite of the fact that in Nepal at least one individual also noted the scientific seminars (especially the second one) included very useful panels on policy and data. Several people were emphatic about the need to make the data and the policy implications of the data available in a more timely fashion. Nutrition Innovation Lab ME documentation and comments from interviews with ME staff, sub awardee PIs and some host country PIs show that, while complex data analysis is not far along in either Lab, there have been a number of presentations of the preliminary analyses of the data in Nepal. There have been fewer data available in Uganda. Furthermore, in both Nutrition Innovation Lab research sites, time sequence data that could address the effectiveness of interventions are not yet available, so presentation of policy implications are premature. In Nepal there were several firm statements that the data and its policy implications should e more widely distributed. This is included here to highlight the degree to which there is a hunger for policy analysis in Uganda and Nepal. However, the ME and the U.S. PIs have presented the data that are available. Communications with Partners, USAID Missions, and USAID Washington Key collaborators in both Nepal and Uganda report a high level of satisfaction with the quantity and quality of communications with the ME. USAID Mission staff also report good communication with the ME, although we do note that the Feed the Future Coordinator in Malawi had not been approached by the Nutrition Innovation Lab/Africa PD. The current Nutrition Innovation Lab AOR reports good communication with the ME including both PDs. 32 QUESTION 2: ASSESSMENT OF THE NUTRITION INNOVATION LAB MODEL The second question posed in the SOW (Appendix A) asked the EET to evaluate the effectiveness of the Nutrition Innovation Lab model including an assessment of the degree to which funding both Labs under a single ME contributes to synergies between the two; the extent which there is “value added” in this model; and the degree to which the model does or does not enhance the ability of the Nutrition Innovation Labs to reach their objectives. Some of these questions will be further discussed in the evaluation of the research. The EET has carefully reviewed research project documents and data collection instruments for all of the data collection activities to date. The EET interviewed ME PDs, sub-awardee PIs and some host country PIs and staff about the interactions between the two Nutrition Innovation Labs with respect to data collection and analysis. Lastly, the EET examined goals for the Nutrition Innovation Labs outlined in the RFA and in the technical proposals. Evidence of Synergies and Collaboration There is clear synergy between the Nutrition Innovation Labs as a result of the shared Management Entity Management procedures, especially with respect to operating procedures, governance and communication are very similar in both Labs. Clearly they work from the same set of management documents. Both the EET and several of the key informants associated with both Labs note that the management styles (and expertise) of the PDs are different. However, the same informants note that they are satisfied with the leadership of both PDs. The PDs both remarked that through being in such close proximity they have learned from each other. During the analysis of the data collection procedures for the research projects, fewer synergies were apparent. While both Nutrition Innovation Labs state the same overall goals, the questions being asked are somewhat different in Nepal and Uganda. The data collection instruments are also different as a result. Both Labs address the set of objectives outlined in their technical proposals. From that point of view, the data provide different slices of the bigger picture. There has been some concern expressed by members of the TAC that there is less coordination of design and data collections methods than would be desirable. A workshop on metrics organized and reported on by members of the ME and one of the TAC external advisors addressed this issue, but the projects continue to have different designs and use different metrics. Review of minutes show little time devoted to this question in BoD meetings. 33 The Single Major Sub Awardee Model The Nutrition Innovation Labs are different from the historical CRSPs in several ways, but most critical to the research is the model of the single major sub awardee in each country, rather than the management of a set of different smaller research projects. Review of the original RFA and an interview with the original AOR make it clear that the Nutrition Innovation Labs were to focus on large scale research projects that addressed the objectives of the research. Historically the CRSPs supported a range of smaller projects taking different approaches to related questions. As noted above, the EET feels that the reliance on a small number of projects addressing similar questions seems to have resulted in projects with different designs but which address the same objectives. There is little research that directly takes context into account. The EET sees that the funding mechanism of the Nutrition Innovation Labs as LWA Awards leaves open the very real potential for Associate Awards to more directly address operational research, and research that addresses the needs of specific countries and intervention strategies. These projects would provide the kind of opportunity for operational research that would appropriately expand the research agenda to address the kinds of questions not currently included in the research. The current AOR reports that several Associate Awards that are now under negotiation would provide this opportunity. The Johns Hopkins University PIs and staff provide historical presence in the country and added value to the research process of the Nutrition Innovation Lab/Asia. However, in the case of the Nutrition Innovation Lab/Africa, there is less evidence that the inclusion of Harvard University as a sub-awardee adds value to the overall research in Uganda. From the EET’s review of management, the ME and the local PIs have taken the lead in managing the on-the-ground operations in Uganda. Makerere University handles the data collection activities and Makerere staff are in more contact with the ME and Nutrition Innovation Lab/Africa PD than the sub-awardee PIs or staff. At this time, the EET conclude that having Harvard as a sub-awardee on the major research programs in Uganda adds an additional layer of management and cost with little additional value added. The skills of the Harvard researchers reproduce rather than complement the research skills of the ME. This project could be more efficiently run directly by the ME. 34 CONCLUSIONS Question 1 Governance Structure • The governance structure of the Nutrition Innovation Labs is clearly laid out in the POPM and the entities outlined in the POPM operate generally in the ways in which they are outlined in the POPM, although the role of the TAC is not as large as the POPM suggests it should be. • The BoD operates as a decision making body, although a number of the implementable decisions are actually taken by the ME following discussion among the members of the BoD. • The BoD does not appear to act as a body that works actively on the coordination of research design, methods or outcomes across the Nutrition Innovation Labs. • TAC membership, especially those members in the role of external advisors, represents a collection of scholars and practitioners of exceptional and diverse expertise. Fiscal management • Overall fiscal management of both Nutrition Innovation Labs is effective and timely, although there is a specific issue with training partners in the Nutrition Innovation Lab/Africa. • Early problems with movement of funds and payment have been addressed through improved practices and training. • Issues in the fiscal management of the training program at Makerere University are the result of more fundamental issues in communication among the several partners involved with training of Ugandan students at Makerere University. Program management • Overall management of both of the Labs is good, but the overall management of the Nutrition Innovation Lab/Asia is more efficient than the overall management of the Nutrition Innovation Lab/Africa. • The ME and the PDs for both Nutrition Innovation Labs have strong collegial relationships with most of the partners, both U.S. based and host country based. • The ME and the Nutrition Innovation Labs currently have good relationships with the host country USAID Missions and the implementation projects with which they partner. Early tensions due to misunderstandings of the Nutrition Innovation Labs objectives have been resolved in both countries (Nepal and Uganda). • The major sub-awardee for the research in Uganda – Harvard University – does not have a strong Nutrition Innovation Lab based presence in Uganda, and there is some 35 tension between the ME and the Harvard PIs. • Miscommunication among the partners involved in training masters’ students at Makerere University is not yet resolved and needs attention. This applies both to fiscal management and in the impact of the program on the training and capacity building missions of the Nutrition Innovation Lab/Africa (see Training section of this report). • Tuskegee University may not have sufficient fiscal management strengths or administrator skills to manage the budget, and provide adequate annual reports. • Delays in the roll out of the cohort study and the second wave of the panel study in Uganda are, in part, the result of a changes in personnel, delayed roll-out of the Community Connector Project and, in part, the very light footprint in Uganda by Harvard University. • Delays in the implementation of the project in Malawi are the result of procedures for new curriculum approvals in Malawi and the inexperience of USAID/Malawi Mission staff with the Associate Award process. • The ME staff is overstretched. There is need for some increase in ME staff and its capacity as the projects attract more Associate Awards. Communications • The Nutrition Innovation Lab website is not currently an effective communication tool and the NCRSP Workspace is very little used as a vehicle for communication among partners. • Research Briefing Papers are timely and readily available. • While it may be too early to be making strong policy statements on the basis of the data collected to date, policy makers in Nepal, Uganda and Malawi indicated that they would like more feedback on policy implications of the projects. Question 2 • Overall, the EET concludes that the model of the Nutrition Innovation Labs under a single ME has provided important advantages, both in the efficient use of resources, but also as a way of addressing the overall goal of the projects to address in depth, and at scale, the critical pathways through which nutrition, health and agricultural interventions are connected. However some of the potential advantages are not fully realized, and there are some weaknesses. • ME members have strong collegial relationships and share insights with each other. • PDs and sub awardee PIs do not share expertise as effectively as they could. The BoD does not serve as a body of exchange of ideas on research design, methods, and assessing results. • Research activities, guided by similar objectives, are more comparable than they 36 might have been if the Labs did not share an ME. However, while projects in Nepal and Uganda provide different cuts through the same set of objectives they do not use the same design or share all survey research instruments. • Management of the projects under one ME, BoD and with the input of a single TAC, has not resulted in a shared design or shared metrics for the projects. • The potential for Associate Awards provides a mechanism to pursue operational research and research that examines the processes and specific pathways through which interventions are effective. LESSONS LEARNED • Timely implementation of a complex research agenda on the ground requires time and effort on the ground by all research partners. • Communication among all partners is critical to avoid problems in project implementation and management on the ground. • Lack of investment in website development and maintenance limits the broader impact of the Nutrition Innovation Labs. RECOMMENDATIONS • The TAC could have a larger role in the review of project activities and research, and some members of the TAC would welcome that. • Management issues with the U.S. partner Tuskegee University need to be addressed. This could be in the form of attention to improved administrative and fiscal management training and support; improved communication; or the termination of the project with Tuskegee in a second phase of funding. • The ME should address the issues related to Harvard University as a sub-awardee. The Harvard project needs to have a firmer footprint in Uganda. PIs and staff should be more present in the project. However, it is not clear to the EET that Harvard adds value to this project and it may be that there should be a shift in partners if an additional grant is awarded. • The Nutrition Innovation Labs should consider more targeted policy presentations to address the perceptions of policy makers in both Nepal and Uganda that the Nutrition Innovation Labs are not yet meeting their policy needs. • Due to tensions between the U.S. and the Uganda Office of the Prime Minister, the Nutrition Innovation Lab/Africa needs to develop strategies for working directly with ministries involved in the multi-sectoral nutrition planning process. • The ME should consider a stronger effort at coordinating design and data collection methods between the two host country sites. The BoD and TAC contain expert 37 researchers and should provide the impetus for the continuation of the development of a set of shared metrics for the projects. • The Nutrition Innovation Labs should continue to pursue Associate Awards that broaden the research questions being addressed and provide opportunities for operations research and more context-driven research. 38 RESEARCH PROGRAM The Scope of Work for the Nutrition Innovation Labs evaluation asked the EET to address the following questions with respect to research: 1. Does the body of research being funded by Feed the Future make strategic contributions to the following high-level research questions: (a) what are the agriculture-to-nutrition pathways; (b) what are the program impact pathways; and (c) what is the value of integrated programming pathways? How might the research design for the two programs be adjusted, if necessary, to better answer the research questions and fill the evidence gaps? 2. What challenges have the two Nutrition Innovation Labs faced during research design and implementation? What impact, if any, have these challenges had on implementation of research activities? How effective have the two research programs been in addressing the challenges? What could they do differently to better address the challenges? A summary of the research programs can be found in Appendix I and a list of project publications and research briefs can be found in Appendix J (as well as in Table 1). BACKGROUND ON RESEARCH OF NUTRITION INNOVATION LABS Agriculture is a clear contributor to human health and nutrition because one of its main roles is to produce food and subsequently essential nutrients necessary for human consumption. Despite much rhetoric suggesting that enhanced agriculture leads to improved nutrition and health, there is very little empirical evidence of the kinds of actions in agriculture that do (or do not) support nutrition and health (or vice versa), or the optimal ways to integrate actions beyond pilot activities to achieve value-added outcomes 1 . While it is important to conduct research on what works biologically to improve human nutrition (and to enhance cultivar productivity in agriculture), higher priority is needed on understanding what works operationally. To date, there is no well-defined package of agriculture and food-based nutrition sensitive interventions that address nutrition and health outcomes2 . Mapping how these interventions would be delivered, through what channels and to which target beneficiaries, may not be 1 Webb, P., & Kennedy, E. (2014). Impacts of agriculture on nutrition: Nature of the evidence and research gaps. Food & Nutrition Bulletin, 35(1), 126-132. 2 Ruel, M. T., & Alderman, H. (2013). Nutrition-sensitive interventions and programmes: how can they help to accelerate progress in improving maternal and child nutrition?. The Lancet, 382(9891), 536-551. 39 straightforward because agricultural systems vary across the world. Designing food-based interventions must take into consideration specific economic contexts at local, regional, and national levels. This further depends on agro-ecological aspects of landscapes and the variations of how food is produced, moved and accessed through food value chains. The focus of the Nutrition Innovation Labs’ research was to identify the kinds of investments in agriculture, agriculture and food systems policies, human health and nutrition that would achieve large-scale and sustainable improvements in nutrition outcomes for vulnerable groups, especially pregnant women and children less than 2 years of age. The research was centered on three strategic pathways: A) Agriculture to Nutrition Pathways: seeking greater clarity on cause and effect pathways (agriculture to nutrition). For examples, what ‘kinds’ of agriculture investment have greatest net impacts on nutrition. B) Program Impact Pathways: seeking clarity on constraints to program implementation fidelity at each layer of operational management from central (national) to local (ward) levels. C) Integrated Programming Pathways: seeking clarity on what combinations work best, in what context, and with what efficiency gains of integration (and costs). Overview of the Nutrition Innovation Lab Research Portfolios in Asia and Africa In both Nepal and Uganda, there was initial consideration of integrating research plans for the Nutrition Innovation Labs with projects relating agriculture, nutrition, and health being implemented by the USAID/Missions. Delays in awarding the contracts for the Nepal Integrated Nutrition Program (now called the Suaahara Project) and the Community Connector Project in Uganda (CC) contributed to the need for the Nutrition Innovation Labs to move ahead independently in both countries. Also, it became apparent that due to differences in core project objectives and the requirements of research sampling strategies, a direct meshing of the projects was not feasible in either country although some programmatic overlaps remained in both Nepal and Uganda. Thus, in order to accomplish their research agenda, Tufts University provided core funding to several U.S. and host countries entities and moved forward. The core research projects, partners, and progress will be described on the following pages, first for Asia and then for Africa. Nutrition Innovation Lab/Asia The proposed research agenda for Nutrition Innovation Lab/Asia (Nepal) was organized into three categories: 40 1. Understanding ‘the how’, not just ‘the what’, of programming to achieve successes at scale; 2. Filling defined knowledge gaps in nutrition, derived from country-defined priorities, and global assessments of the nutrition landscape; and 3. Identifying ‘essential packages’ of actions that link nutrition, health and agriculture around key problems in food, water and disease. Nutrition Innovation Lab/Africa The Nutrition Innovation Lab/Africa stated their research agenda to have three interdisciplinary goals: 1. Determine how, what and with whom changes are needed for simultaneous successes at the national scale in nutrition, health and agriculture; 2. Identify and fill knowledge gaps regarding linkages and potential synergies between nutrition, health and agriculture; and 3. Identify 'essential packages' of actions to address the interlinked problems of food, water, and disease needed to achieve measurable successes on a large scale. As discussed in the Management and Capacity sections of this report, there were significant delays in starting up the project due to changes in PIs at both Tufts University and Makerere University. The first Makerere PI was situated in the College of Health Sciences. The project then moved to the College of Agricultural and Environmental Sciences (CAES). In FY2013 the Makerere Principal Investigator transitioned to a new position as High Commissioner to the United Kingdom for the Republic of Uganda, and another professor at Makerere took over the lead role. FINDINGS To assess the research programs, the EET organized the research projects by location: Asia and Africa and are shown in Table 1. Project names, lead PIs, goals, design, interventions and summaries to EET directed questions 1 and 2 are included in the Table. A background summary of the research projects can also be found in Appendix I. QUESTION 1: STRATEGIC CONTRIBUTIONS AND RESEARCH DESIGN Most of the research projects in both Uganda and Nepal address elements that delineate the first high-level question (A): To better understand the agriculture and nutrition pathways. The other two high level questions (B and C) -- impact pathways and value of integration – will not be as thoroughly addressed due to the timing of the EET’s evaluation (mid-stream for many of 41 the projects), the nature of the design of the projects, as well as the multi-sectorality contributions. Although some of the smaller research projects could be seen as opportunistic, the larger panel and cohort studies (PoSHAN and the project assessing the Community Connector project) are well thought through in their design, and have the potential to strategically address all three aspects of the higher level questions, eventually. Overall, the package of research projects fits well within USAID’s Multi-Sectoral Nutrition Strategy (2014- 2025). The projects will produce data that demonstrate the value of integrated programming pathways based on interventions selected by the implementing organizations. The larger panel, surveillance and cohort studies in Nepal and Uganda will provide significant contributions to the cause and effect pathways of agriculture and nutrition. These projects are well designed and have substantive surveys that are encompassing most aspects of the causal pathway of nutrition (UNICEF 1990) and aspects of food security. The Nutrition Innovation Labs in both countries are primarily evaluating the effects of programs implemented by other USAID Mission projects, governmental initiatives, and NGOs. Thus, most of the projects remain dependent on other organizations in terms of the quality of interventions and implementation, as well as selection and frequency of interventions being implemented. This makes the research more challenging by design. It is always much easier to start a research project from scratch, having complete control of all facets, however in the case of most of these projects, design is led by other organizations that are overseeing the implementation of interventions. Thus, the hallmark randomized control trial (RCT), often considered the best measure of impact, is not possible in the projects undertaken by the Nutrition Innovation Lab. RCTs can be used to evaluate complex and dynamic processes, not just simple and static interventions. However most of the research supported by this grant, does not fit well with a RCT design. Instead, most of the research is designed as “learning by doing” or implementation science design. Agriculture-Nutrition Pathways The PoSHAN observational study in Nepal has the potential to make a significant strategic impact on better understanding the links and pathways between agriculture and nutrition. Primary data are being collected at the household level on demographics and socioeconomic characteristics, caregiver diets and health status, infant and young child feeding practices, health status of an index child, household food insecurity, gender and decision making, hired labor and access to information and infrastructure. The surveys also include questions about crop and livestock production, markets, and agricultural technologies and land management practices. The core panel is being repeated on an annual basis. With the focus on the “first 1000 days”, that annual repetition is important to capture a longitudinal look at the index children within that critical time frame. The second round of data collection began in May 2014 and will reflect changes over time in this population. 42 In Uganda, the cohort and panel are designed differently than the PoSHAN panel and sentinel site studies, but could still provide valuable information on the agriculture-nutrition pathways. In Uganda, the panels serve somewhat as an independent evaluation arm for a few of the Community Connector (CC) project sites comparing CC active and non-active sub-county households. The project is dependent on the CC approach and on a set package of interventions from which the communities choose. Thus, assessing efficacy by teasing apart the causative impact of specific interventions contained in these packages is challenging because the interventions are not uniform across sample communities. Depending on the involvement of the Nutrition Innovation Lab, assessing program effectiveness of the CC is a realistic option. When EET met with CC colleagues, the relationship between implementation and research was perceived as strained. Without the Nutrition Innovation Lab having some control over delivery of interventions, effectiveness studies will be challenging because the design is dictated by programming not the apriority research objectives. While the counterfactual measures should be sufficient, the EET concluded that there is a concern that in the first phase, there may not have been enough households in each sub-district to make a clear distinction between the intervention and comparison groups. The EET was unable to determine whether the sample size is not sufficient due to inadequate CC coverage or due to sample size issues. Furthermore, with the two-year cycle currently planned for the panel survey in Uganda, all index children selected in the baseline study will already be beyond the age of two years at the time of the next panel, which presents an issue in understanding how these pathways impact the first 1000 days. The project being undertaken by Harvard University in Bhaktapur, Nepal is a longitudinal follow-up study of mother-infant pairs. The work and data being collected is rooted mainly in public health and less so in agriculture. It would be of great interest to expand this work for several reasons. First, the lead researcher in Nepal, who is also affiliated with the University of Bergen, is very engaged in the work and the EET concluded that she provides substantive leadership on the project. Second, this project provides an opportunity to examine prospectively the effects of early diet on cognition of school-aged children, which is a major research gap in the nutrition literature. Third, the project location is peri-urban. This is a very important, and an interesting model of agriculture-nutrition linkages to further explore. Fourth, there is a need for more integration of agriculture and food security survey modules for this study. The EET reviewed the survey and the food security/agriculture questions were considered less robust. The resulting data will provide rich information on nutrition and health outcomes but much less on the links between peri-urban livelihoods, agriculture and income￾generation and nutrition outcomes. Most of the other studies in both Asia and Africa are either exploratory or secondary data analysis on “side” or less direct pathways that impact agriculture, food security and nutrition. The Nepalese projects with Heifer International, Helen Keller International and Development 43 Alternative International, for example, although interesting research areas, seemed more opportunistic than strategic. However, these projects can still make important contributions to the evidence base. The work on aflatoxins, climate change and variability, and environmental enteropathy in both Uganda and Nepal are all critical pieces to the agriculture-food-nutrition outcome pathway puzzle, however it may be a challenge to pull all the data together into one cohesive story in each country because of the use of different data sets, diversity in disciplinary approaches among the PIs, and different research designs. While each piece could be important as stand-alone research projects, pulling together these data into one cohesive body of work may be more challenging for the Nutrition Innovation Labs. These bodies of work will still inform the field and USAID’s multi-sectoral nutrition strategy, however the data coming from these individualized studies may be seen as piece meal or one-off projects without clear links to the larger research questions or Nutrition Innovation Lab strategy (A-C above). Purdue University is doing interesting work in both Nepal and Uganda utilizing secondary data analysis and remote sensing data to assess temporal patterns in agriculture and health, as well as on health effects of environmental exposures (indoor smoke pollution). The EET concluded through interviews with Purdue colleagues and others, and after careful examination of the survey protocols, that this approach and work could be better consulted and integrated into the other larger projects moving forward. Program Impact Pathways Unpacking the “active ingredients” of multi-sectoral programming on health and nutrition outcomes in various studies is challenging for all of the projects taking place in both Nepal and Uganda. In the case of Nepal, in the Village Development Committees (VDCs) being sampled, there are many organizations working on nutrition and communities are targeted frequently. It should not be expected that these projects can easily tease apart contribution or impact of single interventions (or specific projects) to the nutrition and agriculture pathways due to the design of the studies, the complex nature of multi-sectoral programs, and the cross over of different organizations working in the same communities. Multi-sectoral approaches to development have been difficult to implement in the past but such approaches are consistent with Feed the Future objectives, the USAID Multi-Sectoral Nutrition Strategy, and both Nepal and Uganda are implementing fledgling multi-sectoral plans. It is clear to the EET that work done in both Nepal and Uganda will make some contribution to the gap in evidence on how best to integrate sectors. The results coming from both the PoSHAN study and the CC Uganda panel and cohort have the potential to assess impact pathways. The design and sample size will allow for impact assessments to be done. However, at this time and stage of the data collection, it is not possible for the EET to evaluate these impact assessments. The 44 data are still being collected and quality of data analysis was not assessed because it is not yet available. While there are important investigations of biological mechanisms planned in terms of relations between aflatoxin and infant and child growth and between sanitation, infectious disease, and growth, unpacking the pathways and how sectors/disciplines contribute, may be a challenge with some of the smaller, observational and secondary analysis projects. The EET found that the agriculture inputs into the research projects, in both Nepal and Uganda were not as robust as they could be. New strategies and metrics need to be added so that agriculture is better integrated into the research programs to avoid underestimating the potential impacts of agricultural interventions on nutrition and health. There are many different assessment metrics being developed by different academic institutions and the Consultative Group for International Agriculture Research (CGIAR) centers. To be on the cutting edge, it will be important to start engaging with new investigators who are not necessarily only at the few institutions traditionally strong in international nutrition and health and expand to agriculture schools working in Asia and Africa. There are a few experts working on tandem projects (from other Innovation Labs and Purdue University for example) who could be better used on the project. Value of Integrated Programming Pathways The integrated research in both Nepal and Uganda are seen by interviewed stakeholders in the countries as visionary and valuable. Most of those interviewed and those being trained, find the research undertaken by the Nutrition Innovation Labs very valuable for future program and policymaking. There are two research projects that are attempting to better understand the linkages between agriculture and nutrition. These studies are examining how approaches to collaboration and the ‘quality’ of policy implementation (determined through the knowledge, attitudes and practices of stakeholders involved in implementing multi-sectoral policies and actions), may impact agriculture, health and nutrition outcomes on the ground. The EET was given a preliminary presentation of the finding from Nepal, but no data have been shared for the work in Uganda. These deep-dive qualitative assessments could provide valuable information for several reasons. First, the sampling for interviews will provide a breadth of opinion on the current nutrition situation from various stakeholders who are intimately working on nutrition and agriculture in the country. Second, the nature of qualitative data allows for a substantive analysis of perceptions, commitments, incentives and transparency of nutrition programs over time in these two countries, which is not captured in other projects as part of the Nutrition Innovation Labs. Third, the studies could provide measures of accountability, which currently are 45 considered an important topic on the global nutrition agenda (accountability is considered one of the main themes at the International Conference on Nutrition 2 (ICN2) and within the Global Nutrition Report.). The Nutrition Lab/Asia PD shared some initial analysis of linking these data with nutrition outcome data to better assess progress and accountability. These data are important. However, it will be important for the teams to note that “scoring” districts on how they progress is political (data are always political!) and the researchers will need to tread lightly on how to present these data. That said, the PD and the co-PD have extensive experience engaging with member states, governments and other relevant stakeholders and their diplomacy has been a strength in the overall project. The EET compared the research projects being undertaken in Nepal and Uganda with the original proposed research agendas for the Nutrition Innovation Labs (shown in the beginning of the Research section): • The Nutrition Innovation Lab/Asia is fulfilling all three agendas: to better understand the how and the why of programming, filing knowledge gaps derived from country priorities and identifying essential packages. The combinations of the larger projects and the smaller projects will address these three agendas. • The Nutrition Innovation Lab/Africa will partially address their three distinct agendas: determine how, what, and with whom changes are needed for simultaneous success at the national scale may be difficult with the current research designs. Initially, the Nutrition Innovation Lab’s original design may have addressed that goal more substantively but the current panels and cohorts paired with the Community Connector project may not address the simultaneous aspects of national scale. The second and third goals will most likely be achieved because the data being collected will address the linkages between food, water and disease. Both projects harness the idea of operations research. Both projects are attempting to answer linkage questions of agriculture and nutrition (both synergies and trade-offs), while fitting within country-driven priorities, which can be challenging. Research design has not been ideal from a pure research perspective because the Nutrition Innovation Labs’ research must fit around what is already happening in the country, what intervention packages are already in place (without having control over when those interventions are given, by whom and for whom), and timing of programs and large-scale projects. This poses a challenge and an opportunity. There has been much less emphasis on how to make agriculture and nutrition work for each other and the EET concludes that the series of projects will shed light on this major research gap. While they will be able to address biological efficacy to a minor degree (perhaps the water studies will provide some of this evidence), the research undertaken will address program 46 effectiveness. In the world of agriculture and nutrition research, the demand is for more effectiveness trials and these projects will make a significant contribution in the local context of Nepal and Uganda. While the value of this work is important (the linkages between agriculture, health and nutrition), translating the findings to a policy context to ensure that they are relevant at the global level will be more challenging. QUESTION 2: CHALLENGES FACED AND SOLUTIONS Both in Nepal and Uganda, the projects encountered the expectation that they would be wrapped-around major USAID Mission-funded projects linking agriculture, nutrition, and health. Particularly in Uganda there was an expectation that the Nutrition Innovation Lab/Africa would serve as the monitoring and evaluation arm for the Community Connector project. Although monitoring and evaluation was not mentioned in the RFA or Tufts University’s response to the RFA, this misalignment of expectations caused tensions and delays in initiating the projects. At the time the projects were initiated, there was a rapid turnover in AORs in USAID/Washington as well as the usual problem of turnover in personnel at the Missions. Mission staff and other stakeholders report that these misunderstandings have been resolved and there is generally satisfaction with the research approaches being taken by the Nutrition Innovation Labs. The EET recognizes that these initial misunderstandings delayed the initiation of research in both countries and would like to emphasize the importance of frequent and transparent communication between all parties associated with projects such as the Innovation Labs and the USAID Missions. Currently, USAID and the government of Uganda are on “strategic pause”. Due to this situation, there were significant delays in getting started on the policy analysis in Uganda. In Nepal, there is a challenge because the Government of Nepal has a policy of frequent rotations of posts for their upper-level government officials. This means that the value of research for building an evidence base for policy decisions needs frequent re-explanation and in the next round of data collection on policy analyses, the information received by the researchers could be dramatically different depending on political priorities. In Uganda, change in the Project Director for the ME at Tufts and changes in PIs from the College of Health Sciences at Makerere University to the College of Agricultural and Environmental Sciences (CAES) have created delays in starting the research and establishing an identity for the project. Furthermore, the first PI from CAES, who was a professor from the Department of Food Technology and Human Nutrition and supervised the first panel study in 2012, was appointed as Uganda High Commissioner to the UK and assumed that post in 2012. Subsequently, a professor from the Department of Agribusiness and Natural Resource Economics and Principal of CAES became PI for the project. Additionally, the original host country project coordinator has begun studies at Tufts University. He is supporting the project 47 from the U.S. side, but there was a five-month gap in hiring his replacement, seemingly without a written manual of operations or formal handover of responsibilities. Many of these changes were outside the control of the ME but have added to disconnects in communication and speak to the need for more direct involvement of the ME Project Director for timely accomplishment of project objectives. Issues related to the relationships between the ME and the sub-awardees suggest that the “all the eggs in one basket” approach, in which a small number of sub-awardees manage larger research projects, may have some other important limitations to achieving the goals of the Nutrition Innovation Labs. While the sub-award process was well conducted and the roles among the partners were negotiated in each of the settings, concerns about the implementation of research activities, data analysis and dissemination by the research partner/sub-awardee in Uganda reveal weaknesses in the approach. In the case of Uganda, the major sub-awardee has a very light presence in the country. Furthermore, while the two major research projects attempt to capture outcomes across a range of interventions, there is little space and funding for smaller, more detailed research projects that can examine process more closely. Another challenge that the EET heard frequently from in-country stakeholders interviewed, was on the objectives of the research and who were the intended beneficiaries. Some individuals see the Nutrition Innovation Labs’ research as an academic exercise with a primary purpose to publish in peer reviewed journals (See Appendix J for publications list). Those stakeholders, particularly in development and government, would like to see data shared and presented more rapidly and in a timely-fashion that corresponds to nutrition events in the country, so that the work can inform programs and policy at a larger scale. While the work is seen as important, the view was that the data were slow to be released. There were also some voiced concerns in interviews about sharing survey instruments and even cleaned data sets to others in the host countries for their own use and analytical purposes. This is always a difficult balance, and a fine line must be made between open access and ensuring that data quality and interpretation remain intact. The EET understands that data cannot be released without thorough cleaning of data so that all analyses are carried out with a clean, error free data set, which can take significant amount of time. Perhaps a protocol guideline for data use could be created by the Nutrition Innovation Labs and shared with countries and partners so the expectations are established early on of when and how data will be released and used. Lastly, it is important to understand whether the research design is being undertaken in the context of sufficient coverage of programmatic interventions so that results stemming from the research can inform what is and what is not effective for nutrition outcomes. While understanding the determinants of malnutrition is important, there is a clear need to better understand what interventions are effective and how those interventions should be delivered. 48 Some of the smaller studies and partially the PoSHAN study in Nepal will help answer some of the determinants of malnutrition. However because some of the projects in both Nepal and Uganda are implemented in the context of fitting around country specific project objectives, more information on intervention effectiveness and their delivery will need to be addressed. There are a few issues that concern the EET: 1. The timeliness of analyzing and releasing these data. There are major gaps and calls for integration of agriculture and nutrition. The evidence in their linkages is far behind the evidence of how public health can improve nutrition. As countries commit to scaling-up nutrition, they cannot wait for evidence on how to effectively improve nutrition through agriculture and the environment. Data must come more quickly from the Nutrition Innovation Labs and be shared with country implementers and policymakers in a more timely way. 2. The identification of probable program impact pathways should always be a prerequisite for the designing of an intervention. In some cases for the Nutrition Innovation Labs, the incorporation of interventions believed to be critical in the literature and the development of a clear set of hypotheses was not possible because the research design had to fit into already pre-existing, pre-defined projects. This could lead to some issues in assessing programmatic fidelity. In the case of the PoSHAN project in Nepal, more controls have been put into place to ensure fidelity. (Fidelity means that researchers need to verify that their interventions were delivered as designed (intervention fidelity), and that variations from the design can be assessed. Infidelity can result in non-significant findings that are not due to the study design but rather to elements that affected the intervention delivery 3 ). Furthermore, USAID Mission offices should be clear on the distinction of what the Nutrition Innovation Lab can deliver on with regard to their research results: effectiveness research as opposed to efficacy research is a much more realistic option. 3 Horner, S., Rew, L., & Torres, R. (2006). Enhancing intervention fidelity: a means of strengthening study impact. Journal for Specialists in Pediatric Nursing, 11(2), 80-89. 49 Table 1: Summary of Research Projects The high-level research questions are below and projects are aligned to either A, B and/or C (See Column 5). A) Agriculture to Nutrition Pathways: seeking greater clarity on cause and effect pathways (agriculture to nutrition). For examples, what ‘kinds’ of agriculture investment have greatest net impacts on nutrition. B) Program Impact Pathways: seeking clarity on constraints to program implementation fidelity at each layer of operational management from central (national) to local (ward) levels. C) Integrated Programming Pathways: seeking clarity on what combinations work best, in what context, and with what efficiency gains of integration (and costs). Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Uganda Makerere University Professor Bashaas ha Empirically understand pathways by which interventions in agriculture impact on nutrition and how such pathways can be enhanced through appropriately designed multi-sector interventions Addressing A and B Multiple intervention packages are available through the USAID Community Connector Project (CC). Selection of interventions to be delivered is driven by community demand Design: Longitudinal w/counterfactual "controls" Sample size: 3630 households in 6 districts; 4500 U5 children Local and international presentation s on baseline results, but no publications to date At this point, it will be important in phase 2 to ensure that the pairing of districts is put into place to create a counterfact ual design. Baseline survey got started quite late into the project (Y3 - 2012) mainly due to delays in identification of CC target districts and to leadership changes for the research at Makerere University. No panel survey was conducted in 2013; Second panel survey scheduled for October-December, 2014. Yes. Professor Bashaasha is now leading the project and two additional competent staff has been hired to oversee the panel. If possible, the same enumerators who performed the baseline survey will be hired for the 2014 panel, which should go more quickly and smoothly. Tablets will be used for data collection. 50 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Uganda Harvard School of Public Health Drs Fawzi and Duggan Generate prospective evidence to better understand the agriculture to nutrition linkages relative to the role of maternal status in child growth and development as well as to examine the effectiveness of large scale integrated/multi￾sectoral programming targeting maternal and child nutrition Addressing A and B Multiple intervention packages are available through the CC. Selection of interventions to be delivered is driven by community demand. In targeted subcounties reaching pregnant women - use of WASH, Antenatal care, iron and folic acid and social support. The birth cohort will be examining this specific cohort. Design: Quasi experimental comparing CC to non-CC sites. Cohort to be established from randomly selected CC Phase 1 and Phase 2 subcounties and in matched non-CC subcounties as counterfactuals. Planned sample size: 5152 pregnant women during an enrollment period of 12 months and continue until child age of 2 years. None No adjustments to the design are needed, but clear timelines for deliverables from ME to HSPH would be appropriate. Initial project enrollment planned for August/early Sept, 2014. With a planned enrollment period of 1 year and a monitoring window of 820-910 days, the project is unlikely to be complete before February, 2018 Enumerators will be hired from the local area so training times may need to be extended. There are communication, transport and cold￾chain issues with a birth cohort study in rural areas. Need substantive co￾ordination from the Nutrition Innovation Lab/Africa PD with project PIs to ensure smooth roll-out of project. Tablets will be used for data collection. 51 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Uganda Tufts University Dr. Eileen Kennedy Empirically determine how approaches to collaboration and the ‘quality’ of policy implementation (determined through the knowledge, attitudes and practices of stakeholders involved in implementing multisector policies and actions), may impact agriculture, health and nutrition outcomes on the ground Addressing C National level stakeholder interviews were conducted in 2014. District and lower level interviews to be conducted in conjunction with the second panel survey scheduled for September￾December, 2014. Design: Qualitative: Interviews using structured and semi-structured questionnaires; data collection from all levels of government and from stakeholders across all sectors of activity relevant to implementation of Uganda’s multi￾sector nutrition plan and the SUN movement Sample size: Approximately 30- 40 stakeholders (although Kennedy and Florence K. have interviewed over 100 stakeholders at the national level). Although the EET has not seen the results of this work, a paper is planned for later this year. No adjustment to be made. There were some challenges in scheduling interviews due to the political situation in Uganda. Interviews were done earlier this year (early 2014) and due to the nature of qualitative research, data took time to collate and analyze. The challenges in doing politically driven interviews in Uganda have not subsided. USAID is currently on "strategic pause" with the government making the entry points for the investigators more challenging. The situation ongoing in Uganda may be out of the hands of the researchers, however they managed to complete the political interviews and hopefully, if data are collected in a second round (the EET encourages this in the second phase of funding), the political situation may be manageable. Uganda Gulu University and Cornell University Barnabas Natamba Examine issues of food insecurity, HIV infection, and depression and their influences on maternal and child nutrition and health Addressing A Not applicable Design: Purposive, longitudinal case￾control, hospital based study. Sample size: 400 HIV infected and uninfected pregnant women. 250 of these women and their infants to be None No adjustments to the design are needed; data on aflatoxin exposure and Early Childhood Developme None noted. Not applicable 52 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? examined for neurocognitive early childhood development up to 12 months of age. nt assessment s are being added to Phase 2 of the project. Uganda Makerere University & University of Illinois Benito Jose Marinas Understand the role of water and sanitation in the pathways linking agriculture, nutrition and health. Specific goal of pilot study was to validate the use of the Aquagenx Compartment Bag Test (ACBT) Addressing A Not applicable Design: Exploratory and validation research None Adding the data from the ACBT to other trials will help to evaluate the effectivenes s of delivery of WASH intervention s as well as effects of diarrhea on growth of infants and children. Large-scale trial not yet implemented. ACBT is a field￾friendly tool but some training will be required for implementation. Uganda Purdue University and NASA Professor Gerald Shively Examination of spatial and temporal patterns in agriculture and health Addressing A Not applicable - Examining associations Design: Secondary data analysis Sample size: Study 1 - 180 households; Study 2 - 1209 women & 598 children 2 - peer reviewed articles No adjustments needed None noted. Not applicable Uganda IFPRI Dr. Nassul Kabunga Examination of relations between fruit and vegetable production, individual F&V intake, household food Addressing A Multiple intervention packages are available through the CC. Examined fruit producers vs non-fruit producers Design: Secondary data analysis from the panel survey (baseline) and will look at second collection as a 1 IFPRI working paper; 1 peer reviewed article No adjustments needed None noted. Not applicable 53 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? security and anemia in women irrespective if in CC area or not. comparison. Sample size: 3630 households in 6 districts Uganda Tufts University and Peanut and Mycotoxin Innovation Lab members Dr. Will Masters and Shibani Ghosh Identify opportunities for new investment & interventions to improve nutrition and livelihoods on a commercial scale. Enhance understanding of crop value chains particularly relevant to women and to the issues around value chains involving aflatoxin-free foodstuffs Addressing A Not applicable Design: Qualitative and analyses of market chains; interviewed aggregators and producers Sample size: Approximately 50 Research Brief published No adjustments needed None noted Not applicable Nepal Johns Hopkins University, New Era Pvt. Ltd. Professor Keith West Empirically understand pathways by which interventions in agriculture impact nutrition, and how such pathways can be enhanced through appropriately designed multisectoral inventions Addressing A and B Participation in selected agricultural, health and nutrition interventions provided by various governmental and non-governmental organizations Design: Observational study; Stratified random sample from the three agro-ecological zones in Nepal (mountains, hills, and terai); 21 districts, 7 from each zone; 2nd panel just collected Sample Size: 4288 households, 5401 U5 children None No adjustments to the design should be made. Field work difficult due to terrain issues and beginning of monsoons. Field surveys will be timed for completion just ahead of monsoons 54 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Nepal Johns Hopkins University; New Era Pvt. Ltd. Professor Keith West Addressing seasonality interactions with effectiveness of multisectoral interventions Addressing A and B Participation in selected agricultural, health and nutrition interventions provided by various governmental and non-governmental organizations Sample Design: Purposive 7 Village Development Committee (VDCs) sampling across 3 agroecologies (mountain, terrai and hills) Sample size: One sentinel VDC (3 wards in each) selected to represent the median in each zone based on population density, demographics and socioeconomic factors None No adjustments to the design should be made. Only one seasonal survey collected during the first implementation. 3 surveys per year are planned Nepal Tufts University with Valley Research Group, Patan Academy of Medical Sciences Drs. Patrick Webb, Kedar Baral Empirically determine how approaches to collaboration and the ‘quality” of policy implementation (determined through the knowledge, attitudes and practices of stakeholders involved in implementing multisectoral policies and actions) may impact agriculture, health and nutrition outcomes Addressing C Not applicable Design: Qualitative Design across 21 VDCs, annual surveys planned. Government officials in 6 defined ‘layers” of governance (national, regional, district, Jlaka, village development committee and ward) and additional non￾governmental stakeholders working on agriculture and nutrition programing were None yet. It will be important to ensure that the data are presented in a politically appropriate way. Scoring regions on their impact can be empowerin g or disengaging . Nepal has government shifts that may be important to take into account in the next data collection round. Although it did not pose a challenge initially, the changing political parties could present an issue in comparability. There is not much they can do in this situation. Other than to provide documentation. 55 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? interviewed. Interviews at the local and VDC levels were conducted in the same field sites where JHU collected community-level data Sample size: ~ 700 key informants (linked to POSHAN panel survey) Nepal Tufts University with Heifer Internationa l Dr. Laurie Miller To determine the value-added of specific nutrition knowledge over and above enhanced knowledge in livestock management Addressing A Interventions being done by Heifer. 1) Heifer livestock training (+ life skills) plus nutrition training – 289 families; 2) Heifer livestock training – 360 families 3) No activities – 304 families Design: Randomized control trial Sample size: One district; 960 households; 1,300 U5 children None yet? No adjustments needed None noted. Not applicable 56 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Nepal Heifer Internationa l Nepal and Nepali Technical Assistance Group (NTAG) Dr. Laurie Miller Determine how measures of dietary diversity correlate with nutrition outcomes Addressing A Interventions being done by Heifer. 1) Heifer livestock training (+ lifeskills) plus nutrition training 2) No activities Design: Randomized control trial 3 sets of paired communities; Communities were randomly assigned to control or intervention status. (Baseline plus four surveys at 6-mo intervals prior to Nutrition innovation Lab support-- Nutrition IL funded additional 2 rounds of data collection during FY2013). Sample size: 415 randomly selected households None yet? No adjustments needed None noted. Not applicable Nepal Harvard School of Public Health: Bergen University, Norway; Institute of Medicine, Tribhuvan University Drs. Fawzi, Duggan and Thorne￾Lyman Assess measures of diet quality in relation to anemia outcomes, child growth and household food security in a panel of mother-infant pairs; peri-urban environment Addressing A and B None although routine health services were monitored Design: Cohort study of 500 mother-infant pairs; doing follow-up survey Sample size: Follow-up survey of 319 (may increase to 367) mother￾child pairs from a cohort study of 500 mother-infant pairs conducted in 2008 Yes. Some joint peer reviewed publications with University of Bergen. No adjustments needed and study is finishing. The agriculture related questions could have been stronger with the assistance of agriculture and food security experts. None noted. Cognitive studies are now being implemented in the same area (Bhaktapur). Will be important to link these data with the upcoming research. Not applicable 57 Location Institution PI Goal of project Does the project make a contribution one or more of the high level research questions? Interventions being delivered Design (including type, sample size and target groups) Number of Publications (major output) QUESTION 1: How might the research design be adjusted? QUESTION 2: Challenges during design and implementation? QUESTION 2: Have the challenges been addressed? If no, what could they do? Nepal Purdue University Professor Gerald Shively Measure the connections between agricultural capacity, technology adoption, nutrition outcomes, and conditioning factors at levels of aggregation ranging from household to district Addressing A Not applicable Secondary data analysis of NLSS and DHS Not clear. No adjustments to the design should be made. None noted. Not applicable 58 Nepal Developme nt Alternative Internationa l Pilot project to build the capacity of a cottage-industry organization in Kathmandu to improve the production of their complementary food product, Sarbottam Pitho Addressing A Not applicable Value chain analysis Not clear. No adjustments to the design should be made. None noted. Not applicable 59 CONCLUSIONS • Projects are on track to answer the three high level questions to varying degrees with the majority being able to address the Agriculture to Nutrition Pathways: seeking greater clarity on cause and effect pathways (agriculture to nutrition). • Some of the projects could have better integration of agriculture, climate and ecosystems and use of current expertise (Purdue University) to ensure that robust questions on the agriculture and food security fronts are included. Currently, most of the work is led by public health experts. • The projects are largely addressing program effectiveness; however it may be difficult to put the findings from the different projects together into a cohesive story that can be effectively communicated. Although the research studies can be seen as distinct, the projects should strategically fit into the higher level questions. At present, some projects are not seen as strategic and considered more opportunistic. This may be a positive attribute, but there needs to be some cohesive thread that is shared in publications and for USAID’s programmatic and investment decision-making. • The political analysis is important but is met with challenges that are based on the stability and consistency of governments. This presents a challenge for the research regarding timing and sharing of results. There is also a need to ensure that data are presented in a sensitive way due to its political nature. • Much of the research being done will fill knowledge gaps and provide a better understanding of how to effectively deliver packages of agriculture and nutrition interventions in various settings. LESSONS LEARNED • Inclusion of local staff in data analysis and sharing of information at the country level is key for local ownership and support. • Fitting research around already existing programs with set interventions is a significant challenge in establishing efficacy and effectiveness of interventions. • Multi-sectoral research paired with implementation is challenging and requires expertise on the ground to manage. • Data collection and analysis need several stages of sharing and dissemination to keep momentum and collaborations running smoothly. • Engaging agricultural scientists, food security experts and climatologists is critical to effectively understand agriculture investment pathways and their impacts on nutrition. 60 RECOMMENDATIONS • The comments from certain stakeholders indicate that they see the Nutrition Innovation Labs’ work as an academic exercise, and that data gathered should be better used to address local policy and programs in a timely manner. To establish their relevancy in the policy realm, there must be more attention to building local capacity to utilize project data for policy purposes. • In the spirit of collaboration survey instruments should be posted in the Workspace to be available for use in the host country for other related projects. • New strategies and metrics need to be included so that agriculture is better integrated into the research programs to avoid underestimating the potential impacts of agricultural interventions on nutrition and health. • Research findings should be made available to local researchers and partners for communicating results in-country in different fora. • Many different assessment metrics are being developed by different academic institutions and the CGIAR centers. To be on the cutting edge, it will be important for the Nutrition Innovation Labs to start engaging with new investigators that are not necessarily only at the few institutions traditionally strong in international nutrition and health. • The EET encourages the Nutrition Innovation Labs to think of creative and sustainable ways to continue communication with government officials currently and previously in relevant sectors. • The website needs to be viewed as a significant source of scientific information as well as a source of evidence-based research that makes a difference. The website could have a particularly important role in building advocacy in Nepal with its unique practice of rotating government officials, but it is not being used in its present state. Data analyses should be available on the website to partners before being presented at a meeting and without waiting for appearance as a publication. 61 TRAINING PROGRAM The SOW (Appendix A) for the Nutrition Innovation Labs evaluation asked the EET to address the following questions with respect to training: 1. Have the Nutrition Innovation Labs met academic and technical capacity strengthening targets? Are the appropriate type and number of people being targeted for the right kind of training? What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? 2. How have trainees put into practice the knowledge and skills acquired? How have the training programs contributed to strengthening institutional capacity in the target countries? FINDINGS QUESTION 1: MEETING ACADEMIC AND TECHNICAL CAPACITY STRENGTHENING TARGETS Overall, the Nutrition Innovation Lab/Asia and Nutrition Innovation Lab/Africa have not been as effective in reaching the targets for peer-reviewed publications as they have for providing training. This is particularly so for papers that are co-authored by investigators from host country institutions. It may be because it is still too early to publish results due to delays in starting up the research projects. However, moving forward, feedback from local partners and stakeholders, collected in interviews, indicated that it would be important to them for the ME to ensure that local scientists serve as first authors on papers and are given the opportunity to perform data analyses and “own” the data as well. The Nutrition Innovation Lab/Asia and the Nutrition Innovation Lab/Africa, predominantly the ME, have been more successful in publishing brief articles and doing presentations. In most cases, in each year of the project, these targets have been exceeded. The Nutrition Innovation Labs have worked toward increasing academic and technical capacity of host countries and U.S. institutions particularly in the area of nutrition and child health but more emphasis and effort can be directed in host country institutions, particularly on other areas of translation research and linkages between agriculture and nutrition. Four indicators on training of people and institutions have been tracked as of 2013 for both the Nutrition Innovation Lab/Asia and Nutrition Innovation Lab/Africa. The indicators are: • Number of people trained in child health and nutrition (nutrition science, dietetics, 62 public health nutrition) through USG supported programs (longer term) • Number of people trained in child health and nutrition (nutrition science, dietetics, public health nutrition) through USG supported programs (short term) • Number of US and host country institutions with enhanced capacity to assess, plan, design, implement, monitor and/or evaluate nutrition programs, policies and practices • Number of U.S. and host country institutions and individuals who have gained enhanced capacity in clinical, operational, agricultural, translational and/or public health nutrition research aimed at the reduction of malnutrition with USG assistance As part of training, the coauthoring of peer-reviewed publications is also important and there are two indicators for the Nutrition Innovation Labs. They are: • Number of peer-reviewed journal articles co-authored with host country institutions and others in country with USG assistance (submitted or published) • Number of brief articles and presentations co-authored with host country institutions and others in country with USG assistance Type and number of people being targeted for training In meeting their training targets, almost all four targets on training of people and institutions were reached in 2013 for both the Nutrition Innovation Lab/Asia and Nutrition Innovation Lab/Africa. The targets and actual results are: • Number of people trained in child health and nutrition (nutrition science, dietetics, public health nutrition) through USG supported programs (longer term) o Asia = 5 actual/2 target o Africa = 19 actual/2 target • Number of people trained in child health and nutrition (nutrition science, dietetics, public health nutrition) through USG supported programs (short term) o Asia = 14 actual/69 target o Africa = 129 actual/100 target • Number of US and host country institutions with enhanced capacity to assess, plan, design, implement, monitor and/or evaluate nutrition programs, policies and practices o Asia = 9 actual/6 target o Africa = 15 actual/12 target • Number of U.S. and host country institutions and individuals who have gained enhanced capacity in clinical, operational, agricultural, translational and/or public health nutrition research aimed at the reduction of malnutrition with USG assistance o Asia = 31 actual/30 target o Africa = 163 actual/100 target The targets for training are focused on research design and surveillance as well as child health and nutrition (nutrition science, dietetics, public health nutrition). The short-term training 63 curriculums have focused on this for the BBNC, Johns Hopkins and Harvard trainings. In the original year one targets, short- and long-term agriculture sector productivity or food security training was included. However, in later years, these targets were removed. It is not clear to the EET why this decision was made. Nutrition Innovation Lab/Asia A number of Nepalese have been involved in the short-term trainings, particularly the Bangalore Boston Nutrition Collaborative (BBNC) training sessions. Some of the trainees found the training beneficial, but are enthusiastic to receive additional, deeper training in research methods and data analysis areas as well as food security and agriculture. Three Nepalese are receiving formal degree granting training at the Tufts University and Tuskegee University, and one student is studying at Purdue University. The two Nepali students interviewed by the EET are content with the quality of these degree programs. Lastly, the Nutrition Innovation Lab/Asia is continuing its collaboration with Nepali academic partners in developing the curriculum of a public health nutrition master’s degree. The process seems slow due mainly to local university bureaucracy and approvals, and as indicated in the Capacity section of this report, ensuring that local partners (Tribhuvan University, Patan Academy of Medical Sciences and the Institute of Medicine (IOM) in Nepal) have ownership of the curricula, will be critically important. Tufts University should be seen as technical advisors to ensuring that quality and capacity is ensured for the degree program. This partnership will prove to be additionally fruitful if student and faculty exchanges and research collaboration are born from this new degree program. Nutrition Innovation Lab/Africa In Uganda, Ugandan trainees who attended the BBNC training sessions viewed them as very beneficial for their professional development and applicable to their current work. Some requested that additional sessions be designed that build on the core curriculum of the first training. On the formal training side, eight Ugandan MSc students at Makerere are being supported by Nutrition Innovation Lab/Africa, but there are some administrative and management issues between Tufts University, Tuskegee University, and Makerere University. The issues center around the financial support of students’ tuition and their research projects as noted above in the Management section. These poor relations have resulted in negative perceptions of the training experience by the eight students interviewed by the EET. Most of the students expressed that they do not get clear directions from their mentor at Makerere University and are doing research that does not squarely fit into the overall objectives of Nutrition Innovation 64 Lab/Africa. They also did not get clear directions on how much should be spent on research and as a consequence, most students over-budgeted their projects, which then had to be scaled back. In addition, some of the students at Makerere University who are supported by Tuskegee University have had considerable difficulty with timely receipt of tuition payments and stipends, which caused personal stress. Finally, after the students had settled on topics for their thesis, the Tuskegee PI sent a list of topics from which they were required to choose. As noted in the Management section, there has been a pattern of poor communication, unclear invoicing and fundamental disagreements among the Nutrition Innovation Lab/Africa PD, the Tuskegee PI and the Makerere supervisor concerning the conduct of research and research budgets which have contributed to a poor training experience for the students enrolled in the masters’ program. Overall, there is also lack of clarity for development of their research topics and budgets; transparent details about availability of research funding and its timely receipt seem to be critical to improve this situation. One solution moving forward could be to take on fewer students and provide better quality mentorship to those who are selected. Another solution could be to change the management strategy for this training relationship between Tufts University, Tuskegee University and Makerere University. However, part of the overarching objective of this project is to build training and capacity at all levels – not just in research – but in management as well. It is important for all three universities to reflect on their own management skills and find better ways to communicate and manage expectations, develop work plans and establish accountability mechanisms. For the formal training by the Nutrition Innovation Lab/Africa, there are three graduate students at Tuskegee University and three graduate students at Tufts University, supported by the Nutrition Innovation Lab. There are also two graduate students at Purdue and Cornell Universities. Nutrition Innovation Lab/Africa is also supporting a researcher at the International Food Policy Research Institute (IFPRI) on a half time basis, who is considered a junior up-and￾coming scientist. This has been an important investment for the Nutrition Innovation Lab/Africa as the IFRPI-based researcher has been very productive and is the first to publish the only two papers stemming from the research. The EET spent some time with this individual and recommend that his work continues to be supported. All of the graduate students at the respective universities are satisfied with their degree programs. It should be noted that at least one of the Ugandan students studying at Tuskegee is conducting thesis research in Alabama rather than in Uganda. In Malawi, a curriculum for a diploma in dietetics has been completed and is moving forward through the approval process at LUANAR/Bunda University, but there have yet to be any trainees under this project. The Malawi project, although small at this time, is critical in a country with no existing dietetics training. 65 The EET examined the strengthening of training of personnel and institutions within the ME as well as their academic partners. This award has significantly strengthened Tufts University’s ability to train and build research expertise based on the EET’s interviews and information received by those collaborating with Tufts. With this award, Tufts University is further seen as a strong nutrition academic institute by many stakeholders interviewed and globally in the nutrition community, with more interesting field research and training experiences for students and faculty. This award is seen as a “draw” for incoming students. From interviews, Harvard, Tuskegee and Purdue Universities have also benefitted from being partners in providing interesting research training experiences for students and meaningful relationships for faculty engagement in new areas of work that cross disciplines (agriculture, nutrition, health and environment). QUESTION 2: PUTTING SKILLS AND KNOWLEDGE INTO PRACTICE Two 10-question surveys were developed by the EET to assess former trainee and student satisfaction and opinions of the courses or training sessions they participated in during the Nutrition Innovation Labs operation. The survey tools were developed using the web-based SurveyMonkey program. One survey targeted short-term training that the Nutrition Innovation Labs have supported. The other survey targeted longer-term formal training at universities with degree granting programs. A range of questions were asked to assess the satisfaction with training, translation of training into professional use, and areas where improvements can be made. For the short-term training survey, there were 24 responses from the BBNC training, the Summer Institute of Biostastics and Epidemiology at Johns Hopkins University training, and the non-degree Summer Session Training at Harvard University. For the formal degree training survey, there were 17 responses from Tufts University, Purdue University, Makerere University, Uganda Christian University, Johns Hopkins School of Public Health, Tuskegee University and Harvard University. The EET asked trainees if the training they received or are receiving is appropriate to their country needs and the majority answered very appropriate or appropriate (Figure 2). From the training curriculums that the EET reviewed, the information and materials provided are relevant to the nutrition challenges for Nepal and Uganda. 66 Figure 2: Appropriateness of Training Many of the trainees feel that the quality of training received, both in the short-term and degree granting trainings were of very high quality (Figure 3). Figure 3: Quality of Training 67 Training contributions to strengthening capacity in countries Of those who participated in the short-term programs, many of the trainees agreed that the training was very useful for their current position (Figure 4). During EET interviews, trainees had indicated that they are incorporating the nutrition theory into their current work portfolios and some are actively engaged in nutrition research, which was perceived as being strengthened with trainings. The trainings also allowed for exposures to new “hot topics” in nutrition and dialogue with other researchers and development practitioners from other countries. The impact of the training on practice and knowledge in the formal programs is too early to evaluate. To better understand the longer-term impact of overall training in the countries themselves is also too early to determine. In the immediate assessment phase, the trainings have injected nutrition knowledge into the current workforce, particularly with the short-term trainings that have taken place. It will be important to sustain these trainings through refresher courses, and building onto the core curriculum with specialized subjects matters and practical experiences where training skills can be applied in the field. Figure 4: Training Relevance For both the Nutrition Innovation Labs in Asia and Africa, there was an overall request to support more workshops in data management and training in research methods (which include 68 epidemiology and biostatistics) (Figure 5). In the online survey, the EET asked students if they were interested in receiving more training and if so, in what topics. Epidemiology, biostatistics and research methods had high level of responses, as did food security, agriculture and food￾based approaches. Less has probably been achieved in the training area of agriculture, most likely due to the expertise of the ME. There are no “card carrying agricultural scientists” who sit at the leadership level of the ME as well as the sub-awardee structures with the exception of one scientist at Purdue University. While there have been attempts to ensure that agriculture remains a central theme in both the Nutrition Innovation Labs, the focus is more centered on nutrition and public health training. Thus, agriculture is less emphasized as compared to nutrition and public health capacity and training (as well as research). Interestingly, for short￾term trainees, many selected public health nutrition as an area of further training interest, although the previous trainings emphasized this topic area. Figure 5: Subject Areas of Future Workshops Lastly, there are some issues, from the trainees’ perspectives with funding and mentoring, as those who responded to the survey did not give as many high-ranking responses (Figure 6; 69 Figure 7). In Uganda, this was noted with the eight students at Makerere University. Although responses to date comprise a small sample size, this survey captured responses from six universities both in the U.S. and in the host countries. Figure 6: Quality of Mentoring Figure 7: Adequate Funding for Training 70 CONCLUSIONS Overall, the Nutrition Innovation Labs are seen as making an important contribution to strengthening the training and capacity building of nutritionists working across sectors, particularly in Nepal, where there is a clear dearth of nutritionists in the country. More can be done to integrate and build capacity for agriculturalists working in public health and nutrition. More training is recommended to continue to build research design and methods capabilities (which includes biostatistics and epidemiology), and following that, training in data analysis and writing publications. As part of training, there also needs to be the ability to harness and translate that training into professional endeavors. The Nutrition Innovation Labs can play an important role, and should do so more than they currently are, to ensure that researchers in the host countries are given the opportunities to be first authors on publications, present findings stemming from the research, and are seen as the “go to” knowledge keepers with those that make a difference in the country: policymakers and programmers. LESSONS LEARNED • Establish early on job descriptions and roles of various institutions for training and institute clear objectives for trainees on research deliverables. • In training capacity for nutrition sensitive approaches, there should be substantive engagement with agricultural scientists and food security specialists to ensure agriculture content is also incorporated into curricula to the same level as public health and nutrition. • Shared high caliber publications authored jointly by north and south collaborators are important but involve extensive and intensive training. Training opportunities on manuscript and grant writing should be established early within research projects. RECOMMENDATIONS • For future trainings, there should be better integration of other sectors including agriculture and food security in trainings for public health nutrition as well as operations research design and methods. • The Nutrition Innovation Labs should ensure that lesser known universities outside the capital cities (Uganda in particular) are engaged in research and should seek to assist with capacity building in areas that are not the “obvious choice.” This could be done through a competitive small grants process. • For the Nutrition Innovation Lab/Africa, thought should be given to creative ways to build on what Nepal has done with the countrywide scientific symposia to garner more attention and understanding of the project in Uganda. 71 • For the Nutrition Innovation Lab/Africa, a new relationship needs to be considered for the Makerere-Tuskegee-Tufts relationship. The current arrangement appears to be ineffective and students currently funded for master’s study at Makerere are very concerned and confused about their ability to complete the degree program on time. • More mentoring of key host-country researchers is needed so they are prepared to analyze the data, publish manuscripts and present results on their own to policy makers. They should not depend on receiving analyses from ME or U.S. partners in the long run. If this capacity is not there, the gap needs to be addressed with long-term training to make “real and sustainable” impact on policy. 72 INSTITUTIONAL COLLABORATION The Scope of Work (Appendix A) for the Nutrition Innovation Labs evaluation asked the EET to address the following questions with respect to collaboration: 1. How have the MEs performed in communicating and establishing productive collaborations with host country governmental and academic institutions? Local NGOs? Other Feed the Future Innovation Labs? Other relevant USAID programs in the target countries? Other relevant international research programs working in the target countries (e.g., Agriculture for Nutrition and Health [A4NH], IFPRI)? How could the MEs improve in building their institutional collaborations? 2. The MEs have received numerous requests for technical assistance from USAID (headquarters and Missions), host country institutions, and other nutrition technical assistance groups (e.g., SPRING). How responsive have the MEs been to these requests for assistance? How useful has their assistance been? What have been the outcomes of their assistance (e.g., new collaborations, new or improved research)? What, if anything, could the MEs do to improve in responding to requests for assistance? FINDINGS QUESTION 1: PRODUCTIVE COLLABORATIONS Overall, the ME has done an effective job in communicating and establishing collaborations with host country governments, academic institutions, international NGOs and local NGOs. Some collaborations have been more successful than others (see Training Section), depending on the Nutrition Innovation Labs, the country, the fora and the time period. When examining the organograms of both the Nepal and Uganda programs (see evaluation plan Figures 1 and Figure 2 in Appendix B), the structures and stakeholders are complex and have evolved. From the organogram structures, there are also many entry points for institutional collaborations – some have been harnessed, but others remain untapped. Global collaboration Collaborations at the “global level” have been very successful in that the two Lab PDs have presented research concepts, strategies and findings in many international fora (Feed the Future 73 meetings and the ICN2 Preparatory Meeting for example) as well as with other Feed the Future Innovation Labs. Through a series of interviews, the EET conclude that these two individuals are well respected in how they have managed the Nutrition Innovation Lab/Asia and the Nutrition Innovation Lab/Africa respectively and their level of engagement in the countries. Many interviewees indicated that the Nutrition Innovation Lab/Asia PD appreciates the links between agriculture, nutrition and health and makes a concerted effort to bridge these disciplines as part of Nutrition Innovation Lab/Asia through their collaborations. Both Principal Investigators are sound choices to continue to lead the project and spearhead new partnerships. They are both by nature, collaborative and willing to share information at international, national and local meetings (see further details in the Management section of this report). Building and expanding junior academic capacity and ensuring they also have the opportunity to communicate and share the Nutrition Innovation Labs information will be very important if/as the project continues. One interviewee felt strongly that it would also be important to develop relationships and collaborations with other institutions that focus on agriculture to develop nutrition-sensitive projects. The EET found that there is less interaction with FANTA and SPRING, two large, global initiatives supported by USAID. Nutrition Innovation Labs/Asia: Government, Academic Institutions and Local Organizations In Nepal, there has been a long and deep history of collaboration with the partner organization, Johns Hopkins University, in the country. This has served as an advantage for the reputation of the project in that the lead JHU Professor has been working in the country over twenty years and has been building capacity in the country for a long while. This made the Nutrition Innovation Lab/Asia project start with a strong foothold in the country as opposed to starting from scratch. Further, the consistent presence and collaborative nature of Lab PD in Nepal, engaging with government and the USAID Nepal Mission has been quite beneficial for the project’s success. He is held in high esteem in the country by all parties that the EET interviewed which is an important factor in the success of the project. In addition, two scientific symposiums, hosted by the Nutrition Innovation Lab/Asia, have been seen as information sharing by both international and local scientists, researchers, government, USAID Nepal Mission, and development practitioners. Interviewees and those that participated in the symposiums were very positive about both symposium sessions. These alone have provided a collaborative environment that engages a variety of stakeholders working on similar issues to learn more about the project and to open new doors of collaboration. 74 Two young professionals oversee the Johns Hopkins University research and the Tufts University work in Nepal as well as the partnerships with local entities in the country. Interviews with other stakeholders demonstrate that both are effective collaborators and engage well with seasoned researchers and policy makers, and especially with a younger generation of Nepalese working in nutrition and food security. It will be important for ME and Johns Hopkins University to continue supporting the careers of these young professionals through scholarly activity, and the networks being created. It was noted, however, that there is some concern by a few individuals interviewed (in government and organizations) that not many Nepalese have taken ownership of data analysis or writing of papers resulting from the research that has taken place thus far by the Nutrition Innovation Lab/Asia. This may be related to the fact that data collection for the PoSHAN community project is carried out by professional data collection companies under the close supervision of John Hopkins and Tufts staff, rather than by the collaborating Nepali universities. In order to enhance high quality and engaged collaboration in the future, there may be a need to ensure more time investment is made to build the research capabilities of local persons in the areas of agriculture-health-nutrition linkages in the country, as, currently, very little exists. The EET recognizes that the Innovation Lab is already working to build better collaborations in the country. The Nutrition Innovation Lab is working with local institutions to develop a curriculum for a master’s degree in public health nutrition. While this is being developed, the EET concludes that more engagement and facilitation of participation in Nutrition Innovation Lab/Asia’s data analysis is needed in the short-term with local partners to raise the level of collaboration. It will also be important that in-country academic institutions take ownership of this curriculum-building process in its initiation and growth as it moves forward. Overall, government sees the Nutrition Innovation Lab/Asia as a strong collaborator and government stakeholder interviewees expressed their keen interest in having the Nutrition Innovation Lab/Asia continue. With the interviews from government, the EET assessed that the government sees the project has having different goals from Nutrition Innovation Lab/Asia and USAID in that policymakers see Nutrition Innovation Lab/Asia as very “academic” and they do not see the practical applications resulting from the research easily. Government interviewees made a strong recommendation to turn around baseline data results (and follow-up studies) rapidly so that project implementers can benefit by incorporating into their core programs. The Nutrition Innovation Lab/Asia has also had strong engagements with NGOs and the relationship, in particular, with Helen Keller International (HKI) is seen as very collaborative. HKI serves as a strong partner in country and data sharing, finances, and cooperation are running smoothly. 75 Nutrition Innovation Lab/Africa: Government, Academic Institutions and Local Organizations In Uganda, there is a strong history of collaboration by many entities with the main university, Makerere University. One USAID Uganda Mission official questioned why there was so little capacity engagement with other relevant universities in the country and suggested that in the future, looking beyond Makerere University may make sense. However, Makerere University is globally known and the strong ties that the Lab PD had with the College of Health Sciences (now working with the agriculture college), made the initial collaboration an easier fit. The Nutrition Innovation Lab/Africa in Uganda had some rough patches in getting started with changes in leadership and capacity at both Tufts and Makerere Universities. The local PI from Makerere changed twice. At Tufts the original PD was replaced with the current PD. It is seen that these shifts in personnel were good decisions by all parties. That being said, changes in management have significant impacts on building and incentivizing capacity. On research collaborations with Nutrition Innovation Lab/Africa, the communication and collaboration between Tufts, Harvard, Tuskegee and Makerere Universities is not optimal. Harvard University has one junior professional, based in Boston, who was spearheading the work but supervised by two more senior scientists. Data collection is carried out directly by Makerere University, and has provided the opportunity for local researchers to increase their experience with large scale data collection. However, not having a local but senior scientist based in Uganda to oversee the data collection, who can then report to the Tufts senior scientists presents capacity and collaboration issues. Recently, Harvard supported a new Ugandan professional to be a point person on ethical approval (IRB) matters and to assist with research implementation. This should alleviate some of the time and communication constraints in carrying out research, led locally. There is also another professional who also serves as a faculty at Makerere University, now leading the day-to-day management of the cohort project and overall administration of the entire portfolio, who is supported by Tufts University. It will be important for Tufts University to support and nourish her capacity as she takes on this enormous role from the previous young professional. There have been fewer opportunities in Uganda to promote collaborative relationships and develop capacity in that there have not been enough significant scientific symposia, such as what was done in Nepal. Perhaps presenting baseline findings and data more frequently and in ways that bring different sectors together, regardless of political difficulties of certain high level issues in the country, would help to build understanding and capacity of the links between agriculture, nutrition, and health and would highlight the work of the Nutrition Innovation Lab/Africa. Some of the interviewees (government and Community Connector as some examples) had concerns that in the past, the research leadership of the Nutrition Innovation Labs in both Asia and Africa have not shared survey tools and instruments with others in the countries. There 76 was concern that by not sharing such information in a timely fashion, local collaboration and capacity for research cannot be strengthened. This lack of sharing was viewed as very “academic” in that information is protected until peer review publications are published. A proprietary policy regarding data forms and access to data by U.S investigators was reported early in the project, however presently all can have access to data and tools, through a password protected database system. The Nutrition Innovation Lab/Africa has a strong collaboration with IFPRI in analyzing data resulting from the panel study by supporting one post￾doctoral young professional at 50% of his time. The EET determined that this IFPRI postdoc has substantive experience in research and could be an important asset. Thus, his capacity should continue to be supported. This partnership was originally established by one of the Tufts University PIs. This collaboration has resulted in the first two published papers to come from the project. The Nutrition Innovation Lab/Africa has had less of an impact on effectively engaging with government in part due to the corruption scandals in the Prime Minister’s Office, which drove the USAID Uganda Mission office to be on “strategic pause” with the government at the moment. This of course has impacted the meaningful engagement that the Nutrition Innovation Lab/Africa has with government and capacity development overall. The Nutrition Innovation Lab/Africa became involved with capacity building in Malawi as a result of a request made by LUANAR/Bunda to the USAID Mission for assistance in developing a diploma in dietetics. Currently there is no dietetics program in the country and no training available for individuals who work in clinical settings, either with the treatment of undernutrition or with patients with non-communicable chronic diseases. In addition, the need for dietetics training was connected with a related felt need for the development of food composition tables specific for Malawi. Upon hearing about the Nutrition Innovation Lab, the USAID Mission staff approached the ME and the USAID AOR with a request to provide assistance in carrying out stakeholder-driven needs assessment and subsequent curriculum development. The Nutrition Innovation Lab/Africa has supported stake-holder meetings and has provided technical assistance in the development of the LUANAR/Bunda University dietetics curriculum. Innovation Lab staff have also participated in collaborations with the Malawian Institute of Medicine, which will act as the initial licensing body for dieticians. The Nutrition Innovation Lab PD, Associate Director, and the Program Manager have visited Malawi at least twice. A primary consultant from Tufts University for curriculum development has also visited Malawi twice to consult on the curriculum. With Nutrition Innovation Lab support, LUANAR/Bunda has held stakeholder meetings and a workshop to inventory existing curricula and identify curricular needs. The curriculum is currently undergoing approval at LUANAR/Bunda, and there are no students enrolled as yet. LUANAR/Bunda anticipates beginning in the in late 2014/early 2015. The Nutrition Innovation Lab activities in Malawi 77 demonstrate a true collaboration and are very appropriate to the capacity building role of the Nutrition Innovation Lab. Through the interviews with the ME and host country partners the EET believes that when implemented, the project will have an impact on capacity within Malawi because of the low availability of existing capacity. U.S.-Based Academic Institutions Beyond the substantive interactions with the sub-awardees, some of which are U.S. based academic institutions, there are fewer collaborations between the Nutrition Innovation Lab and other U.S. Institutions. The model in which the Nutrition Innovation Lab contracts to just a small number of awardees limits the ability for them to foster more collaborations in the U.S. However, it is important for the EET to shed some light on the capacity within the ME itself for their potential future collaborations and expansion. The ME has expressed that there is insufficient capacity within the ME itself for both the Nutrition Innovation Labs as they continue to grow into other countries (Egypt, Timor Leste, Cambodia etc.). The evaluators were surprised with the amount of work and effort the PIs are contributing to the project. Of course, there are benefits to this growth but the ME expressed that the pace and load are not sustainable in the long run with the current staffing. The ME structure for both Labs needs to change by bringing in additional expert staff both at Tufts University as well as in both countries with appropriate, supported formal mentorship for new and young staff. It may be beneficial to bring on more expertise focusing on agricultural sciences and food security. Being intimately involved as a ME does allow for tighter control of the country projects, which is being exercised. It also ensures that the PIs, all academics, are heavily involved in publications derived from the project. While important for academic careers, the perception of authorship and “publication as priority” at the country level is not seen as positive. In countries, interviewed stakeholders are much more interested in how the Nutrition Innovation Labs can contribute to programmatic success that reaches beneficiaries and much less interested in academic publications. The EET heard this message consistently in Nepal and Uganda and it puts the project, which sits at an academic institution, in a conundrum. Feed the Future, Other USAID Programs and International Research Programs The ME has been involved in the CGIAR’s Agriculture for Nutrition and Health (A4NH) (particularly with WorldFish) and IFPRI meetings and the ME continue to collaborate with the CGIAR and UN systems. The two Lab PDs indicated that they have made a concerted effort with other Feed the, Future Innovation Labs including Grain Legumes, Horticulture, Adapting Livestock Systems to Climate Change, and Aquaculture & Fisheries. However one researcher 78 who works on the Grain Legumes Innovation Lab indicated that some other Innovation Labs are unclear on what the Nutrition Innovation Lab does. The Nutrition Innovation Lab/Asia PD did indicate in tangible terms that some projects are now being established across Innovation Labs beyond just idea and knowledge sharing. The PD reports that he is serving on some Innovation Labs as a technical advisor to ensure that nutrition is cross-cutting. As time goes on, it will be important for the global community, USAID and other Feed the Future Innovation Labs to see Nutrition Innovation Lab leaders other than just the two PDs. Scientific Seminars Nutrition Innovation Lab/Asia Symposia In Nepal, the Johns Hopkins University managed sub-award has included two annual scientific research seminars on the topic of Nutrition and Agriculture Linkages. While funding for a third symposium is not available through the Nutrition Innovation Lab, several informants mentioned that the USAID Mission in Nepal has agreed to fund the third annual symposium. The symposia have included keynote presentations from well-regarded experts, as well as presentations by local researchers on their research and programs. In addition, Innovation Lab staff have worked with some presenters to improve their abstracts and presentations. Proceedings are available as Research Briefing Reports (see Appendix J) on the Nutrition Innovation Lab website. These include abstracts for the presentations and include full text and PowerPoint presentations from some of the keynote presentations. From the EET’s interviews with stakeholders, these two large scientific symposia have been viewed as quite successful in improving the quality of future collaborations by contributing the development of capacity in nutrition capacity of the next generation of nutritionists and in building links among agriculture (including markets), nutrition, and health. Another scientific symposium is planned for the coming year. Overall, these symposia are seen by local scientists, development practitioners, and government as important contributions to building collaborations and strengthening capacity in the country. In a series of interviews in the country, these symposia were mentioned by almost each interviewee as important contributions to training of local scientists and development practitioners in the country. The symposia are also seen as important advocacy strategy to elevating and exposing nutrition in the country, bringing together relevant stakeholders and communicating evidence-based work. Lastly, some interviewees mentioned that the symposia brought in stellar scientists who presented new, innovative topics that garnered great interest amongst country stakeholders including environmental enteropathy and aflatoxin contamination of crops in relation to nutrition outcomes. The EET concluded that the symposia agenda and presenters were of 79 excellent caliber. All key informants interviewed who are connected with the Nutrition Innovation Lab/Asia report that the Scientific Seminars organized under the Nutrition Innovation Lab have been exceptionally good and helpful. Several participants in the symposia commented on the importance of incorporating young local researchers into the symposia. A senior Nepali health official commented positively on the way in which the symposia have integrated both seasoned and newer researchers. Although, this person also commented that he would like to see a wider array of stakeholders, including farmers represented. Some specific comments noted by the EET were: • A participant was pleased by the way in which the symposia, particularly the second symposium, focused on policy and the policy implications of research. • A Ministry of Agriculture and Development (MOAD) staff member who attended the symposia found it beneficial to see different views. She did not make a presentation but others from the MOAD did and she served on a panelist. She was impressed that the presentations had to be “evidence based”. She felt a lot of the work by the MOAD was not evidence based. • USAID Mission staff felt that the level of expertise represented in the scientific symposia was “excellent”. They liked that the seminars have included U.S. universities, regional experts, USAID Washington and the USAID Regional Development Mission for Asia. The Mission staff expressed the likelihood that the third symposium would be funded by the mission. • The only comments that could be construed as other than positive appear to come from concerns that the symposia should be expended and incorporate more stakeholders. • Finally, one young professional noted that the symposia were helpful as places to look for a job. Nutrition Innovation Lab/Africa Symposia There have not been similar seminars in Uganda as part of the Nutrition Innovation Lab/Africa. In part, this is because there is much less need for seminars in Uganda. As informants noted, seminars and conferences on development and nutrition are numerous and are often scheduled at the same time. The key issues in which the Nutrition Innovation Lab is involved are routinely incorporated in existing seminars. Project staff and partners have made presentations on the Nutrition Innovation Lab’s design and data in Uganda at several of these events. 80 A high-level meeting to be organized under the Office of the Prime Minister (OPM) has been postponed at least twice due to diplomatic tensions between the U.S. and the Ugandan government. While project staff suggest that they are now looking at strategies that by-pass the OPM, this has not yet taken place. QUESTION 2: TECHNICAL ASSISTANCE In both Nepal and Uganda, there has been a concerted effort to engage with Bureau for Food Security (BFS) and USAID country Missions. There is a balance in how the Nutrition Innovation Labs interacts in the country with established USAID projects. For both countries, the Nutrition Innovation Labs apparently were expected by country Missions to support the evaluations of the large-scale agriculture and nutrition programs however the Nutrition Innovation Labs thought of the project differently in that they were gathering new evidence to support the importance of linkages among agriculture, nutrition and health. This could be seen as a miscommunication with BFS and priorities for the country Mission offices. In Nepal, research is occurring independently from the significant investment USAID is making in the Suahaara program, after some negotiations and slow starts. However the findings from the Nutrition Innovation Lab/Asia will be beneficial for Suahaara as lessons learned and the EET interview with Suahaara indicated that the relationship and sharing of information is improving between the project and the Nutrition Innovation Lab. In Uganda, the Nutrition Innovation Lab/Africa is still linked to the USAID project, Community Connector, but again after some negotiation with the country Mission office, their research objectives are more on evidence gathering and less on evaluating USAID’s program. The Community Connector has shifted their implementation strategy and changes are based on what the needs are. This has made it difficult for the Nutrition Innovation Lab/Africa to get a solid footing with regard to the research, which impacts their ability to develop capacity on the ground with respect to research. The nature of these research shifts impact capacity development differently in who is engaged, how research is done, and what types of collaborations are formed. This has been a source of tension for both projects. However as time has gone on, in-country stakeholders and USAID Mission offices see the value of the research and capacity building that the Nutrition Innovation Lab/Africa has accomplished thus far, and the collaborations are running much more smoothly and efficiently. The Nutrition Innovation Labs have been responsive to technical requests as much as possible however capacity in both ME and in-country is limited. For Nepal, there is one point person to represent the Nutrition Innovation Lab/Asia. Although an additional point person has been hired to oversee the research project led by Johns Hopkins, managing the project and technical demands with one person may not be sufficient. Relying on the Nutrition Innovation Lab/Asia PD and the Associate Director from Boston to address many of the technical needs may not be 81 sustainable and more efforts toward country capacity and autonomy should be established with young leaders currently working on the project. Instead, further building the capacity of in￾country staff is important to effectively provide technical assistance to USAID and partners. Uganda is a similar story. There are two local point persons to manage the entire project from Tufts and Harvard Universities. It will be important to support, incentivize and nurture their capacity. CONCLUSIONS It is too early to say if the current collaborations of the Nutrition Innovation Labs will last beyond the life of the project as the on-going partnerships and research are still young. Partnerships and collaborations are now more solidified after a rougher start. Much of this was due to reorganizing research objectives with USAID country Mission priorities and their respective large-scale project operations research modalities (more in Research section). What is clear is that the Nutrition Innovation Labs are attempting to build evidence through strong research and programmatic collaborations while building capacity through a deep-dive process in select countries. The partners engaged in both countries are extensive and the ME has handled the relationships quite well. LESSONS LEARNED • The other Feed the Future Innovation Labs provide an excellent opportunity to collaborate and build capacity across nutrition sensitive approaches and provide some “proof” on how to work across multiple sectors for nutrition. It is hoped that the Nutrition Innovation Labs will demonstrate that strong partnerships with other Innovation Labs will not only strengthen Nutrition Innovation Labs’ research outputs but other Innovation Lab outputs as well. Cross synergy opportunities should be utilized. There should also be more engagement and collaboration with global USAID projects such as SPRING. • Staffing needs should be assessed early on with a prominent objective of building as much capacity in countries as both Uganda and Nepal focus on scaling up complex nutrition programs. These staffing needs should be built into budgets to ensure that capacity both at ME and in country are matching the needs. • For the long-term development of nutrition sensitive projects, agricultural scientists and front-line extension staff should have a basic understanding of the pathways, and of the major nutritional issues. These projects should be examining how to build better nutrition capacity and collaboration across other sectors. 82 RECOMMENDATIONS • Capacity building is needed in the U.S. as well as in host countries associated with the Nutrition Innovation Labs. If capacity has been built for incorporation of Feed the Future goals into the other Innovation Labs, the Directors of the other Feed the Future Innovation Labs should be able to make a convincing presentation themselves. All presentations should not need to be by the Nutrition Innovation Labs ME. • Symposia in Nepal have been excellent in terms of building awareness of the importance of links between agriculture, nutrition, and health and the importance of multi-sectoral planning. Building on that awareness, there needs to be solid partnerships and collaborations built that can continue to better understand how sectors can effectively work together and engage. • Sets of PowerPoint slides illustrating key points related to Feed the Future, critical messages from the two Lancet series, information derived from host-country analyses, and other important points from presentations currently being made by ME could be made available to others through the website for adaptation to the appropriate context. This would allow for other partners to feel joint ownership of the work being done in the countries. • Presumably host-country officials understand the key objectives of the Nutrition Innovation Labs and of Feed the Future well enough to make strong presentations; in many settings, an outstanding presentation from a host-country official would be more powerful than from a visitor. The Nutrition Innovation Labs could support an enabling environment for such activities by providing assistance for effective communications of project objectives and results. 83 PROGRAM FUTURE As elucidated in the Statement of Work (Appendix A), the EET was asked to evaluate three questions below with regard to the Nutrition Innovation Labs’ future: 1. How well do the two research programs align with the Feed the Future research strategy? What adjustments may be necessary to their research programs to better ensure alignment? 2. Has the way the two research programs have been set up offer strong likelihood of impactful results that justify funding a second phase? 3. If renewed for a second phase, does the evidence suggest that changes are needed to either or both of the Nutrition Innovation Labs’ management, research (i.e., design, implementation, communications, stakeholder involvement) and/or training (i.e., student recruitment and selection, content, location) programs, and/or institutional capacity collaboration? What lessons have been learned that should be taken into consideration if a second phase is funded? FINDINGS QUESTION 1: ALIGNMENT WITH THE FEED THE FUTURE RESEARCH STRATEGY The Feed the Future approach incorporates a focus on 19 target countries chosen on the basis of need, opportunity for partnership, potential for growth, opportunity for regional synergy and resource availability. As a result, the Nutrition Innovation Labs’ research and curriculum development programs currently underway are located in Feed the Future focus countries. The Feed the Future research approach emphasizes research that aims to develop solutions that “enhance agricultural production, with an emphasis on improving nutrition and reducing adverse impacts on natural resources and the environment. The Feed the Future Research Strategy presents a global research portfolio to create more productive crops, sustainably intensify agricultural production systems, ensure food security, and enhance access to nutritionally improved diets.” The initial AOR of the Nutrition Innovation Labs and principal author of the RFA to which Tufts University responded reports that her vision for the Nutrition Innovation Labs was one in which the projects would provide the critical information necessary to respond to the part of the research strategy that corresponds to the generation of data to fill gaps in knowledge related to the pathways through which agricultural research and interventions would have an impact on nutrition and health. In her words, the Nutrition Innovation Labs would “wrap 84 around” the production and value chain oriented Feed the Future projects, which would provide high quality basic data at scale and operations research in Feed the Future countries to identify and assess the impact of both agricultural and health interventions. The research program is seen by the EET to be of good quality and addresses the questions/objectives proposed in the applications, although it is too early to determine how effectively the individual studies will address them. After review of the implementation of the Nutrition Innovation Labs and an assessment of the type and quality of the research carried out to date, the EET concludes that the research program in both Labs is consistent with the original mandate. However, the degree to which the Nutrition Innovation Labs have wrapped around Feed the Future projects is less clear. In both Uganda and Nepal, the Nutrition Innovation Labs have decoupled themselves from the implementation programs with which they initially expected to be paired. In both cases the research program does overlap geographically with some sites of the implementation programs and will provide data that allows the programs to examine the impact of their programs on health and nutrition. However, in neither country do the primary research projects directly assess the impact of programs. Interviews with members of other Feed the Future Innovation Labs, USAID Mission staff involved with the host country coordination of the Innovation Labs and Mission Feed the Future projects provide a mixed story about the degree to which other Feed the Future programs see the Nutrition Innovation Labs as a resource and potential collaborator. The Nutrition Innovation Lab Project Directors have made presentations to other Feed the Future program investigators, but researchers from these other programs express some confusion about what the Nutrition Innovation Labs are doing. A researcher from the Legumes Innovation Lab suggested that the Nutrition Innovation Lab was not very aware of what was already going on in Feed the Future projects with respect to nutrition. The USAID Malawi Feed the Future Coordinator had not been approached by the Nutrition Innovation Lab/Africa and did not know what the Lab did. However, the Feed the Future Coordinator for USAID Uganda was very positive about the Nutrition Innovation Lab and saw the data as important for program planning and evaluation. While the Nutrition Innovation Labs have not carried out much operations research to date the LWA structure of the cooperative agreement provides a strong opportunity to further develop research that more directly addresses the needs of other Feed the Future Innovation Labs through the addition of Associate Awards. Finally, the Nutrition Innovation Labs are uniquely positioned to develop data collection methods and instruments that could be used in the monitoring and evaluation of Feed the 85 Future projects. However the EET concludes that the Nutrition Innovation Labs have not moved enough to develop a core set of methods and instruments. QUESTION 2: JUSTIFICATION FOR A SECOND PHASE The EET’s analysis of the research program is on balance very positive. The EET concludes that both Nutrition Innovation Labs have a strong likelihood of impactful results that justify continued funding. However, both Labs need to consider stronger collaboration with host country agricultural scientists and researchers in other Feed the Future Innovation Labs in order to more closely address the likely impact of innovative interventions and packages of interventions. While both research programs were delayed by a series of understandable circumstances, the Nutrition Innovation Lab/Africa has suffered more from staffing changes and miscommunication among partners. The design of the primary research projects of both Labs have the likelihood of providing key data that addresses gaps in knowledge in the understanding of the pathways between agriculture and nutrition. That the IFPRI led data analysis has yielded useful results is a strong indicator that the data have a high potential for impact in the future. Despite delays, the EET believes that the findings will also be important to inform policy and programs for Nepal and Uganda. However, several of the individuals interviewed suggested that the research agenda would be strengthened by more attention to the nuances of household food systems and food use. One Feed the Future researcher who had attended several presentations of the Nutrition Innovation Lab projects noted that he was not sure the research took into account critical household food processing technologies. Another suggested that more nuanced inclusions of value chain data would enhance the potential for the research to directly address the pathways. Neither Lab is set up as a randomized control trial, nor has a strong set of a priori hypotheses. Thus, the overall impact of the projects are dependent on the questions asked of the data and interpretation of the analyses. Neither Lab is far enough along in data collection to be able to provide sufficient data for the testing of an array of critical hypotheses. CONCLUSIONS Question 1 • The data being generated by the Nutrition Innovation Labs fits directly into the Feed the Future Global Food Security Research Strategy. • The wrap around strategy envisioned when the Labs were developed has not yet gained traction with other Feed the Future programs, but is becoming more recognized. 86 • Associate Awards, including several currently under development have the potential to further enhance the degree to which the Nutrition Innovation Labs directly address operational concerns in the broader Feed the Future Research strategy. • Work on a set of metrics that could be adopted by other Feed the Future projects has not yet been addressed, but could be in future. Question 2 • The current research programs are generating data that has a strong likelihood of impact. • Because of understandable delays in the initiation of data collection, neither Nutrition Innovation Lab has yet completed collection of all the data proposed. The panel survey research in Uganda has experienced further delays in the collection of the second round of data. • As a result of research design, the impact of the research programs will be partially dependent on hypothesis driven data analyses that address the top-level questions posed in the proposals. • A stronger incorporation of information from agricultural research and intervention would strengthen the likelihood that the research programs will provide the information necessary to fill critical gaps in knowledge. RECOMMENDATIONS • The two Nutrition Innovation Labs should be continued for a second five year phase. • The Nutrition Innovation Labs should continue to make a strong effort to engage with other Feed the Future Innovation Labs, and the Feed the Future programs in target countries. • The Labs should continue to seek opportunities to engage in more operations oriented research and collaborations with other Innovation Labs, especially through the judicious and strategic pursuit of appropriate Associate Awards. • Attention to the development of an adaptable methodological tool kit for use by other programs would be a very desirable activity. • In a second phase the two Nutrition Innovation Labs should incorporate a stronger emphasis on analysis and dissemination of data, especially data with policy implications. • In a second phase the projects should more explicitly incorporate collaborators from the agricultural sciences. 87 RECOMMENDATIONS FOR SECOND PHASE QUESTION 3: LESSONS LEARNED AND SUGGESTED CHANGES IF RENEWED FOR A SECOND PHASE The previous sections of this report suggest a series of specific recommendations for the Nutrition Innovation Labs. Drawing on the overall evaluation, the EET would like to layout several broad areas that the EET believes would strengthen the Nutrition Innovation Labs in a second phase of funding. Re-assess Collaborators and Partnerships The ME should take the opportunity to reassess collaborations that are not working well in a second phase of funding. The EET has specifically identified relationships with Tuskegee University for both the Innovation Labs, and Harvard University in the Nutrition Innovation Lab/Africa as problematic. In each case it is unclear if the issues are with the competence of specific individuals, e.g., the project coordinator at Harvard and the PI at Tuskegee, or if the issues are more systematic in the institutions. The ME should either work to fix the relationships by providing technical assistance to Tuskegee and working to develop a stronger supervisory relationship with Harvard researchers, or seek new partners. The individual at Makerere University who is supervising the masters training at Makerere is also in need of training in budgeting and invoicing, as well as communication. Re-assess ME Staffing Current ME staff are over-committed. The addition of Associate Awards should be accompanied by an increase in staffing. Currently the weakest areas are in the management of communication and the maintenance of the website. The website is not an effective means of communication at this time. Broaden the Multi-disciplinary Base of the Research The EET was struck by the apparent lack of input by agricultural researchers in research design and data analysis; and also by the low number of agricultural staff and students included in long￾d short-term training and capacity building. Some of the expertise needed is already included in the Nutrition Innovation Labs research team, but is not being incorporated across research projects. 88 Develop an Integrated Set of Metrics The Nutrition Innovation Labs in collaboration with members of the TAC have conducted a workshop on metrics. However, the EET did not see evidence that a coordinated set of metrics was being used across the research programs. To be clear, the EET is not suggesting that the research programs use identical methods, but that they agree on drawing from a set of appropriate instruments and designs to enhance the degree to which research findings are comparable across research projects and regions. This seems to be a lost opportunity for the Nutrition Innovation Labs to influence a broader research agenda with state of the art methods. Broaden the Scope of the Research Agenda The current approach to the research agenda invests in a small number of large-scale studies designed to generate critical data sets. The EET also sees an opportunity in a second phase, and through the mechanism of Associate Awards, to design and conduct research studies that take a closer look at context, and the ways in which specific pathways operate in specific regions. The EET sees the Nutrition Innovation Labs moving in this direction at this time, but also see the increased opportunity to do so in a second phase of research. Broaden Capacity Building There is an opportunity to broaden the base of capacity within the U.S. as well as in the regions in which the Nutrition Innovation Labs operate through increased inclusion of students, host country researchers and policy makers in data analysis, data presentation, and publication. The Labs also could increase the inclusion of other disciplines in training in nutrition and health, and develop a strategy for cross-training. Move More Quickly to Provide Policy Relevant Analysis The most consistent concern expressed to the EET by policy makers was their perception that policy relevant data were being made available too slowly. This was phrased by several of interviewed stakeholders as a concern that the researchers were more concerned with publishing in peer reviewed journals, than intersecting with policy makers. The EET certainly understands that data collection and analysis take time, and that longitudinal or panel studies may need several rounds of data collection to have meaningful results. However, these data should be carefully presented to policy makers, and a second phase of funding should include explicit strategies to present and interpret project data for local policy makers. 89 APPENDIX A: SCOPE OF WORK Scope of Work: External Performance Evaluation of the Feed the Future Innovation Labs for Collaborative Research on Nutrition4 Awards Numbers: OAA-L-10-00005 (Asia) and OAA-L-10-00006 (Africa) Scope of Work This performance evaluation will provide USAID and the Management Entities (MEs) with constructive feedback on the program management, research program, training program and institutional capacity collaboration of the Nutrition Innovation Labs. Furthermore, since the Nutrition Innovation Labs will be completing their first five year phase in October 2015, the External Evaluation Team (EET) should consider whether a program extension for a second phase is warranted, and if so, make recommendations to USAID on any necessary management adjustments and potential research focus changes during a second phase. Specifically, the EET will objectively evaluate the following questions using an evidence-based and data-driven approach. NOTE: USAID does not expect the EET to answer all the sub-questions listed under each question. These sub-questions are provided to help further define the question’s topic. Program Management 1. Have the Management Entities for the two Nutrition Innovation Labs effectively managed their respective research and training activities in Africa and Asia? How effectively have the MEs and their partners communicated, coordinated and engaged with the Missions? What have been specific challenges faced in terms of management, and how has each ME addressed them? 2. In the past, the CRSP model has been based on one lead university serving as the ME and managing a global program of multiple projects. Have the two Nutrition Innovation Lab awards created value added benefits? Have the two MEs built synergies between their regional programs to ensure comparability among findings? How have these synergies contributed to the Nutrition Innovation Labs’ objectives? How could the synergies between the two regional programs be strengthened? Research Program 1. Does the body of research being funded by Feed the Future make strategic contributions to the following high-level research questions: (a) what are the 4 Formerly called the Nutrition Collaborative Research Support Program (CRSP) 90 agriculture-to-nutrition pathways; (b) what are the program impact pathways; and (c) what is the value of integrated programming pathways? How might the research design for the two programs be adjusted, if necessary, to better answer the research questions and fill the evidence gaps? 2. What challenges have the two Nutrition Innovation Labs faced during research design and implementation? What impact, if any, have these challenges had on implementation of research activities? How effective have the two research programs been in addressing the challenges? What could they do differently to better address the challenges? Training Program 3. Have the Nutrition Innovation Labs met academic and technical capacity strengthening targets? Are the appropriate type and number of people being targeted for the right kind of training? What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? 4. How have trainees put into practice the knowledge and skills acquired? How have the training programs contributed to strengthening institutional capacity in the target countries? Institutional Capacity Collaboration 3. How have the MEs performed in communicating and establishing productive collaborations with host country governmental and academic institutions? Local NGOs? Other Feed the Future Innovation Labs? Other relevant USAID programs in the target countries? Other relevant international research programs working in the target countries (e.g., Agriculture for Nutrition and Health [A4NH], IFPRI)? How could the MEs improve in building their institutional collaborations? 4. The MEs have received numerous requests for technical assistance from USAID (headquarters and Missions), host country institutions, and other nutrition technical assistance groups (e.g., SPRING). How responsive have the MEs been to these requests for assistance? How useful has their assistance been? What have been the outcomes of their assistance (e.g., new collaborations, new or improved research)? What, if anything, could the MEs do to improve in responding to requests for assistance? 91 Program Future 1. How well do the two research programs align with the Feed the Future research strategy? What adjustments may be necessary to their research programs to better ensure alignment? 2. Has the way the two research programs have been set up offer strong likelihood of impactful results that justify funding a second phase? 3. If renewed for a second phase, does the evidence suggest that changes are needed to either or both of the Nutrition Innovation Labs’ management, research (i.e., design, implementation, communications, stakeholder involvement) and/or training (i.e., student recruitment and selection, content, location) programs, and/or institutional capacity collaboration? What lessons have been learned that should be taken into consideration if a second phase is funded? Evaluation Methodology The evaluation will be based on the following tasks, conducted in the following order, and completed by the dates given. USAID anticipates that the EET will treat the two Nutrition Innovation Lab awards as one when it comes to producing the deliverables outlined below and writing the final report with separate sections to distinguish the two awards as needed. 1) Conference call with USAID - between February 17-March 7, 2014 A conference call will be scheduled between the EET and the USAID Evaluation Manager, the Nutrition Innovation Labs’ Agreement Officer’s Representative (AOR), and other officials in the Research and Monitoring & Evaluation Divisions of the Bureau for Food Security to review the scope of work and answer questions concerning the implementation and delivery of the evaluation. 2) Desk review - between February 15-March 12, 2014 The EET will conduct a desk review of Nutrition Innovation Labs’ publications and materials. The purpose of the desk review is to obtain needed background and context about the Nutrition Innovation Labs and USAID in order to complete the Knowledge Gap Table and the Evaluation Plan (see below). Documents to be reviewed will include, but are not limited to, the RFPs (request for proposals), approved program proposals, the Leader Cooperative Agreements, annual reports, work plans, program operation documentation, and funded research proposals. Team members will also familiarize themselves with the Feed the Future Global Food Security Research Strategy5 and the USAID Evaluation Policy6. 5 http://pdf.usaid.gov/pdf_docs/PDACR702.pdf 92 3) Knowledge Gap Table – due April 20, 2014 Based on the desk review, the EET will provide the USAID Evaluation Management the completed Knowledge Gap Table (see Appendix A). 4) Evaluation Plan - due April 30, 2014 Using the Knowledge Gap Table as a guide, the EET will submit to the USAID Evaluation Manager the Evaluation Plan (see Appendix B). The purpose of the Evaluation Plan is, in part, for the EET to present their evaluation design which includes, in part, research questions, methodology for quantitative and qualitative data collection and data analysis, work plan, timeline and proposed domestic and international travel. The Evaluation Plan must be approved by the USAID before the EET can travel and begin their field work. USAID will provide approval or request changes by March 24, 2014. If required, the EET will submit a revised Evaluation Plan by March 28, 2014. 5) Domestic and international travel – to be completed by May 31, 2014 The EET will need to travel domestically and internationally to gather the needed information to implement the evaluation plan and complete this scope of work. Domestic travel is limited to one trip, up to two days excluding transit, to visit the Nutrition Innovation Labs’ ME at Tufts University’s Friedman School of Nutrition, Science and Policy. This visit should precede international travel. International travel is limited to two separate trips to visit international collaborators and stakeholders with the Nutrition Innovation Labs. The USAID Evaluation Manager must pre-approve all travel. All travel will be arranged for the EET by the USDA/Foreign Agriculture Service and must be in accordance with U.S. Government travel regulations. The USAID Evaluation Manager will provide the EET with a travel protocol that outlines the procedures to be followed for all travel. 6) International travel debriefs – prior to country departure A short summary of data collected and preliminary findings will be sent to the USAID Evaluation Manager for each country visited before departure from that country. This is not to be a trip report, nor should time be billed to write a trip report. Instead, it is meant to provide the USAID Evaluation Manager with progress made against the Evaluation Plan. 7) Preliminary findings – due June 9, 2014 The EET will provide in writing to the USAID Evaluation Manager the preliminary findings that will be used to develop the draft evaluation report. 8) Draft evaluation report – due June 23, 2014 A draft of the evaluation report will be submitted electronically in MS Word format to the USAID Evaluation Manager. USAID will review the draft for content. The ME will review the 6 http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf 93 draft for accuracy. All comments, corrections and suggestions for consideration will be sent to the EET by June 23, 2014. 9) Final evaluation report – due July10, 2014 The final evaluation report should sufficiently address all comments and corrections provided to the draft report. Evaluation Report Format The evaluation report will present findings, evidence-based recommendations and conclusions of the topics outlined in this Scope of Work. The EET may include other topics that are deemed relevant and are evidence-based. The report should follow the format and page limits as outlined in Appendix C. The USAID Evaluation Manager will be made available to the EET as a resource person but will not contribute directly to the preparation of the report. Level of Effort The level of effort for the entirety of this Scope of Work will consist of no more than 45 billable days for the Team Leader and 40 billable days for each of the two team members. All billable work is to be performed between March 1 and June 30, 2014. The following is the authorized number of billable days for each team member and leader for each task/ deliverable of this scope of work. Changes of more than two days for a task/deliverable must be authorized by the USAID Evaluation Manger in advance, before the days are worked. Significant changes will require the submission and approval of a new Evaluation Plan work plan (see Appendix B) before additional days are approved. LEVEL OF EFFORT (by billable days) Task/Deliverable Each Team Member Team Leader Conference Call/ Desk Review 4 4 Knowledge Gap Table 1 1 Evaluation Plan 3 3 Travel7 & Travel Debriefs 21 21 Preliminary Findings 4 4 Draft Report 5 8 Final Report 2 4 Total 40 45 7 The EET is expected to work a six day work week while traveling. 94 Payment of Services The University of Missouri will pay the EET for their services. Daily rate of compensation will be in accordance with U.S. Government regulations and based on verifiable past work experience. Payment will be made on a monthly basis in accordance with the billable day limits per task/deliverable outlined in the Level of Effort table above. Team Composition and Qualifications The technical qualifications of EET members must be matched with the technical areas of focus of the Nutrition Innovation Labs. Team members must have the expertise necessary to evaluate the Nutrition Innovation Lab and to address the Scope of Work topics. USAID will designate one team member as the Team Leader. Administrative/management member (1): A senior administrator with a minimum of ten years of experience managing and/or evaluating multifaceted international development research and/or university-based programs. The preferred candidate will be familiar with both university-based programs and USAID (or other donor) funded programs. A background in agricultural development, with technical expertise in a field relevant to agricultural research and nutrition is preferred. The candidate will also have: a) a demonstrated capacity to conduct independent program evaluation; b) an understanding of USAID’s foreign assistance goals, and its particular objectives related to collaborative research, agricultural development and food security; and c) the ability to analyze issues and formulate concrete recommendations orally and in writing. Technical team members (2): Must be recognized experts in international development related to nutrition and agriculture with specific expertise in the use of food-based approaches to improve maternal and child nutrition. Technical team members will also have demonstrated the following: a) the capacity to conduct independent program evaluation; b) a thorough understanding of research methodology; c) experience in effectively conducting outreach and dissemination to policymakers, development practitioners and/or the private sector; and d) the ability to analyze issues and formulate concrete recommendations orally and in writing. 95 APPENDIX B: EVALUATION PLAN Evaluation Plan Feed the Future Innovation Labs for Nutrition FTF Activity/Mechanism Name: Nutrition Innovation Lab: Asia Nutrition Innovation Lab: Africa FTF Activity Country/Countries: Nepal, Uganda, Malawi Evaluation Lead Institution: N/A Evaluation Team: Barbara Stoecker, Kathleen Musante, Jessica Fanzo USAID Counterparts: Carole Levin; Maura Mack Approximate start date: Feb. 16, 2014 Date originally submitted: April 25, 2014 Date of revision (version): May 15, 2014 A. FTF Project Evaluation Design 1. FTF Activity/Mechanism Description a. Introduction In compliance with the Scope of Work for the Evaluation of the Feed the Future Innovation Labs for Nutrition in Asia and Africa (Nutrition Innovation Labs) this document: • presents an overview of the Nutrition Innovation Labs including: current organization of the projects and their leadership, project goals and objectives, and activities to date; • proposes a set of key indicators and data sources necessary to complete the evaluation; • identifies information already available through project documents and interviews with project leadership; • identifies gaps in information necessary to evaluate the programs including questions to be used in evaluation and additional sources of data necessary to address the indicators. • outlines a design for the collection of additional data and a plan for analysis. The Scope of Work for this evaluation instructed the review team to evaluate: (a) the program management by the two Management Entities at Tufts University, (b) the research program, (c) the training program, and (d) the institutional capacity collaboration. In 96 addition, the External Evaluation Team will be asked to provide recommendations to inform the decision on program extension and, if appropriate, provide recommendations as to any suggested program changes or improvements (SOW 2013). These questions have been used to organize key indicators, identify available information and design a plan for collecting additional data. This evaluation scope of work will focus on the Feed the Future Nutrition Innovation Labs for Asia and Africa, which are Leader with Associate (LWA) awards in Nepal and Uganda, and on ongoing activities in Malawi that began in February 2012. The Malawi activities are funded under a USAID/Malawi buy-in to the Africa LWA. The context of this evaluation includes the potential for an extension of funding for an additional 5 year grant period for one or both programs. b. The projects The Nutrition Innovation Lab for Asia (Nutrition Innovation Lab/Asia) and the Nutrition Innovation Lab for Africa (Nutrition Innovation Lab/Africa) are USAID funded Cooperative Agreement, Leader with Associates (LWA) Awards. Originally funded as Collaborative Research Support Programs (CRSPs) both projects were awarded to Tufts University’s Friedman School of Nutrition Science and Policy as the Management Entity (ME) with a start date of October 2010. Both projects are in their fourth year of their initial five-year awards, which end on October 3, 2015. Total funding received, as of September 2013, is $4,800,000 for Africa and $4,500,000 for Asia. The ceilings for the five-year Leader with Associates awards are $7,361,494.89 for Africa and $7,3321,861.42 for Asia. The USAID/Malawi level of funding to date is $450,000. Several other associate awards have been made or in the negotiation phases. Originally funded under the Collaborative Research Support Program format, the Nutrition Innovation Labs were incorporated into the Feed the Future Initiative under the Program for Research on Nutritious and Safe Foods in the Food Security Innovation Center created in early 2013. The goals and objectives of the projects did not change at the time of the reorganization; they remain as they were in the original N/CRSP technical proposals and the work plans for project years 1 and 2. Year 3 Annual Reports and Year 4 Work Plans reflect the reorganization of the projects as Feed the Future Innovation Labs. The Management Entity for both Nutrition Innovation Labs is Tufts University. While each Innovation Lab has their own PIs, the two Innovation Labs share management and logistical staff, a single Co-Project Director, a single associate director, a number of faculty researchers and a combined website. Both Nutrition Innovation Labs share goals and objectives. However, the specific research designs, training and capacity building activities for each are somewhat different. Both Nutrition 97 Innovation Labs also share a Board of Directors, a Technical Advisory Committee and a component that focuses on global policy issues. On the ground, several of the collaborating institutions are distinct for Asia and Africa. The overall aims of both Nutrition Innovation Labs are to (1) discover how policy and program interventions can most effectively achieve large-scale improvements in maternal and child nutrition, particularly by leveraging agriculture; and (2) build human and institutional capacity for applied policy analysis, research and program implementation. In their annual reports, both Innovation Labs state their overarching objective as seeking “… to discover how policy and program interventions can cost-effectively achieve large-scale improvements in maternal and child nutrition. Bringing together resources from host country and US institutions, the research and capacity building activities focus on operationally relevant work that supports national government priorities.” The Nutrition Innovation Labs’ human and institutional capacity building at local and national levels aims to identify problems, apply appropriate research tools, assess intervention options, implement best practices, and document impact. The two Nutrition Innovation Labs have very similar sets of specific objectives. For Nepal (from the Nutrition Innovation Lab/Asia technical application 2010) the objectives are presented as: • Understanding ‘the how’, not just ‘the what’, of programming to achieve successes at scale; • Filling defined knowledge gaps in nutrition, derived from country-defined priorities and global assessments of the nutrition landscape; and • Identifying ‘essential packages’ of actions that link nutrition, health and agriculture around key problems in food, water and disease. For Uganda (from the Nutrition Innovation Lab /Africa technical application 2010) the objectives are presented as: • Determine how, what and with whom changes are needed for simultaneous successes at the national scale in nutrition, health and agriculture; • Identify and fill knowledge gaps regarding linkages and potential synergies between nutrition, health and agriculture; and • Identify 'essential packages' of actions to address the interlinked problems of food, water, and disease needed to achieve measurable successes on a large scale. 98 Nutrition Innovation Lab/Africa project activities in Malawi are funded under an Associate Award from the USAID mission in Malawi. Malawi activities fall solely under the second theme guiding the Nutrition Innovation Labs: capacity building, and focus on curriculum development and long and short-term training. Research, training and capacity building activities are carried through a series of US, regional and host country partners. Figures 1 & 2 below present the organizational charts for the Nutrition Innovation Lab Asia and Africa, and highlight the complex nature of these projects. Figure 1: Organogram of Nutrition Innovation Lab/Asia 99 Figure 2: Organogram for Nutrition Innovation Lab/Africa 2. Program Logic a. Understand nutrition-agriculture linkages Enhancing agriculture in ways that support improved nutrition is not simply about increasing yields. It is also about reducing costs, enhancing stability in output, strengthening resistance to weeds, pests and diseases, promoting more variety or new crops, stimulation of market and value chain activity, targeting efforts to at-risk regions and populations, securing greater female empowerment (in the agricultural realm and beyond), and promoting demand for a high quality, diverse diet. As a result, agriculture as a broad sector of activities must be unpackaged so that relevant components can be assessed for their relative contributions to nutritional enhancement. This means that pathways linking agriculture and nutrition need to be refined through a focus on plausible biological and other mechanisms over different timeframes, and potential gains need to be understood in net terms through a focus on factors that affect nutrition at the interface among soil, plant and human systems. Figure 3 lays out a chain of connections from agricultural programming through direct and indirect impacts on food and income to nutrition. Many of the links across most of the major conceptual pathways remain poorly understood. The Nutrition Innovation Lab seeks to delve deeper into these presumed linkages through carefully designed primary data collection activities. 100 Figure 3: Causal Pathway of Agriculture and Nutrition Linkages (Gillespie et al 2012) b. Understanding How Complex Multi-sector Programs Work: Significant synergy could also be expected from programs that intersect. This represents ‘implementation research’ in that its intent is to investigate and address major bottlenecks (e.g. social, behavioral, economic, management) that impede effective implementation, test new approaches to improve health programming, as well as determine a plausible relationship between the intervention and its impact (See Figure 4). The Nutrition Innovation Lab’s research in this domain will focus on seeking to identify which “active ingredients” of multi-sectoral programming in various study sites have greater or lesser impact on health and nutrition outcomes, either singly or in combination with others. 101 Figure 4: Intervention to Impact Pathway c. Understanding Policy and Programming Processes Implementation science has evolved as a direct response to policymaker demands for evidence on, a) the cost-effectiveness of complex national-scale initiatives, and b) how to best implement large multicomponent programs. In both cases, greater attention is required to how initiatives are managed and implemented, not just to what is done. Interventions do not operate in a vacuum. They are implemented in a context that has spatial, temporal, political, economic and other characteristics. That said, there is little clarity on the most appropriate methodologies for process tracing or implementation research. The Nutrition Innovation Lab’s approach to process research will pursue concurrent or prospective analyses of policy processes by capturing “the views of multiple stakeholders and tracing the influence of each organization’s practices and culture upon the policy process”. 3. Focus of the Evaluation Five areas of interest have been identified for this evaluation (from the Scope of Work): a. Program management Have the Management Entities for the two Nutrition Innovation Labs effectively managed their respective research and training activities in Africa and Asia? How effectively have the MEs and their partners communicated, coordinated and engaged with the Missions? What have been specific challenges faced in terms of management, and how has each ME addressed them? 102 In the past, the CRSP model has been based on one lead university serving as the ME and managing a global program of multiple projects. Have the two Nutrition Innovation Lab awards created value added benefits? Have the two MEs built synergies between their regional programs to ensure comparability among findings? How have these synergies contributed to the Nutrition Innovation Labs’ objectives? How could the synergies between the two regional programs be strengthened? b. Research program Does the body of research being funded by Feed the Future make strategic and meaningful contributions to the following high-level research questions: (a) what are the agriculture-to￾nutrition pathways; (b) what are the program impact pathways; and (c) what is the value of integrated programming pathways? How might the research design for the two programs be adjusted, if necessary, to better answer the research questions and fill the evidence gaps? What challenges have the two Nutrition Innovation Labs faced during research design and implementation? What impact, if any, have these challenges had on implementation of research activities? How effective have the two research programs been in addressing the challenges? What could they do differently to better address the challenges? What resources/conditions would be needed to better address challenges? c. Training program Have the Nutrition Innovation Labs met academic and technical capacity strengthening targets? Are the appropriate type and number of people being targeted for the right kind of training? What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? How have trainees put into practice the knowledge and skills acquired? How have the training programs contributed to strengthening institutional capacity in the target countries? d. Institutional capacity collaboration How have the MEs performed in communicating and establishing productive collaborations with host country governmental and academic institutions? Local NGOs? Other Feed the Future Innovation Labs? Other relevant USAID programs in the target countries? Other relevant international research programs working in the target countries (e.g., Agriculture for Nutrition and Health [A4NH], IFPRI)? How could the MEs improve in building their institutional collaborations? 103 The MEs have received numerous requests for technical assistance from USAID (headquarters and Missions), host country institutions, and other nutrition technical assistance groups (e.g., SPRING). How responsive have the MEs been to these requests for assistance? How useful has their assistance been? What have been the outcomes of their assistance (e.g., new collaborations, new or improved research)? How much additional burden could these types of technical requests from USAID implementing partners (e.g., SPRING) add to Nutrition Innovation Lab MEs’ burden especially if there is no formal collaboration agreement? What percentage of the MEs’ time does it take to respond to these technical requests? What, if anything, could the MEs do to improve in responding to requests for assistance? e. Program future How well do the two research programs align with the Feed the Future research strategy8 ? What adjustments may be necessary to their research programs to better ensure alignment? Has the way the two research programs have been set up offer strong likelihood of impactful results that justify funding a second phase? If renewed for a second phase, does the evidence suggest that changes are needed to either or both of the Nutrition Innovation Labs’ management, research (i.e., design, implementation, communications, stakeholder involvement) and/or training (i.e., student recruitment and selection, content, location) programs, and/or institutional capacity collaboration? What lessons have been learned that should be taken into consideration if a second phase is funded? 4. Methodology for Qualitative and Quantitative Data Collection a. Details of the data collection plans The scope of this review covers five key areas outlined in the Scope of Work (SOW) for the two Nutrition Innovation Labs: Asia and Africa. Each Nutrition Innovation Lab review will be conducted using similar methodologies, guided by the same set of questions. However, the partners and the precise nature of the programs in Nepal and Uganda are different, and specific individuals included as key informants will differ in each location. The proposed associate award was expected to be finalized in June 2014 by USAID/Malawi would be distinct. While it fits clearly within the overall training mission of the Africa Nutrition Innovation Lab it is limited to training. Only the sections dealing with training with the addition of basic administration will be applied to the interviews conducted in Malawi. 8 http://www.feedthefuture.gov/sites/default/files/resource/files/FTF_research_strategy.pdf 104 At least two members of the evaluation team will travel to each of the key countries included in this review: Nepal, Uganda, and Malawi. Team members will spend 5 days each in Nepal and Uganda, and two days in Malawi. The evaluation plan operationalizes the questions outlined in the Scope of Work by identifying key indicators and sources of data. Table 1 presents the overall evaluation questions linked to indicators and proposed sources of data. Appendix Table 1, the Knowledge Gap Table, summarizes the information already gathered and identifies current gaps in knowledge to be addressed through interviews and site visits over the two months. The evaluation will draw from three sets of data. These include program documents, interviews with key informants and responses to a brief questionnaire by a sample of respondents who have participated in short term training programs. 105 Table 1: NUTRITION INNOVATION LABS: KEY KNOWLEDGE INDICATORS AND DATA SOURCES KEY KNOWLEDGE INDICATOR DATA SOURCE Program Management Have the MEs for the two Nutrition ILs effectively managed their respective research and training activities in Africa & Asia? How effectively have the MEs and their partners communicated, coordinated & engaged with: --Missions --Key partners Project management structure at Tufts Degree to which Tufts is committed to the project and its continuation Description of organization Assessment of the work of the PI and staff Assessment of management of financial and logistical support from point of view of sub-awardees Assessment of Tufts commitment to the project and its goals Organogram Annuals reports Interviews with project staff (PI, staff) Interviews with PIs and staff of major Sub-Awards (Harvard, Johns Hopkins, Tuskegee, Purdue, etc.) Interviews with PIs and staff of host country sub awardees and collaborators Interviews with Tufts administration (e.g. Vice Provost for Research, Dean of the School) Integrity and efficiency of sub￾award process Clarity of RFPs with respect to proposed objectives and technical requirements Timing of awards Assessment of the awards process by successful awardees Review of RFPs and both successful Efficiency, transparency, and appropriateness of interaction with key collaborators and partners in host counties (NGOs, universities, ministries,) Annual reports, minutes of meetings Interviews with PIs and staff of major Sub-Awards (Harvard, Johns Hopkins, Tuskegee, Purdue, etc.) Interviews with PIs and staff of host country sub awardees and collaborators Partner assessment of the nature and quality of interactions with the ME, host country program staff. Interviews with Harvard, Johns Hopkins PIs and staff Interviews with host country project 106 What have been specific challenges faced in terms of management, and how has each ME addressed them? PIs and staff Alignment of project with host country national plans Frequency and quality of the interaction with host country government entities Host country officials’ assessment of the alignment and interactions. Awareness of host country officials of FtF goals Interviews with MoHP, MoAD staff, others (who?) Frequency and quality of the interaction with USAID country staff Visits to host country USAID missions Trip reports Interviews with Tufts project staff Frequency and quality of the interaction with USAID country staff Degree to which Nutrition Innovation Lab activities intersect with other FTF activities in Nepal USAID mission staff assessment of the interactions with Nutrition Innovation Lab staff Interviews with USAID mission staff Number and kind of interactions with other FtF projects Trip reports, annual reports Degree to which Nutrition Innovation Lab activities intersect with other FTF activities in Nepal Quality and content of the interactions with “other international partners” such as: University of Bergen (Norway), LCIRAH, UNICEF, Save the Children, Heifer Assessment of in-country staff of other FtF projects Interviews with staff of other FtF projects in country. Assessment of quality and content of meetings with international partners Assessment of other international partners’ staff of the quality and content of interaction with Nutrition Innovation Lab projects Annual reports, trip reports Interviews with host country representative staff members from key partners (HKI, Heifer Project, etc.) 107 Have the two Nutrition Innovation Lab Awards created value added benefits compared to the former CRSP model of one lead university managing a global program of multiple projects? Have the two MEs built synergies between their regional programs to ensure comparability among findings? International, AusAID , University of Jakarta , WorldFish (Bangladesh) Degree to which Nutrition Innovation Lab complements or competes with other USAID funded programs such as FANTA and SPRING Degree to which Nutrition Innovation Lab complements or competes with SUN activities. Assessment of Nutrition Innovation Lab staff of the roles and objectives of the Nutrition Innovation Lab vis-a￾vis other projects. Interviews with Nutrition Innovation Lab PIs and project staff Nutrition Innovation Lab annual reports FANTA/SPRING annual reports Quality and impact of research dissemination at international level Impact of research findings on host country policy Degree to which materials on website meet the needs of a Numbers of publications Placement of publications in peer reviewed journals Number of citation of published articles in peer review journals Number of research reports Number of presentations of project materials at conferences, workshops, trainings Assessment of host country partners and ministry officials’ of Review of materials on website Annual Reports Science Citation Index Interviews with project PIs and staff Review of training documents and workshop agendas and minutes Interview with Ministry staff and host country partners. Interviews with project staff, research 108 How have these synergies contributed to the Nutrition Innovation Labs’ objectives? How could the synergies between the two regional programs be strengthened? broad range of potential users impact of research on host country policy and planning Range of materials on Nutrition Innovation Lab website The degree to which the website is kept current Assessment of the usefulness of the website by project stakeholders partners, host country researchers and trainees. Degree to which shared management results in coordination of research activities Degree to which the Nutrition Innovation Lab are producing data addressing the shared goals and objectives of research Assessment of ME PIs and staff on the level of collaboration between the MEs Assessment of collaborating Institutions on the degree to which the research produced by each Nutrition Innovation Lab address critical questions Review of materials on the shared website Review of research reports and published papers Interviews with members of the Advisory Committee Interviews with PIs and staff of collaborating institutions Research Program Are there strategic contributions to these research questions: --What are the agriculture-to￾To what extent has the research program met the goals set out in the original RFP and Technical Application Implementation of research agenda Goals of the research program Appropriateness, with respect to Feed the Future and Nutrition Innovation Lab goals of the specific research agenda for the programs Alignment with Feed the Future and Nutrition Innovation Lab goals, of research design Involvement of host country PIs in Original RFP Technical Application Research Reports Published papers 109 nutrition pathways? --What are the program impact pathways? --What is the value of integrated programming pathways? Challenges: --What challenges have the two Nutrition Innovation Labs faced during research design and implementation? --What impact, if any, have these challenges had on implementation of research activities? --How effective have the two research programs establishment of the research agenda Degree to which the proposed research agenda has been implemented in country Guidance of students who are to be working in support of research agenda Obstacles overcome in relation to staffing Obstacles overcome in data collection Interviews with PIs and Program staff Interviews with host country project staff, including where appropriate, PIs, graduate students whose work supports research agenda Review of working papers Titles and abstract of theses supported by program funds Progress on data collection and analysis Identification of challenges in implementation of research Identification of solutions to research challenges Degree to which programed data collection has been carried out Clarification of who is collecting, cleaning, and analyzing data Assessment of project staff and PIs and staff of research partners and other collaborating institution on the programs’ success in overcoming challenges Interview with in-country project staff, including where appropriate, PIs, students and research coordinators Review of working papers Review of published papers Interviews with PIs and staff of collaborating institutions 110 been in addressing the challenges? --What could they do differently to better address the challenges? Training Programs --Have the Nutrition Innovation Labs met academic and technical capacity strengthening targets? --Are the appropriate type and number of people being targeted for the right kind of training? Short-term training programs provided Impact of training on the skill set of students Adequacy of follow￾up/monitoring of trainees post training Appropriateness of trainees Number and types of training programs provided Appropriateness of curricula to Nutrition Innovation Labs project goals Diversity of trainees Fairness of selection process Leaders of each training program Participants’ assessment of the impact of their training on knowledge, skills and employment Participants’ assessment of the quality of their training Participants assessment of the appropriateness of their training Participants’ assessment of post training support Review of plans for mentoring trainees and monitoring their post￾training experiences. Interview with local organizers Interviews with trainees Project reports Annual reports Program satisfaction surveys Interviews with past participants in short term training Survey of past participants in short term training Interviews with training staff Curricula for short term training Lists of participants by gender, organization, etc. Program announcements and application procedures Guidelines for selection of applicants Plans for monitoring and evaluation of training activities Long-term/degree training programs provided Adequacy of follow￾Number and types of training programs provided Appropriateness of curricula to Nutrition Innovation Labs project goals Interviews with current (and past, if applicable) students enrolled in masters and PhD programs in the US and abroad Survey of participants in long term 111 Institutional Capacity Collaboration --What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? --How have trainees put into practice the knowledge and skills acquired? up/mentoring of trainees post training Adequacy of assistance to identify post training opportunities and/or employment Diversity of trainees Fairness of selection process Students’ assessment of the quality of their training Students’ assessment of the appropriateness of their training Students’ assessment of the impact of their training on knowledge, skills and employment. Students’ assessment of post training support and mentoring Review of plans for mentoring/monitoring students’ post-training experiences. training Plans for continued mentoring of students and monitoring of their post degree careers Needs of local institutions for capacity building and collaborative opportunities Participation of local institutions in selecting trainees, curricula Assessment of congruence of training provided by project with needs identified by host country partners Review of host institutions needs assessments USAID /Nepal staff assessment of success of capacity building Details of Scientific Symposia w/Institute of Medicine 112 --How have the training programs contributed to strengthening institutional capacity in the target countries? Institutional Capacity Collaboration --What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? --How have trainees put into practice the knowledge and skills acquired? --How have the training programs contributed to strengthening institutional Participation of local institutions in planning research and training activities Adequacy of local institutions to identify a good pool of candidates to select from. Degree of involvement of local institutions in research planning, implementation, analysis and dissemination Assessment of stakeholders on the appropriateness and quality of individuals selected for training Minutes of meetings Project reports Interviews with local research, university and government staff Degree to which host country institutions have benefited from training programs Degree to which students and trainees are entering into employment Extent to which Nutrition Innovation Labs ME and in country staff assist trainees to move forward in their careers Assessment of host country partners on the impact of training and capacity building activities on their capacity for research and training Current employment of past trainees Current involvement of past trainees in ongoing Nutrition Innovation Lab research and training Interviews with staff of host country partners Interviews with past trainees Survey of past trainees Nutrition Innovation Lab/ME data on trainees Degree to which the research programs align with the Feed the Future research strategy? Project documents Interviews with other FtF program staff (esp investigators that work on several -- e.g. Shively) 113 capacity in the target countries? Program Future What adjustments may be necessary to their research programs to better ensure alignment? Degree to which project PIs and ME have identified challenges to the research programs Degree to which project PIs and ME have identified strategies to improve research programs. Interviews with PIs, ME staff and host country staff Has the way the two research programs have been set up offer strong likelihood of impactful results that justify funding a second phase? Assessment of quality of current research Assessment of the degree to which current research addresses key FtF questions Reports of current research, publications, briefs Interviews with project staff Review of current knowledge gaps in nutrition and agriculture If renewed for a second phase, does the evidence suggest that changes are needed to either or both of the Nutrition Innovation Labs’ management, research (i.e., design, implementation, communications, stakeholder involvement) and/or training (i.e., student recruitment and selection, content, location) programs, and/or institutional capacity collaboration? What lessons have been learned that should be taken into consideration if a second phase is funded? Degree to which the Nutrition Innovation Labs have achieved the goals of the original RFP as of year four Degree to which the trajectory of research, training and capacity building activities appears to build towards the future 114 How well do the two research programs align with the Feed the Future research strategy? What adjustments may be necessary to their research programs to better ensure alignment? 115 b. Document review and secondary sources The evaluation team has conducted a preliminary review of project documents, including: • the original USAID RFA • the technical applications for each Nutrition Innovation Lab • the sub-award RFPs • the successful sub-awardee proposals, • minutes from meetings • annual reports The team has also reviewed the ME website, its organization and the materials provided on the website; and read the published papers and research reports for each Nutrition Innovation Lab. In addition, the evaluation team has requested several sets of additional materials from the MEs. These include lists of participants in short and long term training with gender and sending institutions, curricula for training programs, and follow-up information on current employment and activities of past trainees. c. Key informant interviews The Evaluation team will conduct a series of semi-structured interviews with key informants representing project stakeholders in Nepal, Uganda, Malawi, and the USA (Tables 1; 2-4). Stakeholders will include: • Sub-Awardee staff PIs and US based staff • Sub-awardee host country directors some staff • Representative research staff • Directors and staff of the key collaborating projects in each country • Representatives of the host country academic partners • USAID country staff • Representatives of the Ministries of Agriculture and Health, • Representatives of other FTF/IL host country program staff • The in-country staff of the Nutrition Innovation Labs and their sub-awardees, • Regional research coordinators and enumerators • On-the-ground partners such as HKI. Heifer International, etc… The evaluation team already has conducted key informant interview with the Tufts Nutrition Innovation Lab PIs and key project staff. The team has also had the opportunity to interview 116 two students currently in degree programs in the Boston area. One of the students, Edgar Agaba, is a previous project director from Uganda. Before traveling the evaluation team will also interview by phone or SKYPE PIs and key staff of the sub-awardees at Harvard University, Johns Hopkins University, Purdue, and Tuskegee. Tables 2-4 list individuals representing stakeholder groups from whom a sample of key informants to be interviewed will be selected. Section D contains a set of preliminary interview guides to be used in semi-structured interviews with key informants. Interviewers will take notes and record key pieces of material. Team members will meet each evening to review notes, summarize key findings for the day and identify key questions for further exploration. d. Training Evaluation As part of the Nutrition Innovation Lab evaluation, analyses will be done on several aspects of the capacity development elements of the project. It will be important for the evaluators to assess training, skills development and capacity development of in-country as well as university personnel and junior staff who have been involved in Nutrition Innovation Lab. Using an on-line survey tool, Survey Monkey, a brief questionnaire will be sent to individuals who have participated in short-term courses and training as part of Nutrition Innovation Lab. Questions will focus on the type of training or course, the content of trainings, the quality of material presented during trainings and the application of what was learned for research and development practice. It is hoped that this survey will provide an overview of the extent of training, skills development and capacity in Nutrition Innovation Lab supported programs and research. The quantitative analysis will be complemented with qualitative information gathered from documents produced by selected Nutrition Innovation Lab-supported programs, such as progress reports and briefs that diver deeper into training activities. There will also be a concerted effort to examine the impact of Nutrition Innovation Lab training on women in particular. Longer-term training and capacity development will also be assessed. Graduate level training and individual scholars will be examined amongst educational institutions that include promotion, publications, involvement in national and international meetings, and involvement in local, regional and global academic networks. Data gathering for long-term training will be done through semi-structured interviews and brief surveys. This aspect of the evaluation will focus on whether long-term training and capacity development prepared Nutrition Innovation Lab scholars for careers in university teaching, research and development practice. 117 5. Methodology for Quantitative and Qualitative Data Analysis Interviews with key informants will be captured in notes, which will be expanded into field notes at the end of each day of interviewing. When the respondent allows, we will audio record interviews as an aid to memory in the writing of expanded field notes. Interview notes will be entered as documents into the NVivo Program for the computer assisted analysis of qualitative data. Notes will be coded using a priori coding scheme derived from the questions included in the interview guides for each set of stakeholders. Emergent themes from the interviews will also inform coding and analysis. Responses to key questions in the interview guides will be examined using content analysis, focusing the range of opinion and assessment. NVivo allows for a broad analysis of a range of responses that can be reported in narrative form, and also for cross checking responses among respondents for presentation of the range of responses in tabular form organized by key characteristics of respondents. 6. Methodological Limitations a. Sampling and participant selection bias In order to develop a list of potential interviewees from among the various stakeholders in these programs, we have solicited a list of active and past staff, investigators, partners, host country officials and academics. The ME has provided what appears to be exhaustive lists of stakeholders. We have then reviewed the original technical proposals, annual reports, and trip reports from the Nutrition Innovation Labs in order to generate additional names to augment the lists provided by the ME. These lists can be found in Tables 2-4. We have identified a number of critical informants with whom we will be speaking. We will interview all of the PIs and other senior staff of both U.S. and host country sub awardees, senior staff in the participating government ministries in each country, and senior USAID staff in each of the host countries. For other stakeholder groups identified as critical to the evaluation, we will sample informants using a purposive sampling frame based on the criteria of 1) level of involvement with the project, 2) breadth of knowledge of program activities, 3) representativeness of the range of diversity of stakeholders and 4) availability during the time available to the evaluation team in each host country or by Skype. 118 While random sampling from the lists might provide a less biased sample of respondents, in this evaluation we are approaching our respondents as key informants and knowledge experts. When the constraints of time and availability are such that a purposive sample is more feasible and interviewees represent key informants and knowledge experts, carefully drawn purposive samples of respondents are understood to adequately represent the range of variability. However, we will carefully assess the likely sources of bias at the time of analysis. If a problem of sample representativeness is ascertained during preliminary analysis, additional individuals may be interviewed via electronic media after the team’s return from host country travel. b. Recall bias Key informant interviewing will focus primarily on the interviewees’ opinions and assessments of program activities as participants and collaborators over the life of the project. We do not anticipate particular problems with recall bias for questions of opinion. Questions of fact will be cross-checked against documentation of events and activities available in other project materials and comments of other informants. c. Response bias Self-report data (i.e., interview data) is always subject to individual interviewees’ interpretations and perspectives. However, we have structured this evaluation in ways to reduce the likelihood of three types of systematic response bias: acquiescence bias (i.e., the tendency to agree), demand characteristics (i.e., modifications to responses because of being “studied”), and social desirability bias (i.e., ascribing favorable traits, even if this is untrue). We have structured the interview guides to include responses to important questions from each of the stakeholder groups. In analysis we will code responses to interview questions such that the responses of different stakeholder’s and different individuals from within stakeholder groups can be cross￾checked for convergence or disagreement. Through triangulation of assessments and understandings of different individuals we can come to a set of conclusions that identify areas of consensus and areas of disagreement. While getting rid of response bias completely is difficult, our triangulation of responses both across and within stakeholder groups during analysis will greatly reduce this threat to data validity. 119 Table 2: List of Interviews for Nepal Interviewee Institute Project Role Academic Partners Dr. Patrick Webb Tufts Program Director - Asia Dr. Eileen Kennedy Tufts Co-Director Dr. Shibani Ghosh Tufts Associate Director - Technical Dr. Paul Giguere Tufts Associate Director - Communications Terese Daly Tufts Associate Director - Administration Communications Team Tufts Knowledge Management System & electronic comm Ms. Liz Marino-Costello Tufts Program Manager - Boston Dr. Jeevan Sharma Tufts Program Manager -Nepal Dr. Jeffrey Griffiths Tufts Program Director – Africa Dr. Jennifer Coates Tufts Faculty Expert Dr. Timothy Griffin Tufts Ms. Zoya Hamilton Tufts Research Administration Eunice A. Bonsi Tuskegee Wafai Fawzi Harvard PI Christopher Duggan Harvard PI Andrew Thorne Lyman Harvard HSPH Team member Lindsey Locks Harvard HSPH Team member Gerald Ernest Shivley Purdue Keith West Johns Hopkins PI Rolf Klemm Johns Hopkins Co-PI Ramesh Adhikari Johns Hopkins Co-PI Devendara Gauchan Johns Hopkins Co-PI Swetha Manohar Johns Hopkins Co Investigator Ruchita Rajbhandaray Johns Hopkins Co Investigator Raman Shrestha Johns Hopkins Co Investigator Sujay Bhattacharya CHD/UNICEF JHSPH/Tufts Alumni (Capacity Building Trainees) 120 Rajan Paudel IOM JHSPH/Tufts Alumni (Capacity Building Trainees) Shikha Basnet UNICEF JHSPH/Tufts Alumni (Capacity Building Trainees) Tor Stand Harvard affiliate Team member in Norway Sigrun Henjum Harvard affiliate Team member in Norway Ingrid Kvestad Harvard affiliate Team member in Norway USAID – Mission Hari Koirala USAID/Nepal Dev. Program Specialist, HFP Danielle Kneuppel USAID/Nepal FtF Team Leader, SEED Evan Meyer USAID/Nepal Agriculture Officer, SEED Debendra Adhikari USAID/Nepal Dev. Program Specialist, HFP Local CO-Pis Devendra Gauchan NARC Also cross listed as JHU Ramesh Adhikari Kathmandu Medical College/ IOM Also cross listed as JHU Dr. Kedar Baral Rector, PAHS Government Shabnam Shivakoti MoAD Senendra Upreti Child Health Division, DoHS Director Sumit Karn Child Health Division, DoHS Coordinator for the Nutrition Technical Committee Rajkumar Pokharel Child Health Division, DoHS Bishnu Nepal NPC Joint Secretary Praveen Mishra MoHP Secretary Partners: NGOs, Academia and 121 Organizations in Country Kathleen Kurz DAI Principal Investigator Sumi Devkota DAI Kathmandu based consultant Dr. Prakash Sunder Shrestha Listed as Harvard team members Dr. Ram Chandyo Listed as Harvard team members Dr. Manjeswari Ulak Listed as Harvard team members Dr. Merina Shrestha Listed as Harvard team members Saba Mebhratu UNICEF Chief of Nutrition Dale Davis HKI Country representative Deepak Thapa NTAG Programme Manager Sharad Onta IOM Vice Dean Peter Oyloe Suaahara Chief of Party Pooja Pandey Suaahara Deputy Chief of Party Dr. Uma Koirala Nepal Nutrition Foundation Chairperson Narayan Subedi Nepal Public Health Foundation Progarm Manager Madhav Shrestha Aquaculture IL PI Mr. Atmaram Pandey Aquaculture IL Ex Joint Secretary, NPC 122 Table 3: List of Interviews for Uganda & Malawi Interviewee Institute Project Role Academic Partners Dr. Eileen Kennedy Tufts Co-Program Director Dr. Jeff Griffiths Tufts Program Director – Africa Dr. Shibani Ghosh Tufts Associate Director - Technical Edgar Agaba Tufts Local Coordinator Communications Team Tufts Knowledge Management System & electronic comm Ms. Liz Marino-Costello Tufts Program Manager - Boston Eunice A. Bonsi Tuskegee Wafaie Wahib Fawzi Harvard Dr. Christopher Paul Duggan Harvard Nutrition Innovation labupa Gunaratna HSPH Team member Gerald Ernest Shivley Purdue Keith West Johns Hopkins Parul Christian Johns Hopkins Joanne Katz Johns Hopkins Luke Charles Mullany Johns Hopkins James Tielsch Johns Hopkins USAID Uganda Mission Alfred Boyo USAID PHN Team Anne Murphy USAID Prevention Advisor, PHN Andrew Mckim USAID FtF Coordinator Simon Byabagambi USAID Agronomist Sheila Nyakwezi USAID Nutrition Specialist Dianna Darsney de Salcedo USAID Crisis Stabilization & Democracy 123 USAID Malawi Mission Violet Orchardson USAID Nutrition Advisor, HPN Cybill Sigler USAID SEG Team Leader John Edgar USAID SEG Deputy Team Leader Government Dr. Kisamba Mugerwa National planning Authority Chairman Bakunzi Maureen Office of the Prime Minister Ass. Commissioner Policy Baku Chadia Baku Ministry of Health Head of nutrition Alex Bambona Ministry of Agriculture Head, Food & Nutrition Local Orgnaizations and Institutions Dr. Robert Mwadime Community Connector Chief of Party Benjamin Aishya Community Connector M&E Assiimwe Charles and team Community Connector Regional office; Head of western region Patrick Kibaya and team Uganda Chartered Healthnet (UCH) Dr. Van Campenhout IFPRI Head of Kampala Office Naluyiga Annet IFPRI Administrator IFPRI-Kampala) Dr. Nassul Kabunga also with NUTRITION INNOVATION LABA IFPRI Margaret Kabahenda Makerere Univ Coordinator for Tuskegee grant Prof Bashaasha Makerere Univ Principal-Grant Recipient and PI Florence Turyashemererwa Nutrition innovation team Coordinator Annet Kawuma Nutrition innovation team Research specialist Bernard Bashaasha listed as HSPH Team 124 members David Guwatudde listed as HSPH Team members Joyce Kikafunda listed as HSPH Team members Henry Wamani listed as HSPH Team members Julius Twinamasiko Makerere University short term trainee George Omiat Purdue PhD trainee 125 Table 4: List of Interviews USA and Global Interviewee Institute Project Role Global Parnters Anu SPRING Sister USAID Program Heather Danton SPRING Anna Herforth Giles Bergeron FANTA Sister USAID Program Technical Advisory Committee Will Master Tufts Shibani Ghosh Tufts Chris Duggan Harvard Keith West/Rolf Klemm JHU Jerry Shively Purdue Eunice Bonsi Tuskegee Richard Deckelbaum Columbia University Victoria Quinn HKI Ram Shrestha NTAG Shakuntala Thilsted World Fish Shelly Sundberg Gates Stephen Vosti UC Davis Maura Mack USAID Ahmed Kablan USAID Barbara Seligman DAI Board of Directors 126 Patrick Webb Tufts Jeff Griffiths Tufts Wafae Fawzi Harvard Keith West JHU Eunice Bonsi Tuskegee Jerry Shively Purdue Eileen Kennedy Tufts Wil Masters Tufts Shibani Ghosh Tufts Ruth Oniang'o CIAT board, Kenya Maura Mack USAID Ann Tutwiler Bioversity Ahmed Kablan USAID 127 7. Evaluation Work Plan Activities Dates of Activity 1st Month 2nd Month 3rd Month 4th Month 5th Month 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 TASK 1. Develop evaluation design and implementation plan Feb 16 -Apr 30 Activity 1: Desk review & interview with manage - ment entity at Tufts Feb 16 – Mar 12 X X X X Activity 2: Development of evaluation design Mar 16 – Apr 28 X X X X X X Activity 3: Select stakeholders to interview; plan travel April 1 – April 30 X X X X TASK 2: Data Collection May 1 – June 6 Activity 1: Travel to Nepal; interview stakeholders; travel debrief to USAID May 1 – May 9 X X Activity 2: Travel to Africa; interview stakeholders; travel debrief to USAID May 12 –May 24 X X Activity 3: Conduct SKYPE interviews with stakeholders and collect on -line surveys from stakeholders May 1 – May 31 X X X X TASK 3: Data Analysis May 16 – June 9 128 Activity 1: Review data collection for gaps May 16 – May 31 X X Activity 2: Organize stakeholder responses and develop evaluation themes May 25 – June 6 X X Activity 3: Compile preliminary findings for submission to USAID by June 9 June 1 -June 9 X TASK 4: Report Writing June 1 – July 10 Activity 1: Draft report prepared and submitted to USAID by June 23 June 1 -June 23 X X X Activity 2: Final report submitted by July 10 June 21 – July 10 X → 129 B. Data Collection and Management Plan 1. Survey Training Evaluators are experienced in interview techniques. No special training is necessary to use the interview guides but if a translator is necessary, for example, we will provide training and confirm the nature of the translation. 2. Data Management and Security Data collection and management have been described in Section A.4. Evaluators are experienced with the confidentiality requirements needed for work with human subjects. All responses will be collated anonymously and no direct reference will be made to any individual respondent. 3. Data Collection Approvals Data collection approvals will not be needed as this data collection is not for research, and the only information collected is that given by the subjects themselves with their agreement. Confidentiality of responses will be maintained, and no names or identification will be attached to responses. Questionnaires will be coded with the general category of stakeholder, but individual names will not be needed. C. Data Collection Instruments The following section contains lists of questions, which have been developed to obtain data from the various groups. Responses to these questions will be collected and analyzed as detailed in Section A. Interview Questions (guides) for individuals and organizations PIs of major sub awards (HSPH, JH); in-country PIs; in-country staff of sub-awardee project What is your experience with the management of the Nutrition Innovation Lab? Professional/collegial interactions Fiscal interactions (promptness, clarity) Promptness and effectiveness of managing problems Quality of the solutions to problems What is your experience with the management of the Nutrition Innovation Lab with respect to the planning and conduct of research? How was the sub-award process handled? Clarity of the RFP Fairness of the review process Efficiency of the awarding process and fiscal management. Clarity of goals and objectives Degree to which planning is collaborative 130 Degree to which data collection and analysis are supported Degree to which findings are appropriately disseminated What is your experience with the interactions with Nutrition Innovation Lab program PIs and staff in the host country? What is your assessment of the strengths and weaknesses of the Nutrition Innovation Lab in general, and your specific project (s) in particular? In-country project staff, including where appropriate, PIs, graduate students whose work supports research agenda, field supervisors, etc. Please describe the current status of the research program in which you are collaborating? In your opinion is it on time? Progressing? If not, why not? To what extent have you been involved in setting research goals, designing research, conducting the data collection, analyzing data? What is your assessment of the degree to which the research program on the ground addresses the key data goals of the Nutrition Innovation? What is your assessment of the degree to which the research program on the ground addresses key data needs of your organization? The country? Global policy? What do you see as the key challenges in implementing the research plan? To what extent do you feel the challenges have been met? To what do you attribute the success/failure of research program to meet its goals? What are your recommendations for the future? Host country officials What is your experience with the management of the Nutrition Innovation Lab? What is your assessment of the appropriateness of the research and training activities for your country? To what extent does the Nutrition Innovation Lab research and training program meet the needs of your ministry? Of your country? What has been the impact on your institution of the research and training activities of the Nutrition Innovation Lab ? What do you see as the benefits of the Nutrition Innovation Lab for your country? What do you see as not benefiting your country? How would you assess the quality of the interactions with Nutrition Innovation Lab PIs and staff? Appropriateness Collegiality Responsiveness to your interests 8. What processes do the Nutrition Innovation Labs use to keep host country officials informed and what is the frequency of the communication? 9. To what degree do the host country officials feel informed about Nutrition Innovation Lab activities? 131 10. How responsive have the Nutrition Innovation Labs been to host country inquiries and/or needs? Training Partners (e.g. Tuskegee) What is your experience with the management of the Nutrition Innovation Lab? Fiscal Logistical Appropriateness Collegiality Please describe the process of identifying students and trainees supported by the Nutrition Innovation labs. What do you see as the advantages of collaborating in training for your institution? How well prepared are your trainees? How have they performed in your program? What recommendations would you make for future trainees and for the ME? Other partners (HKI, Heifer, etc.) What is your experience with the management of the Nutrition Innovation Lab? To what extent do your mission and goals overlap with the Nutrition Innovation Lab? What is the nature and quality of your collaboration with the Nutrition Innovation Lab? How would you assess the impact of the current and possible future Nutrition Innovation Lab research and training on? The planning and implementation of your current and future projects and development strategies? Agricultural and rural policy at the nationals and international levels? Agricultural and livestock related research nationally and globally? Students Long-term/degree students How did you come to be supported by the Nutrition Innovation Lab? How did you find out about it? How did you apply? What was the process? Please describe your training program. What are its strengths? What are its weaknesses? What is your assessment of the appropriateness of the training you are/have receiving/received to your current work and future work plans? What is your assessment of the appropriateness of the training you are/have receiving/received for the problems in your organization? Region? Country? What is your assessment of the quality of the training you are/have receiving/received? How would you assess the level of support you have received financially? Logistically? In terms of professional development? What recommendations would you suggest for future trainees? For the rest of your program? What type of mentoring did you receive during your program? What kind of assistance have you received for post training opportunities? 132 Short-term trainees Please describe the training program in which you participated. How did you come to know of; were selected for; the training program in which you participated? What is your assessment of the appropriateness of the training you received to your current work and future work plans? Quality of the curriculum? Appropriateness of curriculum? Quality of instruction? What is your assessment of the appropriateness of the training you received for the problems in your organization? Region? Country? What is your assessment of the quality of the training you received? Has the training you received had an impact on your work? How? What recommendations would you suggest for future training programs? SPRING, FANTA, SUN How do you see the mission and goals of the Nutrition Innovation Lab in relation to the mission and goals of your project(s)? What is the nature and quality of your interactions with the Nutrition Innovation Lab staff and projects? What is the frequency and purpose of these interactions? What type of follow up and follow through is there by the ME? 133 D. Curriculum Vitae of EET Barbara J. Stoecker ACADEMIC BACKGROUND: Dates Major Field Degree Institution 1977-78 Nutrition Postdoctoral Iowa State University 1966-70 Nutrition Ph.D. Iowa State University (Minors--Biochemistry, Physiology) 1961-65 Home Economics Educ B.S. Kansas State University PROFESSIONAL POSITIONS: Marilynn Thoma Chair in Human Sciences, 2010 - Present Regents Professor, Department of Nutritional Sciences, Oklahoma State University, 2002-present Professor and Head, Department of Nutritional Sciences, Head 1993-2001 & 2002-2003; Interim Assoc Dean for Research Services, College of Human Environ. Sciences, Oklahoma State University, 2001-2002. Professor, Department of Nutritional Sciences, Oklahoma State University, 1990-1993 Associate Professor, Department of Food, Nutrition & Institution Administration, Oklahoma State University, 1987 - 1990 Coordinator of Academic Affairs, International Center for Arid and Semi-Arid Land Studies, Texas Tech University, 1984-87 Associate Professor, Department of Food and Nutrition, Texas Tech University, 1982-87. Tenured - 1985. Adjunct - 1987 -1992 Associate Professor (part-time), Department of Food and Nutrition, Texas Tech University, 1979-82 Postdoctoral Associate, Department of Food and Nutrition, Iowa State University, 1977-78 (1/2 time) Faculty Member, Nutrition Research Center, Dept. of Pediatrics, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (1/2 time) 1973-77 Consultant, Rural Family Research Project, Ames, Iowa, 1972-73 134 Instructor, Department of Food and Nutrition, Iowa State University (1/3 time position) 1970-73 Research Associate, Lipid Project, Iowa State University (2/3 time position) 1970- SELECTED JOURNAL ARTICLES from >80 refereed articles (*indicates graduate student) *Abebe H, Abebe Y, Loha E, Stoecker BJ. Consumption of vitamin A rich foods and dark adaptation threshold of pregnant women at Damot Sore District, Wolayita, Southern Ethiopia. (Submitted to Ethiopian Journal of Health Sciences). *G/Egziabher T, *Teyikie N, *Mulugeta A, Abebe Y, Hambidge KM, Stoecker BJ. Lack of dietary sources of iodine and the prevalence of iodine deficiency in rural women from Sidama Zone, Southern Ethiopia. African J Food, Nutr Develop (In Press) *G/Egziabher T, Stoecker BJ. Vitamin D insufficiency in a sunshine sufficient area: Southern Ethiopia. Food Nutr Bull (In Press). *Ersino G, Tadele H, *Bogale A, Abuye C, Stoecker BJ. (2013) Iodine status and knowledge of iodine deficiency disorders (IDD) among pregnant women in rural Sidama, southern Ethiopia. Ethiop Med J (In Press). *Bogale A, Stoecker BJ, Kennedy T, Hubbs-Tait L, Thomas D, Abebe Y, Hambidge KM. (2013) Nutritional status and cognitive performance of mother-child pairs in Sidama, Southern Ethiopia. Matern Child Nutr 9:274-84. *Girma M, Loha E, *Bogale A, Teyikie N, Abuye C, Stoecker BJ. (2012) Iodine deficiency in primary school children and knowledge of iodine deficiency and iodized salt among caretakers in Hawassa Town: Southern Ethiopia. Ethiop J Health Dev 26: 30-35. Regassa N, Stoecker BJ. (2012) Contextual risk factors for maternal malnutrition in a food-insecure zone in Southern Ethiopia. J Biosoc Sci 44:537-48. *Aubuchon-Endsley NL, *Grant SL, Thomas DG, Kennedy TS, *Berhanu G, Stoecker BJ, Hubbs-Tait L, Hambidge KM. (2012) Infant responsiveness, alertness, haemoglobin and growth in rural Sidama, Ethiopia. Matern Child Nutr Jan 10. doi: 10.1111/j. 1740-8709.2011.00391.x. [Epub ahead of print] Regassa N, Stoecker BJ. (2012) Household food insecurity and hunger among households in Sidama district, southern Ethiopia. Public Health Nutrition 15:1276-83. *Mulugeta A, Hagos F, Kruseman G, Linderhof V, Stoecker BJ, Abraha Z, Yohannes M, Samuel GG. (2010) Child malnutrition in Tigray, Northern Ethiopia. East Afr Med J 87:248-254. 135 *Bogale A, Abebe Y, Stoecker BJ, Abuye C, Ketema K, Hambidge KM. (2009) Iodine status and cognitive function of women and their five year-old children in rural Sidama, Southern Ethiopia. East Afr J Public Health 6:296-9. Gibson R, Abebe Y, Arbide I, Teshome A, Hambidge KM, Stoecker B. (2009) Inadequate feeding practices and impaired growth among children from subsistence farming households in Sidama, Southern Ethiopia. Matern Child Nutr 5:260-275. *Gharaibeh MA, Stoecker BJ. (2009) Assessment of serum 25 (OH)D concentration in women of childbearing age and their preschool children in Northern Jordan during summer Eur J Clin Nutr 63:1320-6. Mulugeta A, Hagos F, Stoecker B, Kruseman G, Linderhof V, Abraha Z, Yohannes M, Samuel GG. (2009) Nutritional status of adolescent girls from rural communities of Tigray, Northern Ethiopia. Ethiop J Health Dev 23:5-11. Stoecker BJ, Abebe Y, Hubbs-Tait L, Kennedy TS, Gibson RS, Arbide I, Teshome A, Westcott J, Krebs NF, Hambidge MK. (2009) Zinc status and cognitive function of pregnant women in southern Ethiopia. Eur J Clin Nutr 63:916-918. Hubbs-Tait L, *Mulugeta A, *Bogale A, Kennedy TS, *Baker ER, Stoecker BJ. (2009) Main and interaction effects of iron, zinc, lead and parenting on children’s cognitive outcomes. Dev Neuropsychol 34:175-195. Gibson RS, Abebe Y, Stabler S, Allen RH, Westcott JE, Stoecker BJ, Krebs NF, Hambidge KM. (2008) Zinc, gravida, infection, and iron, but not vitamin B-12 or folate status predict hemoglobin during pregnancy in southern Ethiopia. J Nutrition 138:581-586. Kennedy TS, Thomas DG, Wogene T, Abebe Y, Hubbs-Tait L, Stoecker BJ, Hambidge KM. (2008) Growth and visual information processing in infants in southern Ethiopia. J Applied Develop Psych 29:129-140. Abebe Y, Bogale A, Hambidge KM, Stoecker BJ, Arbide I, Teshome A, Krebs NF, Westcott JE, Bailey KB, Gibson RS. (2008) Inadequate intakes of dietary zinc among pregnant women from subsistence households in Sidama, Southern Ethiopia. Public Health Nutr 11:379-386. Abebe Y, Bogale A, Hambidge M, Stoecker BJ, Bailey K, Gibson RS. (2007) Phytate, zinc, iron and calcium content of selected raw and prepared foods consumed in rural Sidama, Southern Ethiopia, and implications for bioavailability. J Food Comp Anal 20:161-168. Abebe Y, Stoecker BJ, Hinds MJ, Gates, GE. (2006) Corn and kocho-based fortified foods for early childhood feeding in southern Ethiopia. African J Food, Agric, Nutr Develop 6:1-19. 136 Hambidge KM, Abebe Y, Gibson RS, Westcott JE, Miller LV, Lei S, Stoecker BJ, Arbide I, Teshome A, Bailey KB, Krebs NF. (2006) Zinc absorption during late pregnancy in rural southern Ethiopia. Amer J Clin Nutr 84:1102-1106. Medeiros DM, Stoecker B, Plattner A, Jennings D, Haub M. (2004) Iron deficiency negatively affects vertebrae and femurs of rats independent of energy intake and body weight. J Nutr 134:3061-3067. *Toure F, Lucas E, Stoecker BJ. (2003) Fish and shrimp added bioavailable iodine to cassava and millet-based diets. Ecol Food Nutr 42:223-239. OTHER RELEVANT PUBLICATIONS: (from 20) Panel on Micronutrients, Subcommittees on Upper Reference Levels of Nutrients and of Interpretation and Use of Dietary Reference Intakes, and the Standing Committee on the Scientific Evaluation of Dietary Reference Intakes, Food and Nutrition Board, Institute of Medicine. (2001) Dietary Reference Intakes for Vitamin A, Vitamin K, Arsenic, Boron, Chromium, Copper, Iodine, Iron, Manganese, Molybdenum, Nickel, Silicon, Vanadium, and Zinc. Washington, D.C.: National Academy Press, 773 pp. Stoecker BJ. (1986) Interdisciplinary components of agricultural planning. In: Behrens JS, Bennett WF, eds, Looking Forward/Looking Backward: The Cultural Readaptation of International Students. Lubbock, TX: International Center for Arid and Semi-Arid Land Studies, pp. 157-166. Stoecker BJ, Montgomery EI, Gott ES, eds. (1982) Developing Nations:Challenges Involving Women. Lubbock, Texas: Texas Tech University Press, 160 pp. Stoecker B, Dhanamitta S, Sirivech S, Valyasevi A. Assessment of Nutritional Status Parameters for Further Use in a Proposed Agricultural-Nutritional Planning Project in Thailand, Final Report to the World Food Institute, Iowa State University. 1979. RECENT PRESENTATIONS: (*indicates student supervised) *Girma M, Stoecker BJ. (2013) Maternal household characteristics and child stunting in Ethiopia: A secondary analysis of the 2011 Ethiopian Demographic and Health Survey. FASEB J 27:lb356. *Bogale A, Peterson S, Stoecker BJ. (2013) Plasma zinc response to supplementation in apparently health women from rural Sidama in southern Ethiopia. FASEB J 27:845.29. *Lasley K, *Changwatpol P, Hermann JR, *Bogale A, Stoecker BJ. (2012) Bone mineral density and vitamin D status in older women with and without metabolic syndrome. FASEB J 26:642.3. *Girma M, Loha E, *Bogale A, Stoecker BJ. (2012) Child nutritional status and cognitive performance in Hawassa Town, Southern Ethiopia. FASEB J 26:652.2. 137 *Nekatebeb H, Smith BJ, Kuvibidila S, Wang Y, *Girma A, Peterson S, Stoecker BJ. (2012) The effect of dietary selenium intake on proliferation of T cells and B cells in C57BL/6 mice with chronic inflammation. FASEB J 26:651.4. *Woltamo T, White SD, Hubbs-Tait L, Stoecker BJ, Hambidge M. (2012) Relation of mother-infant interaction to infant weight and length in rural Southern Ethiopia. FASEB J 26:1028.11. Traore D, Chandra L, French C, Lucas EA, Stoecker BJ, Kuvibidila S. (2011) Effects of Digitaria exilis (fonio) on glucose metabolism and inflammatory cytokines in KK/HIJ diabetic male mice. FASEB J 25:lb294. *G/Egziabher T, Stoecker BJ, Abebe Y, Hambidge KM. (2011) Iron status of women from selected rural areas of Sidama Zone, southern Ethiopia. FASEB J 25:779.5. Regassa N, Stoecker BJ. (2011) Food insecurity among households in Sidama District, Southern Ethiopia. FASEB J 25:lb254. *Joray M, *G/Egziabher T, Stoecker BJ, Hambidge KM. (2011) Seasonal differences in household food insecurity in Sidama Zone, southern Ethiopia. FASEB J 25:986.10. Park L, Kennedy T, Grant S, Aubuchon-Endsley N, Traore D, Stoecker BJ, Thomas D. (2011) Vitamin D status in breastfeeding women in Oklahoma. FASEB J 25:589.5. *Woltamo T, Hubbs-Tait L, Thomas D, Stoecker B, Hambidge M. (2011) Associations of family cattle ownership, mother and father education, maternal depression, and social support with anthropometric measures of breastfeeding infants in rural Ethiopia. FASEB J 25:592.21. FUNDED GRANTS RELATED TO INTERNATIONAL NUTRITION: 2003-2004 B.J. Stoecker, C.G. Neumann. Combating Micronutrient Malnutrition: Assessment of Constraints to Including Animal Source Foods in Children’s Diets in Rural Ethiopia and Kenya. Global Livestock Collaborative Research Support Program. $50,000. B.J. Stoecker, T. Kennedy, L. Hubbs-Tait and D. Thomas. Zinc Nutrition and Brain Development in Southern Ethiopia. NIH, (Subcontract to UCHSC, KM Hambidge PI), $166,659. 2004-2005 B.J. Stoecker. Field Assessment and Training for Iraqi Pediatricians – Children’s Nutritional Status. OU/USAID. $234,008 B.J. Stoecker. Vitamin A Status of Breasfeeding Mothers in Southern Ethiopia. Micronutrient Initiative of Canada, $9,500. B.J. Stoecker and E. Lucas. Enhancing research capacity: An inductively-coupled plasma mass spectrometer, USDA, $50,000. 138 2007-2011 B.J.Stoecker, T Kennedy, L. Hubbs-Tait and D. Thomas. (Subcontract with K.M. Hambidge). Zn and maternal-infant brain function in S. Ethiopia: Randomized Controlled Trials, NIH, $761,942. 2007-2010 D.G. Thomas, T. Kennedy, L. Hubbs-Tait and B.J. Stoecker. Maternal Dietary Nutrients & Neurotoxins in Infant Cognitive Development. USDA/NRI, $369,991. 2008-2009 B.J. Stoecker. Student Support for Sengal through Institute de Technologie Alimentaire. World Bank, $25,310. 2008-2009 S.M. Wilson, A. Tesgaye, B.J. Stoecker, J. Hattey, R. Merkel, G. Emslie. Increasing the capacity of higher education in East Africa through the creation of a Consortium of African and United States Educators (CAUSE) in the focus area Food and Nutritional Security: A strategic plan. Higher Education for Development/US Agency for International Development. $50,000 2012-2014 Tafere G. Belay and B.J. Stoecker. Community salt testing and relation of iodine intake to visual information processing (VIP) of Ethiopian infants. Nestle Foundation. $20,000. 2013-2014 Dawd Gashu Adem and B.J. Stoecker. Association between micronutrient status (iron, selenium and iodine) and health and thyroid metabolism of under five year old children living in the Amhara region, Ethiopia. USAID/Borlaug LEAP Program. $20,000. GRADUATE STUDENT ADVISEMENT: Total Major Professor: M.S. and M.Sc. 42 Major Professor: Doctor of Philosophy 17 HONORS AND AWARDS Oklahoma State University Lela O'Toole Outstanding Research Award - 1992 CHES Outstanding Academic Advisor – 1996 Margueritte Scruggs Outstanding Research Award – 2000 Regents Professor – 2002 – present OSU Faculty/Staff Appreciation Award – 2007 OSU International Education Faculty Excellent Award - 2008 OSU Eminent Faculty Award – 2008 Marilynn Thoma Chair in Human Sciences – 2010 - present Human Sciences Outstanding Graduate Advisor - 2012 Association of Public and Land-Grant Universities Michael P. Malone International Leadership Award – 2010 INTERNATIONAL EXPERIENCE2013 – Training for women cereal processors on improvement of complementary foods for infants and young children in Mali with ACDI/VOCA and Winrock International. 139 2012 – BIFAD CRSP Review Committee 2010 - External Reviewer for European Union Project to Harmonize Nutrition Curricula for East Africa – Meetings in Kenya, 2010. 2012 – Fulbright Senior Specialist in Uganda focusing on departmental organization and maternal nutrition issues 2008 - Fulbright Senior Specialist in Ethiopia for delivery of graduate level nutrition class and MSc student advising. 2008- 2011 – Member of OSU Team for development of the Consortium of African and United States Educators (CAUSE). Strategic planning meetings in Rwanda, Ethiopia and Kenya. 2006 - Present – Support for development of M.Sc. degree program. Graduate instructor for AHuN 513 – Nutrition and Metabolism and a graduate advisor for MSc students. 2006 – Contributor to International Curriculum Workshop for Development of M.Sc. Program in Applied Human Nutrition at Hawassa University, Awassa, Ethiopia. 2004 - 2012 - Research work funded by NIH in Ethiopia, “Zinc and Maternal-Child Brain Function in Southern Ethiopia” and “Zn and Maternal-Infant Brain Function in S. Ethiopia: Randomized Controlled Trials”. 2004 – 2005 Child Nutrition Workshop presented in Jordan for 14 participants from Iraq under the auspices of the Al Sharaka Program for Higher Education in Iraq. Funded by USAID 2003 Project funded by Global Livestock – Collaborative Research Support Program (GL-CRSP) in Ethiopia and Kenya – “Combating Micronutrient Malnutrition: Assessment Constraints to Including Animal Source Foods (ASF) in Children’s Diets in Rural Ethiopia and Kenya” 2001 Curriculum development at Awassa College of Agriculture, Debub University and research project supervision in the Sidamo region of Ethiopia. 2000 Supervised Ph.D. project on complementary food developmed for infants and young children in rural communities in the Sidama region of Ethiopia. 1983 Consulted with and assisted scientists in the National Cancer Institute on cooperative research projects to assess the nutritional status of people in selenium deficient areas of the People's Republic of China. 1981 Member of 12-person Agricultural Production and Nutrition Delegation hosted by the State Scientific and Technological Commission of the People's Republic of China (Visited several provinces primarily in semi-arid regions. Discussed problems of agriculture and nutrition, gave seminars, and made program recommendations.) 1978 Survey of Nutritional and Socioeconomic Status of Children in Thailand (Worked 2 months with team of 6-8 to assess nutritional status of children in North and South Thailand. Developed procedures to incorporate selected indicators of food consumption and nutritional status into Thailand's general farm survey.) 1973-77 Nutrition Research Center, Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand (Extensive work with nutritional assessment of rural children in Northeast Thailand) 140 1965-66 International Farm Youth Exchangee (IFYE) to Jamaica, West Indies (Lived with rural families and worked with the families and extension agents. 141 Kathleen Musante, Ph.D. Professor of Anthropology and Public Health Director, Center for Latin American Studies EDUCATION: University of Connecticut, Ph.D., 1980, Anthropology University of Connecticut, M.A., 1976, Anthropology University of Connecticut, B.A., 1971, Anthropology POSITIONS: Director, Center for Latin American Studies, University of Pittsburgh 2001-present. University Center for International Studies Research Professor 2011- present. University of Pittsburgh, Professor of Anthropology and Public Health 1993 - present. Academic Dean, Semester at Sea, Spring 2006 Voyage, June 2004 – April, 2006. Director of Graduate Studies, Department of Anthropology 2003-2005. Co-Director Interdisciplinary Master’s of Arts in Bioethics 1998-2001. Associate Dean of Arts and Sciences for Graduate Studies and Research 1996- 1999. Chair, Department of Anthropology, University of Pittsburgh, 1995-1996. Faculty Associate, Center for Latin American Studies, University of Pittsburgh, 1993- present. Director, Research Center for Health Risk Reduction in Rural Youth, University of Kentucky, 1990- 1993. University of Kentucky, Professor 1992- 1993, Associate Professor, 1984 - 1992, Assistant Professor, 1978 - 1984, Department of Behavioral Science, College of Medicine (Joint appointment in Anthropology; Member of the Graduate Faculty in Nutritional Science). Cornell University, Visiting Fellow, Division of Nutritional Sciences, Program in International Nutrition, August 1, 1985 - June 30, 1986. AWARDS: Sheth Award for Excellence in International Education Scholar in Residence, Rockefeller Center, Bellagio, Italy, April 2000. Research Award, Fulbright, American Republics Research Program, July, 1992 - August 1994 Praxis Award, Washington Association of Professional Anthropologists, Honorable Mention, 1983 W.H.R. Rivers Prize, Society for Medical Anthropology, November 1976 MEMBERSHIPS AND OFFICES HELD: Society for Applied Anthropology, Fellow President Elect 2014 - present Chair, Malinowski Award Committee 2010-present Member, Nominations Committee 2012-present American Anthropological Association, Fellow Elected member, Nominations Committee of the AAA 2006-2009 Elected member, Committee on the Status of Women in Anthropology 1997-2001, Chair 1999-2001 Annual Meeting Program Board 1986 American Society for Nutrition, Elected Member 142 American Association for the Advancement of Science Society for the Anthropology of Food and Nutrition: President 1986-1988 Nominations Committee 1977-80; Chair 1980 Courses and Curricula Committee Chair 1978-80 Latin American Studies Association Society for Medical Anthropology Executive Committee 1985-1988 Program Co-chair 1986 Chair, W.H.R. Rivers/Steven Polgar Prize Committee 1986-1988 Polgar Prize Committee, member, 2001 Search Committee, Editor of the Medical Anthropology Quarterly 2002 Society for International Nutrition Research, Elected Member GRANTS AND CONTRACTS (total of about $13 M.) Principal Investigator, continuous annual funding from the Roy A Hunt Foundation 2001-present, $5,000/year. Principal Investigator, National Resource Center for Latin American Studies; FLASF, US Department of Education. 2010 – 2014, $2.3 M. Program Co-Coordinator, funding for the exhibition of art: Race and Racism in Cuba, held at the Mattress Factory Museum, Pittsburgh 2010. Ford Foundation, $40,000. Principal Investigator, Dissertation Improvement Grant for Lucia Guerra, National Science Foundation, 2009-2010, $14,999. Principal Investigator, Dissertation Improvement Grant for Penelope Morrison, National Science Foundation, 2007 – 2008, $12,000. Principal Investigator, Support for the Brazilian Studies Program; Mine Safety Appliances Charitable Foundation, 2006-2010, $40,000. Principal Investigator, National Resource Center for Latin American Studies; FLASF, US Department of Education. 2006 – 2010, $1.83 M. Principle Investigator, Fulbright-Hays Group Project Abroad 2006 $80,000. Principal Investigator, National Resource Center for Latin American Studies, US Department of Education. 2003 – 2006, $1.7 M. Principal Investigator, National Resource Center for Latin American Studies, US Department of Education. 2000 – 2003, $1.2 M. Principle Investigator, Hewlett Foundation Support Grant, 2001-2004, $300,000. Co-Investigator, with Jorge Recharte, Susan Poats, (FLACSO - Quito), Ravi Sharma (Pittsburgh), Population, Land Use Water Consumption, and the Environment: A Comparative Exploration of Linkages, Competition, Conflict and Alternatives in Northern Ecuador. John D. and Catherine T. MacArthur Foundation. April 1996 - June 1999, $215,000. Principal Investigator (with Susan Poats - FLACSO-Quito), Women's Economic Decision Making and Child Welfare in Manabi, Ecuador. National Science Foundation NSF/SBR 95-14818. June 1996 - May 1999, $96,678. 143 Principal Investigator (Dissertation Improvement Grant for Coral Wayland), Gender, Urban Poverty and Child Health in Northeastern Brazil. National Science Foundation. August 1995 - July 1996, $12,000. Principal Investigator (with Ravi Sharma, Susan Poats and Jorge Recharte) Population, Land Use and the Environment in Three Regions of the Mira Watershed - Ecuador (Planning grant). Heinz Social Policy Program. July 1995 - April 1996, $10,000 Principal Investigator (with Richard Scaglion, Harry Sanabria, and Michael Siegel) Ethnographic Fieldwork Training Grant, National Science Foundation, May 1995 - April 2000, $50,000. Principal Investigator/Center Director, Exploratory Research Center for Health Risk Reduction in Rural Youth. NIH/NCNR, 1 P20 NR02979-01, 9/91 - 6/93 ($311,000). Principal Investigator (with William Leonard and Billie DeWalt), Farming Systems and Socio-cultural Determinants of Young Child Growth in Two Ecological Zones of Ecuador. NSF, BNS-9106378, 8/91 -1/94, ($93,000). Co-Principal Investigator (with Sara Quandt), Nutritional Strategies of Rural Elderly in Two Kentucky Counties, NIA, AG07999, 5/89 - 4/93 ($450,000). Co-Investigator, Sub-Project Co-Director, Core Leadership Team, Oral History of Kentucky Farm Families, Kentucky Oral History Commission, 1991-1993 ($67,000). Principal Investigator, Food Consumption and Nutrition Concerns in Sustainable Agricultural Development, Individual Planning Grant for the Sustainable Agriculture and Natural Resource Management Collaborative Research Support Program. 10/91-1/92, USAID ($15,000). Co-investigator (with Jane M. Kotchen, Fred Danner and Melody Noland), Childhood Nutrition, Physical Activity and Cardiovascular Health in Bourbon County, Kentucky, NIH/NHLBI, HL-35100 9/85 - 6/91. ($845,000) Principal Investigator, Nutrition in Agriculture, Cooperative Agreement with The University of Arizona and the Nutrition Economics Group of the U.S. Department of Agriculture, USAID Office of Nutrition DAN-5110-A-9095-00, 1986-1990 ($550,000). Co-Investigator (with Milton Coughenhour, Billie DeWalt, Lawrence Busch, and William Lacey), Sociocultural Constraints in the Production and Consumption of Grain Sorghum and Pearl Millet in Less Developed Countries, USAID Collaborative Research Support Program Grant No. AID/DSAN/XII-G-0149: 11/79 to 6/86. ($1,200,000). LANGUAGE PROFICIENCY: Spanish - Fluency in reading, writing and speaking. Portuguese - Reading good, spoken fair. French - Reading only. STUDY SECTIONS, CONSULTING, NATIONAL COMMITTEES, AND INVITED WORKSHOPS AND CONFERENCES 1991-present: Member IIE Fulbright/ National Geographic Digital Story Telling Ward Review Panel, 2014 –present. Member IIE Fulbright Brazil Scholars Review Panel 2011-present. Team Leader, Review team for the Latin American Studies Program at Temple University, April-July 2009. Member, Administrative and Management Review Team for the Integrated Pest Management Collaborative Research Support Program (Virginia Tech) for USAID, November 2007 – April 2008. 144 Member, Administrative and Management Review Team for the Sustainable Agriculture and Natural Resource Management Collaborative Research Support Program (Virginia Tech) for USAID, November 2007 – April 2008. Reviewer of Screen Plays for Carnegie Mellon/Sloan Foundation Program for the Inclusion of Science in Plays and Screenplays 2002-2011. Member, Fulbright Peer Review Committee for the Western Hemisphere 2000-2003. Member, Special Emphasis Panel for Research on Vulnerable Populations, Agency for Health Care Policy Research, May 1999. Member, Dissertation and Post-doctoral Research Review Panel, Agency for Health Care Policy Research, June 1998. Member, Administrative Management Review Team, Integrated Pest Management Collaborative Research Support Program (Virginia Tech), USAID, September – October 1997. Member, National Science Foundation, Dissertation Grant Review Panel, Cultural Anthropology, 1995 -1997 Participant, Invited Workshop: Food-Based Approaches to Preventing Micronutrient Malnutrition: Setting an International Research Agenda. Nov. 5-9, 1995 Member Review Panel for Scholarships for Latin American Students Wishing to Study in the United States, Inter-American Foundation 1994-1996. Member, Special Emphasis Panel, CDC/NCHS Minority Health Grants. September 1995. Member, Independent Review Committee of the Inter-American Foundation Graduate Study Fellowship Program for Latin American and Caribbean Citizens 1994-1997. Member, Special Study Section on Community Prevention and Control, National Institutes of Health, 1993-1994. Member, Expert Panel to Review Year 1 Work Plan for the Integrated Pest Management Collaborative Research Support Project, November 1993. Member, Expert Panel Meeting on Behavioral and Social Factors in Disease Prevention in Developing Countries, National Research Council, Committee on Population, June 14-15, 1993, Washington, D.C. Member, Priority Expert Panel, Health Promotion for Children and Adolescents, National Center for Nursing Research 1992. Participant, "Integrated Strategies for Controlling Micronutrient Malnutrition: A Technical Workshop", International Life Sciences Institute/Centers for Disease Control/Carter Center, Atlanta, GA, November 7-9, 1991. Consultant panel member, Review of the Policy Implications of the Nutrition Collaborative Research Support Program, Office of Nutrition, US Agency for International Development, August 11-13, 22- 23, 1991. PUBLICATIONS: Books and Monographs: DeWalt, Kathleen Musante. Nutritional Strategies and Agricultural Change in a Mexican Community. Ann Arbor: UMI Research Press, 1983. (Later: University of Iowa Press) DeWalt, Billie R. and Kathleen Musante DeWalt. Sistemas de Cultivo en Pespire, Sur de Honduras: Un Enfoque de Agroecosistemas. Estudios Antropologicos e Historicos # 4 del Instituto Hondureño de Antropologia e Historia: Tegucigalpa, 1984. 145 Bryant, Carol, Anita Courtney, Barbara Markesberry, and Kathleen Musante DeWalt. The Cultural Feast: An Introduction to Food, Society and Change. Atlanta: West Publishing Co. 1985. DeWalt, Kathleen Musante and Billie R. DeWalt. Participant Observation. Altimira Press, 2002. Bryant, Carol, Kathleen Musante DeWalt, Anita Courtney, and Jeffrey Schwartz. The Cultural Feast: An Introduction to Food, Society and Change Second Edition. Atlanta: West Wadsworth, 2003. DeWalt, Kathleen Musante and Billie R. DeWalt. Participant Observation Second Edition. Altimira Press, 2011. Chapters: DeWalt, K.M., and G.H. Pelto. Food use and household ecology in a Mexican community. In T. Fitzgerald (Ed.), Nutrition and Anthropology in Action, pp. 79-93. The Hague: Van Gorcum, 1977. DeWalt, B.R. and K.M. DeWalt. Differential agricultural change in a bicultural community. In, F. Sanchez-Camara and Felipe Ayala (eds.) Concepts for Communication and Development in Bilingual and Bicultural Communities. Mouton: The Hague, 1979. DeWalt, K.M., P.B. Kelly, and G.H. Pelto. Nutritional correlates of economic microdifferentiation in a highland Mexican community. In N. Jerome, R. Kandel, and G.H. Pelto (Eds.), Nutritional Anthropology (Vol. 1), pp. 205-221. New York: Marcel Dekker, 1980. DeWalt, B.R. and K.M. DeWalt. Stratification and decision making in the use of new agricultural technology. In P.F. Barlett (Ed.), Agricultural Decision Making: Anthropological Contributions to Rural Development. New York: Academic Press, 1980. DeWalt, K.M. Usos del sorgo en Honduras: El caso de Pespire. Proceedings of the Grain Quality Workshop for Latin American. INTSORMIL, INIA, ICRISAT, 1983. DeWalt, K.M. Nutritional strategies and farming systems research in southern Honduras: The International Sorghum and Millet Project (INTSORMIL). In C.B. Flora, Animals in the Farming System: Proceedings of the Farming Systems Research Symposium. Manhattan, Kansas: Kansas State University, 1984. DeWalt, K. M. El lugar de la investigación en sistemas de cultivos en el tratamiento de asuntos del sorgo como alimento humano. In C. Paul and B.R. DeWalt (Eds.), El Sorgo en Sistemas de Producción en América Latina. México: INTSORMIL/CIMMYT, 1985. DeWalt B.R. and K.M. DeWalt. El contexto socioeconómico para la investigación sobre el sorgo en el Sur de Honduras. In C. Paul and B.R. DeWalt (Eds.), El Sorgo en Sistemas de Producción en América Latina. México: INTSORMIL/CIMMYT, 1985. DeWalt, K.M. Diet and agricultural development: the dietary correlates of changes in agricultural strategies in a Mexican community. In Carol Hill (Ed.), Alternative Health Care Policies: An Applied Social Science Perspective, pp.71-93. Athens, Georgia: University of Georgia Press, 1986. Reeves, E., B.R. DeWalt and K.M. DeWalt. Applied anthropology and farming systems research in the International Sorghum/Millet Project. In R.M. Wulff and S.J. Fiske (Eds.) Anthropological Praxis: Translating Knowledge into Practice. Boulder, CO: Westview Press, 1987. DeWalt, Kathleen M. and B.R. DeWalt. Including nutritional concerns in agricultural research in the International Sorghum/Millet Project. In, C. McCorkle (ed.) Social Sciences in International Agricultural Research: Lessons from the CRSPs. Boulder, CO: Lynne Rienner Publishers, 1989. DeWalt, Kathleen. Integrating nutritional concerns into adaptive small farm research programs. In, D. MacMillan (ed.) Anthropology and Food Policy. Athens, Georgia: University of Georgia Press, 1991. 146 DeWalt, B.R. and K.M. DeWalt. The results of Mexican agriculture and food policy: Debt, drugs and illegal aliens. In S. Whiteford and A.E. Ferguson (Eds.) Harvest of Want:Hunger and Food Security in Central America and Mexico. Boulder, CO: Westview Press, 1991. DeWalt, Kathleen and Billie R. DeWalt. Agrarian reforms and the food crisis in Mexico: Microlevel and macrolevel processes. In, Poggie, J. J., B. R. DeWalt and W. W. Dressler (eds.) Anthropological Research Process and Application. Albany, NY: SUNY Press, 1992. DeWalt, Kathleen M. Nutrition in anthropological research and development: A case study from Southern Honduras. In, J. van Willigen, B. Rylko-Bauer, and A. McElroy (eds.) Making Our Research Useful: Case Studies in Utilization of Anthropological Knowledge, Boulder, CO: Westview Press, 1992 DeWalt, Kathleen Musante and Billie R. DeWalt Participant Observation. In, H. Russell Bernard (ed.) Handbook for Methods in Cultural Anthropology. Altamira Press, 1998 DeWalt, Kathleen Musante Medical anthropology In, Alen Hedblad (ed.) The International Encyclopedia of the Social Sciences, 2nd Edition. Thompson, 2007. Musante, Kathleen Participant Observation. In, H. Russell Bernard and Clarence (ed.) Handbook for Methods in Cultural Anthropology Second Edition. Altamira Press, 2014. Articles in Peer Reviewed Journals and Peer Reviewed Abstracts: DeWalt, K.M. The illnesses no longer understand: Changing conceptions of health and curing in a rural Mexican community. Medical Anthropology Newsletter, 8 (2): 5-11, 1977. DeWalt, K.M. Diet as adaptation: Looking for nutritional strategies. Federation Proceedings, 40 (11): 2606-2610, 1981. DeWalt, K.M. Income and dietary adequacy in an agricultural community. Social Science and Medicine, 17 (23): 1877-1886, 1983. DeWalt, K.M. and K.S.Thompson. Nutritional anthropology and farming systems research in Southern Honduras. Practicing Anthropology, 5 (3): 15-16, 1983. DeWalt, K.M. and J. van Willigen. Research priorities for Medical Anthropology in the 1980's, Social Science and Medicine, 18: 845-846, 1984. DeWalt, K.M. and B.R.DeWalt. Nutrition and agricultural production in Southern Honduras. Food and Nutrition Bulletin, 9:36-45, 1987. DeWalt, B.R., K.M. DeWalt, J.C. Escudero, and D. Barkin. Agricultural modernization and small farmer welfare: evidence from four Mexican communities. Food and Nutrition Bulletin, 9:46-52, 1987. DeWalt, Kathleen and Jorge Uquillas. Potato production and consumption in the sierra of Ecuador: A diagnostic survey by the Nutrition and Agriculture Cooperative Agreement. Culture and Agriculture, No. 39: 6-11. DeWalt, Kathleen, Sandra D'Angelo, Molly McFadden, Frederick Danner, Melody Noland, Jane M. Kothchen. The use of itemized register receipts for analysis of household food acquisition patterns. Journal of the American Dietetics Association, 90: 559-562, 1990. Noland, Melody, Fred Danner, Kathleen M. DeWalt, Molly McFadden and Jane Kotchen. The measurement of physical activity in young children. Research Quarterly for Exercise and Sport, 61(2): 146-153, 1990. 147 DeWalt, K.M., B.R. DeWalt, J.C. Escudero, and D. Barkin. The nutrition effects of shifts from maize to sorghum production in four Mexican communities. Food Policy, October 1990: 395 - 407. Danner, Fred, Melody Noland, Molly McFadden, Kathleen DeWalt, and Jane M. Kotchen. Description of the physical activity of young children using movement sensor and observation techniques. Pediatric Exercise Science, 3:11-20, 1991. Kotchen, Jane M. Fred W. Danner Melody Noland, and Kathleen M. DeWalt. Impact of television viewing time on disease risks in young children. Circulation 85: (2) 878-878, 1992. Moore, AE, William R. Leonard and Kathleen M. DeWalt. A comparison of 2 dietary methods to evaluate food consumption of rural agricultural based communities in highland Ecuador. American Journal of Human Biology 4(1) 146-147, 1992. DeWalt, Kathleen. Nutrition and the commercialization of agriculture: Ten years later. Social Science and Medicine, 36: 1407-1416, 1993. Roos, Gun M., Sara A. Quandt, and Kathleen M. DeWalt. Meal Patterns in the Elderly in Rural Kentucky. Appetite 21:295-298, 1993. Popyach JB, Quandt SA, DeWalt KM The Relationship of gardening and food preservation to food intake and dietary status of the rural elderly FASEB Journal 7 (3): A82-A82 part 1 Feb. 19 1993 Leonard, William R., Kathleen M. DeWalt and Jorge Uquillas. Ecological correlates of dietary consumption and nutritional status in highland and coastal Ecuador. Ecology of Food and Nutrition 31: (1-2) 67-85, 1993. Leonard, William R., Kathleen DeWalt, and Jorge Uquillas. Variation in children's growth status in highland and coastal Ecuador. Ecology of Food and Nutrition, 1994. Leonard, William R., Kathleen M. DeWalt, Jorge E. Uquillas and Billie R. DeWalt. Diet and nutritional status among cassava producing agriculturalists of coastal Ecuador. Ecology of Food and Nutrition, 32:113-127. 1994. Quandt, Sara, Joan Popyach, and Kathleen DeWalt. Home gardening and food preservation practices of the elderly in rural Kentucky, Ecology of Food and Nutrition, 1994. Woodill, LM, VA Galloway, WR Leonard, KM DeWalt, J. Stansbury and MK McCaston. The influence of food acquisition and development of Ecuadorian children. Human Biology Council Abstracts. American Journal of Human Biology 7:138, 1995. Leonard, William R., Kathleen M.DeWalt, James P. Stansbury and M. Katherine McCaston. Growth Differences Between Children of Highland and Coastal Ecuador. American Journal of Physical Anthropology, 98: 47-57, 1995. Quandt, Sara, Mara Z. Vitolins, Kathleen M. DeWalt, Gun Roos. Meal patterns of older adults in rural communities: Life course analysis and implications for undernutrition. Journal of Applied Gerontology, 16(2):152-171, 1997. Stansbury, James P., William R. Leonard and Kathleen M. DeWalt. Caretakers, child care practices, and growth failure in highland Ecuadorian children. Medical Anthropology Quarterly. 14(2): 224-241, 2000. Leonard, William R., Kathleen M. DeWalt, James P. Stansbury and Mary Catherine McCaston. Influence of dietary quality on the growth of highland and coastal Ecuadorian children. American Journal of Human Biology 12:825-837, 2000. Documet, Patricia I., Laura Macia, Richard Scaglion, and Kathleen M. DeWalt. Latinos and collectivism: parallels between health and legal issues. Annals of Behavioral Medicine 43: Pp. S125- S125, 2012. 148 Book Reviews: DeWalt, K.M. and B.R. DeWalt. Review of Medical Decision Making in a Mexican Village by James C. Young, Man, (Journal of the Royal Anthropological Society) 17: 385-384, 1982. DeWalt, K.M. Review of The Anthropology of Food in Rural Igboland, Nigeria by L. C. Okere, American Anthropologist, 86: 716-714, 1984. DeWalt, K.M. Review of Food and the Social Order, M. Douglas (ed.) and Ethnic Foodways, Mussell and Brown (eds.), American Anthropologist 88: 738-739, 1986. DeWalt, K. M. Review of Food Energy in Tropical Ecosystems by D. Cattle and K. Schwerin (eds.), American Anthropologist 89:238-239, 1987. DeWalt, K.M., Review of Diet and Domestic Life in Society by A. Sharman, J. Theophano, K. Curtis and E. Messer (eds.) American Anthropologist, 1993. DeWalt, K.M. Review of The Sage Handbook of Fieldwork by Hobbs and Wright. Journal of the Royal Anthropological Institute, 13 (1) pp.250-251, 2006. DeWalt, K. M., Review of The Struggle for Maize: Campesinos, Workers, and Transgenic Corn in the Mexican Countryside. Contracorriente, 2012. DeWalt, K. M. Review of The Omnivorous Mind: Our Evolving Relationship With Food, American Journal of Physical Anthropology 151(1): 164-165, 2013. DeWalt, K.M.Review of Revolutionary Medicine: Health and the Body in Post-Soviet Cuba. P. Sean Brotherton, American Ethnologist 40 (3), 2013. DeWalt, K.M. Review of Emotions in the Field. Anthropology and Humanism, 38(2), in press Technical Reports: DeWalt, Billie and DeWalt, K. M. Socioeconomic Constraints to the Production, Distribution and Consumption of Sorghum in Southern Honduras. INTSORMIL, Farming Systems Research in Southern Honduras. Report No. 1, 1982. Lexington, KY.: University of Kentucky Experiment Station. DeWalt, K.M. Sorghum consumption and diet in Southern Honduras. In J.F. Winn (Ed.), INTSORMIL: Fighting Hunger with Research. A five year technical report of the Grain Sorghum/Pearl Millet Collaborative Research Support Program, Lincoln Nebraska, 1985. Bouis, Howarth, Kathleen DeWalt, Eileen Kennedy, Per Pinstrup-Andersen, Isabel Nieves, and Joachim von Braun. Conceptual Framework for a Research Network on the Income and Nutrition Effects of Increasing Commercialization of Semi-Subsistence Agriculture. IFPRI, Washington, D.C., 1985. Thompson, Karen S., Kathleen M. DeWalt, and Billie R. DeWalt. Household Food Use in Three Rural Communities in Southern Honduras. INTSORMIL, Farming Systems Research in Southern Honduras. Report No. 2. Lexington, KY.: University of Kentucky Experiment Station, 1985. Fordham, Miriam, Billie R. DeWalt, and Kathleen M. DeWalt. The Economic Role of Women in a Honduran Peasant Community. INTSORMIL, Farming Systems Research in Southern Honduras. Report No. 3. Lexington, KY.: University of Kentucky Experiment Station, 1985. DeWalt, Kathleen. Case Studies in Nutrition in Agriculture. Conceptual Background, Criteria for Case Selection and Case study Format. NEG/TA/OICD/USDA Washington, D.C.: USDA, 1987. DeWalt, Kathleen. Case Studies in Nutrition in Agriculture. Case Study # 1, The Adaptive Crops Research and Extension Project, Sierra Leone. NEG/TA/OICD/USDA Washington, D.C.: USDA, 1987. 149 DeWalt, Kathleen. Case Studies in Nutrition in Agriculture. Case Study # 2, The International Sorghum and Millet Project, Honduras. NEG/TA/OICD/USDA Washington, D.C.: USDA, 1987. DeWalt, Kathleen, Jorge Uquillas, Charles Crissman. Potato Production and Consumption in the Sierra of Ecuador. NEG/TA/OICD/USDA Washington, D.C.: USDA, 1988. Uquillas, Jorge E., Charles Crissman, Kathleen M. DeWalt, and Warren Peterson. Aspects of the Potato Food System of the Sierra of Ecuador. Quito Ecuador: FUNDAGRO Working Paper Series, 1990. Billie R. DeWalt, Jorge Uquillas, Kathleen M. DeWalt, William Leonard, James Stansbury. Dairy Based Production and Food Systems in Mejia and Salcedo: The Research, Extension and Education Project Baseline Surveys, Report #1. FUNDAGRO: Quito, 1990. Uquillas, Jorge, James Stansbury, Billie R. DeWalt, William Leonard, Kathleen M. DeWalt. Coffee Based Production and Food Systems in Manabi and Selected Areas of the Coast of Ecuador: The Research, Extension and Education Project Baseline Surveys, Report #2. FUNDAGRO: Quito, 1990. DeWalt, Kathleen M., William Leonard, Billie R. DeWalt, Jorge Uquillas, and James Stansbury. Food Consumption and Production Systems in the Cassava Program - Manabi: The Research, Extension and Education Project Baseline Surveys, Report #3. FUNDAGRO: Quito, 1990. Frankenberger, Timothy, Kathleen M. DeWalt, Judith Balderston, Eileen Kennedy and Pauline Peters, Proceedings of the Agriculture-Nutrition Linkage Workshop Vols. 1 and 2. Washington, D.C.: USAID. DeWalt, Kathleen, Food Security and Nutrition Impacts of Non-traditional Agricultural Exports in Latin America, report for the Nutrition and Health Sustainability Project, USAID. August 1993. DeWalt, Kathleen, Susan Poats, and Ravi Sharma. Ecological Zone and Poverty: Food Use In Three Regions Of The El Angel River Watershed, report to the Mac Arthur Foundation and available on the project DVD, 2007. Lucas, DeWalt and Ortman, Final Report of the Administrative and Management Review Team for the Integrated Pest Management Collaborative Research Support Program (Virginia Tech) for USAID, April 2008, 135 pp. Lucas, DeWalt, Gregorsen, Final Report of the Administrative and Management Review Team for the Sustainable Agriculture and Natural Resource Management Collaborative Research Support Program (Virginia Tech) for USAID, April 2008, 141 pp. Papers, Colloquia, Invited Lectures (since 1991): Quandt, Sara and Kathleen DeWalt. Nutritional risk among older adults in two Kentucky counties. Paper presented at the annual meetings of the American Anthropological Association, Chicago, November 20 - 24, 1991. Gun Roos, Sara A. Quandt and Kathleen M. DeWalt. Continuity and change in meal patterns of the elderly in rural Kentucky. Paper presented at the Sixth Annual Meeting of the Association for the Study of Food and Society, East Lansing, Michigan, June 4-7, 1992. Popyach, Joan, Sara A. Quandt, and Kathleen M. DeWalt. Gardening and preservation practices of the elderly in rural Kentucky. Paper presented at the Sixth Annual Meeting of the Association for the Study of Food and Society, East Lansing, Michigan, June 4-7, 1992. DeWalt, K.M.. The Nutrition and income effects of policies promoting non-traditional agricultural exports in Latin America. Paper presented at the annual meeting of the American Anthropological Association, Dec. 1992. 150 DeWalt, KM. S. Quandt and J. Popyach. Food security and nutritional risk in the rural elderly. Poster presented at the annual meeting of the American Anthropological Association, Dec. 1992. Quandt, SA, J. Popyach and K. M. DeWalt. Social eating and meal patterns among the rural elderly. Poster presented at the annual meeting of the American Anthropological Association, Dec. 1992. Quandt, Sara A., Kathleen M. DeWalt and Joan B. Popyach. Solitary eating among older adults in rural communities. Paper presented at the Seventh Annual Meeting of the Association for the Study of Food and Society, Pennsylvania State University June 3-6, 1993. Leonard, William, Kathleen M. DeWalt and Jorge E. Uquillas. Ecology of childhood growth and nutritional status in highland Ecuador. Paper presented in the symposium: Landscapes of Health in the Andes, at the annual meeting of the American Anthropological Association, Dec. 1993. DeWalt, Kathleen M., Sara A. Quandt and Joan B. Popyach. Gender differences, living arrangements, and nutritional risk in older adults in rural Kentucky. Poster presented at the annual meeting of the American Anthropological Association, November 1993. DeWalt, Kathleen. Greasy Beans with Canola Oil Seasoning; Qualitative Approaches to Understanding the Diet of Older Adults in Rural Communities. Invited presentation, Virginia Politechnic Institute, February, 1994. DeWalt, Kathleen, William Leonard, Maura Mack and Coral Wayland. Women's and men's education and child health in two household systems. Paper presented at the Annual Meetings of the Society for Applied Anthropology, April 13-17, 1994. Leonard, William R., Kathleen DeWalt, James Stansbury and M. Katherine McCaston. Biocultural determinants of nutritional status and growth failure in rural Ecuador. Paper presented in the symposium: Biocultural Models: A Coming of Age, at the annual meeting of the American Anthropological Association, Dec. 1994. DeWalt, Kathleen M. the health of development: Issues in development and the sustainability of the quality of life. Paper present in the symposium: Dilemmas and Dimensions of Development, Annual Meeting of the American Anthropological Association, Dec. 1994. Stansbury, James, Kathleen M. DeWalt, and William Leonard. Sociocultural and ecological predictors of child growth failure in highland Ecuador. Paper presented at the Annual Meetings of the Society for Applied Anthropology, Albuquerque, NM, April 1995. DeWalt, Kathleen M. Indigenous Stocks of knowledge: participatory approaches to generating knowledge for nutrition. Paper presented in the symposium: "Capacities for Improving Nutrition in the Post-Cold War Climate", at the Annual Meetings of the Society for Applied Anthropology, Baltimore, Maryland, March 27-31, 1996. DeWalt, Kathleen. Greasy Beans with Canola Oil Seasoning. Invited presentation, The Pennsylvania State University, April 1, 1996 DeWalt, Kathleen. Women’s income, social power and child welfare in Manabí. Ecuador . Paper presented in the Symposium: Women’s Associations, Social Power and Child Welfare in Manabi, Ecuador , Annual Meetings of the Society for Applied Anthropology, April 1999. Poats, Susan V, and Kathleen M. DeWalt. Examining the Impact of “Women’s Projects” : Does Ten Years of Development Have and Impact? Paper presented in the Symposium: Women’s Associations, Social Power and Child Welfare in Manabi, Ecuador , Annual Meetings of the Society for Applied Anthropology, April 1999. DeWalt, Kathleen. Gender and Research In Medical Anthropology. Paper presented in the Symposium. Feminism in Anthropology, COSWA Invited Session, Annual Meetings of the American Anthropological Association. November 2000. 151 DeWalt, Kathleen. Women’s Income and Domestic Violence in Manabí, Ecuador. Paper presented at the Annual Meetings of the Society for Applied Anthropology, March, 2004. DeWalt, Kathleen. Becoming a Socia. Invited presentation, University of Arizona, August 2004. DeWalt, Kathleen. Becoming a Socia. Invited presentation, Chatham College 2004. DeWalt, Kathleen Musante and John van Willigen. Old Ham and Sweet Sorghum Syrup: The Political Economy of the Emergence of Kentucky Cuisine. Paper read at the Annual Meetings of the American Anthropological Association, December 2004. DeWalt, Kathleen Musante. Becoming a Socia. Invited presentation, Arizona State University, February 2007. DeWalt, Kathleen Musante. Balancing Career and Family. Invited presentation, American Anthropological Associations Annual Meetings, November 2007. DeWalt, Kathleen Musante. Gender and Administration in Latin American Studies. Invited Presentation, in the Panel: Borrando desigualdades: experiencias de mujeres directivas en universidades de Cuba y EU. Latin American Studies Association Meetings, Rio de Janeiro, Brazil, June 2009. DeWalt, Kathleen Musante, Susan V. Poats, Hernan Caballero. “…Oh, my child, those were the beautiful times...”: Examining the Impact of Participating in Rural Micro-Enterprises on Manaba Women’s Place in the Family and the Community, Twenty Years Later, Paper presented at the Society for Applied Anthropology Meetings March 20, 2011. Musante, Kathleen“…Oh, my child, those were the beautiful times...” Presentation in the Panel Empowering Women, Carnegie Museum of Natural History, April, 2013. 152 Jessica Fanzo, PhD January 2013 – present Assistant Professor of Nutrition, Department of Pediatrics & the Institute of Human Nutrition Adjunct Associate Professor, School of International Public Affairs (SIPA) Senior Advisor of Nutrition Policy, Center on Globalization & Sustainable Development (CGSD) Columbia University, New York, NY November 2011 – December 2012 Program Officer, United Nations REACH (Renewed Efforts Against Child Hunger) The United Nations World Food Programme, Rome Italy June 2010 – November 2011 Senior Scientist and Director of Nutrition Programmes, Bioversity International Consultative Group on International Agricultural Research (CGIAR) Centre, Rome, Italy May 2007 – May 2010 Director of Nutrition, Center for Global Health and Economic Development of the Earth Institute Food and Nutrition Security Program Coordinator, MDG Center Columbia University, Nairobi, Kenya and New York, NY May 2004 – May 2007 Program Officer for Medical and HIV/AIDS Research, Doris Duke Charitable Foundation New York, NY Sept 2000 – May 2004 Postdoctoral Immunology Training Fellow/Stephen I. Morse Recipient, Department of Medicine Columbia University, New York, NY May 1995 – August 1997; June 1998 - June 2000 Research Assistant/Ph.D. Student, Department of Nutritional Sciences University of Arizona, Tucson, AZ CONSULTANCIES and PROJECT COLLABORATIONS 2014: UN World Food Programme, Rome, Italy Authoring discussion paper and strategy that reflects global knowledge and evidence, as well as WFP programming experience, around nutrition-sensitive programming. This strategy serves to outline the conceptual tenets of nutrition-sensitive programming in WFP and an action plan for moving forward. 2014: World Health Organization, Geneva, Switzerland Advising on the Framework for Maternal Infant and Young Child Nutrition and assisted in establishing the WHO/CIDA surveillance project on Accelerating Nutrition Improvements in sub￾Saharan Africa. 153 2014: UNICEF Stunting Country Case Studies and Policy Briefs, New York, NY Authoring two research studies examining how countries reduced stunting over a ten year time period. 2013: Scaling Up Nutrition (SUN) Nutrition Sensitive Modeling, New York NY Led the research on developing a quantitative model for mapping nutrition sensitive interventions to assist with country decision-making. 2013: Myanmar Sustainable Development Center, Yangon, Myanmar Collaborated in a group to do an assessment of the future of sustainable development in the fields of nutrition, climate change, and rural livelihoods for Myanmar and Irish Aid. 2013: UN Standing Committee on Nutrition (UNSCN), Geneva and Nepal Authored a synthesis paper of a policy analysis examining the “nutrition sensitivity” of food and agriculture policies in 8 countries including Brazil, Malawi, Mozambique, Nepal, Senegal, Sierra Leone, South Africa and Thailand. Also led the research and authored the Nepal case study. 2013: Global Alliance for Improved Nutrition (GAIN), Washington DC Advised and supported GAIN’s Agriculture/Nutrition and Monitoring, Learning and Research teams in the design of an overall M&E framework and Key Performance Indicators for the Agriculture/Nutrition work stream, including M&E plan and baseline survey for the REGAL-IR project in Kenya focusing on pastoralist livelihoods. 2012-2013: World Bank Secure Nutrition Platform and Global Forum for Rural Advisory Services (GFRAS), Washington DC Conducted a mapping exercise on home economics and nutrition in extension and advisory services. The report is intended to inform multi-sectoral planning exercises Scaling Up Nutrition (SUN) countries are undertaking. 2012-2013: UNICEF, Nutrition Section, New York, NY Drafted the global strategy for UNICEF’s nutrition programs. The strategy should be ready for dissemination in 2013. 2012: Seeds of Life, Ministry of Agriculture and Fisheries, Dili, Timor Leste Developed a nutrition sensitive strategy for Timor Leste Ministry of Agriculture and Fisheries and provided on-going technical support to the nutrition programmes within Seeds of Life’s overarching food security initiatives in the country. 2012: UN World Food Programme (WFP), Office of Evaluation, Rome, Italy Provided nutrition and food security expertise on impact evaluation of WFP’s Food for Assets for Livelihoods Resilience in five countries including Guatemala, Bangladesh, Nepal, Uganda and Senegal. 2011: Institute for Development Studies, Brighton, England 154 Collaborated in a qualitative study that analyzed the political and institutional determinants of delivering a national multisectoral response in six countries. 2010: UNDP, New York, NY Authored the nutrition chapter for the UN Human Development Africa Report in 2012. 2009: United Nations Development Group Working Groups on the MDGs, Rome Italy Authored the report for the UNDG Working Groups on the MDG1. The report was commissioned by the UN World Food Programme and is entitled: An evaluation of progress toward the millennium development goal one hunger target. EDUCATION PhD, Interdisciplinary Nutritional Sciences: University of Arizona (2000) BS, Nutritional Sciences, University of Arizona (1993) Advisory and Working Groups Sustainable Development Solutions Network, member of Thematic Group 7 on Sustainable Agriculture and Food Production: 2014 The Sackler Institute of the New York Academy of Sciences, Technology and Innovation for Food and Agriculture Working Group: 2014 Feeding the World Ethically International working group, Global Food Ethics Project of Johns Hopkins University/Stavros Niarchos Foundation: 2014 ILSI Research Foundation Center for Integrated Modeling of Sustainable Agriculture and Nutrition Security (CIMSANS) working group: 2014 Earth Institute’s Agriculture and Food Center advisor member: 2013 to present Columbia University’s School of International Public Affairs Masters in Development Practice Faculty Advisory member: 2013 to present Community for Zero Hunger advisor: 2013 to present Food Tank Advisory Board member: 2013 to present Bioversity American Board member: 2013 to present Daniel & Nina Carasso Foundation Scientific Committee member: 2012 to present International Conference on Dietary Assessment & Methods Steering Committee: 2011 Scaling Up Nutrition (SUN) Working Group National Capacities and Systems, 2010 Nutrition & Agriculture Technical Working Group for the UN Special Advisor on the MDGs Secretariat: 2010 CGIAR Agriculture, Nutrition & Health Research Program Steering Committee Member: 2010 Task Force on the Cross Cutting Initiative on Nutrition & Biodiversity under the Convention on Biological Diversity Steering Committee member: 2010 to 2011 United Nations Standing Committee on Nutrition Working Group on Household Food Security Working Group Member: 2009 IASC Global Nutrition Cluster Member: 2008 to present Agriculture and Health Research Platform Member: 2008 to 2009 Columbia University’s Food and Nutrition Council member: 2007 to present Panel Moderator Health Research Alliance Meeting, Program Evaluation Session, 2006 Echoing Green Fellowship Final Reading Committee, Echoing Green Foundation, 2006 155 Editorial Boards Editorial Board, Global Food Security Journal, 2014 Editorial Board, Frontiers in Nutrition and Environmental Sustainability, 2014 Reviewer for Earthscan and Routledge Environment and Sustainability, 2012 to present Associate Editor, Food Security Journal, 2009 to present Ad-hoc peer-reviewer for Journal of Clinical Immunology, 2000 to 2004 Evaluation Teams USAID Feed the Future, Improving Nutrition of the Poor, Young Children and Women, through Grain Legume Consumption, Proposal Review Panel: 2013 USDA National Institute of Food and Agriculture’s Proposal Review Panel for the Agriculture and Food Research Initiative Food Security grant program: 2013 PROFESSIONAL AFFILIATIONS Honorary Fellow, Bioversity International, 2013 BCG Professional Forum, World Food Programme, 2012 Peer leader and Organizer for Global Food Systems Forum, 2008 Terra Madre United States Delegate, 2008 Slow Food NYC, 2008 Global Health Council, 2006 to 2008 American Public Health Association (APHA), 2006 to 2008 Public Health Association of New York City (PHANYC), 2006 to 2008 African Grantmakers Affinity Group, 2005 Health Research Alliance (HRA), 2004 to 2007 American Association for Immunologists (AAAI), 2000 to 2004 New York Academy of Sciences (NYAS), 2003 to present American Association for Cancer Research (AACR), 1999 to 2001 Society of Experimental Biology and Medicine (SEBM), 1997 to 2004 American Society for Nutritional Sciences (ASNS), 1999 to present American Society for Clinical Nutrition (ASCN), 1999 to 2005 Society for International Nutrition Research (SINR), 1999 to 2005 AWARDS & GRANTS 2014: CGIAR Africa and South Asia Food and Agriculture Grant, Homestead Food Production and Irrigation for Nutrition in Senegal, Principal Investigator (200,000 USD). 2013: Global Alliance for Improved Nutrition (GAIN) grant on Development of Behavior change communication intervention in Northern Kenya arid land pastoralist project, Principal Investigator (35,000 USD). 2012: Premio Daniel Carasso Laureate on “Sustainable food and diets for long term human health” (100,000 Euros). 2012: NAKFI Ecosystem Services grant, “Econutrition within REACH: Incorporating an Ecosystems Approach into the United Nations' Partnership to End Child Hunger”, co- Principal Investigator (100,000 USD). 156 2011: GIZ Grant, Agricultural biodiversity and improved complementary foods in rural Kenya, Principal Investigator (60,000 EU). 2011: Gates Foundation Grand Challenges Grant, Round 7, Role of Wild and Underutilized Foods in Daily Costs of Diets in collaboration with Save the Children UK, Principal Investigator (100,000 USD). 2011: GTZ Grant, Agricultural biodiversity and the nutrition transition in rural and urban populations Kenya, Bioversity, Principal Investigator (300,000 EU). 2010: CTA Grant, Sustainable Diets and Biodiversity, Bioversity, Principal Investigator (30,000 USD). 2010: IDRC Grant, Sustainable Diets, Bioversity, Principal Investigator (40,000 CA). 2010: IDRC Grant, Econutrition, Agrobiodiversity, and Health, Bioversity, Principal Investigator (375,000 CA). 2010: Nestle Foundation, Nutrition and the Millennium Villages, Earth Institute, Grant Writer and Originator (450,000 USD). 2010: Sight and Life Infant and Young Child Feeding Grant, Millennium Villages, Principal Investigator (205,000 USD). 2008: Earth Institute of Columbia University Earth Clinic Grant, Principal Investigator (50,000 USD). 2007: Pershan and Cohen Grant, Millennium Villages, Principal Investigator (250,000 USD). 2007: Project Nutrition Leader on Gates Special Initiative Grant, Millennium Villages Project (450,000 USD). 2004: Keystone Symposium Postdoctoral Scholarship Award, Lymphocyte Activation Series. 2003: Immunology Postdoctoral Training Fellowship Grant, Columbia University, Department of Microbiology. 2002: Stephen I. Morse Fellowship Grant, Columbia University, Department of Molecular Medicine. 2000: Teaching and Research Ph.D. Fellowship in Nutrition, University of Arizona. PUBLICATIONS & PRESENTATIONS PEER REVIEWED ARTICLES Fanzo J (2014) Strengthening the engagement of food and health systems to improve nutrition security: Innovative and equitable solutions to address malnutrition. In Review: Global Food Security. Reinhardt K and Fanzo J. (2014) Addressing Chronic Malnutrition through Multi-Sectoral, Sustainable Approaches: A Review of the Causes and Consequences. In Review: Frontiers Journal. Thow, AM, Fanzo, J, and Negin J (2014) A systematic review of the effect of remittances on diet and nutrition. In Review: Globalization and Health Journal. Johnson J, Fanzo, J and Cogill B (2014) Understanding Sustainable Diets. In Review: Advances in Nutrition. Luckett, B, DeClerck, F, Fanzo, J, Mundorf, A, and Rose, D (2014) Nutritional Functional Diversity: An Indicator to Link Sustainable Agriculture to Healthy Diets. In Review: Public Health Nutrition. Sachs, JD, Remans, R, Smukler, SM, Winowiecki, L, Andelman, SJ, Cassman, KG, Castle, D, DeFries, R, Denning, G, Fanzo, J, Jackson, LE, Leemans, R, Lehmann, J, Milder JC, Naeem, S, Nziguheba, G, Palm, CA, Pingali, PL, Reganold, JP, Richter, DD, Scherr, SJ, Sircely, J, Sullivan, C, Tomich, TP and Sanchez, PA. (2012) Effective monitoring of agriculture: a response. J. Environ. Monitor. Mar;14(3):738-42. 157 Remans R., Pronyk, P., Fanzo, J., Palm, C., Chen, J., Nemser, B., Muniz, M., Radunsky, A., Abay, A., Coulibaly, M., Mensah, J., Wagah, M., Quintana, E., Sachs, S.E., Sanchez, P., McArthur, J., and Sachs, J.D. (2011) A multi-sector intervention to accelerate reductions in child stunting: an observational study from nine sub-Saharan African countries. American Journal of Clinical Nutrition. 94(6):1632-42. Remans, R., Flynn, D., Fanzo, J., Declerck, F., Lambrecht, I., Sullivan, C., Gaynor, K., Siriri, D., Mudiope, J., Mutuo, P., Nkhoma P., and Palm C. (2011) Assessing Nutritional Diversity of Cropping Systems in African Villages. PLoS ONE 6(6):e21235. doi:10.1371/journal.pone.0021235 Fanzo, J. and Pronyk, P. (2011) Road map to 2015: Reviewing global progress towards the Millennium Development Goal One Hunger Target. UN FNB 32(2): 144-158. DeClerk, F., Fanzo, JC, Remans, R. and Palm, C.A., and Deckelbaum, R. (2011) Ecological Approaches to Human Nutrition. UN FNB 32(1): 41S-50S. Fanzo, J. (2011) IFPRI’s 2020 conference on leveraging agriculture for improving nutrition and health: keeping the momentum and translating ideas into action. Food Security. 3(2): 263-265. Burchi, F., Fanzo, J., and Frison, E. (2011) The role of food and nutrition system approaches in tackling hidden hunger. International Journal of Environment and Public Health 8(2), 358-373. Sachs, J., Remans, R., Smukler, S., Winowieiki, L, Andelman, S., Cassman, K, Castle, D., DeFries, R, Denning, G. Fanzo, J. et al. (2010) Monitoring the World’s Agriculture. Nature 466: 558-56. Pronyk, P Palm C; Study investigators (New York): Edwin Adkins, Xaioyi An, Yanis Ben Amor, Matt Berg, Jiehua Chen, Prabhjot Dhadialla, Jessica Fanzo, et al (2010) 09PRT/8648: The Millennium Villages Project: integrating the delivery of health and development interventions and assessing the impact on child survival in sub-Saharan Africa (NCT01125618). The Lancet. http://www.thelancet.com/protocol-reviews/09PRT-8648 Negin, J., Remans, R., Karuti, S. and Fanzo, J. (2009) Integrating a broader notion of food security and gender empowerment into the African Green Revolution. Food Security. 1(3): 351-360. Fanzo, J.C., Yang, W., Jang, S.Y., Gupta, S., Chen, Q., Sidiq, A., Greenberg, S., and Pernis, A.B. (2006) Loss of IBP leads to the spontaneous development of systemic autoimmunity. J. Clinical Investigation. 116(3): 703-714. Fanzo, J.C., Hu, C.H., Jang, S.Y., and Pernis, A.B. (2003) Regulation of Fas-Dependent Apoptosis in Lymphocytes by Interferon Regulatory Factor –4 (IRF-4). J. Exp. Med. 197(3): 1-13. Fanzo, J.C., Lynch, M.P., Phee, H., Hyer, M., Cremesti, A., Grassme, H., Norris, J.S., Coggeshall, K.M., Rueda, B.R., Pernis, A.B., Kolesnick, R., and Gulbins, E. (2003) CD95 rapidly clusters in cells of diverse origins. Cancer Biology and Therapy. 2(4): 392-395. **Front Cover Image** Gupta, S., Fanzo, J.C., Hu, C., Cox, D., Jang, S.Y., Lee, A.E., Greenberg, S.A., and Pernis, A.B. (2003) T cell receptor engagement leads to the recruitment of IBP, a novel guanine nucleotide exchange factor, to the immunological synapse. J. Biol. Chem. 278: 43541-43549. Gupta, S., Lee, A.E., Hu, C., Fanzo, J.C., Goldberg, I., Cattoretti, G., and Pernis, A.B. (2003) Molecular cloning of IBP, a SWAP-70 homologous GEF, which is highly expressed in the immune system. Human Immuno. 64: 389-401. Fanzo, J.C., Reaves, S.K., Wu, J.Y.J., Zhu, L., Cui, L.B., and Lei, K.Y. (2002) p53 tumor suppressor gene and apoptotic target genes, caspase-3 and bax, are influenced by dietary zinc supplementation in human aortic endothelial cells. Am. J. Physiol. Cell Physiol. 283: C631-C638. Hu, C., Jang, S.Y., Fanzo, J.C., and Pernis, A.B. (2002). The modulation of cytokine production in T cells by Interferon Regulatory Factor 4. J. Biol.Chem. 277: 49238-49246. 158 Cui, L.B., Schoene, N.B., Zhu, L., Fanzo, J.C., Alshatwi, A., and Lei, K.Y. (2002) Zinc depletion reduced Egr-1 and HNF-3beta expression and apolipoprotein A-I promoter activity in HepG2 cells. Am. J. Physiol. Cell Physiol. 283: C623 - C630. Fanzo, J.C., Reaves, S.K., Wu, J.Y.J., Zhu, L., Cui, L.B., and Lei, K.Y. (2001) p53 tumor suppressor gene expression and p53-target genes are enhanced by zinc deficiency in normal human bronchial epithelial cells. Am. J. Physiol. 281: C751-C757. Reaves, S.K., Fanzo, J.C., Arima, K., Wu, J.Y.J., Wang, Y.R., and Lei, K.Y. (2000) Expression of the p53 tumor suppressor gene is upregulated by depletion of intracellular zinc in HepG2 cells. J. Nutr. 130: 1688-1694. Reaves, S.K., Wu, J.Y.J., Wu, Y., Fanzo, J.C., Wang, Y.R., Lei, P.P., and Lei, K.Y. (2000) Regulation of intestinal apolipoprotein protein B mRNA editing levels by a zinc deficient diet and cDNA cloning of editing protein in hamsters. J. Nutr. 130: 2166 – 2173. Reaves, S.K., Fanzo, J.C., Wu, J.Y.J., Wang, Y.R., Wu, Y.W., Zhu, L., and Lei, K.Y. (1999) Plasma apolipoprotein B-48, hepatic apolipoprotein B mRNA editing, and apobec-1 mRNA levels are altered in zinc-deficient rats. J. Nutr. 129: 1855-1861. BOOKS & BOOK CHAPTERS Fanzo J (2014) Food Policy and Global Nutrition. In: International Food Law and Policy. Forthcoming. Fanzo J and Bloem M (2014) Value chains for nutrition In: Nutrition in Developing Countries. Forthcoming. Fanzo J (2014) Agricultural biodiversity, nutrition and sustainable food systems (chapter 22) in The Handbook of Agricultural Biodiversity, Earthscan from Routledge, UK. Forthcoming. Fanzo, J, Remans, R and De Clerck, F (2014) Chapter 32: Smallholders, agro-biodiversity and mixed cropping and livestock systems In: Routledge Handbook of Food and Nutrition Security. Forthcoming. Fanzo J and Hunter D, editors (2013). Diversifying Food and Diets: Using Agricultural Biodiversity to Improve Nutrition and Health. Earthscan (as part of the series Issues in Agricultural Biodiversity). Fanzo, J., Negin, J., Remans, R., Pronyk, P. Motuo, P., Wariero, J., Masira, J., Duru, W., Nemser, B., Kim, D., Muniz, M., Palm, C., Sachs, SE, Sanchez, P and Sachs, JD. (2011) A 3 year cohort study to assess an integrated food- and livelihood-based approach to better nutrition in rural western Kenya. In: Combating Micronutrient Deficiency: Food-based Approaches. Edited by B Thompson, and L Amoroso, FAO and CABI Italy. Chapter 4. Fanzo, J., Remans, R. and Sanchez, P. (2011) The Role of Chemistry in Addressing Hunger and Food Security. In: The Chemical Element: Chemistry’s Contribution to Our Global Future, First Edition. Edited by Javier Garcia-Martinez, Elena Serrano-Torregros. Wiley-VCH Verlag GmbH & Co. KGaA. Chapter 2. Remans, R., Fanzo, J.C., Palm, C.A., DeClerk, F. (2011) Human Nutrition as an Ecological Service. In Integrating Ecology into Poverty Alleviation and International Development Efforts: A Practical Guide De Clerck, F., Ingram, JC., Rumbaitis del Rio, C. Eds. Springer, New York, Vol. 1. Chapter 2.1.2. 159 REPORTS & WHITE PAPERS Fanzo, J (2014) Ethical Issues for Human Nutrition in the Context of Global Food Security. White paper for: Global Food Ethics Project: Feeding the World, Ethically. Johns Hopkins University. Forthcoming. Fanzo, J and Cohen, M (2014) A Nutrition Analysis of Food and Agriculture Policies: Eight Country case study. UNSCN, WHO Geneva. Fanzo J (2014) Integration of Nutrition into World Food Programmes’s Food for Assets Programs: A review of five countries. UN WFP. Rome, Italy. Acharya, T, Fanzo, J, Schneeman B et al (2014). Sustainable nutrition security: its fundamental role in food security. The ILSI Research Foundation Center for Integrated Modeling of Sustainable Agriculture and Nutrition Security (CIMSANS). Washington DC. Curran, S, Fanzo, J, Remans, R, Mara, V, Fracassi, P and Denning G (2014) Simulating the Impact of Nutrition Sensitive Approaches to Scaling-Up Nutrition Columbia University, New York, NY. For the Scaling Up Nutrition Movement. Fanzo, J and Andrews, D (2013) Nutrition Impact of Food Systems in Nepal: UNSCN Country Policy Analysis. UNSCN, WHO, Geneva. Fanzo, J., Marshall, Q, Wong, J, Merchan, R, Jaber, M, Souza, A and Verjee, N (2013) The integration of nutrition into extension and advisory services: a synthesis of experiences, lessons and recommendations. Lindau, Switzerland: Global Forum for Rural Advisory Services. Fanzo, J, Curran, S, Mara, V, Remans, R and Denning, G (2013) Simulating the Impact of Multi￾Sectoral Approaches for Improved Nutrition: Columbia University Project for the Scaling UP Nutrition Movement (SUN). Columbia University, New York, NY. Fanzo, J. (2013). Monitoring and Evaluation Framework. GAIN’s Agriculture and Nutrition Programme. GAIN Washington DC. Njoro, J, Anthony, N, de Hoogh, I, Fanzo, J, Chahid, N, Fornah, D, Gboku, M, Kamara, M, Kargbo, A, Koroma, A, Ljunqvist, B, Momoh, J, Osiro, A, Sawi, M, Rhodes, E, Scott, S, Torgerson, S, van Dorp, M and Wiegers E. (2013) An analysis of the food system landscape and agricultural value chains for nutrition: Two case studies from Sierra Leone. UN FAO Expert Paper for International Conference of Nutrition 2 (Rome Italy 2014). Fanzo, J, Curran, S, and Denning G (2013). Nutrition Sensitive Agriculture Strategy for Timor Leste. Seeds of Life, Ministry of Agriculture. Dili, Timor Leste. Termote, C., Cogill, B., Deptford, A., Muguro, S., Kimere, C., Grace, J., Mutangah, J., Mattei, F., and Fanzo, J. (2012). Role of wild, neglected and underutilized foods in reducing the cost of a nutritionally adequate diet in the eastern region of Baringo District, Kenya. Gates Grand Challenges. Bioversity, Rome Italy. Fanzo, J, Cogill, B and Mattei, F (2012) Metrics of Sustainable Diets and Food Systems. Technical Brief Madrid Roundtable. Bioversity and Carasso Foundation. Rome. Mejia Acosta, A and Fanzo, J (2012). Fighting Maternal and Child Malnutrition: Analysing the political and institutional determinants of delivering a national multisectoral response in six countries. Institute for Development Studies. University of Sussex Brighton, England. Haddad, L. Acosta, A, M. and Fanzo, J. (2012). Accelerating Reductions in Undernutrition: What nutrition governance can tell us. Institute for Development Studies In Focus Policy Briefing, Issue 22. University of Sussex Brighton, England. Fanzo J. (2012) Nutrition Background Paper for 2012 UNDP Human Development Report for Africa. UNDP, New York, NY 160 Hodgkin, T, Frison, E, Fanzo, J and Lopez Noriega, I. (2011) Biodiversidad Agricola, Seguridad Alimentaria y Cambio Climatico. Ambiente. No 94. Issue No 1. Fanzo J. and Pronyk P (2010) United Nations Development Group Working Groups on the MDGs. An Evaluation of Progress Toward the Millennium Development Goal One--Hunger Target. Commissioned by World Food Programme. Fanzo, J. (2010) Sector Strategies: Nutrition. In: Harvests of Development in Rural Africa: The Millennium Villages After Three Years. The Earth Institute, Columbia University, Millennium Villages Project, and Millennium Promise. New York, NY. Chapter 2. pp 50-54. Fanzo, J. and Pronyk, P. (2010) The Importance of a Multi-Sectoral Approach to Food and Nutrition Security in Africa. The Forum: Discussing International Affairs and Politics. Spring 2010. Pg 108. IDS, England. Fanzo, J. (2010) Millennium Village’s Home Grown School Meals program. MDG Good Practices Report. UNDP, New York, NY. Remans, R., Fanzo, J. Berg, M., Atkins, E., Mohammed, A., and Sachs, JD. (2010) Climate change SCN News #38. UNSCN Geneva, Switzerland. Fanzo, J. (2010) An Evaluation of Progress Toward the Millennium Development Goal One Hunger Target: A country-level, food and nutrition security perspective. Report for the UNDP MDG Task Force for 2010 MDG Summit. http://www.undg.org/index.cfm?P=327 Fanzo, J. (2009) Adequate nutrition is key factor in African development; MDG Centre Report. Fanzo, J. (2009) How a low-cost meal is allowing more village children to be educated; MDG Centre Report. Ad Hoc Advisory Group to the Madrid Conference on Food Security Smallholder Food Production and Poverty Reduction: Principles for a Financial Coordination Mechanism (FCM) to Support Smallholder Farmers. (2009) Chaired by Prof. Jeffrey Sachs. Fanzo, JC and Haddad, L. Box 10 - Addressing Nutrition through Smallholder Agriculture. Fanzo, J.C. and Gallin, E.K. (2006) The Doris Duke Clinical Scientist Development Award: A Seven￾Year Retrospective and Summary. In Enhancing Philanthropy’s Support of Biomedical Scientists: Proceedings of a Workshop on Evaluation. National Research Council of the National Academies. pp. 11 – 20. MANUALS, GUIDELINES & STRATEGIES UNICEF Global Nutrition Strategy, 2014. Served as main author. Farming Matters, October 2011 issue. Regional Food Systems. Served as editor. Bioversity International Nutrition Strategy, 2011- 2021. Served as the author. Bioversity International Field Manual: A Manual on Implementing Food Systems Field Projects to Assess and Improve Dietary, Nutrition and Health Outcomes, 2011. Served as first and main author. Millennium Villages School Meals Program – A Nutrition Guideline, 2008. Served as first and main author. Millennium Villages Project Nutrition Home Garden Manual, 2008. Served as first and main author. Millennium Villages School Feeding Program Cost Analysis, 2007. Served as first and main author. Millennium Villages Nutrition Manual, 2007. Served as first and main author. ABSTRACTS & PRESENTATIONS Lachat, C, Ionata de Oliveira Granheim, S, Mayer AM, Wagah, M, Schonfeldt, HC, Tontisirin, K, Mahy, L, Wustefeld, M and Fanzo J (2014) The inclusiveness of nutrition in food and agricultural 161 policies in 8 countries. Submitted to: 4th Annual LCIRAH Research Conference Agri-food policy and governance for nutrition and health. London, England. Fanzo, J and Bhurtyal A (2014) Agriculture, food systems and nutrition linkages in Nepal’s current policies. Submitted to: University of Alberta’s Food Security Dialogue on, “Enhancing Food Production, Gender Equality and Nutritional Security in a Changing World.” Alberta, Canada. Fanzo, J (2014) Where the drivers in the political economy of food lies? Consensus Conference on nutrition and cardiovascular disease. World Heart Federation. Ontario, Canada. **Invited Speaker** Fanzo, J (2013) Biodiversity and Sustainable Diets. Nutrition and Sustainability. UNSCN and FAO meeting. Rome, Italy. **Invited Speaker** Fanzo J (2013) The Links between Farming Systems and Human Nutrition: Valuation of externalities. Public Health session; Workshop Session 1, Theme 3. True Cost Accounting for Food and Farming systems. London England. **Invited Speaker** Fanzo J. (2013) Strengthening the engagement of food and health systems to improve nutrition security: Innovative and equitable solutions to address malnutrition. Global Food Security Conference. Netherlands. **Keynote Speaker** Fanzo, J. (2013) Nutrition and environmental sustainability (FAO) parallel symposium. Oral presentation: Nutritional diversity of national food systems linked to child nutrition. NPS1-3/Nutrition and environmental sustainability. IUNS 20th International Congress of Nutrition. Granada, Spain. **Oral Speaker** Fanzo, J. (2013) Daniel & Nina Carasso Foundation satellite symposium. Sustainable food and diets: From theory to evidence – Based successful practice. Oral presentation: Case studies from the field: engaging communities. IUNS 20th International Congress of Nutrition. Granada, Spain. Colnar, M, De, S, Leung, M, Wang, M, Fanzo, J and Denning G (2013) P0276/Nutrition Governance: Five Critical Building Blocks and an Application of These building blocks into specific country contexts. IUNS 20th International Congress of Nutrition, Granada, Spain. Meija Acosta A and Fanzo, J (2013) P01229/Effective Governance and policies to improve nutrition outcomes: a cross comparison of nine country cases. IUNS 20th International Congress of Nutrition, Granada, Spain. Johnston, J, Fanzo, J and Cogill, B (2013) P01679/Understanding Sustainable Diets: Past, present and future efforts to advance sustainable diets. IUNS 20th International Congress of Nutrition, Granada, Spain. Fanzo, J, Luckett, B, Clerck, FD and Rose, D (2013) P03203/Econutrition within REACH: A Framework to assist decision-making in large-scale nutrition interventions. IUNS 20th International Congress of Nutrition, Granada, Spain. Fanzo J (2012) Earth Institute presents the second in the 2012-2013 Sustainable Development Seminar Series on Food Security. New York, NY. **Keynote Speaker** Fanzo, J, Ljungqvist, B, Chahid, N and Perez Zaldivar, A (2012). Monitoring and Evaluating the Country led process of UN coordinated, multisectoral approaches to scaling nutrition. ICDAM, Rome. Fanzo, J. (2011) Global Health, Agriculture and Food Security. Global South-South Development Expo. FAO, Rome, Italy. **Invited Speaker** Fanzo, J. (2011) Improving Nutrition Security through Agriculture: Ensuring Access, Quality and Resilience. Committee on Food Security Event. FAO, Rome, Italy. **Invited Speaker** Fanzo, J. (2011) The role of agricultural biodiversity in diets in the developing world: Improving diet diversity, quality and ecosystem sustainability. FENS, Madrid. **Invited Speaker** Fanzo, J. (2011) World Food Day Conference Gorta. Dublin, Ireland. **Invited Speaker** 162 Fanzo, J. (2011) Educating for Biodiversity. Global Health Conference. Montreal, Canada. **Invited Speaker** Fanzo, J. (2010) Ensuring Nutrition and Agricultural Biodiversity remains central to addressing the MDG One Hunger target. Sustainable Diets. FAO, Rome. **Invited Speaker** Fanzo, J. and Remans, R. (2010) An Integrated, Food-based approach to bettering nutrition: A case study in the Millennium Village of Western Kenya. Unite for Sight Meeting, Yale University, Connecticut. **Invited Speaker** Loeffler Peltier, G., Remans, R., Fanzo, J., Palm, C., Ngigi, S., and Walsh, B. (2009) Challenges associated with reducing malnutrition and diarrheal disease in children in developing countries. ISEE, Dublin, Ireland. Remans, R., Fanzo, J., Nemser, B. Sodjinou, R., Sachs, SE. and Palm, C. (2009) Undernutrition, food insecurity and diet diversity across agro-ecological zones in African Millennium Villages. International Congress on Nutrition, Bangkok. Wariero, J.; Fanzo, J; Biko, S.O.; Mutuo, P.; Sachs, S.E. (2009) Review of Deaths in a High Child Mortality Area for Nutritionally Attributable Deaths using Verbal Autopsy. International Congress on Nutrition, Bangkok. Loeffler Peltier, G, Remans, R., Fanzo, J., Palm, C., Ngigi, S., and Walsh, B. (2009) Challenges associated with reducing malnutrition and diarrheal disease in children in developing countries. "Environment, Food, and Global Health". International Society of Environmental Epidemiology, Dublin, Ireland. Remans, R., Fanzo, J., Siriri, D., Mudiope, J., Okongo, M., Namakula, P., Okorio, J., Sullivan, C., Lambrecht, I., Bergers, K. Nemser, B., Sachs, S., and Palm, C. (2009) From Fork to Farm in the Millennium Villages: Addressing links between agriculture and nutrition in Ruhiira, Uganda. The Uganda Nutrition Congress, Kampala, Uganda. Fanzo, J. (2008). Global Food Forum. Earth Institute, Columbia University. **Host and organizer** Siriri, D., Mutuo, P., Maera, J., Fanzo, J., Spicer, N., Rose, A., and Palm. C. (2008) Community-Led School Feeding Programs in the Millennium Village Project. 35th Annual Conference on Global Health, Global Health Council. Washington DC. Fanzo, J.C. and Gallin, E.K. (2005) The Doris Duke Clinical Scientist Development Award: A Seven￾Year Retrospective and Summary. National Academy of Science Markey Committee: Enhancing Philanthropy's Support of Biomedical Scientists: The Role of Evaluation. **Invited Speaker** Fanzo, J.C., Jang, S.Y., Gupta, Siddiq, A., Greenberg, S., and Pernis, A.B. (2004) The Role of IBP in T cell effector function. Keystone Symposium. Lymphocyte Activation and Signaling. January 2004. ** Invited Speaker** Fanzo, J.C., Gupta, S., Hu, C.H., Jang, S.Y., Lee, A., and Pernis, A.B. (2003) A novel GTPase Exchange Factor, IBP, that is recruited to the immunological synapse in T lymphocytes. Keystone Symposium. Cell Biology of the Immune Response. Cui L.B., Wang L.R., Schoene N.W., Alshatwi A., Fanzo J.C., Han Z.T., and Lei K.Y. (2003) Zinc depletion impairs p53 transcriptional activity in HepG2 cells. FASEB J, 17(4): A1663. Fanzo, J.C., Hu, C.H., and Pernis, A.B. (2002) Regulation of Fas-Dependent Apoptosis in Lymphocytes by Interferon Regulatory Factor –4 (IRF-4). Keystone Symposium. T lymphocyte Activation, Differentiation, and Death. **Invited Speaker** Cui L.B., Alshatwi A., Fanzo J.C., and Lei K.Y. (2002) Differentially expressed genes in zinc-deficient human bronchial epithelial cells detected by cDNA microarray. FASEB J, 16(4): A266. 163 Fanzo, J.C., Reaves, S.K., Zhu, L., and Lei, K.Y. (2001) The p53 protein levels, cyclin-dependent kinase inhibitor p21, bcl-2 and caspase-3 activity are altered by zinc status in aortic endothelial cells. FASEB Journal 15(4). Fanzo, J.C., Reaves, S.K., Zhu, L., Cui, L.B., and Lei, K.Y. (2000) The influence of zinc on human p53 tumor suppressor gene expression and p53-target genes in the apoptotic pathway. FASEB Journal 14(4): 164.19, A229. Fanzo, J.C., Reaves, S.K., Arima, K., Wu, J.Y.J., Zhu, L., Wu, Y., and Lei, K.Y. (1999) Human p53 tumor suppressor gene expression is regulated by zinc in HepG2 cells. FASEB Journal 13(4): 215.1, A241. Zhu, L., Reaves, S.K., Fanzo, J.C., Wang, Y., Wu, J.Y.J., Arima, K., Wu, Y., and Lei, K.Y. (1999) Apolipoprotein A-I gene promoter activity is enhanced by improved zinc status in HepG2 cells. FASEB Journal 13(4): 215.2, A241. Wu, Y., Wu, J.Y.J., Reaves, S.K., Zhu, L., Fanzo, J.C., and Lei, K.Y. (1999) The binding of HepG2 nuclear extract to putative metal responsive elements in human apolipoprotein A-I promoter is inducible by zinc. FASEB Journal 13(4): 215.3, A241. Reaves, S.K., Wu, J.Y.J., Wang, Y.R., Fanzo, J.C., and Lei, K.Y. (1997) Regulation of the tumor suppressor gene expression of p53 by zinc status. FASEB Journal 12(3): 1122, A193. 164 E. Knowledge Gaps Table Key Knowledge Knowledge Gaps Program Management Have the MEs for the two Nutrition ILs effectively managed their respective research and training activities in Africa & Asia? How effectively have the MEs and their partners communicated, coordinated & engaged with: --Missions --Key partners -MEs decided synergies and efficiencies justify merging the US side of the management for Nutrition Innovations Labs for Africa and Asia -Management team at Tufts works well together -Technical Advisory Committee members are highly qualified -Trip reports have documented multiple interactions w/missions -Mission turnover is a serious problem NEPAL -Meet with mission in Nepal at every visit -No funding from mission in Nepal UGANDA AND MALAWI NEPAL Main core partner in Nepal is JHU Other partners include -Heifer -DAI -Harvard -Purdue -HKI -Nepal Technical Advisory Group -Commitment of Tufts administration to the Nutrition ILs? -How effective are interactions with core partners? -What are outputs from the communications team? -Does the Tufts team have adequate capacity to handle all of the work? -Do they have the required expertise to handle the complexities of the project? Is the information on the website geared toward both a general audience (e.g., Mission staff) and the scientific community? -Is the expertise of the technical advisory committee used appropriately? Do mission staff understand the goals and organization of the Nutrition Innovation Labs? -What are the expectations of mission staff for the Nutrition Innovation Labs? -Are mission expectations similar or different for the Nutrition IL and other ILs? -What differences between mission interactions in Asia and Africa may have impacted progress of the project? -Do mission personnel have suggestions for maintaining continuity for projects in the face of frequent turnover in the mission? -Does the mission provide funding for the Nutrition IL in Uganda and Malawi? -Do other ILs receive funding from the mission? - Is Nepal mission planning to fund a Scientific Symposium? -What are similarities and differences in expectations for mission interaction? -Is the Nutrition IL integrating their data collection into existing JHU project sites, or are new sites being established? -Does the Nutrition IL use what JHU has already done in Nepal with regard to nutrition research? - What specifically is the role of Heifer with the 165 Key Knowledge Knowledge Gaps What have been specific challenges faced in terms of management, and how has each ME addressed them? Have the two Nutrition Innovation Lab Awards created value added benefits compared to the former CRSP model of one lead university managing a global program of multiple projects? Have the two MEs built synergies between their regional programs to ensure comparability among findings? How have these synergies contributed to the Nutrition Innovation Labs’ objectives? How could the synergies between the two regional programs be strengthened? UGANDA Considerable overlap between projects with funding of core partners They are working with their third AOR Management described by MEs as complicated -MEs state there are efficiencies and synergies with current structure Nutrition IL? - What are the roles of DAI personnel in Nepal? - Who is funded by Nutrition IL for other discreet studies in Nepal? - What specifically is Harvard doing in Nepal with the Nutrition IL? - What specifically is Purdue doing in Nepal with the Nutrition IL? - How does the Nutrition IL interact with Helen Keller International in Nepal? -What are the roles of the NTAG in the project? -What has materialized from the MOUs with Nepal CHD/MOH and Nepal IOM? - What are the specific roles/tasks of partners in Uganda? -What similarities and differences are there between Nepal and Uganda in experiences with selection of partners? - What are specific interactions with University of Bergen and UNICEF? - Is there a stable understanding across USAID of the goals/responsibilities of the Nutrition IL and of other nutrition-oriented projects so that management changes in Washington will not contribute to misunderstandings? NEPAL - What are interactions between Nutrition IL, Suaahara project & IFPRI? -Why did Tufts decouple from Suaahara project? - What led to misunderstanding with Suaahara that Nutrition IL would do baseline and ending data collection for M&E from the Nutrition IL budget? UGANDA - How specifically did the problem with the Office of the Prime Minister impact the project? - How effective and how frequent are the direct contacts between Tufts and the Ministries of Agriculture, Health, Gender etc.? - What created the misunderstanding about sources of funds for the Uganda Community Connector? -May not be possible for our team to make a definitive comparison with previous CRSP model? -What expertise is being built at US universities for design of nutrition sensitive projects with this model compared to the former CRSP model? - What type of relationship does Nutrition Innovation Lab have with other partners working on similar projects with regard to results? 166 Key Knowledge Knowledge Gaps What are specific commonalities between the programs? NEPAL -What inputs do the MEs have to the research design for the Suaahara program? Or, to other research design in Nepal? -What inputs do the MEs have to the research design for the Uganda Community Connector? -Are there specific examples of agriculture-nutrition pathways that are being evaluated in both Africa and Asia? -Can the different agricultural and cultural contexts contribute to enhanced understanding of integrated program impacts? Synergies will be evaluated to derive these suggestions. Research Program Are there strategic contributions to these research questions: --What are the agriculture-to-nutrition pathways? --What are the program impact pathways? --What is the value of integrated programming pathways? Challenges: --What challenges have the two Nutrition Innovation Labs faced during research design and implementation? --What impact, if any, have these challenges had on implementation of research activities? -Agriculture-Nutrition Pathways listed as a priority for Nutrition IL Research Agenda -Integrated Program Impacts listed as a priority for the Nutrition IL Research Agenda -Neglected biological mechanisms also were listed as a research focus -Face limitations regarding low resource tools to do field testing -Have encountered confusion in host countries about roles of the various USAID funded entities -Links with Suaahara NEPAL What is the primary source of data to evaluate these pathways? How will these data contribute to the nutrition sensitive agriculture literature? Are their projects designed to establish mechanisms between nutrition and nutrition sensitive sector linkages? UGANDA -How much input does the Nutrition IL have into the protocols for the Uganda Community Connector? -At what intervals will data generated by the Uganda Community Connector be available to the MEs? -What “neglected biological mechanisms” are being investigated and by whom are they being conducted? -Clarify projects related to: --Effect of exposure to Mycotoxins/aflatoxins on stunting and wasting --Gut microbiome (SHINE research) --Links between animal and human pathogens --Envir Environmental Enteropathy - What will the research implementation ultimately bring about in both Nepal and Uganda without doing rigorous RCT type research programs? 167 Key Knowledge Knowledge Gaps --How effective have the two research programs been in addressing the challenges? --What could they do differently to better address the challenges? project have not been as productive as hoped -Leadership changes in Uganda project and funding misunderstandings have caused project delays To be determined. To be determined. Training Program --Have the Nutrition Innovation Labs met academic and technical capacity strengthening targets? --Are the appropriate type and number of people being targeted for the right kind of training? --What improvements, if any, are needed in how academic and technical capacity strengthening activities are identified and implemented? --How have trainees put into practice the knowledge and skills acquired? --How have the training programs contributed to strengthening institutional capacity in the target countries? -Short-term training provided to 275 students over last 3 years -Long-term training in progress for 15 M.S. students and 4 doctoral students. Students are studying at Tufts University, JHU, Purdue, Harvard, and Tuskegee. Students selected based on More than 100 of the trainees were female. - What have been the topics of the short-term training? - What specific curriculum development has been supported? - What is the plan for continued short-term or refresher trainings once the project is no longer active? What was the impact of these short trainings on the trainees? Did they improve their skill sets, increase their employability, make them of value for the nutrition programs in their own countries, and/or improve their job performance? - How many students at each institution? - Are the students satisfied with their programs? - Are student stipends adequate? - What is the employability of these students in their home country once their training and degree work is finished and they have returned to their home country? Is there any kind of agreement with their national institutions to hire them? - What are the best methods for attracting applicants for training opportunities? - What are the selection criteria for student training? - How are students monitored? What steps are in place to ensure that students’ training is progressing as planned? - What would cause more effective utilizations of the community of practice workspace? To be determined from trainees and their supervisors. Do students publish or have the opportunity to be on published papers? Do students have the opportunity to present their work at meetings? -What kind of post-training career assistance are 168 Key Knowledge Knowledge Gaps students receiving from the ME or others (who?) to help them plan for and identify possible job opportunities after graduation? Is the ME putting students in touch with its network of collaborators in Uganda and Nepal to explore job opportunities with government ministries, academic institutions, and NGOs, among others? To be determined from trainees and their employers Institutional Capacity Collaboration Effective communication and productive collaborations with host country --governmental institutions? --academic institutions? --local NGO’s? --Other Feed the Future innovation labs? --Other relevant USAID programs in target countries? --Other relevant international research programs working in target countries (AA4NH, IFPRI)? How could the MEs improve in building their institutional collaborations? Technical assistance: Frequent meetings are referenced Student thesis research was supported. Representatives of NGO’s were among the trainees Dr. Griffiths is Chair of the Innovation Lab Directors Council The MEs spend a large amount of time responding Assess effectiveness of communication at different levels. What have been the impacts of the Nutrition IL on the institutions? Have preliminary results been shared? - How were projects aligned with Nutrition IL objectives? -How did projects co-ordinate with academic institutions to assist in meeting their expectations for graduate training? - From which academic institutions were students selected for support? - Were needs assessed for training topics? - Who specifically was trained? - Was training used as an opportunity to encourage integration of government. NGO and academic partners? - Clarify all USAID program interactions in host countries? - Is there a clear understanding of the roles appropriate to the Nutrition IL and to roles for other USAID funded projects so that they are synergistic and not competitive? To be determined in host country. Interview host country institutional leaders. 169 Key Knowledge Knowledge Gaps How responsive have the MEs been to requests from --USAID – Washington? --USAID- Missions? --Host country institutions? --Other nutrition technical assistance groups (e.g. SPRING) --How useful have the ME responses been? --What have been outcomes of the assistance (e.g. new collaborations, new or improved research)? What, if anything, could the MEs do to improve in responding to requests for assistance? to requests about linking nutrition and agriculture from USAID-Washington and other organizations -Associate Award in Malawi was added to the portfolio -Associate Award with Egypt is to be issued in the near future -What efficiencies can be created by the “communications team” - How does Nutrition Innovation Lab align with other nutrition initiatives such as SPRING? Need specific examples from recipients Program Future How well do the two research programs align with the Feed the Future research strategy? What adjustments may be necessary to their research programs to better ensure alignment? Goals of the Nutrition ILs are very well aligned with Feed the Future Current research outputs address critical policy questions Research currently underway further addresses critical policy questions -Do research plans for a possible next phase build on research results from the initial 5 year cooperative agreement award? 170 Key Knowledge Knowledge Gaps Does the structure of the two research programs offer strong likelihood of impactful results that justify funding a second phase? If projects are renewed, what changes are needed to either or both of the Nutrition Innovation Labs: --management --research (design, implementation, communications, stakeholder involvement ) --training (student recruitment and selection, content, location) programs --institutional capacity collaboration Lessons learned for a second phase, if funded HQ grant awarded for aflatoxin work in core birth cohort study. Possible buy￾in from USAID/EA for aflatoxin work in smaller Gulu cohort study Discussion with Mali but coup reduced momentum What structure will be put in place to manage the multiple awards that have been received/are being sought? When can the data analysis be completed to inform future studies? -Do research plans for a possible next phase build on research results from the initial 5 year award? 171 APPENDIX C: TRAVEL ITINERARY, LOCATIONS & DATES OF VISITS Location Dates EET Members Purpose No of Days Boston MA, Tufts University Feb 24-25, 2014 B. Stoecker, K. Musante and J. Fanzo To meet with ME to discuss overall Nutrition Innovation Lab 2 Kathmandu and Bhaktapur, Nepal May 1-9, 2014 B. Stoecker, K. Musante and J. Fanzo To meet with Nutrition Innovation Lab/Asia and partners 8 Kampala Uganda May 16-24, 2014 B. Stoecker, K. Musante and J. Fanzo To meet with Nutrition Innovation Lab/Africa and partners 9 Lilongwe Malawi May 13-15, 2014 K. Musante To meet with Nutrition Innovation Lab/Africa and partners 3 172 APPENDIX D: METHODS This evaluation covers the five key areas outlined in the Scope of Work (SOW) for the two Nutrition Innovation Labs: Asia and Africa. It uses a mix of document review, key informant interviewing and on-line surveys to address these questions. Each Nutrition Innovation Lab review was conducted using similar methodologies, guided by the same set of questions. However, the partners and the precise nature of the programs in Asia and Africa are different, and specific individuals included as key informants differed in each location. All three members of the EET traveled to Tufts University, Nepal and Uganda. Only one member was able to travel to Malawi. Team members spent two days in Boston, five days each in Nepal and Uganda, and three days in Malawi. The Scope of Work for this evaluation instructed the review team to evaluate: (a) The program management by the Management Entity at Tufts University, (b) the research program, (c) the training program, and (d) the institutional capacity collaboration. In addition, the External Evaluation Team will be asked to provide recommendations to inform the decision on program extension and, if appropriate, provide recommendations as to any suggested program changes or improvements (SOW 2013, Appendix A). These questions were used to organize key indicators, identify available information and design a plan for collecting data on those indicators. The proposed Associate Award that was expected to be finalized in June 2014 by USAID/Malawi is distinct. While it fits clearly within the overall training mission of the Nutrition Innovation Lab /Africa current activities are limited to training. Only the sections dealing with training and basic management were applied to the interviews conducted in Malawi. The questions posed in the SOW were operationalized into a set of key indicators for each question. Sources of data to addresses each of the indicators were identified. A table of the overall evaluation questions linked to indicators and proposed sources of data can be found in Appendix B: The Evaluation Plan. Sources of Data Following the indicator matrix, the EET drew on three main sources of data for analysis: • Project documents including the Nutrition Innovation Lab website. A full list of documents consulted is found in Appendix F. • Interviews with key informants. A full list of interviewees is found in Appendix E. Interview guides are found in Appendix B, the Evaluation Plan. 173 • On-line surveys administered to samples of short and long term trainees. Document review and secondary sources The evaluation team conducted a thorough review of available project documents, including: • the original USAID RFA • the technical applications for each Nutrition Innovation Lab • the sub-award RFPs • annual reports • the successful sub-awardee proposals, • minutes from meetings of the Board of Directors and Technical Advisory Group • available trip reports • trainee selection guidelines • publications and presentations • research briefs • the procedures and operations policy manual (POPM) • press releases • newsletter articles The team also reviewed the ME website, its organization and the materials provided on the website. Two members of the EET joined the Nutrition Innovation Lab NCRSP Workspace. EET members have read the published papers, working papers and research briefs currently available. In addition, the evaluation team requested and received lists of trainees who participated in short and long term training with information on gender and sending institutions, curricula for training programs, and follow-up information on current employment and activities of past trainees. Several sources of documentary data were not available for the review. The EET did not learn of them until late in the review process. These include trip reports submitted in e-mail messages among the partners; monthly bulleted email updates to the Nutrition Innovation Lab AOR; the full text of newsletters, which the EET was told were circulated through e-mails (but not available on the website). RFAs, technical proposals, annual reports, trip reports, the POPM, and minutes of meetings were imported into an NVivo database. Documents were coded for key indicators outlined in the matrix. In addition, the database was also searched for key works relating to the SOW questions. 174 Key informant interviews The purpose of key informant interview in this evaluation was to establish the saliency and importance of key events, solicit a wide range of opinion on project activities from expert stakeholders, and elicit recommendations for the future from the point of view of key stakeholders. The EET conducted semi-structured interviews with critical informants and a sample of key informants representing Nutrition Innovation Lab project stakeholders in Nepal, Uganda, Malawi, and the USA. Critical informants included the ME Program Director, the Nutrition Innovation Labs Project Directors and staff, PIs of U.S. sub awardees, host country PIs, host country project coordinators from both the ME and sub awardees, and USAID Mission staff in each country. All of these individuals were interviewed either in person or by conference call/SKYPE. With the assistance of the ME, USAID AOR, and a review of project documents, the EET generated and continued to build a list of individuals representing other project stakeholders in each of the Nutrition Innovation Labs. A preliminary list was included in the Evaluation Plan, but was expanded as more information was gathered. The following groups of stakeholders were identified as important: • Members of the Board of Directors and Technical Advisory Committee • Representative research staff • Directors and staff of the key collaborating projects in each country • Representatives of the host country academic partners • Representatives of the Ministries of Agriculture and Health, • Representatives of other Feed the Future Innovation Lab host country program staff • Regional research coordinators and enumerators • On-the-ground partners such as HKI and Heifer International • Long and short term trainees and students EET members interviewed a sample of individuals from each of these stakeholder groups. Sampling used a purposive sampling frame based on the criteria of 1) level of involvement with the project, 2) breadth of knowledge of program activities, 3) representativeness of the range of diversity of stakeholders and 4) availability during the time available to the evaluation team in each host country or by Skype. 175 A total of 82 interviews were conducted for this project. Interviews took place in the U.S. and in host countries either face-to-face or through conference call/SKYPE. All interviews were conducted using the interview guides developed for this evaluation (Appendix B, Evaluation Plan). Interviews were recorded in notes taken at the time of the interview and expanded later. If notes were taken by more than one EET member, the notes were reviewed and consolidated. In very few cases the interviewee gave permission to record the interview. Recordings were used as aids to memory in writing notes. Interview notes were imported into an NVivo database and were coded by the questions in the interview guides, and the indicators in the matrix. Analysis of Documents and Interviews All textual materials – documents and interview notes – were read and reread several times. All were entered into an NVivo database and coded for overall characteristics: which Nutrition Innovation Lab and stakeholder group the text represented, place of the interview; the specific question(s) on the appropriate interview guide to which the material pertained; the indicators represented in the text. A priori codes were constructed using the indicator matrix and interview guides. Several additional themes were identified in the analysis of the data. Word searches focusing on key words identified during the analysis were also conducted to look for material overlooked in the first pass of coding. Analysis consisted of querying the data for specific codes and themes and cross tabulating, where feasible, with characteristics of the respondents. Sampling and Limitations of the Sample The sampling strategy for this review was a form of non-probability sampling that mixed expert sampling with quota sampling. Stakeholder groups were selected a priori to address the indicators outlined in the indicator matrix. Within each stakeholder group we generated a list of individuals suggested by the ME, USAID or by our examination of project documents. Before each site visit, we sent a description of the types of individuals with whom we wished to speak, the list of specific individuals we had identified as critical informants to the host country program coordinators with a request to schedule meetings. Host country coordinators were very responsive to our requests and scheduled a series of meetings with individuals that were available during the time of our visit. In some cases when the person that we had identified a priori was not available in country, an interview with another person in that organization or stakeholder group was identified for interview. In several instances the individual we wished to interview was not available and we did not feel that any of the substitutes had the same level of project involvement. In those cases we conducted conference call/SKYPE interviews after returning to the U.S. The Chief of Party for the Uganda Community Connector was one example. While the selection of key informants from each stakeholder group was not random, we are confident that we were able to interview the most important experts in each category. Every attempt was made to select the most representative 176 sample of key informants possible. The EET is confident that with 82 interviewees the full range of information and opinion has been captured. In order to assess the likelihood that we had an adequate sample of informants we examined the degree to which the interview materials reached saturation. Saturation, the point at which no new themes are emerging from interviews 9 , is one of several ways of assessing the adequacy of the sample size for qualitative research. In this case EET members feel that the saturation point was reached, and subsequent analysis of data suggests that that has occurred. By this the EET does not mean that all of the informants concurred, rather that there was consensus regarding events and the saliency and importance of events. Different stakeholders have rather different opinions on some issues. These are noted in the discussion of research results in each section. Recall Bias Key informant interviewing focused primarily on eliciting experts’ opinions and assessments of program activities as participants and collaborators over the life of the project. Because of this the EET did not anticipate particular problems with recall bias. Opinions are not particularly sensitive to recall bias. Questions of fact were cross-checked against the responses of other participants and against documentation of events and activities available in other project materials. Response Bias Self-report data (i.e., interview data) is always subject to individual interviewees’ interpretations and perspectives. However, the EET has structured this evaluation in ways to reduce the likelihood of three types of systematic response bias: acquiescence bias (i.e., the tendency to agree), demand characteristics (i.e., modifications to responses because of being “studied”), and social desirability bias (i.e., ascribing favorable traits, even if this is untrue). The EET have structured the interview guides to include responses to important questions from each of the stakeholder groups in order to triangulate responses from several points of view on the same events and issues. The EET coded responses to interview questions such that the responses of different stakeholder’s and different individuals from within stakeholder groups could be cross-checked for convergence or disagreement. Through triangulation of assessments and understandings of different individuals the EET came to a set of conclusions that identify areas of consensus and areas of disagreement. In some cases it has also been possible to triangulate description of events and processes with discussions of those same events in project documentation. For example the EET have been able to 9 O’Reilly, M., & Parker, N. (2013). ‘Unsatisfactory Saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research, 13(2), 190-19 177 document differences of opinion in project execution both in the interviews and in the minutes of meetings. However, the EET did not have access to internal memos, e-mail traffic or detailed financial records. For some areas of disagreement the EET has had to draw conclusions after weighing the histories and opinions of several individuals. These are noted in the discussion of the data and the responses to the SOW. On-line Surveys of Long and Short Term Trainees Two 10-question surveys were developed to assess former trainee and student satisfaction and opinions of the courses or training sessions they participated in during the Nutrition Innovation Labs. The survey tools were developed using the web-based SurveyMonkey program. One survey targeted short-term training that Nutrition Innovation Labs have supported. The other survey targeted longer-term formal training at universities with degree granting programs. A range of questions were asked to assess the satisfaction with training, translation of training into professional use, and areas where improvements can be made. A list of trainees with their e-mail addresses was produced by the ME. An invitation to participate in the survey was followed by 2 follow-up requests. For the short-term training survey, there were a total of 24 responses out of 36 requests (response rate of 67%) from the Bangalore Training, the Summer Institute of Biostatistics and Epidemiology at Johns Hopkins University training, and the non-degree Summer training at Harvard. For the formal degree training survey, there were a total of 17 responses out of 24 requests (response rate of 71%) from Tufts University Friedman School, Purdue University, Makerere University, Uganda Christian University, Johns Hopkins School of Public Health, Tuskegee University and Harvard University. Survey responses were analyzed using standard descriptive statistics. 178 APPENDIX E: LIST OF PERSONS CONTACTED Persons Contacted and/or Interviewed: Name E-Mail/Phone Location of Interview Position/Context GLOBAL Carole Levin clevin@usaid.gov 517-243-0240 Phone, conference call & e-mail contacts Program Manager, Bureau for Food Security (BFS)/Office of Agriculture, Research and Policy (ARP), USAID Maura Mack mmack@usaid.gov Phone, conference call & e-mail contacts Nutrition Innovation Lab AOR, BFS/ARP, USAID Saharah Moon Chapotin schapotin@usaid.gov (P) 202-712-4022 (M) 571-243-3916 Conference call & e-mail Research Division Director, BFS/ARP, USAID Ahmed Kablan Conference call AAAS Fellow & Nutrition Innovation Lab Activity Manager, BFS/ARP, USAID Anne Swindale aswindale@usaid.gov Conference call & e-mail Senior Program Analysis, BFS/Office of Strategic Planning & Operations, USAID Tiffanie Ruefly Tiffanie.ruefly@fas.usda.gov E-mail & conference call Travel regulations; USDA Matt Hudson Mat.hudson@fas.usda.gov E-mail & conference call Travel arrangements; USDA Shibani Ghosh Shibani.ghosh@tufts.edu Boston Site Visit, e-mail ME-Associate Director, Nutrition Innovation Labs Tufts University Elizabeth (Liz) Marino-Costello, MS, RD, FADA Elizabeth.Marino_Costello@ Tufts.edu (p) 617-636-3774 SKYPE: liz.marino.costello Boston Site Visit, e-mail ME-Program Manager, Nutrition Innovation Labs Tufts University Kusum Hachhethu Boston Site Visit Student from Nepal Linda Journey journeyl@missouri.edu 573-884-9685 E-mail Consulting Agreement MU CAFNR International Programs Christy Copeland copelandc@missouri.edu E-mail Consulting Agreement MU CAFNR International Programs Patrick Webb Patrick.webb@tufts.edu Boston Site Visit, conference call, e-mail ME – Program Director, Nutrition Innovation Lab/Asia, Tufts University Jeffrey Griffiths Jeffrey.griffiths@tufts.edu Boston Site Visit, conference call ME – Program Director, Nutrition Innovation Lab/Africa, Tufts University Jennifer “Vern” Long jlong@usaid.gov Conference call Former AOR for 179 Name E-Mail/Phone Location of Interview Position/Context Nutrition Innovation Labs, BFS/ARP, USAID Cheryl Jackson Lewis Cheryl.lewis@fas.usda.gov Conference call First AOR for Nutrition Innovation Labs Irvin Widders Widders@anr.msu.edu Conference call Director, Grain Legume Innovation Lab Edgar Agaba Edgar.agaba@tufts.edu Boston Site Visit and Uganda Site Visit Student for MPH at Tufts; former Nutrition Innovation Lab/Africa Uganda Project Coordinator Keith P West, Jr. kwest@jhsph.edu Conference call, Ethiopia Micronutrient Forum Core Partner, Johns Hopkins University Rolf Klemm Rklemm1@jhsph.edu Conference call Core Partner, Johns Hopkins University; Helen Keller International Will Masters William.Masters@tufts.edu Conference call ME Associate Director, Nutrition Innovation Lab/Africa Eunice Bonsi ebonsi@mytu.tuskegee.edu Conference call Core Partner, Tuskegee University Eileen Kennedy Eileen.Kennedy@tufts.edu Conference call ME – Co-Project Director, Global Wafaie Fawzi Conference call Core Partner, HSPH Chris Duggan Christopher.Duggan@childrens. harvard.edu Conference call Core Partner, Harvard University Gerry Shively shivelyg@purdue.edu Conference Call Purdue University NEPAL Diplav Sapkota, MPH Diplav.sapkota@tufts.edu 9840067419/9801101007 015260459 ext 14 Kathmandu Site Visit Project Coordinator Tufts University, Nutrition Innovation Lab/Asia Anushree K.C. kcanushree@gmail.com 9849673538 Kathmandu Site Visit Asst. to Diplav Nutrition Innovation Lab/Asia Dr. Shyam Raj Upreti drshyam@hotmail.com 9851088382 Kathmandu Site Visit Director, Child Health Division, Teku Prof. Dr. Kedar Prasad Baral rector@pahs.edu.np kedarbaral@pahs.edu.np 9851060058 +977-1-5545112 Kathmandu Site Visit Rector Patan Academy of Health Sciences GPO Box #26500 Kathmandu, Nepal www.pahs.edu.np Sumit Karn Link2sumit@gmail.com Kathmandu Site Visit Nutrition Specialist, FAO, Child Health Division, Teku Dr. Sharad Onta sharadonta@gmail.com 9851085061 Kathmandu Site Visit Vice Dean, Institute of Medicine, Maharajgunj Sabnam Shivakoti shabaryal@yahoo.com shabnam.shivakoti@gmail.com 015524352 9841330283 Kathmandu Site Visit Senior Plant Quarantine Officer Ministry of Agriculture & Cooperatives, GON/ADB 180 Name E-Mail/Phone Location of Interview Position/Context Atmaram Pandey patmaram@gmail.com 9851108752 Kathmandu Site Visit Secretary Dale Davis ddavis@hki.org Kathmandu Site Visit Helen Keller International Shika Basnet Shika.basnet83@gmail.com 9841341694 Kathmandu Site Visit UNICEF Pooja Pandey ppandey@hki.org 9851057677 Kathmandu Site Visit Dep. Chief of Party, SUAAHARA Swetha Manohar smanohar@jhsph.edu 9849450354 Kathmandu Site Visit John Hopkins University Madhav Shrestha Madhavshreshta1954@gmail.co m 9851075404 Kathmandu Site Visit Co-PI, Aquaculture & Fisheries Innovation Lab Debendra Adhikari dadhikari@usaid.gov 9801070054 Kathmandu Site Visit Health & Family Planning Team, USAID Mission Amy Prevatt aprevatt@usaid.gov 9801046972 Kathmandu Site Visit Food Security Specialist, USAID Mission Gautam Bajaracharya gbajaracharya@usaid.gov 9801073748 Kathmandu Site Visit M&E Specialist, USAID Mission Deepak Thapa deepakntag@mail.com.np Kathmandu Site Visit Programme Manager, NTAG Sujay Bhattacharya snbhattacharyaji@gmail.com 9841469777 Kathmandu Site Visit CHD/UNICEF Dr. Ramesh Adhikari Ramesh497@gmail.com 9851105737 Kathmandu Site Visit Local Co-PI, KMC/IOM Dr. Manjeswori Ulak manjeswori@gmail.com Bhaktapur Site Visit IOM, University of Bergen, Harvard Project MALAWI Edwin WP Siyame Sylame2002@yahoo.com Lilongwe Site Visit Lecturer in Human Nutrition, Bunda College Campus, Lilongwe University of Agriculture & Natural Resources Agnes Wangwela amwangwela@bunda.luanar.mw agnesmwangwela@yahoo.com Lilongwe Site Visit Violet Orchardson vorchardson@usaid.gov Lilongwe Site Visit Nutrition Advisor, USAID Mission John Edgar jedgar@usaid.gov Lilongwe Site Visit Feed the Future Coordinator, USAID Mission Cybill Sigler csigler@usaid.gov Lilongwe Site Visit SEG Team Leader, USAID Mission UGANDA Florence Turyashemererwa Florence.Kinyata@tufts.edu Kampala Site Visit Nutrition Innovation Lab Country Coordinator, since January 2014 Edgar Agaba Edgar.agaba@tufts.edu Kampala Site Visit and Boston visit Summer in Kampala w/project Since January 2014 Prof. Bernard Bashaasha Edgar Agaba bashasha@agric.mak.ac.ugEdgar. agaba@tufts.edu Kampala Site Visit Followed Prof. Kikafunda as MU PI Summer in Kampala w/project Annet Kawuma a.kawuma@gmail.combashasha @agric.mak.ac.ug Kampala Site Visit Research Assistant Nutrition Innovation Lab 181 Name E-Mail/Phone Location of Interview Position/Context PI Handles IRB, Quality Assurance Miriam Annet Kawuma namiriam@caes.mak.ac.uga.kaw uma@gmail.com Kampala Site Visit Financial Administration Research Assistant General Office Support Margaret Kabahenda Miriam mkabahenda@yahoo.com mkabahenda@gmail.com +256 773 009747 +256 706 691994namiriam@caes.mak.ac.u g Kampala Site Visit MU faculty contact for Tuskegee funded students. Kato Hussein Margaret Kabahenda hkkatebo@gmail.commkabahen da@yahoo.com mkabahenda@gmail.com +256 773 009747 +256 706 691994 Kampala Site Visit MU students/Applied Human Nutrition MU student/Ag Economics Edward Kansiime Kansiime16@yahoo.com hkkatebo@gmail.com Kampala Site Visit MU student/Applied Human Nutrition MU student/Ag Economics Joshua Ssemakila Sirjoshua1@gmail.com Kansiime16@yahoo.com Kampala Site Visit MU student/Applied Human Nutrition MU student/Applied Human Nutrition Elizabeth AsiimweJoshua Ssemakila alizeff@yahoo.comSirjoshua1@ gmail.com Kampala Site Visit MU student/Ag Educ & Extension MU student/Applied Human Nutrition Elizabeth Atim atimelizabeth@gmail.comalizeff @yahoo.com Kampala Site Visit MU student/Applied Human Nutrition MU student/Ag Educ & Extension Henry Galiwango Galiwangohenry2005@gmail.co Kampala Site Visit MU student/Ag Educ & Extension MU student/Applied Human Nutrition Barnabas Natamba Bkn6@cornell.eduGaliwangohe nry2005@gmail.com Kampala Site Visit Makerere University PhD Candidate at Cornell University MU student/Ag Educ & Extension Samalie Namukose snamukoseb@yahoo.com +256-772- 491551Bkn6@cornell.edu Kampala Site Visit Senior Nutritionist, Ministry of Health Arthur Bagonza arthurbagonza@yahoo.comsna mukoseb@yahoo.com +256-772-491551 Kampala Site Visit Makerere University Julius Twinamatsiko Arthur Bagonza juliustwinamatsiko@gmail.coma rthurbagonza@yahoo.com Kampala Site Visit BBNC participants Patrick Kibaya pkibaya@healthnet.or.ug Kampala Site Visit Project Officer Programmer of tablets BBNC participant 182 Name E-Mail/Phone Location of Interview Position/Context Fred Kakaire fkakaire@healthnet.or.ugpkibaya @healthnet.or.ug Kampala Site Visit Medical Information Programmer of tablets Project Officer Elizabeth Namakula lenanaelizabeth@gmail.com Kampala Site Visit Communications Officer; Programmer of Tablets Medical Information Dr. Van Campenhout Kampala Site Visit Director IFPRI Communications Officer Dr. Nassul Kabunga n.kabunga@cgiar.org Kampala Site Visit IFPRI, Post-doctoral Associate, 50% salary from Nutrition Innovation Lab/Africa Director IFPRI Dr. John Ssekamate jssekamatte@npa.ug Kampala Site Visit Head, Social Sector Planning, National Planning Authority (Representing Dr. Kisamba Mugerwa who is in charge of nutrition issues) Musimenta Boazi Kampala Site Visit Policy Analyst; Office of the Prime Minister Coordinating body for Uganda National Nutrition Action Plan Sarah Ngalombi Kampala Site Visit Senior Nutritionist Ministry of Health Policy Analyst Benjamin Aisya baisya@fhi360.org +256 782 876 722 Kampala Site Visit Monitoring & Evaluation Specialist; USAID/ Community Connector Brenda Shenute Namugumya bnamugumya@fhi360.org +256 712 883 529 Kampala Site Visit Nutrition Specialist; FHI360 FANTAanta-3- ProjectMonitoring & Evaluation Specialist; USAID Dianna Darsney de Salcedo ddarsney@usaid.gov +256 414-306-001 ext 6548 Kampala Site Visit USAID Vulnerable Populations Unit Leader Nutrition Specialist Dr. Simon Byabagambi sbyabagambi@usaid.gov +256 414-306-001 ext 6554 Kampala Site Visit USAID Mission Dr. Alfred Mashandich Boyo aboyo@usaid.govsbyabagambi@ usaid.gov +256 414-306-001 ext 6554 Kampala Site Visit Program Mgmt. Specialist, Child Survival/Nutrition & MMed(Paed), MPHProgram Dr. Stephen Nsabiyumba Snsaba200@gmail.com Conference call in Uganda USAID Mission District Health Officer for Kisoro District Program 183 Name E-Mail/Phone Location of Interview Position/Context Kampala Site Visit Charles Asiimwe casiimwe@fhi360.org Conference call during Kampala Site Visit Regional Coordinator for CC in the Southwest Region District Health Officer for Kisoro District Dr. Hanifa Bachou hbachou@fhi360.org Conference call during Kampala Site Visit FANTA Regional Coordinator for CC in the Southwest Region Robert Mwadime rmwadime@fhi360.org Conference call during Kampala Site Visit Chief of Party for CC Project FANTA Nilupa Gunaratna ngunarat@hsph.harvard.edu Ethiopia Conference call from USA Project Scientist, HSPH 184 APPENDIX F: LIST OF MATERIALS REVIEWED List of Nutrition Innovation Lab administrative documents reviewed by EET during the course of the evaluation: 1. USAID original RFA for the Nutrition CRSPs 2. Full applications for the NCRSP/Asia 3. Full application for the NCRSP/Africa 4. Annual Reports Nutrition Innovation Lab /Asia a. Year 1 b. Year 2 c. Year 3 5. Annual Reports Nutrition Innovation Lab /Africa a. Year 1 b. Year 2 c. Year 3 6. Annual reports from partner institutions- Asia a. Asia Year 1 i. Johns Hopkins School of Public Health ii. Harvard School of Public Health iii. Purdue University iv. Tuskegee University v. DAI b. Asia Year 2 i. Johns Hopkins School of Public Health ii. Harvard School of Public Health iii. Tuskegee University (report very brief) iv. DAI c. Asia Year 3 i. Johns Hopkins School of Public Health ii. Harvard School of Public Health iii. Purdue University iv. Tuskegee University (very brief) v. DAI 7. Annual reports from partner institutions – Africa a. Africa Year 1 i. DAI 185 ii. Johns Hopkins School of Public Health iii. Harvard School of Public Health iv. Purdue University b. Africa Year 2 i. Harvard School of Public Health ii. Purdue University iii. Tuskegee University iv. DAI c. Africa Year 3 i. Harvard School of Public Health ii. Purdue University iii. Tuskegee University 8. Publications and Presentations (listed under section on Research) 9. Policy and Operating Procedures Manual (May 2011) 10. Minutes a. Minutes from Board of Directors meetings i. January 6, 2011 (the inaugural meeting) ii. March 18, 2011 iii. June 2012 iv. April 26, 2013 b. Minutes from the Technical Advisory Group meetings i. January 7, 2011 (the inaugural meeting) ii. June 2012 (held jointly with BoD) iii. April 26, 2013 c. Minutes from Partner meetings i. September 17, 2011 ii. June 2012 11. Student management forms a. Checklist of Required Steps for any NCRSP Hosted Exchange Student b. Sponsored Student Travel and Contact Information Tufts University 12. Associate Awards materials and proposals a. Malawi b. Mali c. Timor Leste d. Concept note for Egypt 186 APPENDIX G: PHOTOGRAPHS Deepak Thapa, Program Manager NTAG in Kathmandu, Nepal Masters Students at Makerere University 187 Dr. Manjeswori Ulak, IOM, University of Bergen, Harvard Project in Bhaktapur, Nepal 188 Peri-urban agriculture in Bhaktapur, Nepal The importance of livestock for agriculture and nutrition, Bhaktapur, Nepal 189 APPENDIX H: MANAGEMENT STRUCTURE Current Management Structure of the Nutrition Innovation Labs: Core Management Team 2013 Position Institution Patrick Webb Project Director, Asia Professor and Academic Dean, School of Nutrition, Tufts University Jeffrey Griffiths Project Director, Africa Assoc. Professor, School of Medicine, Tufts University Eileen Kennedy Co-Program Director, Global Professor, School of Nutrition, Tufts University William Masters Associate Director, Africa Professor School of Nutrition, Tufts University Shibani Ghosh Associate Director, Global Asst. Professor, School of Nutrition, Tufts University Position Vacant Associate Director for Communications Asst. Professor, School of Nutrition, Tufts University Elizabeth Marino￾Costello Program Manager, Global Program Manager School of Nutrition, Tufts University 190 Board of Directors Position Institution Patrick Webb (co-chair) Project Director, Asia Professor and Academic Dean, School of Nutrition, Tufts University Jeffrey K. Griffiths (co-chair) Project Director, Africa Assoc. Professor, School of Medicine, Tufts University Wafai Fawzi Core partner representative Professor of Nutrition, School of Public Health, Harvard University Keith West/Rolf Klemm Core partners representative Prof. of Infant & Child Nutrition, School of Public Health, Johns Hopkins University Eunice Bonsi Core partner representative Research Assoc. Prof., Dept. of Food & Nutrition Science Tuskegee University Gerald Shively Core partner representative Professor of Agricultural Economics, Purdue University Eileen Kennedy (ex officio) Co-PD, Global Professor, School of Nutrition, Tufts University Will Masters (ex officio) Associate Director, Africa Professor of Food Policy, School of Nutrition, Tufts University Shibani Ghosh (ex officio) Assoc. Director, Global Asst. Professor, School of Nutrition, Tufts University Ruth Oniang'o External Advisor Founder and Director, Kenya Rural Outreach Programme (ROP) Maura Mack Assistance Officer’s Representative (AOR) AOR,BFS, USAID Ann Tutwiler External Advisor Bioversity International Ahmed Kablan Activity Manager AAAS Science Policy and Technology Fellow, BFS, USAID 191 Technical Advisory Committee Position Institution Will Masters (co-chair) Associate Director, Africa Professor of Food Policy, School of Nutrition, Tufts University Shibani Ghosh (co￾chair) Associate Director, Global Asst. Professor, School of Nutrition, Tufts University Chris Duggan Core partner representative Assoc. Prof. of Nutrition, School of Public Health, Harvard University Keith West/Rolf Klemm Core partner representative Johns Hopkins University Jerry Shively Core partner representative Purdue University Eunice Bonsi Core partner representative Tuskegee University Richard Deckelbaum External advisor Director, Institute for Human Nutrition, Columbia University Victoria Quinn External advisor Senior Vice President, Helen Keller International Ram Shrestha External advisor Founder and Director, Nepali Technical Advisory Group (NTAG) Shakuntala Thilsted External advisor Senior Nutrition Adviser, WorldFish Center Shelly Sundberg External advisor Bill and Melinda Gates Foundation Stephen Vosti External Advisor University of California Davis Ag/Resource Economics Maura Mack Assistance Officer’s Representative (AOR) Nutrition Advisor, BFS, USAID Ahmed Kablan Activity Manager AAAS Science Policy and Technology Fellow, BFS, USAID Ex-officio on TAC Barbara Seligman Core partner representative Principal Specialist, Health Sector, Development Alternatives, Inc. 192 APPENDIX I: SUMMARY OF RESEARCH PROJECTS Nutrition Innovation Lab/Asia Policy and Science for Health, Agriculture and Nutrition (PoSHAN) study – John Hopkins University The goal of the PoSHAN project is to assess and monitor household food security, dietary intake and nutritional status of preschool children and their mothers along with measures of agricultural diversity, local market food prices and exposure to agricultural and microeconomic extension, nutrition and health programs. A nationally representative sample has been drawn from Nepal’s 75 districts based on agro-ecological zones. Twenty-one Village Development Committees (VDCs) are included with seven selected randomly from each of the three agro-ecological zones: mountains, hills and terai areas. Then three of the nine wards (smaller administrative units) within each VDC were randomly selected for study. All households (~4300) in these 63 wards were mapped and all consenting mothers/caretakers of an under five child, women married within the last two years, as well as the household heads were surveyed. The sample size in spring, 2013, included 5,400 children under 5 years of age. The same households were revisited in spring of 2014 (during the hot dry season) and this annual same-season agricultural, food security and nutrition survey is scheduled to be repeated in the same households in spring 2015. In addition, there is a year-round sentinel surveillance system in one representative Village Development Committee (three wards each) from the mountain, hill and terai zones for a total of nine wards. Data from these sentinel sites during the post-monsoon season (approx. mid-August to mid-October) and again during the winter (mid-December to mid-Feb.) will complement the annual surveys and allow examination of seasonality. Data collection for the annual surveys was contracted with the New Era Nepali field research firm, and the Nepal Technical Assistance Group (NTAG) is handling data collection in the year-round sentinel sights. Senior investigators, Drs. Keith P. West, Jr. and Rolf Klemm each plan to be in-country for 8-15 days, 2-3 times a year, and the project is managed on a daily basis by a JHU Project Scientist, Swetha Manohar, RD, MSPH. Several of the randomly chosen districts for the PoSHAN project overlap with districts where the USAID Suaahara project is being implemented. Other locations overlap with early roll-out sites for the Nepali government’s Multi-sector Nutrition Plan, the World Bank’s 1000 Golden Days program, Winrock International’s DFID-funded aquaculture promotion program, the Poverty Action Fund’s 193 nutrition-focused community challenge fund, Heifer International’s integrated livestock and nutrition interventions and a Feed the Future program. In addition to demographic and anthropometric data, information on income and expenditure, cropping, gardening and animal raising methods, food production and its disposition by type of food, level of perceived food security, participation in agricultural extension, microcredit and other programs, and health care and nutrition services over periods of time ranging from the past week to year was collected. The child module included a 24-hour recall of breast feeding and complementary and usual home food intake, recent receipt of semi-annual vitamin A supplements, a 7-day history of symptoms of acute respiratory infection, diarrhea, malaria, ear discharge and fever; a history of vaccination coverage, and a recent child care history. One in four households was eligible for anemia testing and, depending on the composition of the household, a child and its mother/caretaker or a newly married woman were sampled for hemoglobin assessment. With second and third annual surveys, similar descriptive data analysis will be followed by estimations of year-to-year differences in indicators of nutritional status, dietary diversity, food security and production among other indices of interest to discern the direction and quantify rates of change reflecting trends that are expected to find widespread interest in government, international agencies and the public health nutrition and agricultural sectors. Descriptive analysis of the second annual survey is now complete. Local markets are being visited on “market days” once each month to gather data on retail prices per unit on a standardized list of meats, fish, vegetables, fruit, oils and other food items. These data will be compiled to construct average, within-season price indices for foods reported to be purchased and consumed during household dietary interviews. At the district level, officers from the district offices of public health, agriculture and livestock will be interviewed. At the VDC level, key informants, such as female community health volunteers, ward representatives or members, community level health personnel, NGO workers and other service professionals (shopkeepers, post office attendants), will be requested participate in focus group discussions about access of the study communities to services and infrastructure. Econometric analyses of secondary data linking ecology, food systems, and nutrition – Purdue University Literature was reviewed on nutrition, agriculture and relevant topics for Nepal. The paper by Shively, G., Gars, J. and Sununtnasuk, C. (2011) titled “A Review of Food Security and Human 194 Nutrition Issues in Nepal” is expected to be part of a series of Global Nutrition Innovation Lab project working papers. Dr. Shively has been pro-active in utilizing existing data sets to investigate agriculture, nutrition and health issues. Two large country-wide datasets available for Nepal (the Demographic and Health Surveys (DHS) and the Nepal Living Standards Surveys (NLSS)) have been catalogued. DHS survey data contain numerous health and nutrition indicators and the NLSS data includes information on a range of household welfare indicators. GIS-linked data on agroecology and climate are also being utilized. Papers on agricultural diversity and child stunting in Nepal and on roads, market access and poverty in Nepal are under review for Journal of Development Studies and Journal of Agricultural Science and Technology respectively. A special session at the 2013 annual meeting of the Agricultural and Applied Economics Association focused on “Agriculture and Nutrition Linkages.” Among the four papers were "Impacts of Agriculture on Nutrition: Nature of the Evidence and Research Gaps" by Patrick Webb and Eileen Kennedy from Tufts University and "Agriculture and Nutrition in Nepal and Uganda: Evidence from Survey and Remotely-sensed Data" by Gerald Shively, Purdue University and Molly Brown, NASA. Peri-urban diet and nutrition outcomes – Bhaktapur - Harvard University School of Public Health Drs. Fawzi and Thorne-Lyman from Harvard University School of Public Health have been able to build on an already existing relationship between University of Bergen, Norway, and researchers at the IOM in Nepal. The project, a longitudinal follow-up of 500 mother-infant pairs originally surveyed in 2008, from a peri-urban near Kathmandu in Bhaktapur District, has focused primarily on relations between nutritional status during childhood and infectious disease. The Nutrition Innovation Lab/Asia has supported analyses of preserved plasma and food consumption data to examine relations between anemia, iron status, and dietary intakes of women and children. Similar analyses were conducted in 2012 on 319 of the original mother-infant pairs. Basic indicators of income and agriculture have been added to the survey. Heifer International Project – HKI Collaboration with Heifer International has been established in Nawalparasi district, one of the districts targeted by the Nutrition IL/Asia as a research site. Heifer International is engaged in agricultural interventions, primarily small ruminants, and behavior change communications around nutrition in Nepal. The trial is examining nutrition and health outcomes of the Heifer training and skill building package versus those without that package. Heifer International already had two years of data collected every six months for 415 households in this district in the terai on the role of livestock in enhancing income and diets. These data were analyzed and presented at the 195 Experimental Biology meeting. The Nutrition Innovation Lab/Asia facilitated additional rounds of data collection during FY2013. Pilot Complementary Food Project - Development Alternatives International Kathleen Kurz and Cheryl Doty went to Nepal in August 2011 to identify a focus for DAI within the broad category of complementary feeding and foods, including both demand and supply factors. They met with representatives from industry, university, and NGOs. DAI suggests an assessment of market potential for quality low-cost complementary food production in various parts of Nepal, including a qualitative assessment of why opportunities have not been taken to work with entrepreneurs in local complementary food production. They also see the need to conduct value chain assessment. In 2013, DAI worked with a no-cost extension on a pilot project to build the capacity of a cottage-industry organization in Kathmandu to improve the production of their complementary food product, Sarbottam Pitho. They also produced training materials from this capacity building exercise. Nutrition Innovation Lab/Africa Uganda Panel Study: Assessing the Linkage Between Agriculture, Food Security, Nutrition and Health Among Women and Children in Rural Ugandan Households – Makerere University, Harvard University The baseline panel survey, conducted between October 28 and December 15, 2012, is a comprehensive assessment critical for examining linkages/pathways for agriculture, nutrition and health in Uganda as proposed by the Nutrition Innovation Lab/Africa. The team was composed of researchers from Makerere University, Harvard School of Public Health, IFPRI and Tufts University and data collectors from the private sector. The project examined crop and livestock production practices and their associations with nutrition and health status of mothers and children under five years of age. This baseline panel survey was conducted in six different districts across two geographical regions. Three districts (Kisoro, Dokolo and Agago) designated as Phase 1 of the USAID/ Uganda Community Connector Project (CC) and three districts planned as Phase 2 projects for the CC (Lira, Kole, and Kamwenge) were included in the baseline survey. Activities had not begun in the Phase 2 districts at the time of the baseline survey, so the Phase 2 districts were used as counterfactuals for the baseline panel survey. Districts were “paired” by investigators based on agro-ecological characteristics and similarities in culture and ethnicity. Pairs were Kisoro with Kamwenge, Dokolo with Lira, and Agago with Kole. Six hundred households were sampled within each district for a total sample of 3600 households. Households were oversampled from CC active sub-counties, so in the three Phase 1 districts there 196 were 167 randomly selected households from each of three CC active sub-counties and 100 households from a non-active CC sub-county. Thus in Phase 1 districts, the sample included a total of 500 households CC active sub-counties and 100 households from a CC non-active sub-county. In the Phase 2 districts, four sub-counties were selected randomly from each district because the CC had not yet identified the intervention sub-counties; the sampled households were distributed equally across the selected sub-counties because implementation of the CC programs had not begun. All parishes from the selected sub-counties were selected for the survey. However, to minimize costs, only 25% of villages within each parish were randomly selected for participation. Households were selected randomly from a village list. The focus of the survey was on health and nutritional status of the primary caregiver and one index child aged 0-24 months. However data were collected on all children under five for specific outcome indicators. Data are presented for 3630 households, 3450 mothers/caregivers, and 4500 children under five years of age including 1870 children aged 0-24 months. Data collected electronically using tablets included demographics, socio-economic status of the household, diet, health status, household food security, water, hygiene and sanitation, breastfeeding and complementary feeding of the index child, anthropometric measurements of the mother/primary caregiver and all under five children, and hemoglobin measurements on a subsample of mothers and children. The agricultural domain included an in-depth survey of the past year’s crop production, livestock activities and household consumption vs. sales. Data were also collected on gender, decision-making and time allocation, access to information and infrastructure and use of agricultural technologies and management practices. Descriptive data from the study have been reported and several key indices have been calculated. Cross-tabulating data on agricultural variables and stunting has been done but the counterfactual districts have not yet been extensively compared with the CC active districts to further elucidate effects of agricultural interventions. A second panel survey will be conducted during a similar season in 2014 (early Y5). Uganda Core Birth Cohort Study – Harvard University School of Public Health The Nutrition Innovation Lab/Africa’s Birth Cohort Study, also a collaborative effort among HSPH, Makerere University, and the ME, plans to examine the effect of interventions that integrate nutrition, health, agriculture, and livelihoods on maternal and child nutritional outcomes. At this point the study is seriously behind schedule. Investigators hope to recruit and train local enumerators during the month of July and have enrollment of pregnant women begin in August, 197 2014. Enrollment is scheduled to remain open for one year. If continued through the “first 1000 days”, the study potentially will not finish until at least the end of 2017. The Birth Cohort Study will also serve as a platform for evaluating factors such as water and sanitation, as well as the effects of aflatoxin exposure on growth outcomes. This quasi-experimental study will utilize a cohort of mothers and young children followed from pregnancy through the first two years of life to evaluate the effectiveness of CC to improve household livelihoods and maternal and child nutrition and health in rural northern and southwestern Uganda (n=5152 pregnant women). The cohort will be established in randomly selected CC Phase I sub-counties and in matched non-implemented CC sub-counties that will provide counterfactuals to evaluate the CC and its interventions. Each selected CC sub-county has been individually matched to a non-CC sub￾county with the same agro-ecology and predominant language. The matched sub-counties will be in non-CC districts to minimize spillover from intervention areas. The enrollment period for the birth cohort study will be 12 months. Participating pregnant women, their households, and later their infants will be visited periodically through pregnancy, delivery, and until their children reach two years of age. However, if the plan is to carry the study through pregnancy to the first two years of age, either refunding of the Nutrition IL/Africa for a second five year period or a no-cost extension of the current project will be required. Birth Cohort Doctoral Research Project in Gulu - Gulu University, Tufts University, and Cornell University An on-going birth cohort project, which is the dissertation project for Barnabas Natamba, a Gulu University faculty member and PhD student at Cornell University, has received support from Nutrition Innovation Lab/Africa. This study of 400 mothers enrolled in a prenatal clinic assessed food insecurity, maternal depression, and HIV infection. After delivery of the infants of the 400 women, 250 of these women and 250 of their infants were enrolled in a second phase study, which has now begun. This second phase will include neurocognitive early childhood development (ECD) assessments at 6 and 12 months of age. Data from this study will be used to help the Nutrition Innovation Lab/Africa refine its approaches to aflatoxins, ECD, logistical support and cold chains for the larger core birth cohort study. Mr. Natamba has submitted several manuscripts for publication based upon the validation of tools used to evaluate the Gulu birth cohort. These include the development and validation of an individual food insecurity access scale to parallel the household food insecurity access scale currently in wide use. 198 Water Quality Validation Studies - University of Illinois Science and Technology Center The goal was to understand the role of water and sanitation in the pathways linking agriculture, nutrition and health, particularly enteropathy. A second goal of this pilot study was to validate the use of a field-friendly water quality kit called the Aquagenx compartment bag test (ACBT). Water, Sanitation and Hygiene (WASH) has received renewed attention for nutrition and health. Pilot testing in June 2013 of the Aquagenx compartment bag test (ACBT) in Lira District (with partners Makerere University and University of Illinois at Chicago) showed that the ACBT effectively detects water source and household water contamination. In addition to the validation testing, the pilot study found that only borehole water was free of contamination, and that household water contamination was statistically linked to the presence of stunting in children of that household. Testing of both source and household water indicated that significant fecal contamination occurs in the household, providing evidence for interventions aimed at reducing household contamination. Process Research Sites: Kampala (national level analyses), Pader, Nebbi, Lira, Kole, Kanungu, Kabale and Kamwenge - Tufts University This aspect of the research involves collecting unique primary data derived from interviews with policymakers at all levels of governance in Uganda. In FY2013 key informants were identified in Uganda. National level stakeholder interviews have been conducted by Prof. Eileen Kennedy. District and lower level interviews will occur at the same time as the second panel survey (September – December 2014) as well as during the birth cohort study implementation. Uganda policy process research was conducted using structured and semi-structured surveys with policymakers and other stakeholders across eight sectors of activity relevant to the implementation of the Uganda multi-sector nutrition plan and the Uganda Scaling Up Nutrition movement: namely, agriculture (both cropping and livestock), health, nutrition, water supply, sanitation, local development, and other aspects of social welfare. Climate and Integrated Agro-Ecological Modeling – Purdue University Dr. Shively continues to be very productive with at least two peer-reviewed publications and some under review. Ugandan DHS and National Household Survey data were integrated with remote sensing (satellite) data provided by NASA to examine spatial and temporal patterns in agriculture and health. The study provided novel insights into direct pathways linking agricultural practices, biomass fuel for cooking, and human health. This validated approach will be used investigate higher level factors such as economic forces, market conditions and pricing, health infrastructure, and climate and weather which influence nutritional outcomes. 199 Furthermore, important analysis of secondary datasets is underway (led by Purdue University in collaboration with NASA) dealing with national level patterns and trends of agricultural productivity, climate change and health/nutrition outcomes. Econometric analyses of baseline data linking fruit and vegetable production to household food security and anemia in women - Collaboration with the International Food Policy and Research Institute (IFPRI)/Uganda The Nutrition Innovation Lab/Africa and IFPRI/Uganda are co-funding a post-doctoral fellowship for Dr. Kabunga Nassul. Dr. Nassul’s training is in agricultural economics and he is producing analyses from the baseline Uganda Panel dataset. This econometric analysis examined potential relations between F&V production, F&V intake, household food security and anemia outcomes of women in rural smallholder farming communities of Uganda. Household F&V production enhanced female caregiver intake of F&V, significantly improved household food security, and significantly decreased maternal anemia and makes an argument for the intensification of smallholder F&V production. Using propensity score matching and other techniques to control for bias, the study reveals that household F&V production increased F&V intake for female caregivers by 12 percentage points (from 64% to 76%). 200 APPENDIX J: PROJECT PUBLICATIONS AND RESEARCH BRIEFS Peer- Reviewed Publications 1. Jagger P, Shively G, Arinaitwe A. Circular migration, small-scale logging, and household livelihoods in Uganda. Population & Environment 2012;34:235-56. 2. Ainembabazi JH, Shively G, Angelsen A. Charcoal production and household welfare in Uganda: A quantile regression approach. Environment and Development Economics 2013;18:537-58. 3. Debela B, Shively G, Angelsen A, Wik M. Economic shocks, diversification, and forest use in Uganda. Land Economics 2012;88:139-54. 4. Jagger P, Shively G. Land use change, fuel use and respiratory health in Uganda. Energy Policy 2014;67:713-26. 5. Kabunga NS, Dubois T, Qaim M. Impact of tissue culture banana technology on farm household income and food security in Kenya. Food Policy 2014;45:25-34. 6. Webb P, Kennedy E. Impacts of agriculture on nutrition: nature of the evidence and research gaps. Food Nutr Bull 2014;35:126-32. 7. Henjum S, Manger M, Skeie E, Ulak M, Thorne-Lyman AL, Chandyo R, Shrestha PS, Locks L, Ulvik RJ, Fawzi WW, Strand TA. Iron deficiency is uncommon among lactating women in urban Nepal, despite a high risk of inadequate dietary iron intake. British JNutr 2014;112:132-41. 8. Thorne-Lyman A, Spiegelman D, Fawzi WW. Is the strength of association between indicators of dietary quality and the nutritional status of children being underestimated? Maternal & Child Nutrition 2014;10:159-60. 9. Masters WA, Webb P, Griffiths JK, Deckelbaum RJ. Agriculture, nutrition, and health in global development: typology and metrics for integrated interventions and research. Annals of the New York Academy of Sciences 2014. Briefing Papers 1. The Global Nutrition CRSP. The Nutrition Collaborative Research Support Program (Nutrition CRSP): Planned Activities in Nepal. Nutrition CRSP Research Briefing Paper No. 1. Boston, MA: Tufts University, December 2012. 201 2. The Global Nutrition CRSP. Nutrition CRSP Stakeholder Meeting: Selected Conclusions on Research Priorities for Nepal. Nutrition CRSP Research Briefing Paper No. 2. Boston, MA: Tufts University, December 2012. 3. Shrestha S, Manohar S, Klemm R. Nutrition CRSP Scientific Symposium: Agriculture, Food Security and Nutrition in Nepal: Taking Stock and Defining Priorities. Nutrition CRSP Research Briefing Paper No. 3. Boston, MA: Tufts University, May 2012. 4. Webb P, Kennedy E. Impacts of Agriculture on Nutrition: Nature of the Evidence and Research Gaps. Nutrition CRSP Research Briefing Paper No. 4. Boston, MA: Tufts University, October 2012. 5. The Global Nutrition CRSP. Integrating Agriculture and Nutrition actions to Improve Maternal and Child Nutrition: Metrics for Coordinated Research. Nutrition CRSP Research Briefing Paper No. 5. Boston, MA: Tufts University, October 2012. 6. Shrestha S, Manohar S, Klemm R. Agriculture, Food Security and Nutrition in Nepal: Taking Stock and Defining Priorities. Proceedings of the First Annual NUTRITION CRSP Scientific Symposium, Kathmandu, Nepal, March 21/22, 2012. Nutrition CRSP Research Briefing Paper No. 7. Boston, MA: Tufts University, December 2012. 7. Webb P, Shively J, Mulmi P. Sanitation in Nepal: Links to Nutrition and Research Priorities. Nutrition CRSP Research Briefing Paper No. 8. Boston, MA: Tufts University, December 2012. 8. The Global Nutrition CRSP. Nutrition Degree Programs in Nepal: A Review of Current Offerings and Gaps. Nutrition CRSP Research Briefing Paper No. 9. Boston, MA: Tufts University, November 2012. 9. Shively G, Sununtnasuk C, Brown M. Measuring the Links between Agriculture and Child Health in Nepal. Nutrition CRSP Research Briefing Paper No. 10. Boston, MA: Tufts University, December 2012. 10. The Global Nutrition CRSP. Stocktaking: Agriculture Degree Programs in Nepal. Nutrition CRSP Research Briefing Paper No. 11. Boston, MA: Tufts University, October 2012. 11. Magnani R, Gevorgyan A, Kurz K. Market Analysis of Complementary Foods in Nepal. Nutrition CRSP Research Briefing Paper No. 12. Boston, MA: Tufts University, November, 2012. 12. Shrestha R, Manohar S, Klemm R. Proceedings of the 2nd Annual Scientific Symposium, August 13-14, 2013. Nutrition Innovation Lab Research Briefing Paper No. 13. Boston, MA: Tufts University, January, 2014. 13. Webb P, Ghosh S, Kennedy E, West K, Klemm R, Sapkota D, Manohar S, Griffiths J. Research in Asia: Approach, Methods and Protocols. Feed the Future Nutrition Innovation Lab Research Briefing Paper No. 14. Boston, MA:Tufts University, (last updated November 5th, 2013). 14. Webb P, Ghosh S, Sapkota D, Davis D, Kennedy E, Gurung S, Baral, K. Governance of Nutrition Policies and Programming: Preliminary Findings from PoSHAN Process Research in Nepal. Nutrition Innovation Lab Research Briefing Paper No. 15. Boston, MA: Tufts University, July, 2014. 202 APPENDIX K: ME (TUFTS UNIVERSITY) COMMENTS Nutrition Innovation Labs – Asia and Africa Management Entity Technical Response to the External Evaluation Report The Management Entity expresses its gratitude to the External Evaluation Team (EET) for this professional review, and for the team’s thoughtful conclusions and recommendations. We also thank the many USAID staff and other stakeholders who supported the review and responded to requests for interviews. The generally positive tone of the recommendations is very welcome. We note with satisfaction that the EET concludes that the Labs a) have met or exceeded expectations, b) that both Nutrition Innovation Labs should continue into a second 5-year funding phase, and c) that fiscal management of both Nutrition Innovation Labs is effective. We are grateful for these important conclusions. Concurrent with this external evaluation, the ME used Y4 to self￾review progress. We therefore differ at times with the interpretations or weight given to certain conclusions drawn by the EET. Our comments below are therefore focused on clarifications or factual rectification on selected points where warranted, and proposals on how the ME would respond to priority actions suggested by the EET. We divide our responses into three sections: 1) ME, Board/TAC and partner roles, 2) research issues, and 3) capacity building agendas. 1. Management Entity, Board/TAC and Partner Management Roles 1.1 The EET concludes that the ME staff is overstretched (pp. 7, 36, elsewhere). The ME fully agrees. We initially sought a higher staffing level given the scale and complexity of our research and capacity-building activities. However, the original AOTR recommended less ME staffing, with which we have complied without sacrificing work quality. However, as activities have increased, as USAID Missions and government partners increasingly turn to the ME for guidance or assistance, as inevitable challenges have arisen which require more ME time and input, and as new Associate Awards emerge, we agree that the organization of the ME and staffing levels and functions have to be restructured, including an increase in several positions (as recommended by the EET). 1.2 The EET notes that the Technical Advisory Committee (TAC) is not as large or engaged as some TAC members desire (p. 6; pp. 20-22). We would like to clarify that although the TAC meets as a group once a year, the ME constantly interacts with individual members on an ongoing basis – seeking advice and guidance, and having them participate in research-based meetings. TAC members, such as Thilsted, Sundberg, Seligman, Klemm, Shrestha, and Quinn have participated in ME organized workshops distinct from TAC meetings, while Deckelbaum co-authored a NYAS paper with 3 ME members. In other words, TAC members have in fact played key roles beyond the annual gathering. Going forward, the ME proposes to keep the TAC better informed of progress through quarterly updates and conference calls. We will restructure the Board and TAC membership to insure that individuals are not at risk of potential conflict of 203 interest, by requiring that technical partners serve on the TAC but not be on the Board of Directors as well. We will also expand Board external membership. 1.3 The issue of insufficient or ineffective communication among partners is raised by the EET. The ME also identified this issue in its self-assessment and sought to address it; thus, the report does not fully reflect current reality. For example, communications around student training at Makerere (pp. 6, 7, 36) have been improved by the collective actions of the ME, Tuskegee, and Makerere University. The ME has already moved to identify clearer reporting, and other communications responsibilities, for its partners (p. 26). We will further increase clarity about roles and responsibilities among partners in-country, increase mutual regular communications with our US-based partners, hold more formal and regular interactions with national host partners, host more regular email-group updates (including all partners) on progress and issues, and significantly improve the website to allow for real-time updates on events, travel, findings, significant papers, etc. 1.4 The EET expressed concerns about the ME-Harvard relationship relating to Uganda and thus to relations with USAID/Uganda, and the “light presence” of Harvard on the ground in Uganda (p. 48). (It expresses similar concerns relating to the ME-Tuskegee relationship). The EET recommends that going forward the ME should “re-assess collaborations that are not working well in the first phase of funding” and states that the Uganda “project could be more efficiently run directly by the ME” (p. 34). The ME and Harvard have already begun such a restructuring of the relationship. This includes personnel changes, improving team leadership and responsibilities, an enhanced communications strategy, a more formal engagement strategy for the Mission and Uganda partners, and speeding up the process of data analysis to accelerate policy and program relevant information. 1.5 Early difficulties engaging with Missions and host governments is noted by the EET. This caused delays and added start-up costs to research in both countries. This has improved significantly due to a huge investment of ME time, involving trips, briefings and repeat presentations to a large cast of professionals who rotate in and out of their positions in government and mission jobs. This is a reality of development, but the need to constantly meet with, and educate, new policy officials in both countries has represented an unexpectedly large time commitment for the ME (see 1.1). We note the EET correctly identified a “policy hiatus” imposed by USAID/Uganda which had an adverse impact on activities such as policy briefings (2.1, below). 1.6 While the EET is generally laudatory of the ME and of the Project Directors (PDs), we believe the EET has not fully synthesized the data available to it. The EET has underestimated the difficulties of working in Uganda and criticizes the ME for delays in progress (pp. 47, 48) - while stating that much of this has been “outside of ME control.” (p. 48). In response to many challenges (e.g. outlined on page 47), the ME has: (1) restructured the operations on the ground in Uganda (1.4), (2) intervened directly to improve the student experience for the one group which experienced difficulties (1.3), (3) worked directly with the main Ugandan subawardee (Makerere University) to improve fiscal and management responsiveness, (4) insisted on the adherence to timelines and deliverables into all sub-awards (1.4, 2.3 below), and (5) invested heavily in better USAID/Uganda understanding of the project. These actions are not included in the report. The EET states that there is a “need for more direct involvement of the ME Project 204 Director for timely accomplishment of project objectives” (p. 48) yet notes, “The evaluators were surprised with the amount of work and effort the PIs are contributing to the project” (page 78). The EET does not fully recognize the huge energy and resources expended by the ME in being physically present in Uganda to ensure progress and backstopping the research in Uganda – both from the US and in country. The PD actually spent 50% of his time in Africa during one 12-month period of the project. The PD and Associate PD have taken on unexpected large operational roles in Uganda to address challenges there and the “light presence” on the ground of a collaborator. In the specific instance of the gap in hiring a replacement for our coordinator, the PD actually drove the replacement issue with team partners, and interviewed all of the top candidates in Uganda. 2. Research Activity 2.1 The need for more timely and effective dissemination of preliminary findings, analyses, and policy briefings is noted by the EET. However, we believe the EET might have better interpreted comments that the Nutrition Innovation Labs are not having made policy relevant data available in a timely way. Changes put into place by the ME (see for example 1.4 above, 2.3 below) have and will improve the timeliness of analysis and the preparation of policy briefs. As noted by the EET, the main obstacle to policy engagement in Uganda has been a US government “strategic pause” for engagement with the Office of the Prime Minister after a scandal, and our ability to engage with policymakers has been affected as collateral damage (pp. 25, 47, 52, 77). We have constructively engaged with USAID/Uganda to repair this situation (July, 2014) and the report’s conclusions do not reflect today’s reality. We also note that long-term, population-based studies do not produce immediate results and we believe some of these criticisms are off target. The early difficulties noted by the EET relating to Mission and government engagement (1.5) necessarily meant a delay in data production and analysis. The EET, in our opinion, has not adequately filtered or interpreted naïve commentary on this point, and may be at risk of have simply repeated comments rather than provide insightful interpretation. In both Uganda and Nepal, we are providing Mission briefs as soon as data is available, provide landscape policy briefs to each district we work in, host workshops and symposia for data dissemination, have presented our initial data at every opportunity at the Ministry and Mission level. We do note some recognition by the EET of these points (p. 89), and understand that it is important that any perceptions of delay be addressed both now and as we move forward. 2.2 The ME’s global dissemination activities have been not been fully taken into account. At a global level, the ME has invested considerable effort in dissemination for wide audiences in the US (at USAID headquarters, SPRING GLEEs around the world and other learning events (including webinars), as well as meetings hosted by the AAAS, the International Congress of Nutrition, other Innovation Lab global partner workshops). To date, over 3,000 people have heard talks by ME members and partners on Nutrition Innovation Lab research. 2.3 Where in-country dissemination is concerned, the ME has also organized many mission-specific and government-focused dissemination meetings/briefings each year in both countries. In Uganda there has been slower than anticipated production of reports (1.4) and dissemination of findings 205 (1.5, 2.1 above) due to political considerations outside of the ME’s control. The ME is eager to conduct more regular briefings of all stakeholders in both countries. To the ME’s credit, agreements for such briefings and Terms of Reference for the sharing of data have already been made concrete in Uganda after forthright discussions led by the ME. In addition, the ME has worked with USAID/Uganda to devise a mutually satisfactory mechanism for engaging policy makers in dissemination events. The ME has also stiffened its requirements of its partners to provide data for, and participate in analyses, now formally inserted into contractual language. The ME will also internally fund an enhanced ME capacity to generate this information. This change will provide the substrate for increasing the findings, analyses, and briefings to be distributed. In Nepal, there have been multiple large scientific symposia and at least 4 policy briefings of study design and findings from first round of data. In terms of publications, 7 papers have been published so far and several dozen more are in the conceptualization phase. While the policy brief series (available on the website) has so far addressed issues in a somewhat opportunistic fashion, the latest briefs (3 for Nepal, for example, each present early findings directly relevant to USAID interests and programming in the country as well as national government concerns). District-specific feedback briefs are currently being prepared for dissemination to policymakers in each of the field-sites in Nepal on the first round data, as has already been done in Uganda. We concur with the EET that a focus on such activities and making early findings more accessible to more people in a more timely fashion will serve to resolve perceptions of such delays. 2.4 The EET notes that the two labs work well together but “do not use the same design or share all survey research instruments.” The key term in that phrase is “all”. The EET has not achieved a balanced view of our capacity to achieve synergies between Nepal and Uganda, and to develop integrated metrics. Effort has been made to ensure that although the study design and research instruments used in both countries are appropriate to the different programs and policies studies, and able to answer key USAID study questions, not all elements are, or need to be, identical (the programs and context of Uganda and Nepal differ and some elements of the research design have to accommodate this). On page 33, the EET reports that some TAC members are concerned about a lack of “coordination of design and data collections methods” and feel that this has not received enough time during TAC meetings, and concludes that “fewer synergies are apparent.” However, factually the ME has organized two ‘survey harmonization’ meetings with partners for both countries, involving a mapping of indicators and variable coherence, and a cross-referencing of instruments. In its final recommendations (p. 89) the EET suggests there has been a “lost opportunity” to influence the broader research agenda. This is not correct. The research themes are the same in both Uganda and Nepal; we have conducted specific harmonization meetings; and that while the instruments are necessarily different given the two very different settings, there is more than sufficient overlap to assure synergy. Indeed, our concern for making sure the outputs are comparable has meant that we have devoted more time than is available in a TAC meeting to this issue. In addition, we have added identical components (such as assessing aflatoxin exposures) to both research platforms. It is also clear to us that the desire for construction of integrated agriculture-nutrition metrics must be informed by data and evidence – which we believe we are well situated to provide. 206 That said, new efforts will be made to ensure that pooling of data will be possible and that cross￾country analyses will be appropriate. This will include a formal review of the instruments and data to identify when and where opportunities to further harmonize data collection exist. 2.5 Deeper integration of researchers among partner institutions is recommended by the EET, and this is accepted by the ME. So far, there has been direct engagement with researchers at Purdue in instrument design for both country studies, as well as ongoing collaborative analyses of secondary data linked to remote sensing and agricultural outputs (in both countries). Formal analytical and author collaboration is being established across teams as data arrive from the field. All Tufts researchers are now IRB-approved co-investigators for the household survey work led by Johns Hopkins and Harvard, and vice versa for Tufts-led research. Similar arrangements are being elaborated with local academic partners. In Nepal, that means co-authorship and substantive research leadership from Patan Academy, NARC and Tribhubvan University, and in Uganda, from individuals at Makerere and Gulu Universities, and IFPRI/Uganda. The ME will require local partners assume greater responsibility in briefing stakeholders on progress of the Nutrition Innovation Labs’ activities. 2.6 The EET raises a question about local ownership of data and participation in research outputs. The majority of outputs (policy briefs, presentations and published articles) already include at least one Nepali or Uganda co-author (sometimes local and sometimes US-based). The ME and its partners are making a concerted effort to ensure full engagement by local partners in the research endeavor. Their substantive involvement in technical workshops and, in Nepal’s case, the annual Scientific Symposia, demonstrate genuine engagement, and a wider range of local co￾authors will be involved as the rich datasets start being mined for publication. Already there are at least a dozen draft papers in the pipeline which have local researcher involvement in analysis and authorship. In Uganda a formal, written data ownership and authorship agreement exists and was agreed to 2 years ago. A similar agreement is being drafted for Nepal. The pace of research output will grow rapidly going forward, especially as data become available to students for thesis work in partner institutions in both countries. The ME will make data publically available as per USAID policy. 2.7 The EET fails to recognize the important current roles of agriculturists in the project and puzzlingly states the Nutrition Innovation Lab is lacking in this discipline (e.g., pp. 10, 12, 87, elsewhere). The apparent lack of agriculture specialists and agricultural economists in the research teams reflects the fact that papers on the agriculture-to-nutrition pathways are only now emerging from the empirical data. Profs. William Masters and Steven Block at Tufts, Steven Vosti at UC Davis (on our TAC), Shively at Purdue, Prof. Bashaasha (Principal of the College of Agriculture and Environmental Sciences at Makerere University), and Dr. Nassual Kabunga, an Ugandan and half-time Nutrition Innovation Lab member stationed at IFPRI/Uganda are all agricultural economists actively involved in construction of agriculture-specific variables and equations for upcoming analyses. Prof. Devendra Gauchan is an agricultural specialist in Nepal’s Agricultural Research Council. These professionals are directly involved in composite variable construction, analysis and paper writing. Additional agronomists (such as Prof Griffin involved as an adviser here at Tufts) and local partners in the research through our collaborations with the Ministry of Agriculture of Nepal, and agronomists implementing USAID programs in both countries. So the absence of agriculture expertise is more in appearance than in fact. Moreover, 207 we have supported students obtaining degrees in Agricultural sciences including extension. As the research moves forward the contribution of the agriculturalists will be more evident. 2.8 More work is needed on novel metrics. The ME has organized one workshop and has published one theoretical paper in this area. Empirical data soon to be in hand from two full rounds of field surveys will support the testing of innovative indicators or clustering of ‘families’ of variables. Sound empirical data are required to support effective metric development. The ME is currently undertaking data analysis to explore the feasibility of new metrics for measuring the policy process (the commitment and capacity aspects of nutrition governance). Findings on the latter analysis are to be presented at the next Nepal Scientific Symposium. 2.9 The EET incompletely analyzes the single major sub awardee model (p. 34) and criticizes the Innovation Lab for “putting all the eggs in one basket” for our research (p. 48). This is puzzling since the Nutrition Innovation Lab has been required by USAID to conduct “deep dive,” in￾depth population based research, which is not amenable to a traditional CRSP small-grant program. USAID, since the time of our award, has required us to focus on single country questions which could not be dealt with by a set of small sub-awardee projects, and in fact required us to have a single major sub-awardee in both Nepal and Uganda, although there are multiple local partners in each case. We have a diverse portfolio of research questions which use an overlapping set of common operational platforms (see 2.11 below). When the EET suggests a small grants program to address additional research questions, it does so without realistic consideration of the budgetary limitations we must live within. 2.10 While the many pieces of the Innovation Labs’ complex research agenda may appear to be opportunistic and piecemeal, they actually represent a thoughtful balancing of strategic research questions with evolving questions that involve time-bound windows of opportunity. All of the discrete studies that make up our portfolio of research are appropriate to answering defined questions (i.e. fully consistent with the core research agenda). Elements of our research agenda relating to neglected biological pathways connecting agriculture to nutrition, for example mycotoxins and WASH, have been layered atop our primary research vehicles. That said, the ME will seek to more clearly enunciate how the apparently disparate pieces fit together to answer the core questions at the heart of the Feed the Future and Innovation Labs’ agenda. 2.11 It is factually incorrect when the EET states the Nutrition Innovation Lab has not made survey instruments available to local partners or involving local leadership in data analysis. Formal requests for collaboration and for sharing of the instruments have been met. 2.12 The EET has not fully appreciated the leadership and visibility of our partners, nor our efforts to work with other Feed the Future Innovation Labs. We are actively working with the Peanut and Mycotoxin Lab, the Horticulture Lab, the Adapting Livestock Systems to Climate Change Lab, Aquaculture & Fisheries Lab and others. For example, on page 79: “One researcher who works on the Grain Legumes Innovation Lab indicated that some other Innovation Labs are unclear on what the Nutrition Innovation Lab does” and the comment is that more leaders need to be seen than Webb and Griffiths. We are not responsible if despite multiple presentations and meetings these individuals are clueless, and wonder if this is a repeated comment without true substance that has not had the benefit of reflective interpretation. Dr. Griffiths, the 208 Nutrition Innovation Lab-Africa program director, chaired two Innovation Lab Council meetings (in Nepal and Washington DC) and has spoken by telephone and/or met with the program leadership of every new Innovation Lab. The statement about leadership visibility is factually incorrect. Drs. Kennedy, Ghosh, and Shively, and other core partners have made multiple international and national presentations and presented at USAID Webinars and in-person seminars, ICN, Experimental Biology, and other venues. The ME has a lean structure and all members have made many presentations. 3. Capacity Building 3.1 The EET concludes that both Nutrition Innovation Labs are “making an important contribution to strengthening the training and capacity building in nutrition for scientists working across sectors.” This activity will remain a core priority for both Labs. We note with pleasure that participants of both short and long-term training express satisfaction with the quality of their training. The ME will do more to strengthen such experiences, widen the pool of candidates to include lesser-known universities and institutions outside of capital cities, and more systematically evaluate their experiences. 3.2 The EET expressed concern about support for and mentoring of students at Makarere. In response, we note this issue relates to a specific cohort of students and not to prior groups. The EET has very much focused on the difficulties of one cohort group of students in Uganda but not counterbalanced this with the experiences of other Makerere students whom we have supported. We have trained over 20 students in Uganda and the US, and hosted students to attend short term training in India and the US. Their experience has been much more positive. For the Makerere students who experienced difficulties the ME has directly and effectively resolved their problems. The Uganda Principal Investigator, Professor Bashaasha, and the ME have intervened to assist these students, deliver funding for their studies, and propose enhanced mentoring. The report does not reflect this (e.g., pp. 7, 72, elsewhere) and its concerns are outdated. Changes in the support and mentoring structure have already been undertaken so that future cohorts of students will enjoy the same experience their earlier colleagues had. Clearer guidance and ‘rules of engagement’ will be provided to partners, students and faculty at Makerere, thus clarifying expectations, accountability, and satisfaction. This issue is given unbalanced attention in the report. 3.3 The EET underestimates our linkages with local universities, stating that the Nutrition Innovation Lab “should ensure that lesser known universities outside the capital cities (Uganda in particular) are engaged in research and should seek to assist with capacity building … [perhaps through] … a competitive small grants process.” (p. 71). First, we have very substantially supported research at Gulu University and sought out partnership with researchers at Mbarara University. The EET report almost completely neglects our collaboration at Gulu. We have also reached out to faculty from Kyambogo University in terms of capacity building. In Nepal, there are no formal nutrition training programs and the Nutrition Innovation Lab-Asia is developing the curriculum for a training and research program. Thus we do not believe this comment factually represents our efforts. Second, our level of funding does not currently allow a competitive small grants program although we would warmly welcome such resources. As our cohort study rolls out in the north and south of Uganda, publications from our work in Gulu, and the program 209 development work in Nepal matures, the depth of our linkages will be more apparent. In the future, additional effort will be spent assisting local institutions to develop their own technically￾rigorous short-term training workshops, as well as longer term curricular offerings. Examples of such opportunities exist in Malawi, in Uganda at Gulu University, and in Nepal through a replication of the BBNC initiative. The intent remains to build institutional as well as individual capacity to undertake cross-disciplinary research that is policy-relevant and timely. Building links to ongoing networks (such as A4NH, the IMMANA training and small grants activity, and the Global Panel on Agriculture and Food Systems for Nutrition) with stronger local partner engagement with global thinking and dialogues. Enhancing the value of the Nutrition Innovation Lab website will also be a top priority, for which steps have already been taken. Significant upgrades will be made that allow visitors to focus on research themes and findings, explore published and unpublished literature on key issues under the Feed the Future mandate, and better link the research and capacity-building agendas not only of Uganda and Nepal, but also Malawi and new Associate Award countries as they come to fruition. 210