Submitted by: Save the Children Federation, Inc. In collaboration with: CORE Group Food for the Hungry Mercy Corps TANGO International Submitted to: United States Agency for International Development (USAID) Office of Food for Peace Cooperative Agreement No.: AID-OAA-A-10-00006 Adam Reinhart Agreement Officer’s Representative USAID RRB/DCHA/FFP 1300 Pennsylvania Avenue Washington, DC 20523 Tel: 202-712-5365 Disclaimer: The Technical and Operational Performance Support (TOPS) Program was made possible by the generous support and contribution of the American people through the U.S. Agency for International Development (USAID). The contents of this report were created by The TOPS Program and do not necessarily reflect the views of USAID or the U.S. Government. Recommended Citation: The TOPS Program. 2018. The Technical and Operational Performance Support (TOPS) Program Final Report: August, 2010 to January, 2018. Washington, DC: The TOPS Program. Photo Credits: Top left: Andrea Aragon/Save the Children, Top right: Ahmad Baroudi/Save the Children, Bottom left: Colin Crowley/Save the Children, Bottom right: Shashank Shrestha/Save the Children Contact: The TOPS Program c/o Save the Children 899 North Capitol Street NE, Suite 900 Washington, DC 20002 info@thetopsprogram.org www.TheTOPSProgram.org www.fsnnetwork.org The Technical and Operational Performance Support (TOPS) Program Final Report August 2010 to January 2018 Final Report: August 2010 to January 2018 Contents i Contents Executive Summary .................................................................................................................. i 1. An Overview of The TOPS Program.................................................................................... 1 TOPS Structure......................................................................................................................................2 TOPS Strategies.....................................................................................................................................3 TOPS Program Principles ......................................................................................................................3 TOPS Consortium and Specialty Areas..................................................................................................4 TOPS Timeline.......................................................................................................................................4 2. TOPS Achievements and Results........................................................................................ 6 Program Result 1: Knowledge Capture.................................................................................................6 Core Competencies ...................................................................................................................6 Self-Assessments in Core Competencies...................................................................................8 Program Result 1: Knowledge Capture and Program Result 2: Knowledge Generation.....................9 Identify Existing Tools and Gaps in Tools..................................................................................9 Fill Gaps by Modifying or Creating Tools.................................................................................10 ‘Recommended’ Designation for Select Tools ........................................................................11 Identify and Promote Promising Practices..............................................................................14 Promising Practices and Emerging Topics...............................................................................16 Program Result 2: Small Grants..........................................................................................................18 Program Result 3: Knowledge Application .........................................................................................21 Traditional Skills Delivery ........................................................................................................21 Pre- and Post-Test Results for Capacity Strengthening Workshops .......................................22 Aiming for Replicability ...........................................................................................................23 Program Result 4: Knowledge Sharing................................................................................................25 The Food Security and Nutrition (FSN) Network.....................................................................25 Knowledge Sharing Meetings: Ten Signature Events..............................................................29 Other Knowledge Sharing Events............................................................................................32 3. TOPS Performance Monitoring Plan................................................................................. 34 PMP Indicators: The TOPS Small Grants Program ..............................................................................34 PMP Indicators: Knowledge Sharing...................................................................................................35 PMP Indicators: Documentation of Core Competencies and Promising Practices ............................36 PMP Indicators: Technical Capacity Strengthening ............................................................................37 PMP Indicators: Tools.........................................................................................................................38 4. The TOPS Program Effects and Impact ............................................................................. 40 5. The TOPS Program Management..................................................................................... 48 TOPS Management.............................................................................................................................48 Staffing Structure................................................................................................................................48 Program Advisory Committee ............................................................................................................49 The Technical and Operational Performance Support (TOPS) Program ii Contents Technical Contributions of TOPS Consortium Members....................................................................50 Save the Children.....................................................................................................................50 CORE Group.............................................................................................................................52 Food for the Hungry ................................................................................................................52 Mercy Corps ............................................................................................................................53 TANGO International...............................................................................................................53 Associate Awards................................................................................................................................55 Small Grants Program.........................................................................................................................57 6. Comparison of Budget to Expenditures............................................................................ 58 Annex I: The TOPS Program Organization Chart..................................................................... 60 Annex II: TOPS Small Grants Program Summary .................................................................... 61 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators ........................ 81 Annex IV: List of TOPS Products............................................................................................. 89 Annex V: TOPS Leader with Associate Awards 2010-2018...................................................... 98 Annex VI: TOPS Leader with Associate Awards – Final Reports.............................................103 Annex VI a) Sahel Resilience Workshop: Associate Cooperative Agreement No. AID 625-LA-14- 00001 ................................................................................................................................................103 Annex VI b) Humanitarian to Development Resilience Collaboration Training (HADA): Associate Cooperative Agreement No. AID-625-LA-14-00002 .........................................................................103 Annex VI c) Advanced Resilience M&E Training for Field Practitioners (ARMET): Associate Cooperative Agreement No. AID-625-LA-15-000002 .......................................................................103 Annex VI d) Asian Resilience Measurement & Monitoring Experiential Learning Event (ARMMELE): Cooperative Agreement No. AID-486-A-16-00002...........................................................................104 Annex VI e) Baseline Study for Enhancing Resilience and Economic Growth in Somalia Program: Associate Cooperative Agreement No. AID-FFP-LA-15-000001 .......................................................104 Annex VI f) Review of Cash-based Emergency Food Assistance Program: Associate Cooperative Agreement No. AID-FFP-LA 16-00002 ..............................................................................................104 Figures Figure 1: TOPS Framework and Theory of Change .......................................................................................2 Figure 2: Infographic about the TOPS Commodity Management Toolkit...................................................10 Figure 3: TOPS Small Grant-Making Criteria ...............................................................................................18 Figure 4: TOPS Small Grants by Technical Sector .......................................................................................19 Figure 5: Uptake of the TOPS Small Grants Program..................................................................................20 Figure 6: TOPS Capacity strengthening Workshops by Topic .....................................................................21 Figure 7: Average Score Pre- and Post-Test Capacity strengthening Workshops......................................23 Figure 8: FSN Network Website Statistics...................................................................................................28 Figure 9: Subscriptions to FSN Network News ...........................................................................................29 Figure 10: Number of Single Topic Knowledge Sharing Events Held..........................................................32 Final Report: August 2010 to January 2018 Contents iii Tables Table 1: TOPS Consortium Members and Their Focus Areas .......................................................................4 Table 2: TOPS Timeline: Program Years (PY) and Calendar Years.................................................................5 Table 3: FSN Network-Recommended Technical Tools..............................................................................12 Table 4: Task Forces and Year of Formation...............................................................................................26 Table 5: The TOPS Program’s Ten Knowledge Sharing Meetings...............................................................31 Table 6: Associate Awards Made During The TOPS Program .....................................................................55 Table 7: TOPS PY1-PY7 Budget vs. Actuals..................................................................................................59 Acronyms ANRM agriculture and natural resource management APR annual program report CM commodity management FAFSA Food Aid and Food Security Assessment FFP Food for Peace FSN food security and nutrition ICB Institutional Capacity Building IDEAL Implementer-Led Design, Evidence, Analysis and Learning IP implementing partner IPR Intermediate Program Result IYCF Infant and Young Child Feeding LWA Leader with Associate M&E monitoring and evaluation NGO nongovernmental organization PMP performance monitoring plan PR program result PVO private voluntary organization PY program year (October through September) REAL Resilience Evaluation, Analysis and Learning RFA Request for Applications SBC social and behavioral change TOC Theory of Change TOPS Technical and Operational Performance Support Program USAID United States Agency for International Development Final Report: August 2010 to January 2018 Executive Summary i Executive Summary The USAID Office of Food for Peace (FFP) launched The Technical and Operational Performance Support (TOPS) Program1 on August 13, 2010, by awarding the Leader with Associate (LWA) Cooperative Agreement to Save the Children and a consortium of private voluntary organizations (PVO). Reflecting FFP’s strong commitment to organizational learning, TOPS was created to facilitate knowledge exchange among practitioners in the food security community for two primary purposes: to improve the efficiency and effectiveness of U.S. Government-funded development food security activities; and to strengthen the knowledge, skills, and practices of the implementing partners (IPs) in delivering those activities to improve the lives of vulnerable families around the world. Save the Children, the leader on the award, provided overall vision and managerial oversight for TOPS, and led a consortium of four PVO partners experienced in food security and nutrition programming globally. The consortium comprised CORE Group (knowledge management); Food for the Hungry (social and behavioral change); Mercy Corps (agriculture and natural resource management, resilience); and TANGO International (monitoring and evaluation, resilience). Save the Children brought technical expertise to TOPS in nutrition and food technology, commodity management, and gender. The TOPS consortium members worked together to achieve four program results (PRs): Knowledge Capture (identify the knowledge and skill needs of practitioners and existing information and tools to meet those needs); Knowledge Generation (synthesize, test, and produce information to meet unfilled needs); Knowledge Application (deliver skills and information to practitioners via capacity strengthening and other learning approaches); and Knowledge Sharing (facilitate the robust exchange of information and skills, and support knowledge sharing across the food security community). The four PRs contributed to improved knowledge and practices and stronger systems among Food for Peace implementing partners, supporting achievement of the TOPS strategic objective: Highest quality information, knowledge and best practices for improved methodologies in Title II food aid commodity program performance are identified, established, shared and adapted. The TOPS Program operated from August 2010 through January 2018. During seven-plus years, the five consortium members collaboratively engaged with scores of food security actors in hundreds of activities globally. TOPS staff, implementing partner organizations, and USAID/FFP experts worked together to learn and share knowledge, advance thinking, and apply improved practices in food security programming. TOPS addressed the four PRs using a variety of strategies and techniques, from interactive learning/sharing events to small grants, and more. At the end of each program year, TOPS submitted individual annual progress reports (APRs) to FFP describing that year’s successes and challenges in delivering this broad range of activities, interactions, and outputs. This final report, however, addresses the larger question, “What did TOPS accomplish in seven-plus years?” It will cover the full seven years in a comprehensive look at the sum of the work in TOPS’ program activities, achievement of program targets, and overall impact on thinking and practice in the food security and nutrition community. 1 In this report also referred to as The TOPS Program or TOPS The Technical and Operational Performance Support (TOPS) Program ii Executive Summary For its first program result―PR 1 Knowledge Capture―TOPS interacted with food security and nutrition practitioners to identify what they needed to know and what tools and information existed to help ensure they could meet those needs. Each TOPS technical team, with a related task force made up of volunteers from the larger community of practice, researched and defined the minimum core competencies that implementing staff in the field required to perform their jobs. These task forces developed, tested and refined self-assessment tools corresponding to the core competencies. Together, the core competencies and self-assessments helped FFP implementing partners (IPs) conduct more targeted recruiting, develop capacity strengthening plans for their staff, and guide practitioners to understand their own capacities and skill gaps. Identifying core competencies in food security and nutrition technical sectors early in The TOPS Program helped to focus topical priorities for subsequent program activities throughout the life of TOPS. Bridging PR 1 Knowledge Capture and PR 2 Knowledge Generation―identifying and creating tools and information for the food security and nutrition community―was a major component of The TOPS Program. TOPS began with the assumption that useful tools and approaches existed within PVOs, but that these were not widely shared beyond the individual organizations. TOPS’ initial reviews and searches for existing tools netted fewer than expected. Consequently, TOPS devoted more time and resources than originally planned to fill knowledge gaps. TOPS’ technical specialists, often working with the related task forces, researched, created, tested, and modified tools across all sectors. In addition, TOPS awarded small grants to others in the food security and nutrition community to create needed tools. By close-out, TOPS had published over 100 products―including products from the Small Grants Program―for food security practitioners, including field guides, handbooks, and facilitation manuals. Of the total number of TOPS products, 36 were designated on the FSN Network website as ‘FSN Network Recommended,’ indicating that practitioners could consider them state-of-the-art guidance.2 TOPS also devoted time and effort to continually identifying and promoting promising practices in FFP-funded activities, aiming to spark discussion and continuous improvement among practitioners. TOPS PR 2 also included the Small Grants Program (noted above), which filled a real gap in the food security and nutrition community since resource constraints often discouraged IPs from exploring and testing creative ideas for program improvement. Over seven years, TOPS made 71 small grants that recipients used to develop tools, conduct learning events, design and implement new technical content, pilot promising practices, and collect evidence to strengthen and improve the quality of food security programs. The 71 small grants covered all of the TOPS technical areas, many integrating several technical or cross-cutting sectors. Through the FSN Network and other channels, TOPS widely shared the 37 tools resulting from small grants. TOPS disbursed $4,071,604.12 in small grants, with recipients’ cost share contributions of $546,911.50. TOPS intentionally avoided creating standardized, off-the-shelf training and learning “packages.” TOPS focused on working with IP organizations to identify their specific needs or gaps and then designing customized activities or products for face-to-face capacity strengthening events for their field and headquarters staff. This was the focus of work in PR 3 Knowledge Application. TOPS delivered over 120 capacity strengthening workshops, reaching more than 3,000 practitioners. Largely targeting PVO staff at different personnel levels in headquarters or field locations, TOPS conducted workshops and learning events covering all TOPS technical sectors, often integrating sectors or cross-cutting themes (e.g., 2 TOPS reported 44 recommended tools rather than the 36 found on the FSN Network website. The remaining 8 tools were removed from the FSN Network, as some of them were found to need more detailed guidance or were no longer recommended in subsequent years based on the findings and recommendations of further reports, such as the FAFSA II. Final Report: August 2010 to January 2018 Executive Summary iii gender, resilience). TOPS tailored each workshop to the participants, guided by the core competencies. TOPS staff also built replicability into their workshop curricula, enabling participants to cascade their learning beyond the workshop to their colleagues and partners in the field. PR 4 Knowledge Sharing drew from and supported the ongoing knowledge generation and application activities of the other PRs. In PR 4, TOPS established the FSN Network, an open community of peers in which FFP, implementers and other food security stakeholders collaborated to identify, grow, test and share evidence-based tools and promising practices. The mechanisms TOPS used to create this open environment included:  Eight task forces, corresponding to TOPS’ technical areas, formed and guided by the TOPS technical leads. Each task force was a convening mechanism for experts and practitioners from PVOs, USAID/FFP activities, academic and research institutions, and more. As members, themselves, of the larger food security and nutrition community of practice, they were key informants in identifying needs of the food security community and ways TOPS could meet them.  The Food Security and Nutrition (FSN) Network web portal3 , the online home of the global community of practice. The website enables visitors to exchange knowledge for strengthening programming and knowledge and information useful in shaping agendas and influencing donor priorities. The site contains a searchable resource library with 800+ guides, tools, and training materials; an events page for future and past meetings, workshops and webinars, and repository of essential materials and products from past events; discussion forums; and more. By the end of TOPS, the FSN Network site had received over a quarter-million visits and delivered nearly 95,000 downloads to visitors.  The FSN Network Newsletter4 published 280+ issues during the life of TOPS. Reaching approximately 2,600 subscribers, the newsletter provides updates from The TOPS Program and FFP, alerts readers to upcoming events, advertises job openings, and links them to recent resources and articles.  TOPS organized and hosted ten FSN Network Knowledge Sharing Meetings (KSMs)―five held in the Washington, DC area and five in several African countries and in Bangladesh. These knowledge sharing meetings brought together implementers, donors, and researchers from around the world for peer-to-peer knowledge sharing and networking around food security and nutrition. The KSMs provided meaningful, face-to-face interactions to maximize the flow of information among implementers, policymakers and researchers. They also helped participants forge important connections enabling knowledge exchange to continue beyond the events. Over the life of the award, these signature TOPS events became highly valued opportunities for the community of practice for learning and sharing in an open, collegial environment. The flexibility—adaptive management capacity―of TOPS enabled it to respond to many new and emerging interests, generated―often on short notice—by the donor and the community of practice. Over its lifetime, TOPS was able to host dozens of other, smaller knowledge sharing events in response to stakeholder requests. Most took place in Washington, DC, but TOPS adopted mechanisms to enable stakeholders and interested parties in other U.S. or international locations to participate online via 3 The FSN Network web portal is being maintained beyond the end of TOPS under the TOPS Bridge Award. 4 The FSN Network Newsletter, similar to the web portal, is being produced and distributed under the TOPS Bridge Award. The Technical and Operational Performance Support (TOPS) Program iv Executive Summary webinars. TOPS also facilitated over a dozen half-day stakeholder consultations between FFP and implementing agencies on topics such as RFA guidance on proposal preparation, budgeting guidance, M&E indicators, and comments from the food security and nutrition community on the proposed new Food for Peace 2016–2025 Food Assistance and Food Security Strategy. TOPS staff also responded to frequent invitations to contribute to events hosted by others, typically by presenting sessions in workshops or conferences. The TOPS Program was a Leader with Associate (LWA) Cooperative Agreement. This permitted the U.S. Government to issue Associate Awards through the TOPS agreement for subjects related to the overall mission of The TOPS Program. Upon receiving a scope of work (SOW) for a possible Associate Award, often from USAID/FFP, Save the Children shared the SOW with the consortium members for review. Save the Children then issued a sub-grant to the member most qualified to perform the requested tasks. Associate Award topics included: implementing partner and donor staff training on resilience concepts, program design, and measurement; randomized control trials measuring cost-effectiveness in “graduation” programming for refugees in Uganda; a five-year award for agriculture, natural resource management and off-farm livelihoods; a five-year WASH program; an award to collect baseline and endline data to measure activities within the Feed the Future Learning for Resilience project in Ethiopia; and more. Over the life of TOPS, 13 Associate Awards were issued for a total of $26,675,451. The TOPS Program’s monitoring framework was aligned with the TOPS approach and focused predominantly on outputs (see Figure 1). Its 29 indicators, tracked annually, were ‘must know’ items management used to understand progress towards results achievement. In all program years, TOPS met or exceeded at least three-quarters of its targets for the 29 indicators in the program monitoring plan (PMP). No baseline survey was conducted at the beginning of TOPS, but an external consultant conducted a Mid-Term Evaluation. FFP considered conducting a final evaluation of TOPS near the end of the award, but did not arrange it. A final evaluation of TOPS is planned to take place during 2018, under the TOPS Bridge Associate Award. TOPS achieved—often exceeded—most of its technical and output targets. Although not identified or measured as an output target, a unique, overarching TOPS achievement was in creating and facilitating opportunities to involve historically competitive stakeholders in an inclusive, open community of mutual learning. When Save the Children responded to FFP’s Request for Applications (RFA) in 2009 with a proposal for what became The TOPS Program, neither entity had a proven blueprint for facilitating information generation, learning, and sharing among PVOs, FFP and other key stakeholders as an egalitarian community of practice. TOPS’ leadership, with welcome support from FFP and PVO leaders, presented The TOPS Program as an open community, in which all were welcome to participate. As the value and usefulness of TOPS’ work became evident to the food security and nutrition community, cooperating participants shared enthusiastically across organizational boundaries, striving for strengthened internal capacities and excellence in programming for the benefit of food insecure populations. The original TOPS LWA Cooperative Agreement was for five years from 2010 to 2015. In late 2015, recognizing the value of TOPS activities to the overall community of practice, FFP extended the award for two more years to 2017 and added $10 million to the budget. Subsequent short-term no-cost extensions carried The TOPS Program to its close-out at the end of January 2018. Final Report: August 2010 to January 2018 An Overview of The TOPS Program 1 1. An Overview of The TOPS Program The TOPS Program (August 13, 2010 to January 31, 2018) focused on capturing, generating, applying, and disseminating high quality information, knowledge, and promising practices in development food security activities. 5 It was conceived, designed and implemented with and for USAID’s Office of Food for Peace (FFP) and the PVOs that implement FFP’s portfolio of activities that tackle food insecurity and help build resilience in communities facing chronic poverty and recurring crises. Looking well beyond direct food aid, these activities encompass work in agriculture, livelihoods, maternal-child nutrition, gender, social behavior change, monitoring and evaluation, disaster risk reduction, and more. In this report, this body of activities is referred to as FFP-funded development food security and nutrition activities or, simply, FFP activities, and the PVOs that implement them as FFP implementing partners (IPs). Unless specified, these do not include FFP emergency response activities. TOPS’ overarching mission was to strengthen the U.S. Government’s efforts to end global hunger by facilitating knowledge sharing and learning among FFP partners and other stakeholders, leading to improved, more effective practices. While USAID had previously provided Institutional Capacity Building (ICB) grants to individual FFP partners, the knowledge, tools, and good practices that arose tended to stay within PVOs. With no mandate, structure, or incentive to share promising practices with one another, PVOs rarely did so, and opportunities to benefit the larger food security community were missed. With the TOPS Request for Applications (RFA), USAID sought a program that would systematize networking opportunities and inter-partner interactions and learning that could be maximized and directed for the benefit of food insecure populations. Save the Children and its consortium members designed The TOPS Program in response to USAID’s aspiration, embodied in the 2009 RFA, that FFP activities could improve if PVOs learned with and from one another to capture and build on success, strengthen design and innovation, and monitor/evaluate good practices. 5 Originally a five-year program, TOPS was granted a two-year cost extension in 2015 and three no-cost extensions through the end of January 2018. In its 2009 RFA, USAID defined TOPS’ strategic objective as… Highest quality information, knowledge and best practices for improved methodologies in Title II food aid commodity program performance are identified, established, shared and adapted. …and its three aims as: food aid grantees have improved knowledge; food aid grantees use improved practices; and improved systems are established and functioning for ongoing capacity strengthening, knowledge management and networking. The Technical and Operational Performance Support (TOPS) Program 2 An Overview of The TOPS Program TOPS Structure Save the Children and consortium members structured The TOPS Program around USAID’s own, four￾stage, knowledge management cycle. Each TOPS program result (PR) is described below in Figure 1. Each PR was iterative, supporting and benefiting from a knowledge management process that produced quality information and best practices, which were used to strengthen implementing partners’ (IPs) capacity. Together they contributed to improved knowledge, practices and systems within and among IPs, and thereby supported achievement of the strategic objective. Within each PR, Intermediate PRs (IPRs) were delineated, and linked to the TOPS M&E framework as shown in Figure 1. Figure 1: TOPS Approach and M&E Framework Final Report: August 2010 to January 2018 An Overview of The TOPS Program 3 TOPS Strategies The TOPS Program used five strategies to guide its work with FFP partners and a global network of food security and nutrition practitioners, researchers, and other stakeholders: 1. A directed program of capacity strengthening activities for FFP implementing partners and related actors or stakeholders, led by TOPS technical staff. 2. An inclusive community of practice through the Food Security and Nutrition (FSN) Network and its task forces and interest groups; membership was open to all FFP implementing partners and other food security and nutrition stakeholders. 3. A Program Advisory Committee made up of food security and nutrition stakeholder senior manager representatives to provide strategic guidance on TOPS activities on a regular basis. 4. A food security and nutrition website (www.fsnnetwork.org), the foundation of the TOPS knowledge-sharing system. 5. A Small Grants Program that promoted the creation, use, and adaptation of capacity strengthening tools, and encouraged evaluation and documentation, to build the evidence base as part of TOPS’ work to disseminate best practices to food security and nutrition stakeholders, with a focus on field-led and collaborative initiatives. Guided, fostered and grown by TOPS, the global network of food security and nutrition practitioners readily participated in the identification, analysis, and sharing of information, tools and practices to improve the performance of FFP activities. TOPS Program Principles The TOPS Program was unconventional in purpose, creative in approach, and unique in its focus on knowledge management and learning. It was a calculated leap of faith for FFP to base a program on knowledge management and sharing activities, and it was an exciting challenge for Save the Children and the TOPS consortium members, despite having few precedents to guide their actions. The TOPS Program began with two linked principles that guided all actors as the journey progressed:  Openness and inclusiveness: From the start, TOPS adopted a “Big Tent” approach―the entire community of practice was invited to join, share, set communal priorities and follow jointly identified interests. TOPS’ initial concerns that engrained competitive habits among PVOs might stymie sharing were quickly dispelled. The response was positive, enthusiastic, and shared across the community. This open, inclusive atmosphere and lively peer-to-peer consultation among practitioners, donors, and other actors in the food security and nutrition community is a signature TOPS accomplishment.  Dialogue among equals: TOPS and FFP both recognized the need for collaboration, mutual respect and constructive consultation between donor and implementing partners to jointly advance food security activities. Within the FSN Network and TOPS-organized forums, therefore, FFP and PVOs engaged as equals, taking advantage of the opportunities for mutual consultation The Technical and Operational Performance Support (TOPS) Program 4 An Overview of The TOPS Program during activities hosted by TOPS. Both sides expanded the comfort zone of donor-PVO relationships and the overall food security community benefited. TOPS Consortium and Specialty Areas Save the Children led a consortium of five organizations with a wealth of experience in sectors vital to The TOPS Program. Together, the TOPS consortium focused on thematic and technical areas crucial to food security and nutrition programming. Each consortium member led activities and bore responsibility for attaining program results in one (or more) specialties, as shown in Table 1 Table 1: TOPS Consortium Members and Their Focus Areas CONSORTIUM MEMBER THEMATIC AREA AND TYPE Mercy Corps Agriculture and Natural Resources Management (ANRM) Technical Save the Children Commodity Management Nutrition and Food Technology Gender Cross-Cutting CORE Group Knowledge Management TANGO International Monitoring and Evaluation (M&E) Resilience Food for the Hungry Social and Behavioral Change (SBC) The thematic areas were not restrictive or “silo-ed,” rather, their technical focus was a way to convene and organize TOPS’ work. These themes or sectors became platforms for lively cross-topic and multi￾sectoral inquiry. Over time, there was increased integration. For example, five of the thematic areas, such as gender or resilience, were inherently cross-cutting, and applicable to all development food security activities. Save the Children, as the TOPS leader, was responsible for overall program management and direction, ongoing coordination with FFP, and administration of the TOPS Small Grants Program. TOPS Timeline In this report, activities and achievements are identified according to program year, or PY. Correspondence between PYs and calendar years is shown in the timeline below. In January 2018, FFP granted a one-year Associate Award—called TOPS Bridge—to Save the Children and partner CORE Final Report: August 2010 to January 2018 An Overview of The TOPS Program 5 Group. The Bridge Award allows TOPS to continue offering a number of critical services, e.g., the FSN Network, Theory of Change and commodity management capacity strengthening and stakeholder consultations and to provide a “bridge” for a smooth transition of these functions to the new Implementer-led Design, Evidence, Analysis and Learning (IDEAL) award, to be issued later in the year. Activities under the TOPS Bridge Associate Award are not included in this report. Table 2: TOPS Timeline: Program Years (PY) and Calendar Years Original TOPS Award TOPS Award Extensions TOPS Bridge Associate Award AUG 2010 – SEP 2011 OCT 2011 – SEP 2012 OCT 2012 – SEP 2013 OCT 2013 – SEP 2014 OCT 2014 – SEP 2015 OCT 2015 – SEP 2016 OCT 2016 – JAN 2018 JAN 2018 – JAN 2019 PY1 PY2 PY3 PY4 PY5 PY6 PY7 n/a The Technical and Operational Performance Support (TOPS) Program 6 TOPS Achievements and Results 2. TOPS Achievements and Results This section discusses each TOPS program result, including its relationship to the logical structure of The TOPS Program as a whole. Summaries of activities within the PR over the life of TOPS are combined with TOPS monitoring data to capture achievements, course corrections, successes and shortfalls. Program Result 1: Knowledge Capture TOPS’ strategic objective was that the highest quality information, knowledge, and best practices for improved methodologies in Title II food aid commodity program performance be identified, established, shared, and adapted. The first step towards meeting this objective was to answer the question: What do food security and nutrition programmers, across all organizations and relevant sectors, need to know and be able to do if their programs are to succeed? Core Competencies Early on, TOPS technical leads began to refine and focus their workplan through discussions with the key PVO representatives participating in the FSN Network Task Forces (see text box). TOPS consulted with practitioners to examine the knowledge and skill needs of key actors in FFP-funded food security activities. The aim was to narrow these into sets of core competencies, each related to a given sector and each comprising a number of skills representing the basic capabilities and knowledge that technical staff should possess to carry out their positions effectively. TOPS methodically collected input from the task force participants, who also consulted within their own organizations. The TOPS team reviewed the findings of the 2008 Institutional Capacity Building Grant Review6 , the 2002 Food 6 Food and Nutrition Technical Assistance Project. 2008. Institutional Capacity Building Grants (2004-2008) Review. Available at: https://www.fsnnetwork.org/document/institutional-capacity-building-grants-2004-2008-review The Food Security and Nutrition (FSN) Network and Its Task Forces TOPS created the Food Security and Nutrition (FSN) Network, an open community of practice for food security and nutrition implementers, researchers, and donors. The FSN Network provided venues (website, newsletter, meetings, events, interest groups and task forces) for its members to collaborate and learn, while shaping agendas, influencing donor priorities, building consensus on promising practices, and sharing technical knowledge to strengthen programming. Each TOPS specialty area formed a task force, whose volunteer members were PVO staff based in the field and at headquarters, renowned technical experts, academics, and donor representatives. The task forces were considered active elements of the FSN Network, rather than structural elements of TOPS, and were the central mechanism by which TOPS identified areas of common need and interest, gaps in programming, and priorities for action to support capacity strengthening, as defined by the PVO community and other stakeholders. The FSN Network and the task forces are discussed in more detail in PR 4 Knowledge Sharing below. Final Report: August 2010 to January 2018 TOPS Achievements and Results 7 Aid and Food Security Assessment (FAFSA)7 report, and the 2013 Food Aid and Food Security Assessment (FAFSA-2) report. 8 As TOPS teams drafted and revised core competencies, they continually tested them with field staff, then revised and re-tested them. The TOPS Program and FSN Network Core Competency Series Booklet was published in 2015. It was revised in 20179 to take into account the results of continued field-testing and, in particular, to ensure alignment between competencies and the new USAID/FFP Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities with added emphasis on Theories of Change and more rigorous annual monitoring strategies. Readily available on the FSN Network website and widely distributed within the food security community, the booklet defined core competencies for:  Agriculture and Natural Resource Management: Agricultural Program Manager  Commodity Management: Commodity Manager  Commodity Management: Warehouse Manager  Gender: Gender Staff  Monitoring and Evaluation: M&E Officer  Monitoring and Evaluation: M&E Manager  Nutrition and Food Technology: Nutrition Advisor or Specialist  Social and Behavioral Change: SBC Advisor or Specialist When the Core Competency Series Booklet was first published in 2015, it did not contain core competencies for Knowledge Management. Initially, TOPS included core competencies for technical areas that typically had specifically assigned implementing partner staff in the field, but due to the growing emphasis on knowledge management in development food security activities, this core competency was added later by the Knowledge Management Task Force. During the life of TOPS, identification of core competencies set topical priorities for much of TOPS’ activities: identifying and disseminating promising practices and useful tools while filling gaps in tools (PR 1/PR 2), building technical capacities (PR 3), and knowledge sharing and networking (PR 4). 7 Food and Nutrition Technical Assistance Project. 2002. Report of the Food Aid and Food Security Assessment: A Review of the Title II Development Food Aid Program. Available at: http://pdf.usaid.gov/pdf_docs/Pnacp369.pdf 8 van Haeften, R.; Anderson, M.A.; Caudill, H.; and Kilmartin, E. 2013. Second Food Aid and Food Security Assessment (FAFSA-2). Washington, DC: FHI 360/FANTA. 9 The TOPS Program. 2017. The TOPS Program and FSN Network Core Competency Series Booklet. Available at: https://www.fsnnetwork.org/sites/default/files/Core%20Competency%20series%20combined_Version%204_December%2020 17_508.pdf Cover of the Core Competency Series Booklet The Technical and Operational Performance Support (TOPS) Program 8 TOPS Achievements and Results Self-Assessments in Core Competencies As a corollary product to the core competencies, TOPS created a generic self-rating tool10 that could be used with any of the core competencies and adapted to create an individualized TOPS Self-Assessment Toolkit for Effective Development Food Security Activities. Together, the Core Competency Series and Self-Assessment Tool had a dual purpose: TOPS used the results of field-testing the tool to contribute to efforts to identify and fill needs for capacity strengthening and tools, while FFP partners could use them for many other purposes, such as developing criteria for recruiting. TOPS’ perspective was that assessing need for capacity strengthening was a continuous element of organizational management, and a requirement for high-quality food security programming. The self-assessment tools supported that process during The TOPS Program’s lifetime. When used with the core competencies, the self-assessment tool could be used well beyond the life of TOPS to:  Assist project managers when hiring staff by detailing necessary skills (core competencies alone) and/or comparing candidates capacities (with self-assessment)  Guide project managers in developing capacity strengthening plans for staff after review of self￾assessment results  Help practitioners understand their own capacities and seek opportunities to improve their skills The TOPS mid-term evaluation11 corroborated that TOPS’ development of a self-assessment system rooted in task force-recommended core competencies was an important improvement in the food security implementing partner community. Building a global culture of dialogue around core competencies—rather than trying to account for and meet unique or highly contextual capacity strengthening needs—offered some strategic focus for both FFP and the PVO community. The mid-term evaluators also noted that placing the responsibility on implementers and staff to think critically about their own skills and development needs reduced the likelihood of creating a culture of dependency upon The TOPS Program. All core competencies, along with the generic self-rating tool, were translated into French and Spanish to facilitate capacity strengthening efforts among field staff. 10 Available at: https://www.fsnnetwork.org/core-competency-series 11 The TOPS Program. 2017. Mid-Term Evaluation of The Technical and Operational Performance Support (TOPS) Program. Available at http://pdf.usaid.gov/pdf_docs/PA00MFSN.pdf Final Report: August 2010 to January 2018 TOPS Achievements and Results 9 Program Result 1: Knowledge Capture and Program Result 2: Knowledge Generation Initially, The TOPS Program sought to answer two broad questions. The first (discussed above) was: What do food security programmers, across all relevant sectors and program cycle specialties, need to know and be able to do if their programs are to succeed? The second question, growing from the first, was: What best practices exist and what information and tools are available to help put best practices into place? TOPS began with an assumption that documented best practices, and the tools to support them, existed within the food security and nutrition practitioner community, but were not widely shared beyond the individual organizations. TOPS envisioned PR 1/PR 2 as extended exercises in locating those resources, selecting the ones that best corresponded to core competencies identified for practitioners, and filling gaps where no appropriate resources could be found. This exercise would be a precursor to building practitioner capacity (PR 3) and sharing knowledge widely across the food security community (PR 4). In reality, The TOPS Program found little documentation of best practices, fewer existing tools, and detected more gaps in needed tools than expected. While TOPS categorized tool and gap identification as part of PR 1 Knowledge Capture, and tool creation as part of PR 2 Knowledge Generation, in practice the activities were so closely linked that reporting on them separately was not necessary. TOPS annual progress reports provided combined information on PR 1 and PR 2, and they are discussed together here. Identify Existing Tools and Gaps in Tools From PY1 through PY2, The TOPS Program sought and located existing tools that were relevant to food security and nutrition activities. Specifically, tools were defined as documentation of how to implement an evidence-based approach or methodology, in the form of a manual, guide or toolkit. By looking at how each tool corresponded with areas listed in the TOPS core competencies, their relevance for different practitioners (such as agriculture program managers, nutrition advisors, etc.) could be determined. The identification process involved several lines of inquiry. TOPS technical specialists and task forces conducted an online survey through the FSN Network to gather practitioners’ knowledge of tools. They compiled a list of tools developed through previous FFP ICB grants and through the former Food Aid Management (FAM) project. They gathered tools recommended by organizations represented on the task forces. Finally, they reviewed tools cited in final evaluations and other documentation, available on the USAID Development Experience Clearinghouse (DEC) of development food aid programs. The Technical and Operational Performance Support (TOPS) Program 10 TOPS Achievements and Results By the end of PY2, TOPS completed its formal review of existing tools—particularly tools developed with USAID funds—to identify those that could be useful for food security activities. TOPS staff and task force members remained alert throughout the life of TOPS, however, for existing and new tools, including those recommended by other FSN Network members. TOPS evaluated each tool it identified. Did evidence exist to support its value? Did it clearly and comprehensively cover how to implement the methodology in question? Was it appropriate for use by field staff implementing FFP-funded activities? Was it proven, or likely to be useful, in a context wider than originally applied? In some cases, the existing tools identified could be used by the food security community in the form that TOPS found them. In most cases, however, tools on a crucial topic either were wholly absent, had significant gaps, or needed updating. Using the same process as that for identifying existing tools, TOPS identified and prioritized gaps in tools, e.g., where information and guidance were still needed if food security practitioners were to fulfill their core competencies. In these cases, TOPS modified, created, or supported creation of tools to fill the gaps. Figure 2: Infographic about the TOPS Commodity Management Toolkit Fill Gaps by Modifying or Creating Tools The TOPS Program used two mechanisms to generate knowledge to fill gaps in tools to make them more effective. The first was to draw upon the technical knowledge and field experience of the TOPS technical specialists and the capacities of FSN Network task force members. TOPS staff and task force members In PY7, TOPS launched the Commodity Management (CM) Toolkit―a one-stop operational guide, available online, for food security practitioners working with food commodities. As a set of inter-connected flowcharts, it illustrates the entire CM process in simple action steps and provides the user with 147 individual tools to implement all steps. Final Report: August 2010 to January 2018 TOPS Achievements and Results 11 (supplemented occasionally by expert consultants) researched, created, tested and modified tools across all sectors to fill identified gaps. The second, described in the subsection below, was to make TOPS small grants available to others in the food security and nutrition community, which many grantees used to create needed tools. By the close of PY7, The TOPS Program modified or created 87 tools for food security practitioners, including field guides, handbooks, and facilitation manuals. Of these, TOPS staff and task forces, directly or via consultants, created 50. The remaining 37 were created by organizations that obtained small grants funding from TOPS (below). ‘Recommended’ Designation for Select Tools Even as TOPS and the task forces identified, reviewed, modified and created tools for the food security community, TOPS understood the need to limit the number of tools it promoted to those for which it and the community had the highest confidence. Therefore, TOPS created the designation ‘FSN Network Recommended’ for a subset of top-line tools. Each task force developed criteria according to state-of-the-art or evidence-based technical information for its specialty area. TOPS and task forces then rigorously reviewed tools and prioritized those that:  built capacities around the minimum core competencies  were easy to use and field-friendly  were empirically shown to be useful and effective  had been successfully applied in an appropriate context  addressed a need or gap in the practitioner community Table 1 shows the 36 tools that are still available on the FSN Network that received the ‘recommended’ designation12 over the life of The TOPS Program. Note that each of the TOPS technical and cross-cutting themes is represented, and that recommended tools include those that were: (a) created by other entities before 2010 and accepted by TOPS; (b) modified or created during TOPS’ lifetime by TOPS technical staff and task forces; and (c) developed by other entities during TOPS’ lifetime. 12 Unless specifically noted, these resources were not peer reviewed in any formal sense, nor were they officially endorsed by any institution. The contents of these resources did not represent the opinions or policy of USAID, the U.S. Government, or other donors. The Technical and Operational Performance Support (TOPS) Program 12 TOPS Achievements and Results Table 3: FSN Network-Recommended Technical Tools Tool Title Organizational Author Languages Year Published Partnership Defined Quality Facilitation Guide Save the Children English 2003 Barrier Analysis Facilitator's Guide Food for the Hungry, CORE Group, USAID English 2004 The Care Group Difference World Relief, CORE Group English 2004 ProPack III - The CRS Project Package: A Guide to Creating a SMILER M&E System CRS English 2010 Private Sector Engagement Toolkit Mercy Corps English 2011 Quality Improvement & Verification Checklists: Online Training Module, Training Files, Slides, etc. The TOPS Program English, French 2012 A Practical Guide to Conducting a Barrier Analysis Kittle, BL English, French, Spanish 2013 Designing for Behavior Change: For Agriculture, Natural Resource Management, Health and Nutrition The TOPS Program English, French, Spanish 2013 Designing Participatory Meetings and Brownbags: A TOPS Quick Guide to Linking Development Practitioners The TOPS Program English, French, Spanish 2013 Guide to Developing a Scope of Work for Mid-Term Evaluation of Title II Development Food Assistance Programs The TOPS Program English 2013 Supporting Communities of Practice: A TOPS Quick Guide to Linking Development Practitioners The TOPS Program English, French, Spanish 2013 Care Groups: A Training Manual for Program Design and Implementation The TOPS Program, World Relief English, French, Spanish 2014 Five Skills Set for Smallholder Farmers CRS English 2014 Formative Research: A Guide to Support the Collection and Analysis of Qualitative Data for Integrated Maternal, Child and Nutrition Program Planning CARE English 2014 Integrating Extremely Poor Producers into Markets, Field Guide, Third Edition FHI360, World Vision English 2014 Community Video for Nutrition Guide: Using Participatory, Community-Led Videos to Improve Maternal, Infant, and Young Child Nutrition SPRING, Digital Green English 2015 How to Hold a Successful Share Fair K4Health, Johns Hopkins Center for Communication Programs, IntraHealth International, FHI 360, English 2015 Final Report: August 2010 to January 2018 TOPS Achievements and Results 13 Management Sciences for Health Make Me a Change Agent: A Multisectoral SBC Resource for Community Workers and Field Staff The TOPS Program English, French, Spanish 2015 Monitoring and Evaluation Facilitator’s Guide The TOPS Program English 2015 Nutrition Program Design Assistant: A Tool for Program Planners Reference Guide and Workbook CORE Group, Save the Children, FANTA English 2015 Participatory Facilitation Techniques Workshop Curriculum The TOPS Program English 2015 A Guide to Developing a Knowledge Management Strategy for a Food Security and Nutrition Program The TOPS Program English 2016 Designing for Behavior Change: For Agriculture, Natural Resource Management, and Gender The TOPS Program English 2016 Reaping the Benefits: Technical Guidance on Reducing Post-Harvest Losses in Smallholder Farming Systems The TOPS Program English 2016 A Training Workshop Guide: Planning for Qualitative Research for Nutrition The TOPS Program English 2017 Activities for Stimulating Knowledge Sharing and Learning: A TOPS Quick Guide to Linking Development Practitioners The TOPS Program English 2017 Commodity Management Handbook The TOPS Program English, French, Spanish 2017 Commodity Management Toolkit The TOPS Program English, French 2017 Farming for a Profit: Technical Guidance for Smallholder Financial Planning The TOPS Program English 2017 Gender and Commodity Management: A Field Guide The TOPS Program English, French 2017 Key Resources for Agriculture and Natural Resource Management Practitioners The TOPS Program English 2017 Knowledge Management and Monitoring & Evaluation Collaboration Package The TOPS Program English 2017 REALIZE: Social and Behavioral Change for Gender Equity and Diversity The TOPS Program, CARE International English, French 2017 Theory of Change Training Curriculum The TOPS Program English 2017 Tool for Framing a Discussion between Nutrition and Agriculture Specialists The TOPS Program English, French 2017 Core Competency Series The TOPS Program English, French, Spanish 3rd ed. 2017 The Technical and Operational Performance Support (TOPS) Program 14 TOPS Achievements and Results In TOPS PY1 and PY2, FFP included the majority of FSN Network-recommended tools as reference materials in several of its RFAs, validating the TOPS review, assessment and recommendation process. By PY6, FFP included links to 93 percent of recommended tools in its USAID/DCHA/FFP Technical References for Development Food Assistance Programs. Field practitioners’ awareness and use of tools are discussed in Section 4 of this report on TOPS effects and impact, below. By the close of PY7, the FSN Network website’s resource library contained over 800 items: TOPS also made non-priority tools, guides, manuals and other resources readily available to food security practitioners. The designation ‘FSN Network Recommended,’ however, identifies the tool as a top choice for review and possible use by FFP implementing partners and others in the food security and nutrition community. Identify and Promote Promising Practices The TOPS Program initially assumed that a number of approaches for project design and implementation already existed―some of which could contain best practices―but not shared due to lack of a system for sharing, or to competitiveness among PVOs. In reality, as TOPS started searching for existing documentation in PY1, it found little evidence-based documentation of best practices. Evaluation reports rarely provided detailed information on design and implementation, and rarely contained sufficient analysis or evidence of the impacts of specific approaches—best or promising practices―on project implementation and outcomes. TOPS, therefore, focused efforts to identify, document, package, and share “promising practices,” aiming to spark discussion and increased documentation for continuous improvement within the food security community. TOPS defined promising practices, broadly, as specific project actions for which some evidence exists to show that they contributed to (positive) impacts. As with the search for useful tools, work to identify promising practices was especially intense during TOPS’ first four years, but continued throughout its lifetime. By the end of PY3, TOPS had identified 30 programs that, based on evaluations, achieved significant reductions in stunting and/or significant increases in household income or other higher-level food security indicators. Promising practices were identified in these narrative reports and were linked to achievements in intermediate indicators that would support attribution. In 2014, TOPS published The TOPS Review of Promising Practices in FFP Development Food Assistance Projects13 , gathering its findings in ANRM and maternal and child nutrition, cross-cut with gender and SBC. TOPS held several consultative presentations for implementers and donor representatives around the publication, which became one of the most frequently downloaded documents from the FSN Network web portal. TOPS considered the review a ‘living document,’ and a strong starting point, rather than an end in itself. The intent was for the publication to serve as a basis for discussion among the food security community, leading to identification of additional promising practices, further field‐testing and/or documentation of 13 See https://www.fsnnetwork.org/tops-review-promising-practices-food-peace-development-food-assistance-projects Final Report: August 2010 to January 2018 TOPS Achievements and Results 15 promising practices, and consensus on promising practices that TOPS and stakeholders would promote. TOPS also encouraged members of the food security community to seek TOPS Small Grants (below) to field-test and document any potential promising practice they identified. Because of the lack of detailed information in final evaluation reports, TOPS tried other means for identifying and sharing promising practices. For example, task force members were asked to share promising practices from their organizations. TOPS staff were also able to identify promising practices in FFP-funded programs when they were visiting countries to conduct capacity strengthening workshops. TOPS often invited staff from these development food security activities to share these practices, processes and results-to-date in the annual TOPS/FSN Network Knowledge Sharing Meetings. As TOPS’ familiarity with FFP-funded country programs increased over time, this strategy produced an agenda particularly rich in practical details from many development activities by the time of the PY6 Knowledge Sharing Meeting in Uganda. In PY6 and PY7, the ANRM team published the TOPS ANRM Case Study Series, sharing promising practices from a half-dozen FFP implementing partners on cases ranging from the role of behavior change in aflatoxin reduction, to the use of seeds e-vouchers. TOPS also used video to document and share promising practices. TOPS’ specialists in technical and cross-cutting areas that were less obvious as direct contributors to impact (such as M&E, Commodity Management, and Knowledge Management) found other ways to identify promising practices. The Knowledge Management team, for example, reviewed knowledge management practices at PVO headquarters, while the SBC specialists hosted numerous knowledge sharing opportunities to disseminate formative research results from Barrier Analyses. In 2016, M&E staff from USAID Missions and IPs in the field gathered in Ouagadougou, Burkina Faso, to share and learn about promising practices for M&E in emergency food security programs. The learnings from this event were expounded further in a workshop in Washington, DC the following year. As the Knowledge Management team developed their skills in video, they assisted the Nutrition & Food Technology Specialist to use video to document Save the Children’s successful community mobilization approach to maternal anemia, child stunting and 1,000 Days as a partner of the SPRING project in Southwest Uganda. 14 14 See http://www.fsnnetwork.org/tops-programs-video-springs-community-nutrition-activities-southwestern-uganda Screen shot from the video documenting the SPRING Project’s community nutrition activities in Uganda. Available at: https://youtu.be/jcCZt65dFOA The Technical and Operational Performance Support (TOPS) Program 16 TOPS Achievements and Results Early in The TOPS Program, there was some doubt that PVOs would openly share the operational details of their best or promising practices. Some believed that PVOs might consider these as their competitive advantages in bids for future USAID funding. Happily, this did not happen. In fact, PVOs proved eager to share their practices or tools at TOPS events or through other TOPS information channels. This was also due―in part, perhaps―to the TOPS practice of highlighting several practices on a given topic or sector. In this way, the efforts of many actors were equally recognized and community members could choose the approach that best suited their context. This not only emphasized a sense of common purpose and community, it also minimized the perception that there was only one or a few “best” practices. Promising Practices and Emerging Topics TOPS was active in defining and refining approaches to several emerging topics over seven-plus years, including climate-smart agriculture and resilience, defined as “The ability of people, households, communities, countries, and systems to mitigate, adapt to, and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.”15 The TOPS Program became involved in the resilience conversation in December 2011, when FFP asked TOPS to work with them, the USAID Bureau of Food Security, and the International Food Policy Research Institute (IFPRI) to organize the USAID-sponsored workshop, “Enhancing Resilience in the Horn of Africa: Evidence-Based Summit on Strategies for Success.” From its third year onward, TOPS, led by consortium member TANGO International, was an active participant in the global development of resilience as an operating concept in food assistance program design, implementation, and measurement. The TOPS Program contributed to USAID’s ongoing examination of promising practices that could inform the development of a resilience measurement framework and in PY6 formed an FSN Network task force dedicated to resilience, led by TANGO. As donor and implementers increasingly saw the importance of resilience—understanding it, refining the concepts, creating measures—to successful food security activities and sustainable life improvements for beneficiaries, TOPS responded by facilitating events, dialogues, and experiences in which members of the food security community could help each other learn and then apply these concepts. With its adaptive management structure and program design, The TOPS Program was able to respond quickly and direct resources towards resilience-themed learning activities on ways to incorporate resilience into development initiatives. Over time, the increased focus on resilience was absorbed into the FFP development food security activity design and, hence, into the TOPS learning/capacity strengthening portfolio. 15 As defined by USAID. See: http://www.usaid.gov/resilience In PY5, the TOPS ANRM team undertook an exploratory study of climate-smart agriculture in FFP-funded activities, intending to identify and document promising practices. Researchers found, however, that several challenges prevented the emergence of promising practices. As described in the TOPS report Climate-Smart Agriculture in USAID Office of Food for Peace Programs (2015), these included lack of clarity around what climate-smart agriculture was, varied awareness of and capacity in climate change in general, disagreement over the scale of change needed, and challenges in balancing climate-smart agriculture with other project needs and demands. Final Report: August 2010 to January 2018 TOPS Achievements and Results 17 TOPS’ work on developing resilience themes, concepts, and approaches to measurement, principally led by consortium partner TANGO International, generated several Associate Awards from USAID. These awards were sub-granted to TANGO and other members of the consortium to orient implementer and donor technical staff on crucial resilience concepts and practices. These Associate Awards are discussed in Section 5 of this report. TOPS-facilitated work helped define (and continues to define) the world of resilience. Resilience has not been “codified” yet, and continues to evolve as concepts, practices, and measurement tools are honed and understood and field-collected evidence analyzed. Over the past several years, TOPS provided an organized process and technical expertise to move the dialogue forward, to convene key thinkers, and to create structured hypothesis of the evolving outputs—a hypothesis that food security practitioners will further test, refine, share, and apply. Screen shot from a video documenting the ANRM team’s capacity strengthening event in Zimbabwe where they introduced the Resilience Design in Smallholder Farming Systems Approach. Available at: https://www.youtube.com/watch?v=bK_WdO4UdOI&feature=youtu.be The Technical and Operational Performance Support (TOPS) Program 18 TOPS Achievements and Results Program Result 2: Small Grants The TOPS Small Grants Program aimed to promote the use and adaptation of capacity strengthening tools, and to ensure that field-led and collaborative projects disseminate best practices to FFP implementing partners and other food security and nutrition actors, as appropriate. The TOPS Small Grants Program served as an incubator in which practitioners could develop and test innovative ideas and promising practices. In its 2010 application to FFP, Save the Children proposed that TOPS would use small grants to fund ‘proven and innovative capacity strengthening tools and activities, especially field-developed and collaborative projects involving several partners.’ It predicted that small grants would be the ‘principal means for supporting collaboration between institutions, ensuring field-driven development, and bringing tested products to completion and scale.’ Each grant would support a defined output, rather than fund ongoing activities. The small grants function filled a real gap in the food security and nutrition community, where resource constraints often discouraged exploration and testing of creative ideas that could improve programs and contribute to reducing global hunger. The scope of the small grants function expanded beyond initial expectations when TOPS’ early assumption—that numerous, useful tools existed within existing organizations—proved incorrect (see PR 1/PR 2). Rather, in most cases, existing tools either were wholly absent, had significant gaps, or needed substantial updating. Over seven years, TOPS made 71 small grants in two categories: Micro Grants (maximum of $50,000 per award, 50 awarded) to develop tools or conduct topic-specific learning events; and Program Improvement Awards (PIA—maximum of $100,000, 21 awarded) to design and implement new technical content, pilot promising practices, and collect evidence to strengthen and improve the quality of food security programs. The TOPS Program website contained a tab that explained the small grants, and guided interested organizations through the application process, which included preparing a proposal narrative and a detailed budget. Applicants had to demonstrate that their proposed activity, whether research, piloting, tool creation, or workshop, (a) contributed to TOPS’ overall goal of improved quality, efficiency, and effectiveness in FFP programming and/or stronger capacity among grantees and implementers, and (b) responded to an important gap in the food security body of knowledge.  Alignment with FFP, FSN Network and TOPS priorities  Clear link between gap in body of knowledge and objectives of project  Field staff engagement with project development  Demonstrated past experience  Collaboration between two or more grantees and other agencies  Description of how lessons will be shared with implementing partners  Clarity and specificity of project output  Quality of impact measurement plan Figure 3: TOPS Small Grant￾Making Criteria Final Report: August 2010 to January 2018 TOPS Achievements and Results 19 In selecting proposals to fund, TOPS prioritized those that demonstrated multi-stakeholder involvement and undertook activities from which results could potentially be useful to many implementing partners in other countries. All applicants were required to consider gender implications across their proposed activity. TOPS required all recipients to have a plan for documenting their work and for publically presenting their results to the overall food security community. Over its seven-year life, The TOPS Program received 165 small grant applications, and funded 43 percent of them. The 71 funded projects covered all of the TOPS focus areas, and in most cases combined two or more. Figure 6 shows small grants by technical sector. 16 At the close of TOPS, the FSN Network Resource Library contained 53 publications that were products of TOPS small grants, including toolkits, training aids, facilitation guides, research reports and more. 17 The number of tools created, field-tested or validated via the small grants function totaled 37 (eight more than the targeted number of tools). In all, 32 organizations received one or more small grants from The TOPS Program. Seven of the 71 grants were made to two or more organizations working jointly; of these, two included collaboration with an academic institution. In 30 cases (42 percent of all grants made), recipients provided match funding for the proposed activity, research or tool, thereby underscoring their commitment to the project. The range of these recipients’ contributions was from two to 127 percent of the value of the TOPS Grant. 16 The total is greater than 71 because so many grants were for multi-sectoral work 17 While all recipients documented the use of their small grants, not all documentation was in the form of a resource useful to others, and hence not all documentation is featured in the FSN Network resource library. Figure 4: TOPS Small Grants by Technical Sector The Technical and Operational Performance Support (TOPS) Program 20 TOPS Achievements and Results Tools created through small grants funding were widely disseminated by TOPS: via the FSN Network Resource Library, the FSN Network newsletter, workshops, and knowledge sharing meetings. Grantees themselves were required to document and share the results of their work with the food security community via webinars or in￾person presentations. The TOPS July 2017 Knowledge Sharing Meeting featured a plenary session, TOPS Small Grants Program: Learning through Practice, consisting of a “marketplace” that highlighted a dozen small grant projects across a range of technical areas, geographic locations, and types of organization. It was highly participatory and received rave reviews by participants. Finally, The TOPS Program issued two editions (2010-15 and 2015-17) of the publication, Small Grants, Big Impact, 18,19 in which every small grant project funded by TOPS is described by purpose, location, grantee, importance and implications (including gap or gaps filled), sources of further information and tools (if any) produced. The TOPS Program disbursed $4,071,604.12 in Small Grants over seven years. Recipients’ match contributions totaled $546,911.50. See Annex II for a detailed listing of the 71 small grants made by TOPS. 18 The TOPS Program. 2015. Small Grants, Big Impact. A Retrospective of the TOPS Small Grants Program 2010-2015. Available at: http://www.fsnnetwork.org/small-grants-big-impact-retrospective-tops-small-grants-program-2010-2015 19 The TOPS Program. 2017. Small Grants Big Impact II: An Overview of the Small Grants Program, 2015-2017. Available at: http://www.fsnnetwork.org/small-grants-big-impact-ii-overview-small-grants-program-2015-2017 Initially, the TOPS Small Grants Program only offered one type of grant with a maximum amount set to $100,000. The uptake of these grants was relatively slow in the beginning, with IPs not aware of the program or its purpose. After TOPS increased promotion efforts and created the Micro Grant, with a maximum amount of $50,000, submissions picked up, and when the initial small grant was redefined and announced as a Program Improvement Award or PIA (still with a maximum amount of $100,000), the process really took off. In addition, TOPS worked to streamline the application processes, such that by the final phase of grant distribution, the average number of days from initial submission to the fully executed subagreement was 154 days, compared to 235 at the beginning of the program. Figure 5: Uptake of the TOPS Small Grants Program Final Report: August 2010 to January 2018 TOPS Achievements and Results 21 Program Result 3: Knowledge Application TOPS implemented capacity strengthening activities to improve the quality and effectiveness of food security and nutrition programming, by equipping practitioners and managers with:  the technical knowledge and skills to achieve or surpass the core competencies (PR 1) in their specialty areas, along with  the relevant tools, including those identified or created by The TOPS Program (PR 1/PR 2), to achieve high quality programming Traditional Skills Delivery The TOPS Program specialized in customized, face-to-face trainings for field and headquarters staff of FFP implementing partners. TOPS delivered 130 capacity strengthening workshops, reaching over 3,000 trainees, in its seven-plus years. These workshops took place in Africa, Asia, Central and North America; they gathered on average 24 participants per workshop; and they targeted implementing staff in FFP countries, their program managers, and headquarters staff of implementing PVOs. TOPS technical staff and consultants tailored each training workshop to meet the interests and needs of trainees and their supervisors. In planning for these workshops, TOPS consulted extensively in advance with PVO technical staff, particularly with field-based workers who were responsible for ensuring that field personnel, community volunteers, and staff of other partners possessed the skills and abilities to deliver high quality programs. Using a growing portfolio of TOPS-identified or created training materials and other tools, TOPS was able to remain flexible and responsive to changing needs within the food security community over the life of the award. Figure 6: TOPS Capacity-Strengthening Workshops by Topic ANRM, 25 CM, 21 Gender, 1 KM, 10 M&E, 20 Nutrition, 14 Resilience, 2 SBC, 19 TOC, 18 0 5 10 15 20 25 30 ANRM CM Gender KM M&E Nutrition Resilience SBC TOC The Technical and Operational Performance Support (TOPS) Program 22 TOPS Achievements and Results Where possible, TOPS engaged PVO staff, FFP staff, and others in the food security communitythose with specific expertise or strengths in core competency areasas co-facilitators or moderators. Further, TOPS generally designed trainings to have participants from more than one PVO or stakeholder organization to promote cross-learning, knowledge sharing, cross-organizational networking, and an interactive atmosphere for transfer of technical knowledge, skills, and experience. The exception to this practice occurred in PY7, when TOPS held a series of TOC workshops with individual PVOs for a review, analysis, or a post mortem of their actual TOCs for a proposal submission to FFP. These review workshops were offered to all implementing organizations equally, and conducted separately for all who requested them. Although a specific TOPS technical team usually led each workshop, many events integrated one or more technical sectors with one or more cross-cutting topics. For example, ANRM and SBC were a frequent workshop combination, as were SBC, Nutrition, and Food Technology. From PY2 onward,20 TOPS integrated Gender into capacity strengthening workshops for all other technical sectors. Figure 6 likewise separates TOC workshops from others provided by the M&E / Resilience Task Forces under the leadership of consortium member TANGO International. Pre- and Post-Test Results for Capacity Strengthening Workshops TOPS used pre- and post-tests at every workshop to monitor the effectiveness and quality of its capacity strengthening events, and to improve them over time. TOPS grew expert at developing effective pre￾and post-tests that looked at many factors of the learner/participant experience, including: (a) whether to include varied structures among the questions (such as multiple choice or single answer only) and/or how to highlight awareness of these varied question structures to the participants; (b) appropriate level or complexity of language/terminology to define unique technical terms; (c) appropriate number of questions; and more. TOPS usually shared the results of the pre- and post-tests immediately with workshop participants. In Figure 7, comparison of average pre-test and post-test scores for all participants and all workshops shows an improvement of roughly 20 percentage points or more each year. Trainee satisfaction with TOPS’ trainings consistently exceeded targets, with composite satisfaction scores ranging from 87% to 90% annually (Annex III, PMP). TOPS technical specialists reviewed all satisfaction survey comments and used these for continuous improvement and redesigning of workshops. For example, early on, TOPS identified some participant dissatisfaction with workshops that gathered individuals with significantly different capacity strengthening needs (e.g., technical coordinators with field supervisors and field staff). The TOPS team gained experience in providing workshop information that was useful for all levels, and increasingly included sessions in which higher￾level staff assisted in workshop facilitation. 20 This integration explains the appearance of only one gender workshop in Figure 6. Final Report: August 2010 to January 2018 TOPS Achievements and Results 23 Aiming for Replicability The TOPS Program encouraged its technical staff to build replicability into their workshops so that skills delivery could exceed TOPS’ temporal and geographic limitations. This included providing all participants with TOPS-created materials, such as facilitation guides, slides, handouts, and more, to share the training content with their colleagues and peers, and to fully replicate and cascade the capacity strengthening activity, as desired. The several examples below illustrate just some of the ways that TOPS designed capacity strengthening workshops for replicability. Care Groups: One of the TOPS SBC team’s overarching goals included increasing the quality of Care Groups implementation. At TOPS’ onset, organizations were adopting the Care Groups approach, but few training materials were available, and practitioners were unaware of guidelines and standards. The TOPS SBC team heard concerns that implementation was haphazard, that quality was uneven, and that the repercussions for this highly effective behavior change approach could be serious. The team facilitated a multi-faceted response, which included: developing several technical guides (including Care Groups: A Training Manual for Program Design and Implementation21 and Care Groups: A Reference Guide for Practitioners22), holding capacity strengthening workshops, inviting practitioners to share experiences and innovations at Knowledge Sharing Meetings, creating a Care Groups Interest Group for practitioners (it met online, and sometimes in person), and hosting webinars. Moreover, several TOPS Small Grants made on the topic of Care Groups led to a policy paper, two journal publications, several webinars, several single-topic knowledge sharing events, and a document with recommendations on the use of Care Groups in emergency settings. At the close of The TOPS Program, SBC staff and task force members were confident that they had addressed the initial concerns, and achieved good organizational uptake of standards and recommended practices. Several Ministries of Health adopted the Care Groups approach for entire countries. 21 The TOPS Program, Food for the Hungry, Core Group, World Relief. 2014. Care Groups: A Training Manual for Program Design and Implementation. Available at: http://www.fsnnetwork.org/care-groups-training-manual-program-design-and￾implementation 22 The TOPS Program, Food for the Hungry, Core Group, World Relief. 2016. Care Groups: A Reference Guide for Practitioners. Available at: http://www.fsnnetwork.org/care-groups-reference-guide-practitioners 0% 20% 40% 60% 80% 100% PY1 PY2 PY3 PY4 PY5 PY6 PY7 Pre-test Post-test Figure 7: Average Score Pre- and Post-Test Capacity-Strengthening Workshops The Technical and Operational Performance Support (TOPS) Program 24 TOPS Achievements and Results Resilience Design in Smallholder Farming Systems: Early on, the TOPS ANRM team incorporated local service providers into its resilience design trainings. The team invited organizations with expertise in the training content to present and/or take the lead on a practical training element. This created local connections between FFP implementing partners and service providers with context-specific expertise. In Zimbabwe, for example, TOPS invited the Muonde Trust, a community-based organization founded in the early 1980’s to implement high-quality resilience agriculture, to present at the TOPS Resilience Design in Smallholder Farming System training workshop in PY6. Several participants took the initiative to organize cross-visits between the Muonde site in south-central Zimbabwe and their own projects in the region. The Permaculture Research Institute (Kenya) and Kusumala Institute of Agriculture and Ecology (Malawi) joined in the Zimbabwe training as regional resources. TOPS created a Facebook page to link all those who had attended a TOPS resilience design training to facilitate information sharing and learning. Participants in resilience design trainings received an extensive set of curated resource and curricular materials (available on the FSN Network site). A TOPS Small Grant to Mercy Corps resulted in the 2017 publication Resilience Design in Smallholder Farming Systems: Measurement Toolkit. 24 A New Behavior Change Communication Activity for Mothers’ Infant and Young Child Feeding Choices: The TOPS Nutrition and Food Technology Specialist developed this new methodology, designed to increase the overall quantity of food consumed by infants and young children via increased frequency of feeding, quantity of food per feeding, and use of responsive feeding techniques. Drawing on some of the essential elements of the Trials of Improved Practices (TIPS) approach, TOPS sought input from mothers in Malawi and Madagascar on small, doable actions that that were used to design a behavior change activity that could be implemented by field staff with little external technical assistance. The TOPS Nutrition team then facilitated an eight-day training-of-trainers (ToT) workshop, inviting staff from all FFP implementing partner organizations in Malawi and Madagascar, with the pre-negotiated agreement that they would replicate the trainings for other project staff and partners. The TOPS workshops included classroom learning, role-plays, and three half-day field experiences during which trainees practiced each step of the new methodology. Equipped with all training materials used in the workshop, participants provided cascade training to field staff and community volunteers. In Malawi, trainees replicated the training for other project staff from all implementing agencies of the FFP-funded project, and these staff then implemented the behavior 23 Glenzer, Kent G. 2013. “Mid-Term Evaluation of The Technical and Operational Performance Support (TOPS) Program Cooperative Agreement No. AID-OAA-A-10-00006.” Available at: https://pdf.usaid.gov/pdf_docs/PA00MFSN.pdf 24 Mercy Corps. 2017. Resilience Design in Smallholder Farming Systems: Measurement Toolkit. Available at: http://www.fsnnetwork.org/resilience-design-smallholder-farming-systems-measurement-toolkit “TOPS’ overwhelming reliance on traditional training – face-to-face – is a major strength and this fundamental strategy should continue. The […] majority of Title II field staff have little experience with […] distance learning strategies, these are not institutionalized within Human Resources departments of PVOs, and training venues themselves forge social connections, social capital, and ongoing relationships that are important. [T]hose responsible for implementing the trainings all report a high level of ongoing engagement – queries, questions, additional requests for guidance – from trainees after the workshops.” Excerpt from TOPS Mid-Term Evaluation23 Final Report: August 2010 to January 2018 TOPS Achievements and Results 25 change activity with community volunteers. Field staff later replicated the activity throughout the project coverage area. In Madagascar, trainees replicated the training-of-trainers workshop for the eight partners in that country, who then supported each other to replicate the training, following a similar cascade model as in Malawi. This several-stage rollout occurred with only minimal support, via email, from The TOPS Program. When different sets and components of TOPS activities were combined, the potential and opportunities for long-term sustainability increased. The TOPS Program envisioned and implemented its components as linked elements: tools corresponded to core competencies, then helped to identify some of the subject material for capacity strengthening workshops, most of which were designed with replicability in mind. Technical materials were featured in knowledge sharing events, which in turn fostered networking and created social capital among implementers with shared interests. Shared interests, in an atmosphere of peer-to-peer community, increased the likelihood that a successful approach or promising practice would be replicated, expanded and/or improved upon. Program Result 4: Knowledge Sharing TOPS’ fourth Program Result stated that information, skill and knowledge exchange would be supported and expanded. Knowledge sharing both capped and contributed to the ongoing knowledge generation and application activities of PR 1, PR 2, and PR 3. In all its work, but especially in PR 4, TOPS aimed to create an open, inclusive community in which FFP, implementers and other food security stakeholders worked together as peers to grow, test and share evidence-based tools and promising practices. Two critical components in PR4 were:  An inclusive community of practice through the FSN Network and its task forces, whose membership was open to all FFP grantees and other food security and nutrition stakeholders, and  A food security and nutrition website (www.fsnnetwork.org), the foundation of the TOPS knowledge-sharing system. The TOPS Program, with direction from its Knowledge Management technical team, created several channels to promote continuous, cross-organizational and cross-sectoral knowledge sharing (described below). The effective use of these channels, and the creation and maintenance of neutral and open spaces in which food security stakeholders learned and shared, were perhaps The TOPS Program’s most influential and far-reaching accomplishments. Tool creation (PR 1, PR 2) and capacity strengthening (PR 3) are common enough activities for USAID-funded technical assistance providers, but the unique contribution of TOPS lay in creating and facilitating regular, new, or unique opportunities to learn and share together. The Food Security and Nutrition (FSN) Network The FSN Network was an open community of practice for the broad group of individuals and organizations that TOPS brought together with its many activities. Sharing a common commitment to food security and nutrition programming, network participants included FFP staff and implementing The Technical and Operational Performance Support (TOPS) Program 26 TOPS Achievements and Results partners, other PVOs, global USAID activities, like Feed the Future, academic institutions, multilateral agencies, and for-profit actors in the food security and nutrition sphere. The FSN Network engaged members in several ways: Task Forces TOPS technical staff formed and guided eight task forces, corresponding to TOPS’ original technical and cross-cutting focus sectors. One additional topic―resilience―was added to the TOPS portfolio in early PY6 in response to a growing global demand for a platform to share information about the ability of countries, communities, and households to mitigate, adapt to, and recover from shocks and stresses, such as drought or flood. Each task force was a convening mechanism for experts and practitioners from a broad array of PVOs (including all FFP IP PVOs), global USAID activities, academic and research institutions, and more. As these participants were, themselves, primary actors in the overall food security community of practice, they were crucial informants in helping TOPS identify how to meet the needs of the community. The task force members provided TOPS with live, direct connections to the leaders and implementers of food security activities. Table 4: Task Forces and Year of Formation Task Force Year Formed  ANRM  Gender  M&E  Nutrition and Food Technology  SBC PY1  Knowledge Management  Commodity Management PY2  Resilience PY6 Structurally, the task forces were active components of the TOPS FSN Network, with potential to live beyond the life of TOPS. They led work in each of TOPS’ technical areas, guided by the corresponding TOPS technical specialist. The TOPS M&E team guided both the Resilience and the M&E Task Forces. Task force members’ voluntary participation and contributions were essential to many TOPS achievements. They defined the core competencies and recommended tools for their respective areas, and promoted use and analysis of results from the TOPS self-assessment tools within their own organizations. They helped to identify and fill knowledge gaps in their technical focus areas, created or improved tools, and provided and adapted information to meet the needs of FSN Network members. They assisted TOPS’ technical teams to host ‘special topic’ knowledge sharing events and webinars that focused on specific tools and promising practices. They were a key resource for the design and execution of concurrent topic sessions during the TOPS FSN Network Knowledge Sharing Meetings. The Final Report: August 2010 to January 2018 TOPS Achievements and Results 27 task force structure promoted collaboration among FFP implementing partners, as well as between implementing partners and other actors in the food security and nutrition arena. While enthusiastic participation in task forces in TOPS’ early years was an indication of need and interest, it was a challenge to sustain voluntary participation over time. At no time, however, did actual task force participation drop below expected target levels. Acting on suggestions from CORE Group, with its years of experience hosting working groups, each task force scheduled a variety of activities to engage members. Examples include: contributing to or reviewing a tool generated through the TOPS Small Grants Program; hosting presentations by researchers on recent studies in the field; brainstorming ideas for special topic knowledge sharing events; presenting on tools used within their own field programs; and more. The TOPS Program would not have achieved all it did without the task force structure and the voluntary participation of so many members of the food security and nutrition community. Each annual progress report (APR) offers more details on the task forces’ activities and accomplishments. The FSN Network Website The FSN Network website is the online home of the FSN Network community of practice. Created and maintained by the TOPS Knowledge Management team, with continual content collaboration from technical staff and task forces, the website provides a venue for members to exchange and share knowledge to strengthen programming, while shaping agendas and influencing donor priorities. The FSN Network website has proven so valuable to the food security and nutrition community that FFP is committed to continuing it beyond TOPS. The website is currently fully active and managed by the TOPS Bridge Associate Award, and will later transition to the upcoming Implementer-led Design, Evidence, Analysis and Learning (IDEAL). The website landing page offers highlights and alerts users to news, events, materials and opportunities in the food security community. Its Resource Library was initially populated by the Knowledge Management team with tools mainly from PVO members of the CORE Group network, USAID and other donors and institutions. Over time, it has grown to include over 800 practical, implementation-focused guides, tools, and training materials. Through a tagging system, users can sort resources according to several categories, such as “FSN Network Recommended” (see PR 1/PR 2 for discussion of the ‘recommended’ designation), funded through the TOPS Small Grants Program (PR 2), and TOPS￾produced materials. Visitors can also search by subject matter, organizational author, and language. The resource library is a truly new and important treasury for the food security and nutrition community, offering vetted materials whose quality users can trust. The Technical and Operational Performance Support (TOPS) Program 28 TOPS Achievements and Results The website’s Events page alerts users to upcoming meetings, workshops, and webinars. Importantly, it allows users to search past events, and provides them essential information and materials from those events: summaries, handouts, presentations, agendas, recordings, and more. It is a repository for all materials from the large Knowledge Sharing Meetings and the stakeholder consultations hosted by TOPS and the FSN Network, including all presentations given by FFP, skilled practitioners, academics, and other stakeholders. The website houses Discussion Forums: one for every TOPS technical area, plus several other focus areas, including water, sanitation and hygiene (WASH), and several linked to SBC. As FFP developed its global 2016-2025 Food Assistance and Food Security Strategy the FSN Network website created a special discussion forum for input from the implementing community. In its Community section, each task force maintains a page featuring an array of tools for members to contribute, collaborate and consult with one another. FSN Network Newsletter The TOPS Program published the electronic FSN Network News for seven years (and continues to do so under the TOPS Bridge Associate Award). The Knowledge Management team produced its regular biweekly publications, plus numerous special editions, numbering over 280 issues during the seven-year life of TOPS. The newsletter provides updates from The TOPS Program and from FFP, advertises Screenshot of the FSN Network homepage Figure 8: FSN Network Website Statistics Final Report: August 2010 to January 2018 TOPS Achievements and Results 29 upcoming trainings and events, job openings, and recent resources, includes topic-specific articles, and accepts for publication notices and other contributions from the community at large. The TOPS technical staff also contributed items of interest. The newsletter’s expanding audience (Figure 9) encouraged readers to contribute content. The FSN Network News reached 2,580 subscribers in the last year of The TOPS Program. The newsletter reaches out to subscribers via listserv and email, while the FSN Network website attracts members of the food security community. Together, they form the hub where the global food security and nutrition community connects and engages, finds new technical resources, learns about upcoming events, and stays up-to-date on emerging topics and current happenings. Knowledge Sharing Meetings: Ten Signature Events TOPS organized and hosted ten TOPS/FSN Network Knowledge Sharing Meetings (KSMs) in North America, Africa, and Asia from 2011 through 2017, bringing together implementers, donors, and researchers from around the world for peer-to-peer knowledge sharing and networking concerning food security and nutrition. These Knowledge Sharing Meetings became widely known, well-attended signature events for The TOPS Program. They arose from a deeply felt need within the overall community for opportunities to engage in meaningful, face-to-face interactions at field and headquarters levels to maximize the flow of information among implementers of food security activities, and between implementers, policymakers and researchers. These gatherings also founded important relationships and created networks that enabled knowledge exchange to continue beyond the events. 750 1175 1630 2005 2314 2512 2580 750 950 1150 1350 1550 1750 1950 2150 2350 2550 2750 PY1 PY2 PY3 PY4 PY5 PY6 PY7 Figure 9: Subscriptions to FSN Network News TOPS was pleased to be able to recognize FFP’s anniversary: “60 Years of Help and Hope” during the July 10-11, 2014 Knowledge Sharing Meeting From the left: former Rep. Sam Farr (D-CA), Dina Esposito, former Director of the Office of Food for Peace, and Rep. James McGovern (D-MA). The Technical and Operational Performance Support (TOPS) Program 30 TOPS Achievements and Results The TOPS Program, led by the Knowledge Management team, developed the content of each Knowledge Sharing Meeting by engaging directly with practitioners via the FSN Network and its task forces. TOPS also consulted with stakeholders from FFP, academic institutions, and PVOs to ascertain knowledge gaps and emerging topics of interest in food security and nutrition programming. This information gathering was essential to ensure that Knowledge Sharing Meetings were relevant and timely to food security and nutrition implementers. Drawing on CORE Group’s successful tradition of creating and managing knowledge sharing events, TOPS established a model for these meetings that combined knowledge exchange, participatory learning activities, and small group discussions to stimulate cross-organizational collaboration. Each TOPS technical specialist played a vital role, shaping a global learning agenda and ensuring the technical validity and applicability of the content presented. True to TOPS’ principles, the events both drew from, as well as extended, the network of food security practitioners, from FFP and its implementing partners in the larger community of practice. For topic areas outside the scope of the TOPS technical specialists, the Knowledge Management team sought the engagement of FSN Network champions in the broader community. ACDI-VOCA, CARE, Catholic Relief Services, World Vision, and many other organizations, although not part of the TOPS consortium, willingly contributed extensively to the Knowledge Sharing Meetings. For example, of the presenters at the first three meetings, about 36 percent were TOPS technical staff, 27 percent were other TOPS collaborators (mostly FFP grantees), 10 percent were from other NGOs, nine percent were from academia, and eight percent were FFP staff. In addition to the headline and plenary sessions, each TOPS Knowledge Sharing Meeting included 18 to 28 concurrent sessions on carefully selected topics, plus numerous learning/sharing mechanisms, such as lunchtime table topics and knowledge cafés, task force meetings, and informal networking sessions. A full package of information, including agenda, synopses, keynote addresses, handouts and other materials, and video / audio recordings, is available for each of the ten Knowledge Sharing Meetings on the FSN Network website.25 The Knowledge Sharing Meetings became The TOPS Program’s marquee events. They created crucial— and exciting―opportunities for peer-to-peer learning and sharing among implementers, donors, academics and government representatives. TOPS collected feedback and evaluation input from participants at every Knowledge Sharing Meeting for continuous improvement of all subsequent meetings. 25 See http://www.fsnnetwork.org/knowledge-sharing-meetings “TOPS’ regional knowledge sharing events are one of the programs’ most important – and high profile – contributions. Participants are nearly unanimous in their praise for these events. They provide crucial opportunities to bring PVO, FFP, and other actors together to share experiences and [by] doing so build social capital in the larger Title II and food security and nutrition epistemic communities. Participants appreciate the engaging, participatory, and active nature of the…events, [and] the careful preparation done with presenters/facilitators. While anecdotal in nature, the data from an online survey points in a heartening direction regarding transfer of contents to the workplace.” Excerpt from TOPS Mid-Term Evaluation Final Report: August 2010 to January 2018 TOPS Achievements and Results 31 Table 5: The TOPS Program’s Ten Knowledge Sharing Meetings WHEN WHERE # ATTENDEES COUNTRIES ORGANIZATIONS THEME / PLENARY SESSIONS May 2011 USA 150 50 TOPS held this event in conjunction with the Community Health Network Spring Meeting, supported by CORE Group. Ensuing Knowledge Sharing Meetings were wholly TOPS products. Sep 2011 Mozambique 130 17  Dialogue between Food Security Implementers and Policy￾Makers/Donors  Sharing What We Know: Strengthening Implementer Capacity  Networking Together in the Future Jun 2012 Ethiopia 144 11 37  Enhancing Resilience to Food Security Shocks: Resilience 101  Pastoralism at the Crossroads: Exploring the Opportunities in Vulnerable Livelihood Systems  From Humanitarian Assistance to Feed the Future: Long-Term Resilience Building through Strengthened Linkages Nov 2012 USA 152 11 37 Resilience in Action:  The Road to Resilience  Enhancing Resilience to Food Security Shocks in Africa Nov 2013 Burkina Faso 136 17 39 Resilience in Action:  Enhancing Implementer-Donor Collaboration Jul 2014 USA 193 10 60 Learn, Adapt, Apply:  Food for Peace Strategy Update: A Consultation Session Mar 2015 Bangladesh 151 10 40 Learning from the Past, Shaping the Future:  Resilience, Climate Change and Disaster Risk Reduction: Definitions and Differences  Reflecting on Five Years of Food Security and Nutrition Programming in Bangladesh  Updating Food for Peace’s Strategy: A Consultation Jan 2016 USA 174 52 Strengthening Connections: Inquiry, Innovation & Implementation  The New Draft Food for Peace Strategy: 2016-2025 Sep 2016 Uganda 201 99 Changing Landscapes of Food Security  FFP’s New 2016-2025 Strategy  Food Assistance Modalities: Evaluating Benefits & Trade-offs to Meet Humanitarian Needs  Taking the Learning Home: Messages and Meaning for Your Programs Jul 2017 USA 208 12 60 Successes, Challenges, and the Way Forward: Seven Years of Community Building  FSN Network Task Forces: The Building Blocks of a Community  TOPS Small Grants Program: Learning through Practice  The Food for Peace Learning Agenda: A Consultative Process The Technical and Operational Performance Support (TOPS) Program 32 TOPS Achievements and Results Other Knowledge Sharing Events In its initial design, The TOPS Program expected that, as TOPS increasingly became a vehicle for convening the food security community, serving as both producer and clearinghouse of high quality food security knowledge and tools, it would be called upon to organize, feature, or facilitate other knowledge sharing events beyond those formally planned (described above). This expectation was correct. Over its lifetime, TOPS received many requests to host such Single Topic Knowledge Sharing Events (Figure 10). The TOPS team, with support from members of the FSN Network, flexibly responded to stakeholder interests, considering these events to be valuable contributions to programming excellence in the food security community. Many of these events were coordinated with little advance notice, but TOPS staff adjusted to meet stakeholder needs. The majority of these events took place in the Washington, DC area, but over time, TOPS created opportunities for online participation by stakeholders in other countries. Some events blended in￾person and online attendance. Others were held wholly online: these ‘webinars’ increased in frequency and importance in the later years of TOPS. Several of these events were held overseas, particularly in response to requests from FFP in support of regional and country-level programming. While events in the first years of TOPS often focused on sharing and discussing outputs of other USAID global projects, such as FAFSA-2 and the Women’s Empowerment in Agriculture Index, for deeper community awareness and comment, community demand (and TOPS’ growing skills in designing and delivering these events) resulted in a growing number of events over time. Every TOPS technical team offered knowledge sharing events, ranging from several hours to several days in length. Attendance ranged from fewer than 10 to 70 or more people, and TOPS frequently arranged for presentations and, occasionally, event co-sponsorship by PVOs, academic institutions, global projects and for-profit actors in the food security community. Over time, more than 5,000 practitioners participated in such events. From 2012 onward, The TOPS Program facilitated over a dozen half-day stakeholder consultations between FFP and members of the food security community. The consultations provided implementing partner agencies and others with opportunities to gain clarity from FFP on specific topics affecting their work, and enabled FFP to consult with implementing agencies on compliance issues related to policies, regulations and guidelines. Before each consultation, TOPS staff interviewed both sets of stakeholders Figure 10: Number of Single Topic Knowledge Sharing Events Held 2 5 5 5 5 12 8 3 3 14 16 19 29 45 0 5 10 15 20 25 30 35 40 45 50 PY1 PY2 PY3 PY4 PY5 PY6 PY7 Anticipated Actual Final Report: August 2010 to January 2018 TOPS Achievements and Results 33 to collect detailed information about the consultation topic and participant needs. They then created a consultation plan that promoted a solutions-oriented dialogue between the two parties. Consultation topics included proposal preparation and budgeting guidance, USAID’s Multi-Sectoral Nutrition Strategy Indicators, the Refine and Implement Model, and input from the food security and nutrition community during the development of the FFP 2016–2025 Food Assistance and Food Security Strategy. Typically held in the Washington, DC, area, each consultation attracted, on average, 50 participants, from as many as two dozen organizations. TOPS made the contents of all consultations, including detailed notes of presentations and discussions, publically available on the FSN Network website.26 Finally, The TOPS Program also responded to frequent invitations to contribute to events hosted by others, typically by presenting one or more sessions in workshops or conferences. Because each one-off request (and TOPS response) was unique, this category of activity is difficult to summarize. In PY6, as a typical example, PVOs, global projects (FANTA, SPRING), professional networks (the American Evaluation Association and the Society for International Development), and academic institutions (University of North Carolina), asked TOPS to contribute to events online and in Washington, DC, Chicago, Chapel Hill, Kampala, Portland, and Bangkok. TOPS specialists shared their expertise with over 750 total participants in sessions on ANRM (3), Commodity Management (1), Gender and Resilience (1), Knowledge Management (6), M&E (3), Nutrition and Food Technology (6), and SBC (2). 26 See http://www.fsnnetwork.org/past-events In May 2017, The TOPS Program hosted a two-and-a-half day Breastfeeding Seminar, where 42 participants and 26 presenters reviewed actions at the global level to protect, promote, and support breastfeeding. Participants also learned from innovative practices in the field and reflected upon breastfeeding practices in Food for Peace￾targeted countries over the past two decades. Photo Credit: Anne-Sofie Helms, Save the Children In January 2017, The TOPS Program produced a Global Gender Summit in Lusaka, Zambia. The need for this event was first identified through the Gender Task Force. The event brought together 22 gender advisors and advocates from ten countries in Africa and Latin America, representing Emergency and Development Food Assistance Programs. Photo Credit: CORE Group The Technical and Operational Performance Support (TOPS) Program 34 TOPS Performance Monitoring Plan 3. TOPS Performance Monitoring Plan When initiated in late 2010 as a new knowledge management program within USAID’s Office of Food for Peace, The TOPS Program did not have an extensive body of indicators or performance monitoring plans (PMP) to draw upon. TANGO International, the consortium member that led M&E, drafted TOPS’ PMP in PY1. Its three levels of indicators were: 1. Program Strategic Objective (SO); 2. Program Results (PR) for Knowledge Capture (PR 1), Knowledge Generation (PR 2), Knowledge Application (PR 3) and Knowledge Sharing (PR 4); and 3. Intermediate Program Results (IPR) for PR 1 through 4. Some targets were annual while others were cumulative, and some targets were percentages while others were numbers (outputs). With the fluidity of The TOPS Program’s capacity to respond rapidly to emerging needs, the targets set in PY1 were estimates; nonetheless, a comparison of PMP results to targets does illustrate the strong performance, many achievements, and several challenges of The TOPS Program. The full PMP table for the life of The TOPS Program is in Annex III. PMP Indicators: The TOPS Small Grants Program Two Strategic Objective (SO) indicators and one Intermediate Program Result (IPR) indicator tracked the performance of the Small Grants Program, a component that assumed increasing importance during the life of TOPS. Annual targets of 80 percent increasing to 90 percent for Strategic Objective 4.0, percent of small grants that meet their objectives, were met or exceeded, with results of 80 percent in PY4, 100 percent in both PY5 and PY6, and 96 percent in PY7. During PY4 and PY6, TOPS fell slightly short of meeting annual targets for Strategic Objective 5.0, numbers of organizations (directly) served by Micro Grants, but greatly exceeded targets for PY2, PY5, and PY7. The Micro Grants were submitted, reviewed and accepted or rejected on a rolling basis, and this explains why, for example, TOPS fell short of the target for PY4, while the target for PY5 was surpassed. The sum of all annual results for Strategic Objective 5.0 (83527) exceeded the sum of all annual targets (410). Similarly, annual targets for IPR 2.4.1, number of tools created, field-tested or validated using small grants, were nearly met or exceeded year by year, but the sum of all annual results (37) surpassed the sum of all annual targets (29). 27 Organizations’ headquarters and country offices were counted separately. For example, service to World Vision headquarters, World Vision Ethiopia, and World Vision Burkina Faso counted as three organizations for any small grant activity; some of these same organizations would be counted more than once if they participated in multiple small grants. Final Report: August 2010 to January 2018 TOPS Performance Monitoring Plan 35 PMP Indicators: Knowledge Sharing A gap in the PMP is the lack of a Strategic Objective-level indicator to capture the impact of the extensive and highly successful knowledge sharing activities implemented through The TOPS Program. A key strength was TOPS’ capacity to respond to demand to create knowledge sharing opportunities—a demand that far surpassed expectations when TOPS developed the PMP. The TOPS Program hosted large, multi-sectoral Knowledge Sharing Meetings in the U.S. and overseas; organized single topic knowledge sharing events in response to emerging interests; and facilitated stakeholder consultations in support of FFP’s commitment to open dialogue with the food security and nutrition community. These activities became extremely important within TOPS and the FSN Network. Results exceeded targets for all five indicators at the PR level and for three of four indicators at the IPR￾level. Highlights include:  IPR 4.1.1, number of FSN Network knowledge sharing events held. This became an ever-expanding area of activity for The TOPS Program in response to extensive interest and demand from the food security community. After annual targets of five such events per year were exceeded for PY2 through PY5, TOPS increased targets to 12 for PY6 and 8 for PY7. Still, results far exceeded the modified targets: TOPS coordinated 30 events in PY6 and 46 in PY7.  The results of IPR 4.1.1 contributed to the success of The TOPS Program in greatly exceeding annual targets in all years for PR 4b, number of participants attending FSN Network technical knowledge sharing meetings. This indicator tracked participation in the large, multi-sectoral TOPS Knowledge Sharing Meetings, plus the single topic meetings and stakeholder consultations (noted above). After increasing the annual targets of 250 (PY1 to PY4) to 500 for PY6 and PY7, TOPS still exceeded targets, at 1,017 and 1,538 participants, respectively. Satisfaction survey results (PR 4c) of this large number of participants were consistently maintained at or above 80 percent.  The increase in annual targets for PR 4e, number of downloads from the FSN Network website, were eclipsed each year, particularly in PY6 (target 8.0k, result 23.2k) and PY7 (target 8.0k, result 43.8k), demonstrating the growing awareness and relevance of resources offered through this channel. Similarly, results for IPR 4.1.3, number of hits on the FSN Network website, exceeded targets (PY7 target 45.0k, result 78.0k).  Despite some challenges, TOPS was able to maintain active and engaged membership in task forces (PR 4a) throughout its lifetime, meeting or surpassing the annual target of 40 members: annual results averaged between 40 and 70 members. It was a greater challenge to develop and maintain regional interest groups (IPR 4.2.1). Annual targets set at five were achieved in PY2 through PY4. The posting of two TOPS staff to Uganda in early 2016 supported the development of regional interest groups in the East and Southern Africa Regions: three were achieved in both PY6 and PY7. The Technical and Operational Performance Support (TOPS) Program 36 TOPS Performance Monitoring Plan PMP Indicators: Documentation of Core Competencies and Promising Practices An achievement of The TOPS Program was the development of minimum sets of core competencies for field staff responsible for the implementation of FFP non-emergency activities by TOPS thematic area: ANRM, Commodity Management, Gender, M&E, Nutrition and Food Technology, and SBC.28 TOPS staff developed these in collaboration with the members of the FSN Network task force for each sector. The core competencies were extensively field-tested early in The TOPS Program and the results contributed to the finalization of a TOPS self-assessment toolkit that included the core competencies and a self￾rating tool. This toolkit served as a mechanism for quickly assessing implementing partners’ needs or gaps, helping TOPS prioritize activities. Two relevant indicators that captured cumulative progress were achieved: PR 1a, an effective system is in place for periodic knowledge and skills self-assessment of Title II functional areas by Title II implementing organizations; IPR 1.1.1, percent of Title II core competency areas defined by the task forces. Another area of achievement for The TOPS Program was the production of a report in PY4 that documented promising practices in maternal and child health and nutrition, gender and ANRM: TOPS Review of Promising Practices in Food for Peace Development Food Assistance Projects. 29 The practices were identified during a review of FFP programs active between 2006 and 2010 that achieved reductions in child stunting; practices were selected that seemed likely contributors to that reduction. TOPS disseminated the report widely among FFP implementing partners for further discussion, including at several TOPS events in PY4 and PY5. The review was downloaded 57 times from the FSN Network website and TOPS ensured the document reached the key members of the PVO implementing community through dissemination at two specific events and distribution during all large Knowledge Sharing Meetings. TOPS anticipated continuing this type of review in search of promising practices with a plan to add to the initial report after each round of reviews. Instead, TOPS and task forces created other opportunities to identify and disseminate promising practices through TOPS’ many channels. Facilitation of capacity strengthening workshops enabled TOPS technical specialists to become more familiar with the successful practices self-identified and highlighted by each FFP-funded activity. They then continued discussions with activity staff, often remotely but occasionally through field visits. As additional promising practices were identified, TOPS invited affiliated project staff to present at Knowledge Sharing Meetings, to create and host webinars, and to develop case studies that were posted as resources on the FSN Network web portal and disseminated through the FSN Network newsletter. The ANRM team, in particular, developed eight case studies, focusing on promising practices within ANRM, such as 28 See http://www.fsnnetwork.org/core-competency-series 29 The TOPS Program. 2014. TOPS Review of Promising Practices in Food for Peace Development Food Assistance Projects Available at: http://www.fsnnetwork.org/tops-review-promising-practices-food-peace-development-food-assistance-projects Final Report: August 2010 to January 2018 TOPS Performance Monitoring Plan 37 permagarden, micro-irrigation, water-smart agriculture, natural resource management, and other technical topics. TOPS, led by the Knowledge Management team, also initiated video documentation of promising practices: two videos were produced in PY6, one on mobilizing key district administrative actors in support of actions to reduce child stunting, 30 and one from a workshop on promising practices in ANRM. 31 The TOPS Program originally anticipated that there would be two indicators to measure promising practices, one indicator (IPR 1.3.1) focused on capturing the number of promising practices, and another indicator (IPR 2.3.1) focused on producing and disseminating promising practices. As The TOPS Program progressed, these indicators eventually measured the same promising practices from capture to documentation stage, thus the results were the same for IPR 1.3.1 and IPR 2.3.1. Through the case studies, videos, and promising practices document described above, The TOPS Program exceeded the cumulative target of four for indicator IPR 1.3.1, number of reports documenting promising practices produced, with a cumulative 10 in PY7. PMP Indicators: Technical Capacity Strengthening One SO indicator, one PR indicator and three IPR indicators were included in the PMP to track the capacity strengthening activities of The TOPS Program. Targets of 80 percent for PR 3c, average score (normalized) on post-training satisfaction survey, were exceeded all years, reaching from 87 to 90 percent. Annual targets for IPR 3.1.1, number of regional or global multi-sectoral training events held, were met--or nearly so-- each year. Over time, this indicator assumed less relevance for The TOPS Program than expected. Initial plans were to hold multi-sectoral technical training events before or after the large regional Knowledge Sharing Meetings to take advantage of participants’ presence. Requests for capacity strengthening in the field resulted in many workshops that generally focused on one technical sector or a combination of two, such as nutrition and SBC, so this indicator did not capture the breadth and sum of training efforts. IPR 3.1.2, number of staff trained through country level, regional or global training events, however, did capture the breadth of activities, and annual targets were exceeded in all but one year. The sum of all annual results (3,151) greatly surpassed the sum of all annual targets (2,033): TOPS achieved 155 percent of the target for this indicator. Annual targets for SO 2.0, percent of Title II implementers scoring 70 percent or higher in post-test assessment on key knowledge, skills, and practices for improved program performance, were set at 60 percent in PY1 and steadily increased to 80 percent in PY4 through PY7. These targets were mostly met or exceeded in all but PY7 when language comprehension issues in two capacity strengthening workshops depressed the average for this indicator.32 Gender was not included as a technical sector in the TOPS RFA or in Save the Children’s proposal. TOPS added the position of Gender Specialist in PY2 after the PMP was established and IPR 3.1.3, percent of 30 See http://www.fsnnetwork.org/tops-programs-video-springs-community-nutrition-activities-southwestern-uganda 31 See https://www.youtube.com/watch?v=bK_WdO4UdOI&feature=youtu.be 32 The low results were in Bangladesh and Ethiopia, and were due to lower English language skills than self-reported by participants when registering for the workshops. Importantly, participatory activities during the training workshops did reflect participants’ acquisition of new knowledge and skills. The Technical and Operational Performance Support (TOPS) Program 38 TOPS Performance Monitoring Plan TOPS training events that integrate gender considerations / principles in content of sessions, was added to the PMP with targets of 100 percent to show TOPS’ commitment to gender integration. TOPS would have benefited from additional indicators that provided more information on the impact of its efforts in this area. PMP Indicators: Tools TOPS developed indicators to track its work to identify useful tools, fill gaps in tools, and disseminate a limited number of highly relevant tools. Over time, it became apparent that the PMP contained too many indicators for tool activities relative to their overall importance in TOPS (8 of 29 total indicators, excluding the indicator monitoring the production of tools through the TOPS Small Grants Program). The original intent was to track progress in the several phases relevant to tools (identification, generation, recommendation, and dissemination). The PMP also included indicators to capture the impact on end users of tools. Two of the five Strategic Objectives focused on tools and both were difficult to monitor on an annual basis. Strategic Objective indicator 3.0, percent of FSN Network Task Force recommended tools incorporated into FFP guidelines and information bulletins was difficult to report, both because the target was developed as a percentage of all tools disseminated through The TOPS Program, a denominator that shifted over the life of TOPS, and because FFP documents often referred readers to the FSN Network website for sets of resource tools rather than individual tools. One difficulty in tracking Strategic Objective 1.0, number of tools in each phase, was that it was a composite of multiple phases of tool capture (PR 1), generation (PR 2), and application (PR 3). Tools did not necessarily advance linearly over time, either. For example, TOPS-created tools that had advanced through the phases and were field-tested through ‘application’ often returned to the ‘modification’ phase after additional use in a greater variety of contexts―or as FFP updated its strategy or revised its guidelines. The PMP also contained multiple IPR-level indicators tracking the tool phases as cumulative results compared to cumulative targets. In hindsight, including the PR-level indicators that tracked impact on end users (see PR 3a and PR 3b, below) at the Strategic Objective level might have been preferable. Among the indicators for tool capture (PR 1) and generation (PR 2) tracked at the PR and IPR levels were the following, in order of the above-mentioned phases:  IPR 1.2.1, number of tools relevant to food security programs identified. TOPS identified 78 tools, short of the cumulative target of 113. This was largely because fewer relevant tools existed than TOPS expected when it developed the PMP. In addition, tools that were identified often had critical gaps or required updating.  IPR 2.2.1, number of tools modified or new tools created (based on identified gaps) and in process to be endorsed. This became a larger area of effort for The TOPS Program than was expected when the PMP was developed. The cumulative result of 50 exceeded the cumulative target of 45. Final Report: August 2010 to January 2018 TOPS Performance Monitoring Plan 39  IPR 2.1.1, number of effective and existing, user-friendly tools endorsed / recommended by FSN Network Task Forces. The annual--cumulative--targets for this indicator were estimates at the time the PMP was developed, and were partly based on an overly optimistic expectation of existing tools that would receive a ‘recommended’ designation. The cumulative target of 55 was not reached. In PY7, the result was 44.  PR 2a, average score on user perception survey of appropriateness, efficacy and satisfaction in use of FSN Network recommended tools. This composite indicator was monitored through surveys conducted in PY4, PY5 and PY7. 33 Annual targets of 70 percent were set for PY1 through PY5 and increased to 75 percent for PY6 and PY7. The target for PY4 was not quite achieved, but results for PY5 and PY7 exceeded targets, at 82 percent and 84 percent, respectively. As noted, the indicators under PR 3, Knowledge Application, might have better served as Strategic Objective-level indicators:  PR 3a, percent of Title II implementers aware of at least 50 percent of key (‘relevant to their sector’) FSN Network-recommended tools. As noted, TOPS tracked this indicator via surveys of FFP grantee field staff. Survey results in PY2, with a limited number of tools recommended at that time, exceeded targets (63 percent versus target of 50 percent). Survey results in PY4 and PY5 did not quite achieve targets (69 and 60 percent, versus targets of 70 and 75 percent). However, by the end of TOPS, the annual result for PY7 was 80 percent, surpassing the target of 75 percent.  PR 3b, percent of projects (that are aware of the tools) reporting use of FSN Network Task Force recommended tools. Survey results remained within the range of 43 to 53 percent in PY4, PY5 and PY7, short of annual targets of 70 percent. TOPS encountered various challenges to monitoring this indicator, perhaps linked to the low response rates for surveys done through remote mechanisms. The TOPS Program had its highest success with this survey in PY5 with 253 respondents. In other years, response was much less. In other situations, projects valued a tool but did not yet have opportunity to use it, based on their program cycle. Among the several elements tracked for the composite indicator of PR 2a, user perception of tools (above), 90 percent of those surveyed deemed the tools appropriate for their projects and would recommend them to colleagues. 33 TOPS conducted a survey in PY2 to report on indicators PR 3a (awareness of tools), and PR 3b (reported use of tools), but felt that survey respondents had not yet had sufficient time using the tools to provide useful information on the composite indicator PR 2a. Hence, information from the PY2 survey was only tabulated for the PR 3a and PR 3b indicators. In summation, in all program years The TOPS Program met or exceeded more than three-quarters of its targets for 29 indicators in the PMP. Targets were exceeded for almost all of the indicators for PR 4, Knowledge Sharing. The number of organizations served by micro grants within the TOPS Small Grants program also greatly exceeded expectations. In its final years, The TOPS Program identified innovative ways to identify and disseminate promising practices, including case studies and video. Challenges remained for establishing and maintaining regional interest groups actively sharing knowledge, and for use of TOPS recommended tools by project staff in the field, although results for these indicators were 50 percent or greater. The Technical and Operational Performance Support (TOPS) Program 40 The TOPS Program Effects and Impact 4. The TOPS Program Effects and Impact As the preceding sections illustrate, The TOPS Program reached widely across geography, organizations, and sectors within the food security and nutrition community of practice. TOPS’ work across the four Program Results directly touched countless individuals and organizations working in the many FFP focus countries. The TOPS’ strategic objective, as stated in the 2009 RFA34 , however, was not designed to measure these wide reaching impacts, placing emphasis on identifying knowledge, information, and best practices and ensuring these were widely shared and adapted. TOPS found it important to look beyond the boundaries of the relatively few stated outputs and deeper into effects and impacts of its actions. First, this comprehensive view helped TOPS plan and manage: Were audience needs being met with the right tools, trainings, and forums for knowledge sharing? If not, what must change? Second, more broadly, it reflected TOPS’ and FFP’s long-term vision of improving people’s lives via better U.S. Government-funded food security and nutrition programming. This section discusses several lines of inquiry into TOPS’ effects beyond those measured by its PMP and discussed in the preceding sections of this report. First, it features anecdotal information from key FFP, TOPS and non-consortium PVO actors, highlighting their views on TOPS-induced changes in the food security and nutrition community over time―and, by extension, in the community’s work. Finally, it describes evidence of TOPS’ contributions to technical advancements in food security and nutrition programming. Did the Food Security and Nutrition Community Change as a Result of TOPS? The TOPS Program arose from the unsatisfactory outcomes of FFP’s Institutional Capacity Building (ICB) grant program, in which individual PVO grantees received funds to build in-house capacity, and develop tools for use by their programs. A 2008 meta-evaluation of the ICB program found that, while PVOs successfully created tools covering a wide area of technical areas, they fell short in disseminating tools and building field capacity to use them. M&E systems were of variable quality and rarely supported a learning cycle within projects, much less among separate organizations with a shared interest in food security. An absence of collaboration and networking mechanisms between grantees in the ICB program led to an array of unlinked approaches, duplication of effort, and little sharing of best practices for application at scale across grantee programs. In the final months of TOPS, the Knowledge Management team and communications specialist interviewed dozens of stakeholders35 as they created two videos summarizing TOPS’ scope and achievements.36 Among other topics, interviewees offered thoughtful observations on change over seven-plus years in the food security and nutrition community. Anecdotal in nature, the comments’ 34 TOPS’ objective as stated in the 2009 RFA was: “Highest quality information, knowledge and best practices for improved methodologies in Title II food aid commodity program performance are identified, established, shared and adapted.” 35 From FFP, PVO grantees, and TOPS consortium members. 36 See: https://youtu.be/qKxZU9csiMg and https://youtu.be/3U1j-mjyzdc Final Report: August 2010 to January 2018 The TOPS Program Effects and Impact 41 themes are remarkably similar from person to person, suggesting a shared perception and understanding of the changes that TOPS fostered and their value. Repeatedly, stakeholders spoke of the neutral, safe space that TOPS created encompassing grantee organizations that heretofore were, at best, irregular collaborators and, often, competitors for donor funding. When interviewed, FFP staff in particular remarked that the challenge was to create an inclusive community of practice and overcome “that attitude that people had…that implementing partners will never work together, speak to each other, share their best practices.” Confounding these expectations, FFP grantees (and others, such as academic and governmental entities) proved remarkably open to collaboration. TOPS provided multiple opportunities for collaboration through large Knowledge Sharing Meetings, multi-organizational capacity strengthening workshops, and stakeholder consultations, based on grantees’ needs expressed in task force meetings, online discussion groups, and informal discussions with technical leads. From the beginning, TOPS staff intentionally sought to build a collaborative spirit with many opportunities for interaction, while maintaining an environment of open dialogue. One FFP staff member observed that stakeholders readily participated in opportunities to work on shared interests. This collaboration also resulted in members benefitting from mutually addressing challenges that are common across organizations. From a practical point of view, TOPS undertook tasks that implementers simply could not. TOPS staff noted that a significant value of TOPS was having an adequately funded program with dedicated technical staff who have the support, opportunity, and time to consult throughout the community to identify and share promising practices. The TOPS team sought to create opportunities and channels for people to talk, to meet, to share knowledge and explore new approaches, and to help increase the number of tools available to organizations implementing food security and nutrition programs. The TOPS domestic U.S. and regional Knowledge Sharing Meetings were major opportunities for practitioners to meet and share. FFP staff observed that these events had a particular value for field staff: “The vast majority of participants attending the TOPS-hosted Knowledge Sharing Meetings in Asia and Africa are the local technical staff who would never have the opportunity to attend something like this, experience all this knowledge, get to meet their compatriots two countries over, discuss ideas in a way that shares information and builds the capacity of everyone in the room, and creates a much more substantial and hopefully lasting community of practice.” Another FFP staff, commenting on the collaborative environment fostered by TOPS, said simply: “Now the implementing community view TOPS as their own, instead of seeing it as led by a consortium.” Nor was the community spirit limited to FFP grantees. Another common theme among those interviewed at FFP was TOPS’ success at extending the neutral space to encompass FFP as a technical peer: “Another big achievement of TOPS is creating the bridge between FFP and the implementing community…a safe space where the implementing community and the donor can discuss things, not wearing a contractual hat but more as professional colleagues and partners in development.” TOPS staff noted that this was a strategy from the beginning, to facilitate two-way communication and mutual learning to advance the understanding of food security and nutrition programming. The Technical and Operational Performance Support (TOPS) Program 42 The TOPS Program Effects and Impact One TOPS staff reflected that, when appropriate, this non-hierarchical communication between FFP and grantees was ‘the kind of thing TOPS would like to see as a legacy: increasing opportunities for sharing and communication between donor and implementer, a peer-to-peer technical and management communication opportunity, so the challenges that both grantees and FFP face are better understood. The donor wants to be sure that its money is being well spent, that positive effects are being achieved, and that vulnerable people are being served. Implementers want to do a good job--a better job--and to report to donors and have their efforts appreciated. I give credit to the donor for expending the amount of time they did to be participants in the events that the TOPS group put together.” The above quotes indicate that TOPS provided opportunities and set the tone for FFP partners and FFP itself to interact as peers, in a neutral environment for knowledge pursuit and sharing. The interviewees went further, however, to note that this safe environment was itself a factor in several important changes that took place during TOPS’ lifetime. Two key examples are highlighted below:  FFP 2016–2025 Food Assistance and Food Security Strategy. As FFP developed and, later, rolled out its new ten-year strategy, TOPS facilitated a number of consultations between donor and implementers. These included specific events focused on elements of the strategy, sharing the draft strategy at four Knowledge Sharing Meetings, and ensuring many opportunities for discussion and input to the strategy by the implementing community. That process supported the creation of a strategy designed to meet both donor and implementer interests in serving beneficiaries with the best possible programming. When introducing the final strategy at the January 2016 Knowledge Sharing Meeting, the Director of FFP credited The TOPS Program with creating the “theme of collaborative learning, partnership and even collective action/collective impact [that] are … woven throughout our new strategy.”  Theory of Change. The frank and collegial collaboration between FFP and TOPS helped TOPS foresee and respond to FFP needs, even as FFP flexibility in terms of budget and workplan allowed TOPS to adjust its program activities—sometimes quickly—based on need. For example, in 2014, FFP began requiring grant applicants to develop a TOC to describe the logic of their proposed projects. Initial efforts were weak, however. According to one FFP staff: “In that first round of proposals, the TOC was not as advanced as it should have been. TOPS sensed a need and […] brought communities together, provided capacity strengthening support, organized workshops, talked to the donor and tried to understand the major issues. TOPS then designed a training program around TOC that significantly improved the quality of TOCs that partners develop as part of their projects.” “[The impact of the Knowledge Sharing Meetings is] huge, and not the easiest to measure. It is direct, in that you have people walking away knowing things they didn’t know before, having access to materials they didn’t have before. They have a stronger network of practitioners to discuss their ideas with…they have a face behind headquarter staff, FFP staff. The social aspect of it cannot be underestimated for its importance in programming: the fact that you have a larger community of practice, that people know each other, can learn from each other, email each other. Those intangibles are difficult to measure. Without a doubt, the programming of FFP would be worse without these meetings. The quality of implementation, quality of staff, would be decreased without them.” FFP Interviewee Final Report: August 2010 to January 2018 The TOPS Program Effects and Impact 43 Sustaining the open, collaborative community of practice was a topic of concern to all interviewees, including participating FFP grantees. Said one: “Going forward, it will be important to continue the momentum that has been built, in getting people to share their experiences on implementation and application, and seeing that these are built into the new FFP strategy.” Did TOPS Contribute to Technical Advances in Food Security and Nutrition? While measuring technical advances in food security programming was outside The TOPS Program’s scope, TOPS consortium members and the task forces created and promoted such advances in the course of their work to capture, generate, apply and share knowledge. TOPS was a vehicle for promoting and disseminating state-of-the-art tools and promising practices, many of which its own staff and peers on task forces created. With the entire food security community as its audience, TOPS was also able to promote a ‘new and better normal’ by gathering, distilling and broadly disseminating quality standards and practices. For this final report, the lead staff of each TOPS technical area were asked to cite an example of advances in their sector, attributable to The TOPS Program. The ANRM technical team cited the Permagarden Toolkit, 37 developed by consortium member Mercy Corps, which guides implementers through a design methodology to test and use home garden technologies that suit a specific location, as a clear advance in ANRM technology. This toolkit combines elements of permaculture (an agricultural approach whose design principles use natural systems for production) and bio-intensive agriculture (which maximizes production on a small amount of land, using sustainable practices that increase biodiversity). The permagarden method helps smallholder farmers produce nutritious food year-round, improve soil health, conserve water, and learn basic principles and practices. The toolkit was tested and refined into a suite of materials, including training guides and training of trainer (ToT) guides, which enabled participants to replicate the trainings with their peers in their field program areas. All materials are available in English, French, Spanish and Amharic. The first TOPS permagarden training took place in 2014, and interest in the method began to accelerate in PY5, PY6 and PY7. The method is spreading in countries where TOPS itself delivered permagarden training (in DRC, Nepal, Uganda and more) and in countries, such as Sudan, where training support was provided outside of TOPS. The Ethiopian government adopted the permagarden methodology and now promotes these practices nationwide. TOPS’ Commodity Management team and task force gathered stakeholders from around the world to discuss needs, test new ideas, and undertake research activities. Perhaps their greatest contribution to technical excellence in the sector was to create the meticulously researched Commodity Management Toolkit, 38 a single source operational guide that gathers best practices and promotes compliance with 37 The TOPS Program, Mercy Corps. 2017. The Permagarden Toolkit. Available at: http://www.fsnnetwork.org/tops￾permagarden-toolkit 38 The TOPS Program. 2017. The Commodity Management Toolkit. Available at: http://www.fsnnetwork.org/commodity￾management-toolkit The Technical and Operational Performance Support (TOPS) Program 44 The TOPS Program Effects and Impact USAID regulations for commodity handling. The user-friendly toolkit’s interactive flowcharts, supported by 147 tools, provide a road map of the entire commodity management cycle from ordering and procurement of commodities to distribution and close-out. The toolkit can be used for self-study, as a training resource, and as a quick-reference guide in the field. Readily available on the FSN Network website (and USB memory stick), USAID also offers the toolkit to those who visit its food assistance resources page, stating that it is ‘a one-step operational guide for anyone working on programs using food commodity. It provides staff quick access to the specific information they need on minimum requirements, whenever and wherever they need it.’ Commodity Management staff now have a vetted, single source of guidance that reflects industry standards and donor requirements. It is available in English and French. The TOPS Gender Specialist and task force considered that their work and their collaboration with FFP were leading factors in FFP’s decision to specify a budget line for gender analysis in its subsequent RFAs. Today, all organizations submitting applications in response to FFP RFAs must plan to analyze the gender implications of their proposed programs. This new requirement (its effects on the ground are too early to determine) capped a chain of decisions dating to 2012, when USAID published its updated Gender Equality and Female Empowerment Policy. FFP subsequently approved TOPS hiring a full-time Gender Specialist. Although The TOPS Program originally designated gender a cross-cutting area affecting all technical sectors and formed a Gender Task Force, it did not originally have a dedicated staff specialist to advance its gender work. The Gender Specialist strengthened task force participation, especially with staff of FFP partners, and advocated for stronger gender programming integration in food security and nutrition activities. The Gender Task Force often invited FFP to join its meetings and to respond to implementers’ questions and concerns. This type of consultation occurred especially around measurement indicators, which then led the task force to ask FFP for guidance on how much implementers should budget for gender analysis and staffing. This resulted in the addition of the gender budget line in subsequent RFAs. TOPS’ Knowledge Management team created a series of skills-building workshops and tools with which FFP partners could learn to manage knowledge, internally and with other stakeholders. As FFP increasingly requested that partners work more collaboratively and include more formalized processes for knowledge sharing and exchange in programming, TOPS responded by developing workshops and events that transferred skills to incorporate knowledge management practices into their programming. The Knowledge Management staff and their Task Force created and carried out workshops on: in-person and virtual facilitation techniques, using video as a learning mechanism, building a knowledge management strategy, and developing a learning agenda; accompanying tools were created to reinforce learning and provide the opportunity for those who could not attend to develop the skills. Before TOPS, skills building for knowledge management was not available to the food security and nutrition community. Now, with the opportunity to gain knowledge management skills, food security and nutrition programming has become more learning-focused, and partners are more willing to share their successes and challenges. The TOPS M&E team used a demand-driven approach to strengthen monitoring and evaluation capacity and the application of M&E data among FFP implementing partners, and to create opportunities for effective consultation and collaboration across the FSN Network. Final Report: August 2010 to January 2018 The TOPS Program Effects and Impact 45 To strengthen M&E capacity, TOPS created the Monitoring and Evaluation Facilitator’s Guide39 , a comprehensive resource to facilitate an interactive M&E training course. The guide provides a resource for facilitators and practitioners alike, complete with PowerPoint slides, exercises and group work, and reference materials. In PY5, this was the most downloaded resource on the FSN Network. This guidance complemented TOPS’ efforts to ensure M&E Officers attained the Core Competencies for M&E, aligned with FFP’s Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities. The TOPS FFP self-assessment tool highlighted the need to strengthen capacity in all core competency areas, and particularly in data quality assurance and the use of qualitative methods. In Liberia, Bangladesh, Uganda, Burkina Faso and Niger, TOPS facilitated workshops for DFSA and Mission staff, which included intensive training on quantitative and qualitative sampling and analysis tools, and guidance on how to appropriately measure FFP indicators. One of these workshops, for example, focused on tools to measure the outcomes and impacts of DFSAs. Another workshop series applied training and tools in qualitative research design and analysis to an actual intervention intended to promote gender equity. In its final year, TOPS worked with FFP to facilitate a successful Annual Monitoring Workshop, which brought together DFSA M&E officers and FFP staff from Washington DC and Missions, and set a foundation for similar workshops in the future. TOPS also developed the Theory of Change (TOC) Facilitator’s Guide40 to support rigorous and evidence￾based planning, implementation, monitoring and evaluation of FFP activities. To complement the guidance and support M&E colleagues in the development and application of a TOC, TOPS facilitated a series of TOC workshops in Burundi, Madagascar, Malawi, Uganda, and the U.S., each tailored to meet the needs of PVO/FFP-grantee organizations. Some focused on creating a theory of change, and others guided participants to use their TOC and M&E data as an adaptive management tool to adjust activities over the life of FFP awards. To further enhance the effectiveness of TOPS activities, the M&E team led a series of highly valued Stakeholder Consultations to bring together FFP staff and implementing partners to discuss critical issues, such as FFP M&E policy and guidance and challenges and opportunities related to FFP indicators. While consultations strengthened capacity in core areas, they also, and importantly, fostered a process of collaborative learning within the TOPS community of practice. Moreover, the M&E team responded to emergent interests and needs of implementers to initiate the Resilience Task Force in 2016. Through the task force and a series of workshops, TOPS provided practitioners with cutting-edge guidance and resources for resilience measurement and analysis, including a collaborative and implementer-led session on promising practices for activity-level monitoring. TOPS’ Nutrition and Food Technology team and task force used TOPS’ multiple channels to emphasize several elements of optimal nutrition programming. Information from surveys, repeat visits to programs, personal communication and anecdotes support the idea that FFP-funded projects have expanded their basic set of nutrition interventions in these aspects: 39 The TOPS Program. 2015. Monitoring and Evaluation Facilitator’s Guide. Available at: https://www.fsnnetwork.org/monitoring-and-evaluation-facilitators-guide 40 The TOPS Program. 2017. Theory of Change Facilitator’s Guide. Available at: https://www.fsnnetwork.org/theory-change￾training-curriculum The Technical and Operational Performance Support (TOPS) Program 46 The TOPS Program Effects and Impact  The Nutrition Programming Design Assistant41 tool, developed initially through the CORE Group Nutrition Working Group and revised with input from The TOPS Program, was used to support programmers to review secondary source data or project baseline indicators and choose one or two Infant and Young Child Feeding (IYCF) practices for ‘extra’ effort over the life of their project. Almost all projects promoted, through their central community approach (mothers’ groups), all the basic minimum set of recommendations for IYCF: early initiation of breastfeeding, exclusive breastfeeding to six months of age, continued breastfeeding to 23 months, introduction of complementary foods at six months, and six optimal complementary feeding practices. Prior to The TOPS Program, the focus was on coverage and scale of the minimum set of recommended IYCF practices, but TOPS emphasized the need for additional efforts and suggested choosing among the poorest performing indicators or choosing a practice close to being sustained by the community (typically, over 80 percent of mothers practicing). FFP also stressed the importance of thoughtful nutrition strategies with an emphasis on quality before quantity, which supported the strategies promoted by TOPS.  Among the six optimal complementary feeding practices (density, frequency, quantity, diversity, responsive feeding and food hygiene), most PVOs emphasized dietary diversity and the link between agriculture and nutrition. TOPS chose to focus on the complementary feeding practices that were not receiving as much attention. Several projects reported establishing, at the community level, the participatory exercises disseminated by TOPS that promoted appropriate density, frequency and quantity with responsive feeding practices.  Food security programmers were encouraged (in knowledge sharing events, technical capacity strengthening workshops, and via TOPS Small Grants) to develop SBC strategies for nutrition that would reach beyond women with infants and young children to identify other ‘persons of influence’ for optimal IYCF practices. Husbands and grandmothers were emphasized. To date, more projects have focused on fathers than on grandmothers, but recognition of grandmothers’ importance was increasingly expressed by PVOs. For TOPS, this emphasis was a continuation of technical assistance previously provided, in collaboration with The Grandmother Project, through the Infant and Young Child Nutrition Project, a USAID-funded project that ended when The TOPS Program began. One of the most important shifts that TOPS’ Social and Behavioral Change (SBC) team saw over seven years was the transition toward mainstreaming SBC: the growing awareness of its cross-cutting, cross￾sectoral utility and effectiveness. In 2010, SBC was typically seen as a side duty for staff in the health or nutrition sectors. Now, it is recognized as applicable to and useful for agriculture, food security, livelihoods, and gender activities. Creation of the core competencies helped propel this advance, along with general SBC visibility in TOPS and in USAID’s SPRING Project. 42 41 CORE Group Nutrition Working Group. 2010. Nutrition Program Design Assistant: A Tool for Program Planners. Available at: https://coregroup.secure.nonprofitsoapbox.com/storage/documents/Workingpapers/NPDA_workbook_web.pdf 42 The Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING) was a USAID-funded project dedicated to strengthening global and country efforts to scale up high-impact nutrition practices and policies. Final Report: August 2010 to January 2018 The TOPS Program Effects and Impact 47 A key TOPS contribution to mainstreaming SBC was the development of Make Me a Change Agent: A Multi-sectoral SBC Resource for Community Workers and Field Staff, 43 which provides freestanding lessons to build the skills of community-level workers, such as community health workers and agriculture extension agents, so that they can be more effective behavior change promoters in their communities. During PY2, the Social and Behavioral Change Task Force and the CORE Group SBC Working Group identified gaps in providing SBC skills to community-level workers. As a result, these groups worked together to draft, review, and complete the guide in PY5. The lessons are not sector specific, and are tried and true generic skills, such as communication and storytelling, that can help a development worker in any sector become more effective as an agent of behavior change. 43 The TOPS Program. 2015. Make Me a Change Agent: A Multisectoral SBC Resource for Community Workers and Field Staff. Available at: https://www.fsnnetwork.org/make-me-change-agent-multisectoral-sbc-resource-community-workers-and-field￾staff The Technical and Operational Performance Support (TOPS) Program 48 The TOPS Program Management 5. The TOPS Program Management TOPS Management The TOPS Program adopted a flexible, adaptive management approach on the principle that changes in the needs of the food security community could be expected over time and that the specific areas of change were unknowable at the outset. TOPS established itself with solid expertise in the areas requested in the RFA: management of FFP development food assistance programs, knowledge management, M&E, natural resource management, behavioral change communications, agriculture, and nutrition/food technology. Headquartered in Save the Children’s Washington, DC offices and with several remote-working staff, TOPS benefited from highly productive professionals from the outset, and the flexibility to add expertise in commodity management, gender, resilience, and other areas as TOPS evolved. Save the Children, as the leader on The TOPS Program Cooperative Agreement and overall manager, brought a long history of effective collaboration with other agencies. Its experience in bringing together diverse stakeholders to reach consensus on, for example, best practices and standards, paid dividends throughout TOPS to foster collaborative program learning and program innovation. Save the Children’s experience in institutional capacity strengthening, exemplified by the successful implementation of its earlier ICB grant and two prior Institutional Support Project grants, enabled the TOPS consortium to establish productive and collaborative relationships with partners throughout the development community. The organization’s experience in awarding funds to other agencies through a number of mechanisms, including umbrella grants, contributed to its management of the TOPS consortium overall, and to building and managing the successful Small Grants Program during the life of TOPS. Staffing Structure The original TOPS RFA called for six key personnel positions: Project Director, Knowledge Management Officer, M&E Officer, Agriculture Officer, Social and Behavioral Change Officer, and Nutrition and Food Technology Officer. The Project Director was one of two individual staff members who remained with The TOPS Program from its beginning in 2010 to close-out in early 2018. The Director ensured technical and managerial oversight of all program elements, and set the performance standards and organizational culture of TOPS. To ensure consistency across organizations and reflect level of expertise, the titles for the key technical personnel were adjusted to be Senior Specialist rather than Officer. The other Save the Children key position was the Nutrition and Food Technology Senior Specialist. CORE Group managed the Knowledge Management function, providing the Knowledge Management Senior Specialist. This position turned over just once. TOPS consortium partner Mercy Corps was responsible for the Agriculture function, which turned over twice, with the third incumbent serving for over five years until close-out. Consortium partner TANGO International covered M&E, with one position turnover during the course of TOPS. Food for the Hungry (FH) was responsible for the Social and Behavioral Change (SBC) technical component in TOPS and changed their technical lead only once. Final Report: August 2010 to January 2018 The TOPS Program Management 49 TOPS made adjustments in the staffing structure over time, touching on all thematic areas: a Gender Senior Specialist position was created in the third year, managed by Save the Children. Save the Children also established a Commodity Management (CM) team in response to a request from FFP to meet a clear need for commodity management training within the implementing community. The team consisted of a CM Senior Specialist (hired in PY2), a CM Senior Training Specialist (hired in PY6), and a CM Operations Manager (hired in PY3). The Operations Manager position was later changed into a Senior Operations Coordinator. TOPS (Save the Children) added a Program Deputy Director position in PY5, for various operations management tasks and for oversight of the Small Grants Program, which had expanded greatly. The Knowledge Management team (CORE Group) expanded over the course of TOPS, ultimately including a Knowledge Management Specialist, a Knowledge Management Officer, and a Communications Associate, in addition to the key personnel position of Knowledge Management Senior Specialist. Mercy Corps’ ANRM team also grew, ultimately including a field-based (Kampala) Regional Agriculture Specialist and an Agriculture Officer in Washington, DC. TANGO International’s expertise in M&E expanded to include an M&E and Gender Senior Specialist and a Senior Research Specialist during the latter part of TOPS, in addition to the key position of M&E Senior Specialist who transferred from Washington, DC, to Kampala in PY6. Food for the Hungry’s SBC staff included a Senior Specialist for most of TOPS, one full-time and one part-time Specialist, and one part-time Coordinator. In the first few years of TOPS, the SBC team also had a part-time position dedicated to ANRM. Save the Children was responsible for the Small Grants Program management, the communications function, and overall program management support. An Associate Director for Grants and Finance oversaw the Small Grants Program, along with a Small Grants Specialist. The Small Grants Specialist position turned over at the end of PY6 and was filled by a Small Grants Coordinator, a position more focused on grant administration, documentation, and close-out of the Small Grants Program in the last year of TOPS. Several staff persons in succession filled the Communications and Documentation Specialist role at TOPS. A Program Associate and Program Specialist rounded out TOPS’ staffing. See the organization chart as Annex I. Program Advisory Committee In the original TOPS proposal, Save the Children proposed forming a Project Steering Committee (PSC) in Year 1 “to provide overall direction and guidance to TOPS.” Each principal and collaborating partner would provide one senior representative for the PSC; Save the Children would provide two. TOPS/Save the Children would invite representatives from FFP, FANTA and other implementing PVOs to participate. TOPS planned for the PSC to meet face-to-face once a year and have an intervening teleconference. The PSC would not take part in the day-to-day management of TOPS. Its role was to:  Develop and maintain the overall project vision and advise on strategic direction  Provide feedback on annual plans prior to submission to FFP  Suggest programming priorities  Suggest criteria for Small Grant approval and Associate Award allocation to the PMT  Mobilize partner resources to support TOPS The Technical and Operational Performance Support (TOPS) Program 50 The TOPS Program Management One of the early actions of TOPS’ management, following launch of the award, was to form and convene this committee. Among its first decisions was to change the name from Project Steering Committee to Program Advisory Committee (PAC). This was to better reflect the commitment for this group to be a strategic “think tank” for The TOPS Program and an interactive forum for the donor, implementing partners, and other stakeholders to collaborate and help guide TOPS for the overall benefit of the development food security community. The PAC fulfilled the functions intended for the PSC (noted above), while adding significantly in strategic thinking and program insight. Participation was enthusiastic and one of the first decisions by the group was to meet quarterly, rather than only once or twice a year. The formation and operation of the PAC was a great success for TOPS. From the beginning, the meetings were public and drew eager participation by senior food security managers of the major implementing partners, with occasional participation by other interested members of the community of practice. It reflected and modeled the open, inclusive nature of The TOPS Program. Of particular value was the faithful attendance of the FFP Agreement Officer's Representative (AOR) and Activity Manager at nearly every meeting for the entire seven years. This provided a close, face-to-face consultative experience for the implementing partner representatives. The quarterly meetings enabled donor and implementers to establish a comfortable, professional familiarity and helped them build and maintain an atmosphere of open communication and collaboration throughout the life of TOPS. The PAC provided valuable advice on strategic trends and often brought issues to TOPS’ attention that were later adopted as topics for TOPS’ learning or knowledge sharing events. The group was a valuable source of information about the overall community, development trends, experiences, and “heads-up” alerts about issues that TOPS and the community might be facing in the near future. Technical Contributions of TOPS Consortium Members Save the Children A leading provider of development and humanitarian assistance around the world, Save the Children brought significant technical expertise and experience to TOPS, in addition to its managerial oversight described above. Save the Children is the leading independent organization for children and their families, currently reaching 125 million children in 120 countries. Its staff are foremost technical experts with decades of experience designing and delivering successful interventions in food security, nutrition, nutrition￾sensitive agriculture, maternal and child health, youth livelihoods and employment, and humanitarian response. Save the Children has a proven record of efficiently designing and managing large, complex projects, managing 520 cooperative agreements, grants, and contracts valued at $1.7 billion. Its development and emergency food security, youth livelihoods, financial services, and employment portfolio–funded by USAID, foundations, and corporate donors–totals over $690 million. With a vast range of relationships with industry practitioners, research institutes, universities, governments, policymakers, and community groups, Save the Children is adept at mobilizing and leading diverse actors in well-functioning, high- Final Report: August 2010 to January 2018 The TOPS Program Management 51 achieving consortia. As a leader in nutrition programming and commodity management, as the first NGO recipient of a local and regional purchase commodity program from FFP, and as an organization long involved in gender-based programming, Save the Children brought global expertise in three important focus areas of the food security and nutrition community: commodity management, nutrition and food technology, and gender. Commodity Management Direct food assistance is an important part of USAID’s efforts to fight global hunger and improve the nutritional status of children and vulnerable households around the world. FFP allocates more than $2 billion annually to several types of food assistance programming to meet both chronic and acute food needs in vulnerable populations. Except in emergency contexts, food distribution is rarely a standalone activity. Instead, food commodities support activities in nutrition, agriculture, and health programs. While food is a non-cash resource, it must be controlled as carefully as cash. The management of food commodities is a complex activity whose many steps must comply with strict U.S. Government laws and regulations. The TOPS Program supported and strengthened the food commodity implementing community with workshops and tools to improve efficiency, decrease losses, strengthen monitoring and reporting, and ensure that food reaches the families and households who need it most. Nutrition and Food Technology Deficiencies in vitamins and minerals and other forms of undernutrition contribute to nearly a third of all child deaths, impede healthy development, and reduce life-long productivity. Maternal undernutrition leads to poor fetal development and higher risk of pregnancy complications. The TOPS Program supported practitioners in the maternal, infant and child nutrition community by creating and delivering specialized workshops and tools for successful program design, implementation, and monitoring and evaluation. Examples of topics covered in tools and trainings include Infant and Young Child Feeding (IYCF), the value of continued breastfeeding, calculating the minimum acceptable diet, feeding during and after illness, and improved maternal diet, among others. Gender Gender roles, norms, and relative power positions are closely connected to food security and nutrition. In developing countries, rural women are commonly responsible for most childcare, for growing and preparing most of the food consumed in the home and for raising small livestock, yet rural men commonly decide how to use food and its proceeds. It is important that women participate in decisions regarding crops, livestock and diet, as a crucial precondition for improving the family’s food security, livelihood and nutrition. Gender norms are deeply rooted but, because they are socially governed, they can be changed. As a cross-cutting area, TOPS integrated gender into all trainings and across all technical areas. An area of particular focus was gender and resilience. The concepts and practices of resilience are widely discussed and analyzed within in the food security community, but often with no regard for gender. To address this gap, TOPS designed and delivered several sessions and workshops to increase knowledge and awareness of how gender can and should be integrated into resilience training. The Technical and Operational Performance Support (TOPS) Program 52 The TOPS Program Management CORE Group CORE Group began in 1997 as an informal collaboration of health technical experts sharing best practices in child survival programming. It developed into an independent 501c(3) organization that, for 20 years, has fostered collaboration among USAID implementing partners and experts, academics and advocates, and a globally linked group collectively known as CORE Group’s Community Health Network. CORE’s efforts have produced improved quality, equity, scale and sustainability of USAID-funded work. CORE Group’s technical momentum is driven by its 74 NGO members. CORE’s technical working groups were a model for the TOPS task forces, and synergy between CORE and TOPS groups was evident through TOPS implementation. CORE’s focus on developing practical guidance and tools that establish new standards also served TOPS well in identifying technical knowledge gaps and in creating resources to address them. Knowledge Management Knowledge Management—the successful capture, sharing and analysis of knowledge among members of the food security and nutrition community—was at the center of The TOPS Program and CORE brought its own experience to the TOPS consortium’s organizational mix. Led by CORE, the TOPS Knowledge Management team ensured that knowledge from all sectors was successfully shared across the FFP-funded food security community via the FSN Network website and e-newsletter, and the TOPS signature Knowledge Sharing Meetings. The TOPS/CORE team helped strengthen the community’s own capacity for knowledge management by supporting the development of a number of tools and resources on topics such as the design of participatory meetings, techniques for participatory facilitation, and ways to link development practitioners through communities of practice. It also devised and conducted workshops and trainings on developing knowledge management strategies, learning agendas, and video products. These highly participatory events gave participants practical experience, skills, knowledge, and tools they could use immediately to improve their own knowledge management practices in food security programming. Food for the Hungry Food for the Hungry’s (FH) organizational purpose is to facilitate sustainable development and provide emergency relief to people in need, while recognizing their dignity, creativity and ability to contribute to solving their own challenges. With specialization in agricultural production and marketing, natural resource management, and community-based health and nutrition, FH’s expertise in Social and Behavioral Change (SBC) methodologies, including Barrier Analysis, Positive Deviance, Care Groups, and Quality Improvement Verification Checklist, improved many stakeholders’ programming during the course of TOPS implementation. Social and Behavioral Change Closely connected to the other cross-cutting technical areas (knowledge management, gender, and M&E) of The TOPS Program, SBC looks at how knowledge and awareness of good practices or new technologies can be translated into sustained behavioral change and improved food security and nutrition in the long term. Simply increasing knowledge or awareness about a new way of doing things rarely leads people to apply their new knowledge. The TOPS Program and the FH SBC team helped implementers design their programs in ways that emphasized productive behavioral change. From promoting breastfeeding and helping parents choose nutritious foods for their children, to stimulating Final Report: August 2010 to January 2018 The TOPS Program Management 53 farmers’ interest in redesigning their farms to make better use of natural resources and increase resilience, FH’s SBC work was a crucial component of all TOPS food security and nutrition activities. Mercy Corps Mercy Corps is an international relief and development organization whose mission is to alleviate suffering, poverty and oppression by helping people build secure, productive and just communities. Mercy Corps helps people in the world’s toughest places turn the crises of natural disaster, poverty and conflict into opportunities for progress. In recognizing that access to adequate food and nutrition is integral to the success of relief and development programming, Mercy Corps’ agricultural development initiatives take a market-driven approach that maps and strengthens relationships between buyers, sellers, producers and end consumers. The organization implements approximately 60 agricultural development projects in 27 countries. Agriculture and Natural Resource Management MC brought to TOPS crucial knowledge and experience in ANRM and food security, focusing on how farming technologies and strategies like irrigation methods, farm site design, and use of fertilizers can make agriculture more effective and resilient to shocks, such as droughts and floods, while sustaining a healthy physical environment. Programming in ANRM also involves teaching marketing and financial planning skills to farmers so they can improve their incomes, and designing strategies for communities and farms so they can better manage their land, water, soil, and vegetation. Led by Mercy Corps, the TOPS ANRM team organized multiple field-based technical trainings for implementers, and produced a series of comprehensive Technical Guides and Case Studies. Following the FFP extension of TOPS for an additional two years, in PY6, Mercy Corps placed a Regional Agricultural Officer in Kampala, Uganda, who became the first TOPS overseas regional focal point. The Regional Agricultural Officer established links with local and regional African agricultural practitioners, FFP and USAID Missions, and implementing partners for FFP-funded development food security activities in the larger region. Mercy Corps also has an established reputation and expertise in resilience. MC collaborated with TANGO International, the TOPS consortium resilience lead, on a number of related activities, learning events, training workshops, and Associate Awards during the life of TOPS. TANGO International TANGO provides institutional and technical support to organizations that work to reduce vulnerability and to alleviate hunger, poverty, and discrimination. It builds a bridge between research and practice that supports institutional capacity development to achieve sustained impact. TANGO has extensive experience in all aspects of development programming, but specializes in food security, M&E, and resilience. Other areas of expertise are project design, objective and indicator development, risk and vulnerability assessment, quantitative surveys and qualitative assessments, data analysis, tool development, policy and guideline development, and guidance for multi-sectoral implementation. In addition to its core team of specialists, TANGO has built a global network of experts, consultants, educators, and advisors that it accesses to carry out its highly regarded research and evaluation work. TANGO has also built trusted working relationships with U. S. Government and multi-lateral donors and The Technical and Operational Performance Support (TOPS) Program 54 The TOPS Program Management agencies, and with staff, implementers, academics, and leading thinkers and practitioners in many fields, including food security programming, resilience, and theories of change. TANGO’s deep research and resulting published analyses and papers have given it an influential voice in global development policy and practice. M&E Programs and activities must be monitored and evaluated to determine if they are working as intended. There are many ways to gather this information and new methods are developed constantly. The TOPS M&E team, led by TANGO, helped implementers and donors improve the effectiveness of M&E of food security and nutrition programs by identifying and disseminating successful methods, tools, and practices. The TOPS M&E team improved skills of staff at headquarters and on the ground by conducting many U.S.-and field-based capacity strengthening workshops and practical exercises; The TOPS M&E team also worked closely with FFP and implementers to ensure that implementing agency priorities were reflected in donor guidance. Theory of Change Workshops: A TOPS focus that strengthened program planning, monitoring, and evaluation Theories of Change (TOCs) have gradually replaced results frameworks and similar planning and evaluation tools as a preferred development hypothesis. Beginning in 2014, FFP required that TOCs be submitted as part of project proposals, and that approved projects review their TOCs annually as implementation progresses. To help FFP grantees develop their proficiency in the relatively new TOC process, The TOPS Program began offering workshops in 2015. Ultimately, TOPS offered two distinct types of TOC workshop, both building on real-life examples rather than simulation and customized for each trainee organization. One type of workshop guided staff through the necessary steps for testing and modifying an existing project’s theory of change. The other type examined the theories of change that were included in rejected project proposals, with the objective of learning from failure. Participants emerged from these workshops with the skills to create stronger models and knowledge to accurately articulate a theory of change for a food security program. Consistently positive evaluations testified to the success of this initiative. Moreover, the Theory of Change Facilitators’ Guide44 and accompanying tools supported FFP partners to design and use TOCs, including in pre-design workshops following FFP RFAs. Resilience TANGO had previously established high-profile leadership in defining, monitoring and evaluating resilience as applied to food security and nutrition programming, which led to TOPS’ involvement in a number of important resilience-themed initiatives. Most of the 13 Associate Awards that Save the Children received through the TOPS Leader with Associate (LWA) award were used to access partner expertise in resilience and M&E. 44 The TOPS Program. 2017. Theory of Change: Facilitators’ Guide. Available at: http://www.fsnnetwork.org/theory-change￾training-curriculum Final Report: August 2010 to January 2018 The TOPS Program Management 55 Associate Awards As a Leader with Associate Award Cooperative Agreement, TOPS provided a mechanism for Associate Awards to be made by FFP, USAID Missions, Bureaus, and Regional Offices, or other U.S. Government funding sources. No further competition or waiver of competition was required for Associate Awards issued within the terms of the Leader Award, making them an attractive mechanism to fund important activities related to TOPS’ objectives relatively quickly. Each Associate Award was a separate agreement between a U.S. Government entity and Save the Children as the TOPS Leader Award Recipient, to support a distinct activity that fit within the scope of the overall TOPS Program. Depending on the scope of work for a particular Associate Award, Save the Children could partner with other organizations – including those in the consortium – to implement the award/s. As shown in Table 6, 13 Associate Awards were made through the TOPS mechanism from 2010 to 2018. Although somewhat slow to start—the first award was approved in February 2014—Associate Awards became important components of The TOPS Program, totaling $26,675,451 over the life of TOPS. In the case of several awards such as REAL, SCALE, WASH, and Bridge, these Associate Awards will continue functioning from one to five years beyond the end of the TOPS award itself. These awards enabled TOPS consortium members, particularly TANGO International and the TOPS Leader, Save the Children, to extend the reach of their technical and managerial expertise and to address programming priorities and new initiatives identified by USAID offices in various parts of the world. Eight of the 13 awards focused on some aspect of resilience, signaling the important role that TOPS and its consortium members played in advancing the thinking and promoting good practice in resilience among food security implementers around the world. Another Associate Award reviewed FFP-funded, cash-based emergency food assistance programs, reflecting the growth of emergency programming in FFP overall. Each Associate Award is briefly described below. Please see Annex VI for more details and Annex VI a) through f) for the final reports of awards that were completed. Table 6: Associate Awards Made During The TOPS Program Date Issued Brief Description of Each Associate Award Implementing Partner (s) Feb. 2014 A weeklong USAID workshop in Niamey, Niger to launch three new resilience-themed awards for Niger and Burkina Faso— REGIS AG, REGIS ER, and SAREL. CORE Group Sep. 2014 A Humanitarian to Development Resilience Collaboration Training (HADA) for USAID field staff in West Africa. TANGO International Apr. 2015 Rapid, targeted research to identify shocks and stressors, and to contextualize resilience indicators for USAID-funded programs in Nepal. TANGO International Apr. 2015 A baseline study for enhancing resilience and economic growth in Somalia program. TANGO International, Save The Technical and Operational Performance Support (TOPS) Program 56 The TOPS Program Management the Children International Sep. 2016 Advanced Resilience M&E Training (ARMET) for Field Practitioners in Ouagadougou, Burkina Faso, built upon the HADA training held in Senegal in February 2015. TANGO International Mar. 2016 Two workshops in Asia intended to advance resilience measurement practices in rural and urban environments by providing a hands-on learning opportunity for select USAID and implementing partner staff. TANGO International Jul. 2016 Review of FFP’s Cash-Based Emergency Food Assistance Program through 2017. TANGO International Sep. 2016 (ending in Sep. 2021) Funded by the USAID Bureau for Food Security and managed by the USAID Center for Resilience, the Resilience Evaluation, Analysis and Learning (REAL) Award works to provide USAID missions, host governments, implementing organizations, and other key stakeholders with rigorous yet practical monitoring, evaluation, strategic analysis, and capacity building support related to resilience. Save the Children, Food for the Hungry, Mercy Corps, TANGO International Jan. 2018 (ending in Jan. 2019) TOPS Bridge Associate Award: A one-year award designed to continue essential functions of The TOPS Program―The FSN Network, commodity management training, stakeholder consultations, Theory of Change design, and other strategic and adaptive learning activities—until passing these functions on to the TOPS follow-on award, IDEAL, to be issued by FFP later in 2018. Save the Children, CORE Group Jan. 2018 (ending in Jan. 2023) Strengthening Capacity in Agriculture, Livelihoods, and Environment (SCALE): Issued by FFP, SCALE is designed to further improve the impact, sustainability, and scalability of FFP’s activities in agriculture, natural resource management, and livelihoods. Save the Children, Mercy Corps Jan. 2018 (ending in Jan. 2023) Water, Sanitation and Hygiene (WASH): Designed to improve the quality of activities, strengthen the capacity and skills of FFP implementing partners in WASH, and improve the level of knowledge, tools, and practices in WASH components of FFP￾funded development and emergency food security activities. Save the Children Jan. 2018 (ending in Jan. 2022) Learning for Resilience: Managed by USAID/Ethiopia, this award will collect baseline and endline data to measure activities within the Learning for Resilience project against their intended outputs and outcomes. Kimetrica, TANGO International Dec. 2017 (ending in Dec. 2021) Uganda RCT: This award will conduct a randomized controlled trial (RCT) to measure the cost-effectiveness of different variations of graduation programming for refugees and host communities in Uganda, as part of the Pilot Development Food Security Activity (DFSA) there. Innovations for Poverty Action Final Report: August 2010 to January 2018 The TOPS Program Management 57 Small Grants Program The Small Grants Program’s activities and achievements were a significant proportion of TOPS ―approximately $4 million over the life of The TOPS Program, or roughly 13 percent of the total budget. The Small Grants Program was a prime instrument through which TOPS supported a highly diverse set of food security-related research, learning, and testing activities. The small grant-funded work by a wide variety of organizations contributed to all TOPS PRs. TOPS made grants not only to FFP implementing partners but also to lesser-known, innovative development organizations, such as Trees for the Future and Helvetas, a Swiss-based water and sanitation-focused PVO. This was to bring new collaborators into the FFP IP community to share their knowledge and experience. Small grants were a principal means of supporting collaboration between institutions, stimulating field-driven development, and bringing tested products to completion and scale. To encourage field-led activities, collaboration among implementing partners (both FFP-funded and otherwise), TOPS prioritized field-developed proposals and those that included several partners. Within TOPS, Save the Children managed the Small Grants Program. Initially, start-up was slow: only three grants were awarded in the first three years. In the beginning, IPs were not familiar with the Small Grants Program and it was not clear how IPs could integrate these modestly funded awards into their overall programming portfolios. In addition, grant approval and awarding procedures at both TOPS/Save the Children and FFP were cumbersome and time-consuming, resulting in slow turnaround for grant applications. With cooperation and substantial streamlining of the processes at both FFP and Save the Children—and adoption of recommendations by the Mid-Term Evaluation―program activity greatly improved. Concurrently, TOPS began promoting the Small Grants Program through the PAC and at learning events, such as the annual Knowledge Sharing Meetings. Consequently, interest and uptake grew markedly from the third year of TOPS. See Figure 5 for additional details. With an improved approval process and the addition in PY5 of a TOPS Deputy Director position with responsibility for overseeing and strengthening the Small Grants Program operations, grant making proceeded more swiftly. TOPS issued over three dozen grants from late PY3 through mid-PY5. All these grants had close-out dates at the end of PY5 (2015), the end date of the original five-year TOPS award. When FFP extended TOPS two more years to 2017, TOPS re-launched the Small Grants Program. TOPS started issuing new grants by the second quarter of PY6 and over the next 12 months, TOPS awarded 29 small grants, all of which successfully closed by the fall of 2017 (PY7). Annex II provides a detailed listing of the 71 grants. TOPS produced two booklets describing the small grant-funded projects and summarizing their results for the periods of 2010-2015 and 2015-2017. The Technical and Operational Performance Support (TOPS) Program 58 Comparison of Budget to Expenditures 6. Comparison of Budget to Expenditures As shown in Table 7 below, a total of $29,780,421 of the total award budget of $29,999,967 was spent between August 13, 2010 and January 31, 2018. This represents over 99 percent of the total estimated budget TOPS had planned to spend through the end of the award. Given Save the Children’s close-out protocols and timeline, and due to the subawardees’ reporting cycles, this table includes an accrued payment to one of the implementing partners (TANGO) that was made at the time of this report being prepared, for expenses incurred for activities performed during PY7. Comparing the budget estimated and approved to be spent through the end of TOPS with the actual expenditures by line item, it can be observed that the program expenditures concluded on track. Slight overspending in salaries and fringe is related to staff continuing to work on TOPS during several no-cost extension periods, which extended TOPS’ end date from August 12, 2017 to January 31, 2018. Similarly, subawards to CORE, Mercy Corps, and Food for the Hungry are also overspent, as these implementing partners were extended to work during the no-cost extension, completing more activities than originally planned. These increased expenses due to continued operations were covered by shifting funds from other budget lines. Given the consultative nature of TOPS’ work and the dependence on community demand for TOPS’ trainings and workshops, there were several activities that were envisioned for PY7 but did not end up happening – for example in the area of nutrition. This resulted in underspending not only on the program expenses budget line, but also the related travel as well as contractual budget lines. In addition, TOPS was able to leverage some savings when planning and executing several Commodity Management activities (for example the Experts’ Summit in Dubai), freeing up the funds for the continued programming during the NCE period. Final Report: August 2010 to January 2018 Comparison of Budget to Expenditures 59 Table 7: TOPS PY1-PY7 Budget vs. Actuals Line Item Year 1 to Year 7 Budget45 (USD)46 Actual through January 31, 201847 (USD) % Spent 1 Personnel 6,026,730 6,162,599 102% 2 Fringe Benefits 1,464,776 1,571,731 107% 3 Travel 827,302 723,732 87% 4 Supplies 41,356 40,990 99% 5 Contractual 558,818 494,476 88% 6 Other Direct Costs 175,683 172,867 98% 7 Small Project Grants 3,940,488 3,921,099 99% 8 Training/Workshops 2,171,267 1,661,875 77% 9 CORE Group 3,668,416 3,932,762 107% 10 Food for the Hungry 2,065,607 2,083,573 101% 11 Mercy Corps 2,726,875 2,857,549 105% 12 TANGO International 2,815,215 2,732,076 97% 13 IYCF-E Toolkit 39,360 28,066 71% 14 IYCF-E Training 40,924 41,265 101% 15 Make me a Change Agent 30,362 29,999 99% Total Direct Costs 26,593,178 26,454,659 99% Indirect Costs 3,406,789 3,325,762 98% Total Project Costs 29,999,967 29,780,421 99% 45 The TOPS Program performance period was August 13, 2010 through January 31, 2018. 46 Numbers may be slightly off due to rounding; each budget line total is shown as per the approved Mod 10 budget. 47 Note that due to subawardee reporting cycles and Save the Children close-out processes, the total includes an accrual for a final payment to an implementing partner that was made after 1/31/18 for activities performed during Year 7 of the program. The Technical and Operational Performance Support (TOPS) Program 60 Annex I: The TOPS Program Organization Chart Annex I: The TOPS Program Organization Chart Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 61 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Micro Grants > $50K Catholic Relief Services/ CaLP Researching Multi-Modality Programming Multi￾modality Conduct research to document conditions under which multi￾modality programming is appropriate and feasible and the best practices for its implementation. Global 7/5/2016- 6/30/2017 Global $16,208.50 CORE Group Emergency Toolkit for Food Security and Nutrition Protection: Cholera Disease Preparedness Care Group Module Nutrition Develop a Cholera Disease Preparedness module to equip mothers and others to engage in optimal health and nutrition practices that mitigate potential negative impacts of cholera on children’s nutrition and overall health. US/Global 10/1/2016- 5/31/2017 Global $42,117.14 FHI 360 Developing and Disseminating Essential WASH Actions Tools WASH, Nutrition Develop and disseminate a set of Essential WASH Actions including a Toolkit and information about key concepts to integrate into existing reference materials. Global 4/1/2016- 6/30/2017 Global $49,553.16 Grandmother Project Stories Without an Ending: An Adult Education Tool for Dialogue SBC, Nutrition Produce a guide on the development and use of stories￾without-an-ending in community nutrition programs to promote change in norms and practices Senegal 4/12/2016- 2/28/2017 West Africa & Sahel $49,085.00 The Technical and Operational Performance Support (TOPS) Program 62 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount and Social Change through a process of adult learning. Helen Keller International Developing Francophone Master Trainers in Essential Nutrition and Hygiene Actions Framework for Health & Community Workers Nutrition Organize a training of trainers in Francophone West Africa to equip participants to use training guides and reference materials and enhance their counseling skills through practice. West Africa 8/9/2017- 3/31/2017 West Africa & Sahel $45,925.00 HELVETAS Behavior Change Approach in WASH Projects WASH, SBC Share lessons learned and mainstream an innovative WASH methodology into the design and implementation of food security projects based on tested approaches, tools, and best practices. Côte d'Ivoire, Niger, Madagascar 8/1/2016- 5/31/2017 West Africa & Sahel $49,840.00 Helen Keller International Designing for Behavior Change on Barrier Analysis SBC Organize a seven-day combined Designing for Behavior Change and Barrier Analysis workshop in Hanoi involving 23 individuals from various PVOs and countries across the region. Vietnam 8/9/2016- 4/30/2017 Asia & Near East $49,525.66 International Medical Corps Camp-based Barrier Analysis of Breastfeeding, Complementary Feeding, and Nutrition, SBC The purpose of the Barrier Analysis is to identify determinants of key behaviors promoted in the Syrian refugee population: exclusive Jordan 4/25/2016- 9/30/2016 Asia & Near East $46,001.97 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 63 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Early Antenatal Care Seeking Behaviors among Syrian Refugees in Azraq Camp, Jordan breastfeeding for the first 6 months, timely introduction of complementary feeding, and accessing antenatal care during the first trimester. International Medical Corps Camp-based Cost of the Diet Assessment among Syrian Refugees in Azraq Camp, Jordan Nutrition, SBC Conduct a Cost of the Diet assessment in Azraq Camp, Jordan to quantify the extent to which available foods are able to meet energy and nutrient requirements for individuals or the entire household and the extent to which households can afford to feed their children with a diet meeting minimum requirements for macro- and micronutrients. Jordan 4/25/2016- 9/30/2016 Asia & Near East $38,097.65 International Medical Corps Barrier Analysis Assessment on Breast Feeding and Antenatal Care Behaviors of Syrian Refugees in Lebanon Nutrition, SBC Lead a Barrier Analysis training and implementation with Syrian refugees in Lebanon to understand constraints and to promote exclusive breastfeeding and antenatal care behaviors for mothers and children under six months of age. Lebanon 4/6/2016- 9/30/2016 Asia & Near East $42,001.77 International Medical Corps Barrier Analysis of Breastfeeding, Complementary Feeding, and Maternal Nutrition of Nutrition, SBC Identify determinants of key behaviors (exclusive breastfeeding for the first six months, complementary feeding practices, and increased consumption of food during pregnancy) that have Turkey 4/25/2016- 9/30/2016 Asia & Near East $49,146.00 The Technical and Operational Performance Support (TOPS) Program 64 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Adolescent Mothers among Syrian Refugees in Turkey been promoted in the Syrian refugee population, in particular among adolescents. Land O'Lakes Livestock– Household Nutrition Learning Series ANRM Address gaps and deepen practitioner knowledge of livestock interventions’ impact on human nutrition through a yearlong series of four learning events. Global 5/16/2016- 6/30/2017 Global $49,876.08 Medical Teams International Development of Video of Partnership Defined Quality in Practice SBC Produce a high quality instructional video on the Partnership Defined Quality methodology while demonstrating its application to the Liberian setting. A set of discussion questions called a Learning Tool will also be developed to accompany the video. Liberia 8/11/2016- 2/28/2017 West Africa & Sahel $49,517.10 Mercy Corps Mobile Money Tool Multi￾modality Develop a set of resources to help humanitarian agencies and Mobile Network Operators communicate and collaborate more effectively. The tools will assist stakeholders in establishing productive mobile money partnerships for future humanitarian assistance programs. Senegal, Rwanda 1/12/2016- 9/30/2016 West Africa & Sahel; Central & Southern Africa $45,992.45 CARE Gender Training of Trainers Gender Build a cadre of Master Trainers to develop gender expertise across Bangladesh 12/11/2016- 5/31/2017 Asia & Near East $49,999.97 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 65 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount current FFP-funded programs in Bangladesh, who will then be responsible to conduct follow-on trainings and provide mentoring throughout their respective teams. Mercy Corps Youth in Agriculture Research ANRM Research the roles that youth play, or could play, in the agricultural development of their countries. The study will identify opportunities, constraints, perceptions and aspirations of youth when engaging in agriculture and will identify points of leverage to facilitate change among this group. Liberia 4/1/2016- 2/28/2017 West Africa & Sahel $49,065.95 Project Concern International Fathers Involvement Promoting Reproductive, Maternal, Newborn, and Child Health SBC Conduct a study to determine how father involvement via Fathers Groups functions as a supportive mechanism and positively contributes to reproductive, maternal, newborn, and child health behavior change practices among beneficiary families. Malawi 4/8/2016- 6/30/2017 Central & Southern Africa $50,000.00 Project Concern International Integrated Pest Management & Fumigation Safety Training CM Conduct a three-day hands-on training program to familiarize participants with the Programmatic Environmental Assessment and its annexes, including techniques for management of pests and proper US/Global 6/27/2016- 5/31/2017 Global $50,000.00 The Technical and Operational Performance Support (TOPS) Program 66 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount use of residual insecticides, phosphine, respiratory protection, phosphine monitoring devices, and warehouse fumigation. Project Concern International Dataset Policy Compliance Guide M&E Develop, test, disseminate, and foster the utilization of a Dataset Policy Compliance Guide by food security practitioners funded by USAID. Malawi 4/1/2016- 6/30/2017 Central & Southern Africa $50,000.00 Save the Children Community￾Based Management of Acute Malnutrition Toolkit Revision Nutrition Revise the Community-Based Management of Acute Malnutrition Toolkit to enhance its relevance and promote wider dissemination. Global 3/3/2016- 6/30/2017 Global $40,493.22 Project Concern International A Resource Guide for Enhancing Potential for Sustainable Impact KM Develop, test, disseminate and foster use of a sustainability toolkit by food security practitioners in PCI operational areas of Sub-Saharan Africa, Asia and Latin America. Global 12/3/2013- 11/30/2014 Global $49,571.77 Mercy Corps Barrier Analysis SBC, ANRM Identify barriers and facilitators in order to promote desired behavior among small agriculture families to prevent mycotoxin contamination via improved maize harvest and post-harvest practices. Guatemala 6/18/2014- 2/15/2015 Latin America & the Caribbean $49,778.36 International Medical Corps Case Study on Adolescent Nutrition, SBC Describe the experience of including adolescent girls in Care Nigeria, global 4/10/2015- 6/26/2015 West Africa & Sahel $20,190.00 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 67 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Inclusion in the Care Group Approach: The Nigeria Experience Groups and highlight barriers, facilitators, and lessons learned from the Care Group approach. Food for the Hungry Commodity Management Echo Training CM Five-day training to address the lack of capacity in commodity programming and management in the Democratic Republic of the Congo (DRC). Thirty-one staff attended from all five PVOs implementing FFP-funded II programs in DRC: Food for the Hungry, ADRA, World Vision, Mercy Corps, and CRS. Democratic Republic of Congo 4/16/2014- 9/30/2014 Central & Southern Africa $45,620.12 Global Alliance for Improved Nutrition Designing the Future of Nutrition Social and Behavior Change Communication SBC, Nutrition Fund travel of representatives to attend an innovative SBC Conference. These participants work in policy, development and implementation of nutrition programs, and bring critical perspectives from the Global South (including from Title II and Feed the Future priority countries) to enrich the learning process and follow-on documentation that will be disseminated broadly throughout the food security community. Global 10/20/2014- 12/31/2014 Global $32,043.00 The Technical and Operational Performance Support (TOPS) Program 68 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Mercy Corps Developing a Resilience Threshold using the Household Economy Approach Resilience Develop, apply and share a resilience threshold using Household Economy Approach (HEA) outcome analysis that will systematically identify when households have reached a point where they should be able to respond to shocks. Niger 6/5/2014- 5/31/2015 West Africa & Sahel $33,661.86 Mercy Corps Energy-Water￾Food Nexus Benchmarking ANRM Test and refine an assessment tool to help analyze “nexus” (energy￾water-food) intersections within the program’s focus areas in Northern Uganda. Uganda 1/20/2015- 5/31/2015 Central & Southern Africa $49,998.27 GOAL Flood and Drought through a Gendered Lens: Mitigating Strategies for Women/Girls/Bo ys/Men in Malawi Gender Implement a project that will provide evidence based recommendations to improve resilience and Disaster Risk Reduction (DRR) programming tailored to groups’ differing experiences, needs and strengths during floods and droughts. The project will be implemented in Malawi. Malawi 12/22/2014- 4/30/2015 Central & Southern Africa $48,517.00 The Grandmother Project Focus on Families and Culture: A Guide for Conducting a Participant Assessment on Maternal and Child Nutrition Nutrition Improve the quality and effectiveness of community programs to promote sustainable change in nutrition norms and practices by providing training on culturally-relevant and effective grandmother inclusive approach. Global 6/10/2014- 12/31/2014 Global $44,642.00 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 69 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount AgriTechTalk International U.K. Functional Numeracy for Food Security and Nutrition ANRM Pilot aims to achieve increased level of resilient food security, improved income and reduced poverty through improved decision-making and increased efficiency in rural enterprises, through improved numeracy. Uganda 1/1/2014- 12/31/2014 Central & Southern Africa $49,055.38 CORE Group Gender Sensitive Social and Behavior Change Gender, SBC, Nutrition Analyze interventions related to nutrition with the broader lens of gender equality to produce more learning about the extent to which gender-sensitive programming leads to improved nutritional outcomes from pregnancy to two years. Global 1/20/2015- 4/30/2015 Global $43,686.00 Catholic Relief Services with CORE Group Getting the Knack of NACS: a State-of-the-Art Event on Nutrition Assessment, Counseling, and Support Nutrition Technical meeting to advance the SOTA on Nutritional Assessment, Counseling, and Support (NACS) in the context of HIV, with the overall aim of building organizational capacity by 1) providing technical updates and review of current evidence, 2) sharing best practice and experience, and 3) facilitating in￾depth discussion and analysis of selected themes Global 1/10/2012- 4/30/2012 Global $19,961.63 CORE Group Improving Program Management Skills within Title KM Develop a field-based training on leadership and management that is focused on Title II consortia. Global 12/3/2013- 11/30/2014 Global $25,780.00 The Technical and Operational Performance Support (TOPS) Program 70 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount II Projects through the Development of Needs-Based Training Modules Save the Children US IYCF-E Toolkit: Rapid Start-Up for Emergency Nutrition Personnel Nutrition Implement the “Infant and Young Child Feeding in Emergency (IYCF￾E)” toolkit. Global 2/3/2014- 1/31/2015 Global $28,066.33 Save the Children US IYCF-E Training for East Africa Nutrition Trainings for participants from the Middle East and government ministry staff in Nairobi, Kenya on infant and young child feeding in emergencies (IYCF-E). Turkey; Kenya 4/16/2015- 5/31/2015 Asia & Near East; East Africa & Horn $41,264.71 Lutheran World Relief Learning for Gender Integration Gender With multiple stakeholders, gather and disseminate lessons on integrating gender in the project cycle. Global 12/3/2013- 11/30/2014 Global $49,840.00 Vetwork U.K. Livestock Emergencies Guidelines and Standards (LEGS) Training of Trainers Course ANRM Training of trainers held in Burkina Faso and adapted to the West African context. Based on previous LEGS trainings of trainer, used to teach participants about the LEGS approach. Burkina Faso 2/20/2015- 4/30/2015 West Africa & Sahel $49,946.22 Catholic Relief Services, with Cornell University, Land O’Lakes, Local and Regional Procurement Learning and Knowledge CM, KM LRP Learning Alliance workshop to educate food security stakeholders regarding their experiences with LRP programs and the effectiveness and Global 2/20/2015- 4/30/2015 Global $18,432.48 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 71 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Mercy Corps, and World Vision Sharing Workshop challenges of such programs in addressing food insecurity – 87 participants from 37 organizations. Save the Children US Make Me a Change Agent Workshop on SBC Lessons and Practice for Community￾Based Volunteers, Maradi, Niger SBC Workshop using the Make Me a Change Agent (MMCA) draft curriculum in Niger and Burkina Faso. MMCA curriculum seeks to build the skills of community-level workers so they can be more effective behavior change promoters in their communities. Niger, Burkina Faso 4/23/2015- 5/31/2015 West Africa & Sahel $29,998.70 CARE USA Measuring and Monitoring Gender Behavior Change Gender, SBC, M&E Finalize a set of standardized, but context-specific, gender monitoring indicators and formats for tracking intra-household and social norm changes. Broadly disseminate CARE’s innovative monitoring and evaluation processes for gender￾transformative food security programming. Global 1/28/2015- 4/30/2015 Global $49,998.70 Catholic Relief Services Post-PRIZE Monitoring for Resilience Project M&E A pilot to measure the sustainability of infrastructure and livelihood activities and overall resilience of communities, determining enabling and inhibiting factors within them. Zimbabwe 1/1/2014- 12/31/2014 Central & Southern Africa $49,310.00 The Technical and Operational Performance Support (TOPS) Program 72 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Concern Worldwide Potential of Nutrition￾Sensitive Conservation for Agriculture in Zambia ANRM, Nutrition Explore the potential, in Zambia, of nutrition-sensitive conservation agriculture, whose primary goals revolve around the sustainable intensification of agriculture through better soil fertility management. Zambia 2/4/2015- 4/30/2015 Central & Southern Africa $36,150.00 Project Concern International Promoting a Female-Focused, Savings-Led Microfinance Model in Food and Nutrition Security Programming KM Practitioner conference to build the capacity of participants to implement cost-effective and locally viable practices to reduce food insecurity through the implementation of successful “Village Savings and Loan” (VSL) methodologies – 61 participants from 39 organizations. Ethiopia 4/1/2012- 10/15/2012 East Africa & Horn $20,000.00 International Medical Corps Regional Designing for Behavior Change Workshop SBC, KM Design and hold a workshop to design a behavior change strategy. Sessions include training of trainers, information sharing and two case studies. Kenya, Ethiopia, Somalia, Jordan, Sudan, Yemen, South Sudan 4/1/2014- 4/30/2014 Global $46,341.00 Project Concern International Resource Guide for Building Effective Collaboration and Integrated Programming KM Develop guidelines to improve collaboration and integration among Title II Grantees and Implementers delivering Title II food aid programming. Guatemala 6/5/2014- 5/31/2015 Latin America & the Caribbean $50,000.00 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 73 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount CORE Group Technical Advisory Group Meeting on Current Innovations, Scale-Up, and Research on Care Groups in Title II Programs SBC Host a Technical Advisory Group (TAG) meeting and develop a position paper to review the implications of current innovations, scale up and research on Care Groups in Title II programs. Global 10/30/2014- 12/3/2013 Global $47,040.00 International Medical Corps Theory of Change Training M&E Training to strengthen the capacity of staff to use Theory of Change throughout the project cycle to support program planning, implementation, monitoring, and evaluation and to adapt programs when shocks occur. Washington, DC/HQ 3/25/2015- 5/31/2015 US $33,260.00 CORE Group Training of Master Facilitator￾Trainers in the Essential Nutrition Actions and Essential Hygiene Actions Nutrition Host a training of trainers workshop in the Essential Nutrition Actions Framework and Tools for Health Providers and Community Volunteers. Burkina Faso, Tanzania, Mali, Niger, Senegal, Burundi, Ethiopia, Zimbabwe, Haiti, South Sudan, Ecuador, Peru, Senegal 11/1/2014- 4/30/2015 Global $45,377.08 The Technical and Operational Performance Support (TOPS) Program 74 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Mercy Corps Vulnerability Assessment ANRM Assess the vulnerability of farming communities to variable and shifting environmental conditions, a disabling governance and policy environment, and inefficient agricultural markets. Myanmar 1/1/2014- 8/31/2014 Asia & Near East $49,959.00 Project Concern International Warehouse Staff Safety Guide and Tools CM Develop a CM Warehouse Staff Safety Guide, field test and disseminate it to Title II implementers. Global 12/18/2013- 12/18/2014 Global $50,000.00 Program Improvement Awards ACDI/VOCA Development of a Comprehensive Theory of Change Adaptive Learning & Validation Workshop Curriculum M&E Support the identification, testing, verification, promotion, sharing, and dissemination of innovative and robust methodologies for learning from and validating the processes and results of development programs’ Theories of Change. Ethiopia 6/27/2016- 6/16/2017 East Africa & Horn $91,215.72 Action Contre la Faim MAM'Out Project SBC, Nutrition Conduct a research project to assess the effectiveness and cost effectiveness of a context-adapted preventive approach of multiannual and seasonal cash transfers, likely to influence several underlying causes of under-nutrition. Burkina Faso 6/27/2016- 6/30/2017 West Africa & Sahel $94,079.40 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 75 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount AgriTechTalk International Increasing Numeracy in Rural Karamoja ANRM, SBC Tackle low functional numeracy levels among rural development and agricultural field staff and farmers, herders, and traders in rural communities in Karamoja, an area particularly affected by long￾term conflicts and developmental neglect. Uganda 6/27/2016- 6/30/2017 East Africa & Horn $92,949.62 Lutheran World Relief Learning for Gender Integration Plus Gender Increase the capacity of FFP grantees and other food security program implementers to conduct gender-focused evaluations to contribute to stronger evidence￾based delivery of food security programming. India, Nicaragua, Uganda 6/27/2016- 5/31/2017 Asia & Near East, Latin America & Caribbean, East Africa & Horn $99,999.00 Mercy Corps Whole Systems Resilience Design Framework M&E Develop a participatory monitoring and evaluation tool to measure the impact of applying a whole systems resilience design framework by projects and test and collate initial impact data in applying this methodology. Uganda 6/27/2016- 6/30/2017 East Africa & Horn $79,692.63 Farm Radio International Innovation for Food Security Program Implementation SBC, ANRM Design and test an interactive rural radio program strategy in Mali. The program will increase the accessibility, availability, and quality of key information and best practices in order to improve household food production, preparation, and consumption of nutritious food. Mali 8/15/2016- 6/30/2017 West Africa & Sahel $99,949.52 The Technical and Operational Performance Support (TOPS) Program 76 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Trees for the Future Agroforestry Training: Packaging a Proven Forest Garden Model into a Shareable, Adaptable Training Program ANRM Compile and share proven agro￾forestry training resources and launch a new web-based Forest Garden Resource Center. Malawi/ Global 11/24/2016- 5/31/2017 Global $97,899.01 World Vision Developing Guidelines to Improve Management of Agriculture Demonstration Sites ANRM Design and carry out a study that documents lessons learned and makes recommendations for improving technical and operational efficiency in management of demonstration plots. Bangladesh, Niger, Zimbabwe 8/15/2016- 5/31/2017 Asia & Near East, West Africa & Sahel, Central & Southern Africa $99,494.00 World Vision Evaluation of Resilience Approaches M&E, Resilience Conduct a comparative analysis and evaluation of shock-response approaches’ contribution to the Resilience Framework in the Zimbabwean context. Zimbabwe 8/15/2016- 6/20/2017 Central & Southern Africa $100,000.00 GOAL Building on Community Strengths: Identifying and Addressing the Social and Cultural Aspects of Maternal, Infant and Young Child Nutrition in West Hararghe, Nutrition Implement evidence-based SBC interventions, strengthen the household and community based family health promotion, and create an enabling environment for adoption of positive health behaviors in West Hararghe, Oromo Region, Ethiopia. Ethiopia 11/11/2014- 5/31/2015 East Africa & Horn $94,325.06 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 77 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Zone of Oromia, Ethiopia International Medical Corps Care Groups in Emergencies: Evidence on the Use of Care Groups and Peer Support Groups in Emergency Settings SBC Use of the Care Groups approach in emergencies appears to provide additional benefits including increasing social cohesion and providing psychosocial support in the midst of traumatic situations. Global 11/10/2014- 4/30/2015 Global $98,243.00 Catholic Relief Services Dietary Diversity and Determinants Survey M&E, Nutrition Through pilot testing and analysis, improve a survey-based assessment tool that is adapted from the Household Dietary Diversity Score and is used to assess beneficiaries’ diet quality, identify nutritional gaps, and provide programming recommendations for addressing these gaps. Burkina Faso, Burundi, Democratic Republic of Congo, Ethiopia, Malawi 1/1/2013- 12/31/2013 West Africa & Sahel; Central & Southern Africa; East Africa & Horn $56,624.34 World Food Logistics Organization Evaluation of a Post-Harvest Extension under the HORT CRSP in Sub-Saharan Africa ANRM, M&E Gather information on capacity building outcomes, best practices, gender issues, and evidence of success or failure of the five major components of the recently completed HORT CRSP pilot project on extension of Global 8/1/2014- 4/30/2015 Global $91,807.08 The Technical and Operational Performance Support (TOPS) Program 78 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount postharvest technologies in Sub￾Saharan Africa. ACDI/VOCA Information System for Management, Analysis, and Reporting Timeliness (I￾SMART) M&E Develop and pilot a cost-effective electronic beneficiary registration and tracking system to improve the accuracy of reporting and allow for analyses to determine impact associated with different interventions. Sierra Leone 2/1/2013- 1/31/2014 West Africa & Sahel $46,571.00 ACDI/VOCA I-SMART Version 2.0 M&E Improve the Information System for Management, Analysis & Reporting Timeliness (I-SMART). Zimbabwe, Liberia, Burkina Faso, Global 8/15/2014- 4/30/2015 Global $94,042.73 Project Concern International Integration of the Care Group MCHN BCC Model with the GROW Social and Economic Development Model: Synergies in Food and Nutrition Security Programming Nutrition, SBC Evaluate the effectiveness of integrating Care Group and GROW methodologies on health and nutrition behaviors through a quasi-experimental (case/control) study design; identify lessons learned and assess their implications for taking this pilot experience to scale and improving future food aid programming; document and disseminate the results of the study. Liberia 5/1/2013- 4/30/2014 West Africa & Sahel $87,487.00 Land O’Lakes Integrating Gender throughout the Gender Develop the second edition of this toolkit via a collaborative process with multiple stakeholders and expanding sections on food Tanzania 10/20/2014- 4/30/2015 East Africa & Horn $91,142.00 Final Report: August 2010 to January 2018 Annex II: TOPS Small Grants Program Summary 79 Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Project Life Cycle Version 2.0 security, nutrition, integrating men through a gender equality lens, and best practices in women’s programs. CARE USA with Catholic Relief Services Monitoring, Analysis and Response Kit (MARKit) M&E Develop the Market Analysis and Response Kit (MARKit) to empower development food assistance organizations to design, monitor, and evaluate programs that are responsive to local market conditions; pilot by Local Regional Purchase (LRP) Learning Alliance members around the world, then disseminate to the wider food security community. Niger 1/1/2013- 12/31/2013 West Africa & Sahel $99,984.50 Hellen Keller International Nurturing Connections: Adapted for Homestead Food Production and Nutrition ANRM, Nutrition, SBC HKI will fill a gap by revising and piloting the Nurturing Connections curriculum, a behavior change communication intervention that aims to increase women’s empowerment via group critical discussions of the domains of empowerment that have the highest impact on women’s and households’ health and well￾being. These may include: control over resources, control over income, unequal food distribution, and women’s mobility and participation in markets. HKI originally piloted Nurturing Bangladesh 11/12/2014- 5/31/2015 Asia & Near East $88,478.00 The Technical and Operational Performance Support (TOPS) Program 80 Annex II: TOPS Small Grants Program Summary Name of Organization Project Name Tech. Area Project Summary Location Dates Region Amount Connections in Bangladesh (2013) and notes continued interest in Bangladesh and internationally] Food for the Hungry Reading and Responding to Your Baby Nutrition, SBC Conduct a pilot to explore feasibility of teaching recognition of infant behavior and feeding cues to Q’ueqhi’ mothers in rural Guatemala in order to improve breastfeeding and complementary feeding practices. Guatemala 4/30/2015- 5/31/2015 Latin America & the Caribbean $99,641.28 OIC International with World Initiative for Soy in Human Health Programming Regional Agricultural Market Production of Super Gari (RAMPS) ANRM, SBC Promote the production and consumption of the fortified food product Super Gari, produced in Liberia, to improve nutrition and enhance local economic development as it becomes more widely used and commercially available. Liberia 1/1/2013- 12/31/2013 West Africa & Sahel $94,043.00 MGs: $2,119,936.23 PIAs: $1,897,667.89 Total: $4,017,604.12 Final Report: August 2010 to January 2018 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators 81 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 Program Strategic Objective: Highest quality information, knowledge, and best practices for improved methodologies in Title II food aid commodity program performance identified, established, shared, and adapted 1.0 Number of tools in each phase: Identification; Modification/ Creation; Recommenda￾tion/ Share; Adaptation/ Use 5 28 28 28 28 40 45 n/a 32 24 26 35 56 57 Through the FSN Network, identify existing tools, review tools and select for recommendation or modification, identify gaps for creation. Source: Internal tracking of progression among the phases. Annual targets Tools are only counted again each year in this target if they continue to move through the four phases. 2.0 Percent of implementers scoring 70% or higher in post￾tests on key knowledge, skills, and practices for improved program performance. 60% 60% 70% 80% 80% 80% 80% 56% 87% 75% 83% 83% 86% 68% Source: Pre- and post- test survey of participants. Annual targets Each participant will complete a pre-test and post-test at each training workshop to assess knowledge and skills. 3.0 Percent of FSN Task Forces recommended tools incorporated into FFP guidelines, information bulletins. n/a 20% 40% 50% 60% 65% 65% n/a 64% 50% 67% 78% 93% n/a Source: FFP guidelines, information bulletin, and other policy statements and notes. Annual targets Measures the number and percent of tools recommended by the FSN Network that are explicitly recognized by FFP in guidelines. The Technical and Operational Performance Support (TOPS) Program 82 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 4.0 Percent of small grants (≤US$ 100,000) that meet their planned objectives. n/a n/a 80 % 80 % 80 % 90 % 90 % n/a n/a n/a 80 % 100 % 100 % 96 % Indicator measures if objectives and deliverables were as per approval documents. Source: Review of small grant final reports to assess whether objectives have been met. Annual targets At the end of each PY, TOPS staff review available reports from awardees to compare results against proposal objectives and deliverables. 5.0 Number of organizations served by micro grants. n/a 15 45 75 105 85 85 n/a 122 n/a 69 242 69 333 This indicator tracks the number of organizations that participate in any event organized by the awardees through TOPS micro grants. Source: Review of final reports. Annual targets TOPS staff rely on the list of participants provided by the micro grant awardees. Program Result 1: KNOWLEDGE CAPTURE: Knowledge and skill needs of audiences identified. 1a. An effective system is in place for periodic self￾assessment of functional areas by Title II implementing organizations. n/a 1 1 1 1 1 1 n/a Partially complete Partially complete Partially complete Complete Complete Complete Elaboration of core competencies for the various technical areas for staff working on FFP￾funded projects. Self￾assessment to identify knowledge and or skills gaps. Cumulative target A set of tools for PVO adaptation is available on the FSN Network. IPR1.1: Skill levels of food security practitioners assessed through participatory and external approaches. 1.1.1 Percent of Title II core competency areas defined by the Task Forces. 75% 80% 100% 100% 100% 100% 100% 75% 80% 83% 100% 100% 100% 100% Task forces (by technical area) identified core competencies required for effective FFP program implementation. Cumulative target Includes: ANRM, CM, gender, M&E, nutrition, and SBC. Final Report: August 2010 to January 2018 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators 83 Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 IPR1.2: Relevant and current universe of appropriate tools identified. 1.2.1 Number of tools relevant to food security programs identified. 5 32 59 86 113 113 113 5 32 56 56 72 72 74 Number of tools identified through reviewing ICB reports, reports produced by other organizations, discussions with the task force members and interviewing implementers. Cumulative target This was a continuous process during the life of the program. IPR1.3: Promising practices identified and documented. 1.3.1 Number of reports documenting promising practices produced. n/a 1 1 1 1 3 4 n/a Semi complete draft 1 1 1 1 10 Number of promising practices identified through interviews, and literature review. Source: review of reports, interview practitioners and consultants, and field visits. Cumulative target New methods (case studies and video) adopted in addition to literature review. Program Result 2: KNOWLEDGE GENERATION: Reliable, high-quality information synthesized and produced in user-friendly, appropriate formats. 2a. Average score on user perception survey of appropriate￾ness, efficacy, satisfaction in use of FSN Network recommended tools. n/a n/a 70% 70% 70% 75% 75% n/a n/a n/a 64% 82% n/a 84% Survey administered in person or remotely by TOPS staff or consultant. Computation of average score for the three attributes: appropriateness, efficacy, and satisfaction. Annual targets Data is gathered under the supervision of TOPS M&E Specialist. The Technical and Operational Performance Support (TOPS) Program 84 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 IPR 2.1: User-friendly tools identified, modified or developed, and recommended by the FSN Network. 2.1.1 Number of effective and user friendly existing tools endorsed/ recommended by FSN Network Task Forces 5 16 27 38 49 55 55 5 12 16 19 27 31 44 Review of tools, task force meeting notes and discussion with implementers. Cumulative targets Effective and user-friendly existing tools identified under 1.2.1 are reviewed by task force members to recommend. IPR 2.2: Tools adapted (created to fill gaps or modified) for use in appropriate formats. 2.2.1 Number of tools modified or created (based on identified gaps) and in process to be recommended. n/a 6 12 18 24 45 45 n/a 4 16 28 39 44 50 Modification is when an existing tool is revised to meet a need or to reflect new developments or policies; creation is the development of a new tool to meet an identified need. Source: task force meeting notes/ tool documentation. Cumulative targets Task force members (with consultants if needed) create and modify tools based on the gaps identified. IPR2.3: Packaged information on promising practices shared with food security community for discussion and continuous improvement. 2.3.1 Number of packaged approaches and promising practices generated and disseminated. n/a 3 4 5 5 3 4 n/a 0 0 1 1 1 10 Existing approaches or new approaches with essential elements identified and packaged into guides. Cumulative targets Focus in PY6 and PY7 is on developing consensus within the FSN Network on essential elements within commonly used approaches. Final Report: August 2010 to January 2018 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators 85 Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 IPR2.4: Knowledge and tools generated and disseminated through the TOPS small grants. 2.4.1 Number of tools created, field￾tested or validated using small grants. n/a 1 2 3 3 10 10 n/a 0 2 2 18 1 14 Information from TOPS Small Grants report. Validated tools are those that have undergone field-testing in various contexts and produce expected results. Annual targets Small grants can be used to create a new tool, field-test a tool and/or validate a tool. Program Result 3: KNOWLEDGE APPLICATION Effective and appropriate traditional and non-traditional skill delivery approaches and systems/applications used. 3a. Percent of Title II implementers aware of at least 50% of key FSN Network recommended tools. n/a 50% 60% 70% 75% 75% 75% n/a 63% n/a 69% 60% n/a 80% The indicator tracks awareness of FSN Network recommended tools by technical area. May be collected through survey or interviews. Annual targets Survey may include field￾and HQ staff of FFP-funded programming. This indicator measures awareness. 3b. Percent of projects [aware of tools] that report use of FSN Network recommended tools. n/a 40% 50% 60% 70% 70% 70% n/a 79% n/a 53% 43% n/a 51% Includes FFP funded projects operational at least for two years and reporting use of tools recommended by FSN Network. Annual targets This indicator measures reported use of recommended tools. 3c. Average score (normalized) on post￾training satisfaction survey. 70% 80% 80% 80% 80% 80% 80% 87% 90% 88% 87% 87% 89% 89% Participant survey scores on appropriateness, satisfaction and user￾friendliness of the training collected through a perception-based survey at the end of the training. Annual targets TOPS staff designed a standardized satisfaction survey for use at the end of in￾person and e￾learning events. The Technical and Operational Performance Support (TOPS) Program 86 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 IPR3.1: Capacity of food security practitioners strengthened through focused skill transfer. 3.1.1 Number of regional or global multi￾sectoral training events held. 1 2 3 4 5 5 6 1 2 2 3 4 5 5 Multi-sectoral training events organized by TOPS regionally or globally. Source: TOPS annual reports. Cumulative target Targets were set based on PY1 experience of organizing technical training jointly with regional knowledge sharing events. 3.1.2 Number of staff trained through country level, regional or global training events. 20 232 292 312 192 535 320 26 316 311 350 935 539 48 674 A count of program staff who participated in (completed) trainings organized by TOPS. Source: workshop reports and participant database. Annual targets Disaggregated by: Organization, type of event, topic. 3.1.3 Percent of TOPS training events that integrated gender considerations / principles in sessions. 80 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % 100 % Number of trainings that incorporated topics on gender integration or considerations. Source: TOPS workshop reports. Annual targets Gender considerations are cross-cutting for all technical areas. Program Result 4: KNOWLEDGE SHARING Information, skill and knowledge exchange supported and expanded. 4a. Number of members regularly participating in FSN Network task forces. 30 40 40 40 40 40 40 70 122 65 50 41 39 62 To be considered a regular participant a member must have participated at least twice in a year. Source: meeting notes/attendance lists. Annual targets Data is disaggregated by technical area and by field and HQ for internal annual review. 48 Note incorrect value of 400 submitted in PY6 Annual Progress Report. Final Report: August 2010 to January 2018 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators 87 Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 4b Number of participants attending FSN Network knowledge sharing meetings. 250 250 250 250 250 500 500 282 251 780 776 825 1017 1538 Participants in knowledge sharing meetings include those who attend the regional events as well as the single topic events organized by technical leads to meet a need. Source: participant attendance lists. Annual targets Disaggregated by: Organization, type of event, topic. 4c Average score (normalized) on post FSN knowledge sharing meeting satisfaction survey. 80% 80% 80% 80% 80% 80% 80% 74% 80% 87% 80% 79% 82% 82% Perception-based survey at the end of day/ session of the entire knowledge sharing event. Source: completed satisfaction surveys. Annual targets TOPS staff designed standardized satisfaction survey to be administered at the end of in￾person and e￾learning events. 4d: Average number of members participating in FSN Network listservs and discussion groups. 500 850 1000 1200 1200 2400 2400 750 1175 1550 2005 2335 3527 4347 Source: FSN Network listserv. Annual targets Membership of the listserv will be disaggregated by sex and location (i.e. field vs. HQ). 4e Number of downloads from the FSN Network website. n/a 1k 1.5k 1.5k 1.5k 8.0k 8.0k n/a 2.1k 7.4k 8.5k 9.4k 23.2k 43.8K Source: web-based analytic systems. Annual targets Disaggregated by month, tool, topic, and other for annual internal review. The Technical and Operational Performance Support (TOPS) Program 88 Annex III: TOPS Progress on Performance Monitoring Plan (PMP) Indicators Indicator Targets Achievements Method/Source and Frequency of Data Collection Comments PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY1 PY2 PY3 PY4 PY5 PY6 PY7 IPR4.1: Physical and virtual community exchange forums and knowledge sharing mechanisms supported and expanded. 4.1.1 Number of FSN Network knowledge sharing meetings held. 2 5 5 5 5 12 8 4 4 15 17 20 30 46 Regional and DC based multi-sectoral knowledge sharing meetings as well as topic specific meetings. Source: meeting notes. Annual targets These include regional or DC￾based meetings and additional 1- day or ½-day meetings that respond to technical topics of high interest. 4.1.2 Number of FSN Network newsletters generated that share FSN Network information, products and services. 4 20 20 20 20 45 30 7 26 41 61 48 55 50 Bi-weekly and special newsletters produced by TOPS and distributed through the listserv. Source: Web analytics for simple count of newsletters. Annual targets Newsletters have a regular publication schedule and special editions are sent as needed. 4.1.3 Number of hits on FSN Network website. n/a 10k 25k 40k 50k 50k 45k n/a 20.7 k 32.9 k 39.7 k 39.9 k 54.0 k 78.0 k Source: web analytics for a count of views on FSN Network website. Annual targets Disaggregated by region and month. PR4.2: Local and regional knowledge sharing and networking strengthened. 4.2.1 Number of regional interest groups actively sharing knowledge on key imple￾mentation issues. n/a 5 6 8 10 5 5 n/a 6 5 5 3 3 3 Interest groups share information around a topic of interest through direct contact or virtual contact supported by TOPS. Annual targets Multiple communication channels are in use. Final Report: August 2010 to January 2018 Annex IV: List of TOPS Products 89 Annex IV: List of TOPS Products PRODUCED BY THE TOPS PROGRAM Title Technical Cluster(s) Languages Link to FSN Network First Edition Published Latest Edition Published A look into SPRING's Community Nutrition Activities in Ntungamo, Uganda (video and talking points) Nut English Yes PY6 PY6 A Training Workshop Guide: Planning for Qualitative Research for Nutrition Nut English Yes PY7 PY7 Activities for Stimulating Knowledge Sharing and Learning: A TOPS Quick Guide to Linking Development Practitioners KM English Yes PY7 PY7 Advanced Workshop on Designing for Behavior Change: Focus on Activities SBC English Yes PY7 PY7 Branching Out: Technical Guidance for Agroforestry ANRM English Yes PY6 PY6 Care Groups: A Reference Guide for Practitioners SBC English, French, Spanish Yes PY6 PY6 Care Groups: A Training Manual for Program Design and Implementation SBC English, French, Spanish Yes PY5 PY5 Climate-Smart Agriculture in USAID Office of Food for Peace Funded Programs ANRM English Yes PY6 PY6 Core Competency Series ANRM, SBC, CM, M&E, Gender, Nutrition, KM English, French, Spanish Yes PY7 PY5 Decision Guide for Program Managers: What You Need to Know About the Designing for Behavior Change (DBC) Approach SBC English Yes PY6 PY6 The Technical and Operational Performance Support (TOPS) Program 90 Annex IV: List of TOPS Products Designing for Behavior Change: A Practical Field Guide SBC English Yes PY7 PY7 Designing for Behavior Change: For Agriculture, Natural Resource Management, and Gender SBC English Yes PY7 PY7 Designing for Behavior Change: For Agriculture, Natural Resource Management, Health and Nutrition SBC English, Bangla, French, Spanish Yes PY3 PY3 Designing Participatory Meetings and Brownbags: A TOPS Quick Guide to Linking Development Practitioners KM English, French, Spanish Yes PY3 PY6 Designing Participatory Sessions for Large Scale Knowledge Sharing Meetings (narrated presentation and video) KM English Yes PY5 PY5 Farming for a Profit: Technical Guidance for Smallholder Financial Planning ANRM English Yes PY7 PY7 Gender and Commodity Management: A Field Guide Gender, CM English, French Yes PY7 PY7 Guide to Developing a Knowledge Management Strategy for a Food Security and Nutrition Program KM English Yes PY6 PY6 Guide to Developing a Scope of Work for Mid￾Term Evaluation of Title II Development Food Assistance Programs M&E English Yes PY2 PY2 Karamoja Seasonal Analysis and Intervention Guide ANRM English Yes PY4 PY4 Key Resources for Agriculture and Natural Resource Management Practitioners ANRM English Yes PY7 PY7 Knowledge Management and Monitoring & Evaluation Collaboration Package KM, M&E English Yes PY7 PY7 Learning that Lasts: Technical Guidance for Farmer Field Schools ANRM English Yes PY4 PY4 Make Me a Change Agent: A Multisectoral SBC Resource for Community Workers and Field Staff SBC English, French, Spanish Yes PY6 PY6 Final Report: August 2010 to January 2018 Annex IV: List of TOPS Products 91 Monitoring and Evaluation Facilitator’s Guide M&E English Yes PY5 PY5 Participatory Facilitation Techniques Workshop Curriculum KM English Yes PY6 PY6 Quality Improvement and Verification Checklist (QIVC) SBC, M&E English, French Yes PY2 PY2 REALIZE: Social and Behavioral Change for Gender Equity and Diversity SBC English, French, Spanish Yes PY7 PY7 Reaping the Benefits: Technical Guidance on Reducing Post-Harvest Losses in Smallholder Farming Systems ANRM English Yes PY6 PY6 Resilience Design in Smallholder Farming Systems - A Practical Approach to Strengthening Farmer Resilience to Shocks and Stresses ANRM English, French, Spanish Yes PY7 PY7 Resource Repositories related to Agriculture and Natural Resource Management ANRM English Yes PY6 PY6 Small Grants Big Impact II: An Overview of the Small Grants Program, 2015-2017 All English Yes PY7 PY7 Small Grants, Big Impact: A Retrospective of the TOPS Small Grants Program 2010-2015 All English Yes PY5 PY5 Strengthening Capacity in Zimbabwe with The TOPS Program: Resilience Design in Smallholder Farming Systems (video) ANRM English Yes PY6 PY6 Supporting Communities of Practice: A TOPS Quick Guide to Linking Development Practitioners KM English, French, Spanish Yes PY3 PY3 Theory of Change Facilitator’s Guide (and Complementary Training Materials) M&E English Yes PY6 PY7 Tool for Framing a Discussion Between Nutrition and Agriculture Specialists Nut English, French Yes PY7 PY7 TOPS ANRM Case Study Series (8 Case Studies) ANRM English Yes PY7 PY7 TOPS Commodity Management Handbook CM English, French, Spanish Yes PY3 PY7 The Technical and Operational Performance Support (TOPS) Program 92 Annex IV: List of TOPS Products TOPS Commodity Management Toolkit CM English Yes PY7 PY7 TOPS Permagarden Toolkit 2nd Edition (updated materials from 2015) ANRM English, French, Spanish, Amharic Yes PY5 PY7 TOPS Review of Promising Practices in Food for Peace Development Food Assistance Projects ANRM, Gender, Nutrition English Yes PY4 PY4 TOPS: Creating Pathways from Learning to Practice in the Food Security and Nutrition Community All English Yes PY7 PY7 PRODUCED THROUGH A SMALL GRANT Title Technical Cluster(s) Languages Organization(s) Link to FSN Network Year Published 2014 Storybook: INGO Experience with Gender Integration Gender English ACDI/VOCA, Catholic Relief Services, Lutheran World Relief, Mercy Corps, Project Concern International, World Vision Yes PY4 A Cost of the Diet Analysis in Azraq Refugee Camp, Jordan Nutrition, SBC English International Medical Corps Yes PY6 A Gender Perspective on Agricultural Development and Food Security in Nicaragua Gender English Lutheran World Relief Yes PY4 A Look Back on the TOPS-funded Livestock￾Household Nutrition Learning Series ANRM English Land O’Lakes Yes PY7 A Resource Guide for Enhancing Potential for Sustainable Impact General English PCI Yes PY4 Barrier Analysis of Exclusive Breastfeeding, Minimum Dietary Diversity and Early Antenatal Nutrition, SBC English International Medical Corps Yes PY6 Final Report: August 2010 to January 2018 Annex IV: List of TOPS Products 93 Care Seeking Behaviors of Syrian Refugees in Lebanon Barrier Analysis of Infant & Young Child Feeding and Maternal Nutrition Behaviors Among Adolescent Syrian Refugees in Urban Turkey Nutrition, SBC English International Medical Corps Yes PY6 Barrier Analysis on Maize Harvest and Post￾Harvest Practices ANRM English Mercy Corps Yes PY5 Behavior Change Manual for WASH Projects WASH, SBC English, French HELVETAS Swiss Intercooperation Yes PY7 Camp-based Barrier Analysis of Early Initiation of Breastfeeding, Iron-rich Food Consumption, and Early Antenatal Care Seeking Behaviors of Syrian Refugees in Azraq Camp, Jordan Nutrition, SBC English International Medical Corps Yes PY6 Care Groups – An Effective Community-based Delivery Strategy for Improving Reproductive, Maternal, Neonatal and Child Health in High￾Mortality, Resource-Constrained Settings: A Guide for Policy Makers and Donors SBC English CORE Group Yes PY4 Care Groups – An Effective Community-based Delivery Strategy for Improving Reproductive, Maternal, Neonatal and Child Health in High￾Mortality, Resource-Constrained Settings: Summary Report SBC English CORE Group Yes PY4 Care Groups in Emergencies: Evidence on the Use of Care Groups and Peer Support Groups in Emergency Settings SBC English International Medical Corps Yes PY5 Case Study on Adolescent Inclusion in the Care Group Approach - the Nigeria Experience SBC English International Medical Corps Yes PY5 Cash or In-Kind? Why Not Both? Response Analysis Lessons from Multimodal Programming Humanitarian English CRS, CaLP, Samaritan’s Purse Yes PY7 Community-based Management of Acute Malnutrition (CMAM) Toolkit Revision Nutrition English Save the Children No, but available via this link. PY6 The Technical and Operational Performance Support (TOPS) Program 94 Annex IV: List of TOPS Products Comparative Analysis of Shock Response Programs & Their Impacts on Resilience in Zimbabwe M&E English World Vision Yes PY7 Consortium Management and Leadership Training Facilitator's Guide General English CORE Group Yes PY4 Designing a Common Gender Indicator Framework for CARE's Pathways Program Gender English CARE Yes PY5 Developing a Resilience Threshold: Lessons Learned from Niger ANRM English Mercy Corps Yes PY5 Dietary Diversity and Determinants Survey Guide and User's Manual Nutrition English CRS Yes PY2 Elan Mobile Money Workshop Report CM English Mercy Corps Yes PY6 Emergency Toolkit for Food Security and Nutrition Protection: Cholera Disease Preparedness Community Group Module Nutrition, WASH, SBC English CORE Group Yes PY7 Energy-Water-Food Nexus Assessment and Benchmarking Tool ANRM English Mercy Corps Yes PY5 Enhancing Nutrition and Food Security during the First 1,000 Days through Gender-sensitive Social and Behavior Change (Technical Brief) Gender, SBC, Nutrition English TOPS, CORE Group, WI-HER Yes PY5 Enhancing Nutrition and Food Security during the First 1,000 Days through Gender-sensitive Social and Behavior Change (Technical Resource Guide) Gender, SBC, Nutrition English TOPS, CORE Group, WI-HER Yes PY5 Essential Nutrition Actions and Essential Hygiene Actions Framework Nutrition, WASH English USAID, JSI, Helen Keller International, CORE Group Yes PY5 Essential WASH Actions: A Training and Reference Pack to Supplement Essential Nutrition Actions WASH English FHI 360 Yes PY7 Evaluation of the Building on Community Strengths SBC English GOAL Yes PY5 Final Report: August 2010 to January 2018 Annex IV: List of TOPS Products 95 Extension of Small-Scale Postharvest Horticulture Technologies - A Model Training and Services Center ANRM English World Food Logistics Organization Yes PY5 Facilitation Guide for an Integrated Evaluation Methodology: Most Significant Change and PhotoVoice Gender English Lutheran World Relief, Cultural Practice, LLC. Yes PY7 Father Involvement in Promoting Reproductive, Maternal, Newborn, and Child Health (RMNCH) SBC English PCI Yes PY7 Field Numeracy - Uno How's Farm Visits™ ANRM English AgriTechTalk CIC Yes PY4 Flood and Droughts through a Gendered Lens; Mitigating Strategies for Women, Girls, Boys and Men in Malawi Gender English GOAL Yes PY5 Focus on Families and Culture: A guide for conducting a participatory assessment on maternal and child nutrition Nutrition English, French Grandmother Project - Change through Culture Yes PY5 Food Security Vulnerability and Capacity Assessment Guidance ANRM English Mercy Corps Yes PY4 Forest Garden Training Center ANRM English Trees for the Future Yes PY7 Getting the Knack of NACS: Highlights from the State of the Art (SOTA) Meeting on Nutrition Assessment, Counseling and Support (NACS) Nutrition English CORE Group Yes PY2 Growing A Future: Liberian Youth Reflect on Agriculture Livelihoods ANRM English Mercy Corps Yes PY7 Improving the Management of Agriculture Demonstration Sites in Food Security Programs ANRM English World Vision Yes PY7 Infant and Young Child Feeding in Emergencies (IYCF-E) Toolkit: Rapid Start-up Resources for Emergency Nutrition Personnel Nutrition English Save the Children Yes PY4 Information System for Management, Analysis & Reporting Timeliness (I-SMART) 2.0 M&E English, French ACDI/VOCA Yes PY4 Instruction Video of the Partnership Defined Quality Methodology Learning Tool M&E, SBC English Medical Teams International Yes PY7 The Technical and Operational Performance Support (TOPS) Program 96 Annex IV: List of TOPS Products Integrating Gender throughout A Project's Life Cycle 2.0 Gender English Land O'Lakes Yes PY5 IYCF-E Toolkit: Rapid Start-up Resources for Emergency Nutrition Personnel Nutrition English Save the Children, TOPS, USAID Yes PY5 LGI Photobook Gender English Lutheran World Relief, Cultural Practice, LLC. Yes PY7 Mobile Money Assessment and Contracting Guide CM English Mercy Corps Yes PY6 Myanmar Dry Zone Vulnerability and Capacity Assessment ANRM English Mercy Corps Yes PY4 Nurturing Connections - Adapted for Homestead Food Production and Nutrition Nutrition English Helen Keller International Yes PY5 Open Data Policy Compliance Guide: A Practitioner’s Guide to Interpreting and Complying with USAID’s Open Data Policy All English, French, Spanish PCI Yes PY7 Potential for Nutrition-Sensitive Conservation Agriculture in Zambia Nutrition, Agriculture English Concern Worldwide Yes PY5 Project Evaluation Summaries for LGI India, LGI Nicaragua, LGI Uganda Gender English Lutheran World Relief, Cultural Practice, LLC. Yes PY7 Reading and Responding to Your Baby Flip Chart Nutrition, SCB English, Spanish Food for the Hungry, Carolina Global Breastfeeding Institute Yes PY5 Reading and Responding to Your Baby Lesson Plan Nutrition, SBC English, Spanish Food for the Hungry, Carolina Global Breastfeeding Institute Yes PY5 Resilience Design in Smallholder Farming Systems: Measurement Toolkit ANRM English, French Mercy Corps Yes PY7 Final Report: August 2010 to January 2018 Annex IV: List of TOPS Products 97 Resource Guide for Building Effective Collaboration and Integrated Programming General English PCI Yes PY5 Stories Without An Ending: An Adult Education Tool for Dialogue and Social Change SBC English The Grandmother Project - Change Through Culture Yes PY7 Supporting Farming as a Family Business: A Gender-Equitable Approach to Improve Food Security for Smallholder Maize Farmers in Uganda Gender English Lutheran World Relief Yes PY4 Theory of Change Based Project Monitoring, Measurement, Learning and Adaptation: Guidance and Methodology M&E English ACDI/VOCA Yes PY7 Warehouse Staff Safety Guide CM English, French, Spanish PCI, TOPS Yes PY4 What to Know: Working with Humanitarians on E-Cash Programs CM English Mercy Corps Yes PY6 The Technical and Operational Performance Support (TOPS) Program 98 Annex V: TOPS Leader with Associate Awards 2010-2018 Annex V: TOPS Leader with Associate Awards 2010-2018 Name of Implementing Partner (s) Award Name Award Summary Dates Total Amount CORE Group Sahel Resilience Workshop: Associate Cooperative Agreement No. AID 625-LA￾14-00001 USAID’s Sahel Regional Office in Dakar hosted a five-day workshop from March 17-21, 2014 in Niamey, Niger. The first three days were for USAID and implementing partner staff, where they focused on strategy and results framework, debated questions on implementation approaches, and devised visual representations of what a resilient community looks like. The final two days introduced key external stakeholders to USAID’s new investments for reinforcing resilience, other donor strategies, and the host country government plans to integrate resilience. 2/6/2014 – 6/30/2014 $27,944 TANGO International Humanitarian to Development Resilience Collaboration Training (HADA): Associate Cooperative Agreement No. AID-625-LA-14-00002 The purpose of the HADA Training was to ensure that USAID’s field staff is prepared to assess, plan, and implement effectively to achieve humanitarian and development integration. A workshop was held February 2-6, 2015 in Dakar, Senegal to promote understanding of the processes involved in joint assessment, program planning, design and implementation of crisis modifiers, and collaborative learning. 9/29/2014 - 3/31/2015 $73,986 TANGO International Rapid Exploratory Study to Identify Shocks and Stressors and to Contextualize Resilience Indicators for Nepal: Associate Cooperative Agreement No. AID-FFP-LA￾16-000001 This award undertook targeted research to identify shocks and stressors and contextualize resilience indicators for Nepal. The objectives were to: 1) identify shocks and stressors that have the greatest impact on the food security and well-being of most vulnerable individuals, households and communities and/or occur with the greatest frequency; 2) identify the capacities critical to absorb and adapt to the impact of shocks and stressors and fill critical gaps in the existing knowledge on the absorptive, adaptive, and transformative capacities of resilience; 3) determine the minimum set of indicators needed to develop absorptive, adaptive and transformative capacity indices that will be integrated into the baseline and final evaluation studies of PAHAL and SABAL, and; 4) develop questions that will allow for these indicators and indices to 4/15/2015 – 7/15/2015 $37,046 Final Report: August 2010 to January 2018 Annex V: TOPS Leader with Associate Awards 2010-2018 99 be collected and calculated. The activity was disrupted by the major earthquake that hit Nepal on April 25, 2015. TANGO International, Save the Children International Baseline Study for Enhancing Resilience and Economic Growth in Somalia Program: Associate Cooperative Agreement No. AID-FFP-LA￾15-000001 USAID, through the Office of U.S. Foreign Disaster Assistance (OFDA), Office of Food for Peace (FFP), and the East Africa Regional Mission (EA), funded a three-year program for Enhancing Resilience and Economic Growth in Somalia, which aimed to 1) increase the capacity of households and communities to adapt to recurrent shocks; 2) build on the ecological, social, and economic capital of households and communities; and 3) increase learning of communities, the implementers, USAID, and other stakeholders. The purpose of this award is to conduct a mixed-methods baseline study for the program performance evaluation. The baseline aimed to identify and measure key indicators of household and community resilience for comparison at end-line, to explore and test relationships across indicators to develop context-specific indices and to contribute to better understanding and future programming strategies on resilience. 4/15/15 – 3/31/17 $778,938 TANGO International Advanced Resilience M&E Training for Field Practitioners (ARMET): Associate Cooperative Agreement No. AID-625-LA￾15-00002 The objective of this training, built upon the HADA Training held in Senegal in February 2015, is to ensure that stakeholders in the Sahel￾Enhanced (RISE), USAID/Senegal, Food for Peace, U.S. Office of Foreign Disaster Assistance, as well as staff based in the field offices of USAID/Niger and USAID/Burkina Faso, RISE implementing partners, and relevant staff from the governments of Niger and Burkina Faso (3N and Burkina Permanent Secretariat of the Coordination of Sectorial Agricultural Policies M&E staffs) have a common understanding of key resilience M&E aspects, and the necessary skills to better carry out resilience monitoring and evaluation activities at field level in the RISE zone of intervention. The activity took place in Ouagadougou in June 2016. 9/30/15 – 12/31/16 $74,990 TANGO International Asian Resilience Measurement & Monitoring Experiential Learning Event (ARMMELE): Cooperative Agreement No. AID-486-A￾16-00002 The purpose of this activity was to advance resilience measurement practices in rural and urban environments by providing a hands-on learning opportunity for select USAID staff and implementing partners to assess resilience capacities, develop resilience indices and indicators, and apply recurrent monitoring techniques. Two workshops were held to achieve these objectives, in Siem Reap, Cambodia from April 18-22, 2016 and in Manila and Batangas, Philippines from June 6-10, 2016. 3/28/16 – 8/15/16 $80,097 The Technical and Operational Performance Support (TOPS) Program 100 Annex V: TOPS Leader with Associate Awards 2010-2018 TANGO International Review of Cash-based Emergency Food Assistance Program: Associate Cooperative Agreement No. AID-FFP-LA 16-00002 This award aims to gather evidence to inform FFP’s determination of modalities and to identify best practices and lessons learned using an array of methodologies including primary data collection to identify major trends and secondary data review to evaluate cost efficiency and rationale for selecting modalities. Given that EFSP funding and several of its modalities were new tools to FFP and its partners, it is critical to understand the evolution and how these were implemented in the field and what, if any, were the key takeaways. 7/1/16- 2/28/18 $859,997 TANGO International, Save the Children, Mercy Corps, CORE Group, Food for the Hungry Resilience Monitoring, Evaluation, Assessment, Strategic Analysis, and Capacity Building (REAL): Associate Award Cooperative Agreement No. AID-OAA-A￾16-00033 Working with USAID and implementing partner staff, the award targets the following objectives: 1. Capacity building to close critical gaps in resilience design, program monitoring and evaluation; 2. Reviewing local context and providing targeted technical assistance in response through an assessment to inform future resilience programming; 3. Strengthening food security networks for enhanced collaboration, coordination and information sharing through learning events, presentations, technical meetings, and “emergent measurement thought leadership”; 4. Field testing new and adapted methods, tools, and approaches via Feed the Future impact evaluations. The REAL implementing team, working closely with USAID’s Center for Resilience (C4R), reported significant progress in its initial year of activities. Overall, 45 percent (34 of 75) of activities initially planned were completed or nearing completion at end September 2017. Of the remaining activities, most were carried forward to Year 2, and some were eliminated. Particular success was achieved in:  Project Result 1 (Knowledge Capture): Deep dive analysis of data from Ethiopia, Kenya and the Sahel; analysis of FFP Baseline resilience modules for Nepal and Bangladesh; resilience analysis support for USAID/Malawi; REAL Associate Award logo created.  Project Result 2 (Knowledge Generation): Guidance Notes on Risk and Resilience Assessment, Measurement of Shocks and Stresses, Measuring Resilience Capacities, Recurrent Monitoring Surveys, collection of qualitative data for RISE Midterm Evaluation. 9/2/16 – 9/1/21 $7,999,315 Final Report: August 2010 to January 2018 Annex V: TOPS Leader with Associate Awards 2010-2018 101  Project Result 3 (Knowledge Application): Horn of Africa Resilience M&E for Programming Training. Project Result 4 (Knowledge Sharing): Learning Agenda Workshop; Expert Consultations and Technical Meetings on Resilience and Fragility, Value for Money, Positive Deviance; Asia Resilience Monitoring, Evaluation and Learning Workshop; development and launch of REAL Award web page. PY1 progress in other activities, including development of the REAL Knowledge Management Strategy and Learning Agenda, serves as a foundation for continued knowledge generation and sharing throughout the life of the REAL Associate Award. Innovations for Poverty Action Evaluative Research on Graduation Pilot Development Food Security Activity in Kamwenge: Uganda Associate Award Cooperative Agreement No. 72DFFP18LA00001 FFP awarded a pilot graduation project in Kamwenge District in Uganda in 2017, which aims to improve food and nutrition security and self-reliance among extremely poor households in refugee settlements and host communities. The implementing partner will deliver three variations of the graduation program, with the aim to learn whether a stripped down version(s) perform better per dollar spent in improving beneficiaries’ welfare. The goal of this associate award is to finalize the three treatment programs, conduct a randomized evaluation of the programs on beneficiaries’ welfare, and compare cost-effectiveness of impact per dollar spent for each version of the program. 12/22/17 – 12/31/21 $2,423,109 TANGO International, Kimetrica Livelihoods for Resilience Learning Activity: Baseline Survey, Recurrent Monitoring System and End Line Performance Evaluation: Associate Award Cooperative Agreement No. 72066318LA00001 This award aims to assess the progress and effectiveness of Feed the Future Ethiopia’s Livelihoods for Resilience (L4R) activity, which focuses on four priority areas: 1) on-farm income generating activities (IGAs) and crop and livestock market systems; 2) off-farm IGAs and non-farm enterprise development; 3) non-farm labor and wage employment; and 4) collaborative learning for scaling and sustaining gains made in on-farm, off-farm, and employment livelihoods pathways. This learning activity has two basic elements of baseline/endline data collection, comparison of baseline and endline results, and an overall evaluation of performance, effectiveness of the L4R activity; and design and implementation of a Recurrent Monitoring System (RMS) for the L4R activity. 1/9/18 – 1/8/22 $3,070,770 Save the Children, CORE Group Technical and Operational Performance Support (TOPS) This award aims to continue the specific and high-value TOPS interventions and practices for one year until the establishment of a 1/30/18 – 1/29/19 $2,499,793 The Technical and Operational Performance Support (TOPS) Program 102 Annex V: TOPS Leader with Associate Awards 2010-2018 Bridge: Associate Award Cooperative Agreement No. 72DFFP18LA00004 new USAID/FFP learning mechanism. The TOPS Bridge award will focus on maintaining the Food Security and Nutrition (FSN) Network and platforms, convening peer-to-peer knowledge sharing events, convening single-topic learning events, facilitating stakeholder consultations, and facilitating capacity strengthening workshops for implementing partners of food security programs, and creating field￾friendly tools. Mercy Corps Strengthening Capacity in Agriculture, Livelihoods, and Environment (SCALE): Associate Award Cooperative Agreement No. 72DFFP18LA00003 The Strengthening Capacity in Agriculture, Livelihoods, and Environment (SCALE) Associate Award aims to further improve the impact, sustainability, and scalability of FFP’s programming in agriculture, NRM, and livelihoods, with a specific focus on sustainable and climate sensitive approaches that foster more resilient agricultural systems and enhance income opportunities within agricultural systems and off-farm livelihoods. 1/30/18 – 1/29/23 $4,999,493 Save the Children Water, Sanitation, and Hygiene (WASH) Associate Award Cooperative Agreement No. 72DFFP18LA00005 The Water, Sanitation, and Hygiene (WASH) Associate award aims to strengthen the impact and sustainability of WASH activities within the context of FFP’s emergency and development food security programs. The WASH Associate award will build a platform of WASH capacity strengthening activities and engage in relevant applied research activities that target key unknowns in the WASH-Nutrition paradigm and field-viable solutions to WASH service delivery for chronically vulnerable and food insecure populations. 1/30/18 – 1/29/23 $3,749,973 Final Report: August 2010 to January 2018 Annex VI: TOPS Leader with Associate Awards – Final Reports 103 Annex VI: TOPS Leader with Associate Awards – Final Reports Annex VI a) Sahel Resilience Workshop: Associate Cooperative Agreement No. AID 625-LA-14-00001 usaid_sahel_region al_resilience_workshop_final_report.docx Annex VI b) Humanitarian to Development Resilience Collaboration Training (HADA): Associate Cooperative Agreement No. AID-625-LA-14-00002 Post Humanitarian and Development Resilience Collaboration Training Workshop Report_RE.docx Annex VI c) Advanced Resilience M&E Training for Field Practitioners (ARMET): Associate Cooperative Agreement No. AID-625-LA-15-000002 The Technical and Operational Performance Support (TOPS) Program 104 Annex VI: TOPS Leader with Associate Awards – Final Reports Annex VI d) Asian Resilience Measurement & Monitoring Experiential Learning Event (ARMMELE): Cooperative Agreement No. AID-486-A-16-00002 Annex VI e) Baseline Study for Enhancing Resilience and Economic Growth in Somalia Program: Associate Cooperative Agreement No. AID-FFP-LA-15-000001 AE_USAID Somalia Resilience Baseline Report_FINAL23Dec.docx Annex VI f) Review of Cash-based Emergency Food Assistance Program: Associate Cooperative Agreement No. AID-FFP-LA 16-00002