MIDTERM PERFORMANCE EVALUATION OF THE JOHN OGONOWSKI AND DOUG BEREUTER FARMER-TO-FARMER PROGRAM February 25, 2022 February 2022 This document was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. MIDTERM PERFORMANCE EVALUATION OF THE JOHN OGONOWSKI AND DOUG BEREUTER FARMER-TO-FARMER PROGRAM February 25, 2022 Prepared for the United States Agency for International Development under USAID’s Feed the Future Global Program Evaluation for Effectiveness and Learning (PEEL) Contract Number: AID-OAA-TO-16-00008 Submitted to: USAID/Bureau for Resilience and Food Security Lindsey Anna, lanna@usaid.gov and USAID/Bureau for Resilience and Food Security Peggy Carlson, mcarlson@usaid.gov Evania Robles, erobles@usaid.gov Submitted by: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: (301) 652-4334 Evaluation Team: Ziad Antonios, Team Leader Sara Duran, Agriculture Monitoring and Evaluation (M&E) Specialist Emily Miller, Development/Gender Specialist Elizabeth Whitcher, Research Analyst Photo Credit: Local volunteer explaining effective pruning techniques to the host farmer, Tajikistan. DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The Farmer-to-Farmer (F2F) program provides voluntary technical assistance to farmers, agribusinesses, and agricultural institutions to generate sustainable economic growth in the agricultural sector with a secondary goal of building awareness of United States (U.S.) development programs within the international community and the American public. Through eight Leader with Associates (LWA) Cooperative Agreements and one additional Cooperative Agreement, volunteers are deployed in 43 countries. Since its inception, F2F has completed 20,303 volunteer assignments with 13,312 host organizations, directly assisting more than 1.5 million agricultural professionals and impacting over 178 million indirect beneficiaries. The evaluation team (ET) conducted an online survey of 549 host institutions and volunteers and 119 key informant interviews with host institutions, volunteers, implementing partners (IPs), and United States Agency for International Development (USAID) staff in 24 countries to ascertain stakeholder perceptions of the program. Overall, stakeholders are highly satisfied with the program’s responsive, needs-based approach. Ninety-four (94) percent of hosts and 90 percent of volunteers hope to continue working with F2F in the future. Stakeholders spoke positively about F2F’s flexible, adaptive approach, which enabled F2F to address the changing needs of hosts during the Coronavirus Disease 2019 (COVID-19) pandemic. After a pause in implementation at the start of the pandemic, F2F worked quickly to transition to alternative models of remote and paired remote support that enabled the program to continue assistance. The pandemic highlighted the value of building local networks of in-country support and investing in innovations that support program resilience to shocks and stressors that disrupt international travel. TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... I 1.0 EVALUATION OVERVIEW.............................................................................................................................. 1 1.1 Evaluation Purpose and Audience ........................................................................................................... 1 1.2 Evaluation Questions.................................................................................................................................. 1 2.0 ACTIVITY BACKGROUND............................................................................................................................. 1 2.1 Activity Description.................................................................................................................................... 1 3.0 EVALUATION METHODS AND LIMITATIONS ....................................................................................... 2 3.1 Data Collection Methods.......................................................................................................................... 2 3.2 Data Analysis................................................................................................................................................ 5 3.3 Methodological Limitations....................................................................................................................... 5 4.0 FINDINGS & CONCLUSIONS........................................................................................................................ 6 4.1 EQ 1: How well do F2F standard indicators capture the effects of F2F activities and provide useful information for program management?..................................................................................... 6 4.2 EQ 2: How well does the F2F program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs?..................12 4.3 EQ 3: What progress has the FY 2019 to FY 2023 F2F program made toward achieving its objectives?...................................................................................................................................................19 5.0 RECOMMENDATIONS...................................................................................................................................41 6.0 ANNEXES ............................................................................................................................................................44 Annex 1: Peel Task Order Expression of Interest .....................................................................................45 Annex 2: Bibliography........................................................................................................................................63 Annex 3: List of Persons Interviewed............................................................................................................65 Annex 4: Interview Guides...............................................................................................................................69 Annex 5: Volunteer and Host Online Survey Questionnaires................................................................83 Annex 6: Signed Conflict of Interest Forms................................................................................................. 91 LIST OF TABLES Table 1: Data Sources for EQs....................................................................................................................................... 2 Table 2: Countries, IPs, and Sectors Included in Qualitative Data Collection ................................................... 4 Table 3: Distribution of KIIs Disaggregated by Type of Respondent.................................................................... 4 Table 4: F2F Program Standard Performance and Impact Indicators.................................................................... 6 Table 5: Outreach/Cultural Exchange Indicators.....................................................................................................11 Table 6: Capacity Development Indicators...............................................................................................................11 Table 7: Frequency of Post-Assignment Communication......................................................................................16 Table 8: Primary Purposes of Post-Assignment Host Communication with Volunteers...............................16 Table 9: Host Satisfaction in Its Involvement in the F2F Volunteer Assignment (N=186)............................20 Table 10: F2F Volunteering Models.............................................................................................................................29 Table 11: Volunteer Satisfaction with Paired Remote Volunteer Mechanism ..................................................31 Table 12: Demographics of Volunteers for FY 2019, FY 2020, and FY 2021. ..................................................35 LIST OF FIGURES Figure 1: F2F Program Theory of Change .................................................................................................................... i Figure 2: Role of Hosts in SOW Development (N=182) ......................................................................................21 Figure 3: Satisfaction of Volunteers with IP Competency (N=364)....................................................................21 Figure 4: Main Challenges Faced by Volunteers (N=358)......................................................................................22 Figure 5: Main Reasons for Not Completing All SOW Deliverables (N=102).................................................23 Figure 6: Type of Recommendations Received from Volunteer (N=184).........................................................24 Figure 7: Factors Limiting Implementation of Recommendations Received from Volunteer (N=180)......25 Figure 8: Main Challenges Faced by Hosts in Working with Volunteers (N=198)..........................................25 Figure 9: Host Satisfaction with Volunteers..............................................................................................................30 Figure 10: Volunteer Challenges Working with Paired Counterpart (N=171)................................................32 Figure 11: Primary Place of Employment During Volunteer Experience............................................................36 Figure 12: Reasons for Volunteering with F2F..........................................................................................................37 Figure 13: How First Heard About F2F......................................................................................................................37 LIST OF ACRONYMS ACDI/VOCA Agricultural Cooperative Development International/Volunteers in Overseas Cooperative Assistance AET Agricultural Education and Training AVOP Agricultural Volunteer Opportunity Program BGGC Browse & Grass Growers Cooperative CNFA Cultivating New Frontiers in Agriculture CoP Community of Practice COVID-19 Coronavirus Disease 2019 CRS Catholic Relief Services ECCA Europe, Caucasus, and Central Asia EQ Evaluation Question ET Evaluation Team F2F Farmer-to-Farmer FY Fiscal Year HQ Headquarters IESC International Executive Service Corps IP Implementing Partner KII Key Informant Interview LWA Leader With Associates M&E Monitoring and Evaluation ME&A ME&A, Inc. NCBA CLUSA National Cooperative Business Association Cooperative League of the USA NGO Non-Governmental Organization NRM Natural Resource Management ODI Organizational Development Index PEEL Program Evaluation for Effectiveness and Learning POA Partners of the Americas RFA Request for Application RFS (USAID Bureau for) Resilience and Food Security SOW Scope of Work (a.k.a. Assignment) SPSP Special Program Support Project ToC Theory of Change U.S. United States USAID United States Agency for International Development USG United States Government i EXECUTIVE SUMMARY INTRODUCTION This report provides an evaluation of the United States Agency for International Development (USAID) Farmer-to-Farmer (F2F) program. It was commissioned by the USAID Bureau for Resilience and Food Security (RFS). The evaluation was independently conducted from July 2021 through February 2022 by the Program Evaluation for Effectiveness and Learning team, led by ME&A, Inc (ME&A).1 The evaluation and its results will be used for program improvement and evolution and setting the future strategic direction for the F2F program or USAID at large. The primary audience for this evaluation is the USAID F2F management and staff and its implementing partners (IPs).2 ACTIVITY BACKGROUND The F2F program was first authorized by the United States (U.S.) Congress in 1985 and has been re￾authorized in every subsequent Farm Bill. The program seeks to provide for the transfer of knowledge and expertise of U.S. agricultural producers and businesses, on a voluntary basis, to lower- and middle￾income countries and emerging democracies. A main goal of the program includes the generation of sustainable and broad-based food security and economic growth in agricultural sectors. A secondary goal is to increase the American public’s understanding of international development issues and programs and to advance international understanding of the United States and U.S. development programs. F2F’s Theory of Change (ToC) is presented below in Figure 1. Figure 1: F2F Program Theory of Change Theory of Change Inputs: Volunteer technical assistance with some additional staff support. ↓ Outputs: Volunteer recommendations and training. ↓ Outcomes: Adoption of recommendations or host change in behavior, management practices, and technology (innovation). ↓ Impacts: Productivity and income increases. Improved social, human, and environmental capital. EVALUATION DESIGN AND LIMITATIONS According to the F2F midterm evaluation Scope of Work (SOW), the evaluation team (ET) was tasked with addressing the following questions: ● Evaluation Question (EQ) 1: How well do F2F standard indicators capture the effects of F2F activities and provide useful information for program management? ● EQ 2: How well does the F2F program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs? ● EQ 3: What progress has the Fiscal Year (FY) 2019 to FY 2023 F2F program made toward achieving its objectives to date? The ET applied a rigorous mixed methods approach, which included: 1) a literature review of F2F project documents and performance reports; 2) 110 key informant interviews (KIIs) across 24 countries in three 1 Program Evaluation for Effectiveness and Learning (PEEL) Contract/Task Order Number: AID-OAA-I-15-00024/AID-OAA-TO￾16-00008. 2 USAID F2F is in the RFS/Center for Agriculture-Led Growth/Production Systems Division. ii global regions; and 3) two online surveys completed by 549 host organizations and volunteers. The ET conducted all KIIs remotely via Microsoft Teams. Due to time, resources, and design constraints, not all F2F stakeholders are equally represented in the evaluation. The qualitative data was limited by time and resource constraints, which prevented more countries from being included in data collection. Time zone differences and poor internet connections posed challenges for the ET and interviewees. MAIN FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS A summary of the evaluation’s main findings, conclusions, and recommendations are presented below. A more detailed presentation can be found in the main report. iii Findings Conclusions Recommendations Training is needed to better define the indirect beneficiaries-related indicators and simplified Organization Development Index (ODI), data collection templates, and reporting. Current standard indicators are useful tools to inform program progress, but definitions can be further improved to guide IPs on data collection and reporting. USAID and the F2F Community of Practice (CoP) should provide further guidance on defining indicators and best practices to strengthen monitoring and evaluation (M&E) capacity and standardize data collection strategies among all IPs. The ODI is appropriate for organizations that are just forming or being established, but it is not adequate for describing the capacity of advanced organizations. The ODI should more accurately reflect the capacity of hosts at all development stages. USAID should explore how to make the ODI more adaptable to different types of hosts so it can be used across the entire range of organization levels. IPs have created custom indicators and recommend creating others on topics related to leadership, business empowerment, and resilience. IPs have developed custom indicators to respond to specific needs and project scopes, and these could become part of the standard indicators list. USAID, through the CoP, should assess the relevance of IPs’ custom indicators and consider their potential to become standard indicators to tell the program’s story. Poor host recordkeeping is a challenge when collecting host data, particularly from farmers on sales and income. Hosts provide data requested by IPs, but it is primarily anecdotal and does not use a standard template. This highlights the need to improve hosts’ recordkeeping capacity. IPs should provide hosts with more recordkeeping guidance and tools for standardizing data collection to strengthen hosts' decision-making processes. There are challenges collecting data along the value chain for women and youth, as hosts often present data in aggregate. It is preferable to provide guidance with indicator definitions about how to classify or distinguish women- and youth-led organizations. While IPs showed confidence in the capacity development indicators, there is a need for improvement to inform social inclusion disaggregates. USAID should further analyze IPs’ recommendations regarding indicator disaggregates for gender, youth, and women-led organizations to respond to F2F’s social￾inclusion focus and capture its impacts. Cultural exchange and natural resource management (NRM) are particularly difficult results to measure. Stakeholder engagement studies could be a tool for capturing cultural exchanges and specific/technical environmental assessments for NRM, but these require specialized human resources that are beyond IPs’ capacity. Cultural exchange and NRM indicators are not currently sufficient in describing the program’s performance and impact. USAID should collaborate with the CoP on developing meaningful indicators on NRM and cultural exchange impacts, including reviewing relevant Feed the Future NRM indicators on their suitability for F2F. The CoP is beneficial. USAID’s involvement in the CoP can drive increased participation by IPs, and since late 2020, it has organized and led CoP meetings. IPs recognize the need for participation and commitment to regular CoP meetings to accomplish objectives and recommend more participation of in-country teams, which can be accomplished using the remote meeting platforms used during the COVID-19 pandemic. The CoP is a valuable capacity-building resource for IPs and a mechanism to drive program learning and evolution. However, it needs more involvement from USAID, IPs, and in-country team members to accomplish tangible results. Without a systematic approach to analyzing data and information shared with the CoP, implementation ideas will remain untested. USAID should maintain strong involvement and participation in the CoP to support IP engagement. USAID should encourage IPs to increase inclusion of in￾country team members and develop a systematic approach for addressing data and ideas generated by the CoP along with a mechanism for follow-up and accountability. iv Findings Conclusions Recommendations There is a need for more data-driven approaches to learning. IP recommendations include outcome mapping, stakeholder engagement studies, case studies, qualitative data collection, and development of new outcome and impact indicators. IPs are piloting innovative data-driven approaches to learning with strong M&E programs; however, resources available for M&E are limited. Innovative data-driven approaches to learning exist or are being developed by IPs but are not widely implemented. There is a need for more resources allocated to program learning and M&E. USAID and IPs should consider investing additional resources to develop data-driven approaches to learning. The CoP should leverage the support of IPs that are already piloting these methods. USAID should work with IPs to strategically identify opportunities where trade-offs between investment in program implementation and additional investment in M&E are worthwhile. There are untapped opportunities to leverage learning between F2F stakeholders, including having IPs coordinate meetings and develop virtual platforms that leverage stakeholder-to-stakeholder learning opportunities. Examples include host-to-host exchanges and forums for communication between F2F stakeholders and external development partners. Opportunities exist to leverage knowledge-sharing between F2F stakeholders, expand the impact of the program to more organizations, facilitate learning from past assignments and recommendations, and foster sustainability of program outcomes in the regions where F2F operates. USAID and IPs should invest in meetings and virtual platforms to leverage learning and knowledge-sharing between stakeholders within F2F countries or regions. In cases where stakeholders are in remote areas or have limited access to virtual platforms, additional resources will be necessary to facilitate in-person meetings. There is generally good communication between IPs and volunteers. However, many volunteers have not received sufficient follow-up from IPs regarding host outcomes. Volunteers have a strong interest in the outcomes of their assignment and the success of the organizations they support. There is a need for improved communication from IPs to volunteers after assignments end. Post￾assignment communication enhances the volunteers’ sense of involvement, ensures they feel valued, and supports retention and expansion of the talent network for future assignments. USAID should encourage the CoP to synthesize best practices for communication from IPs to volunteers after the assignment ends. Improved communication should focus on providing information to volunteers on host outcomes. It is important to provide information on local context, challenges, and resources available to volunteers prior to assignments. Providing orientation on the local context and operating environment once volunteers arrive on-site is time consuming and limits the time to work on the assignment. Volunteers desire more information from hosts pre-assignment to better plan and target assistance. Improved communication between the hosts and volunteers before the assignment provides volunteers a strong orientation to the local context, challenges, and operating environment. Investing in communication up front supports volunteers in providing targeted support and maximizes the effectiveness of the time spent on assignment. USAID should encourage the CoP to develop best practices for communication between hosts and volunteers before the assignment begins, while IPs should facilitate more extensive communication between hosts and volunteers before the assignment begins to orient volunteers to the local context and the host's operating environment. Hosts feel that their priorities are valued and respected, and volunteers feel that their inputs were valued. Hosts desire more communication from volunteers after the assignment. Sixty-nine (69) percent of volunteers and 70 percent of hosts are touch with their counterparts after the assignment, but communication is infrequent. Volunteers and hosts are sometimes unclear about whether and how they should communicate after the assignment. In general, hosts and volunteers appreciate being connected and desire extended communication after the assignment. However, norms and expectations for communication between hosts and volunteers are not always clear, which can inhibit continued contact. Strong communication after the assignment ends can support future assignment development. IPs should facilitate conversations between hosts and volunteers to clarify norms and expectations around post-assignment communication, supporting continued relationships between volunteers and hosts after the assignment ends. In cases where language barriers or limited access to virtual platforms inhibit communication with hosts, additional resources may be needed to support post-assignment communication. v Findings Conclusions Recommendations Inadequate staffing and funding at the Agricultural Volunteering Opportunity Program (AVOP) are major constraints to implementing the knowledge management and sub-awards management components The numerous activities under each component require more staff and funds to better respond to program needs. Managing different sub-awardees, following up with them on their activities, and continuously building their capacity is an excessive workload for a single organization. The lack of financial resources and the inadequate number of staff potentially hinder innovation in both components. In the next F2F phase, USAID should seek either to split AVOP into two projects, preferably with two different partners, or have IPs with extensive experience in F2F manage and guide one to two sub￾awardees each. The final decision on sub-awardee selection does not reside with AVOP despite numerous improvements to the competition structure of the Requests for Applications (RFAs) that Partners of the Americas (POA) made during the program’s first two years. Concurrence and approval from USAID/Washington and the Missions are needed to proceed with the application. The decision-making process for approving sub￾awardee applications involves multiple levels of review, with final approval outside of POA’s remit, which limits AVOP’s responsiveness in supporting potential sub-awardees. The IP managing the next AVOP award should have clear-cut responsibilities in the decision-making process for choosing the sub-awardees based on their initial applications. Concurrence from USAID and the Missions should come at an early stage to better enhance the process of choosing sub-awardees. IPs, hosts, and volunteers are satisfied with F2F’s flexibility and adaptive market-driven approach, which enabled innovative responses to the COVID-19 pandemic. F2F was delayed by the outbreak of COVID-19, affecting deployment of volunteers and assignment initiation. It is important to improve program resilience to shocks and stressors that inhibit travel by building networks of local support, particularly in the context of the global pandemic. F2F’s flexible and adaptive approach enabled it to modify implementation and delivery to address hosts’ changing needs during the COVID-19 pandemic. The pandemic highlighted the need to invest in program implementation innovations that support program resilience and build local networks of support. Flexibility and innovation are key to the success of the program. Building on the success of the paired remote volunteer model, USAID should work with IPs to continue to identify innovative approaches to improve resilience in program implementation and build local networks of support. Hosts are highly satisfied with F2F’s implementation processes, from initiating the SOW through the implementation of recommendations and follow-up. Most IPs use custom needs assessments to inform SOW development. F2F practices for needs assessments and SOW development are generally strong, but not standardized with diverse approaches and methods across IPs and projects. USAID should encourage the CoP to develop guidance around best practices for needs assessments and SOW development to leverage the richness and diversity of the strong practices that exist and ensure the program’s continued responsiveness. USAID should ensure that sufficient resources are provided to IPs to allow for investment in rigorous needs assessments. The COVID-19 pandemic led to the development of alternative volunteering models, including paired remote and fully remote volunteer assignments. The response to paired remote and remote volunteering was strongly positive among IPs, hosts, and volunteers. Despite the demonstrated value and success of local experts in volunteer positions and the success of fully remote assignments, to date F2F has not counted assistance provided independently by a local expert or fully remote assistance toward assignment targets. The alternative volunteering models strengthened the program by providing innovative approaches to achieving F2F objectives that serve to support hosts and volunteers, allowing F2F to invest in improved networks of in-country support, strengthening program resilience, providing new opportunities for cultural exchange between paired volunteers, and increasing the diversity of U.S. volunteer demographics. F2F should retain fully remote and paired remote models of volunteering as an alternative delivery mode in addition to resuming in-person assignments with U.S. volunteers when conditions permit and facilitating collaboration with local experts through paired in￾person assignments with U.S. and local volunteers. USAID should work with the CoP to develop updated guidance ensuring assignments conducted under alternative models are counted toward volunteer targets, including assignments conducted independently by local experts and fully remote assignments. The success of alternative assignment models demonstrates that innovation should always be a part of programming and encouraged. vi Findings Conclusions Recommendations Volunteers, hosts, and IPs expressed that the use of remote platforms during the COVID-19 pandemic for volunteer orientation and information-sharing prior to the assignment led to improved coordination between hosts and volunteers. Respondents emphasized the value of improved remote coordination before the assignment begins in maximizing the efficiency and effectiveness of the limited time available to accomplish the SOW. Lessons learned from the remote and paired remote volunteering models demonstrate the potential to use remote and in-person models of communication between hosts and volunteers to maximize the efficiency and effectiveness of assignments. USAID should support IPs to adopt a hybrid assignment model that incorporates remote and in-person support. In this model, IPs would guide volunteers to prepare their assignment remotely with the host and complete initial portions of the SOW remotely, as appropriate. Volunteers would then travel to deliver assistance in person, using time in country to focus on aspects of the SOW that require in-person support. The volunteer would then finalize the SOW and provide follow-up remotely. For hosts with access to virtual platforms, this hybrid model would serve to maximize the effectiveness of time spent in person and address the common constraints of limited time. IPs experienced challenges recruiting volunteers for paired remote and fully remote assignments, particularly those who had previously provided in-person assistance and preferred the in-person mode and sometimes local experts. Paired assignments can also increase the recruitment burden by doubling the recruitment effort. The paired remote and fully remote models of volunteering are new to F2F and present new challenges to IPs, specifically around volunteer recruitment, support, and retention. USAID should support IPs to address challenges related to volunteer recruitment for paired remote and fully remote assignments. Support should include tasking the CoP with synthesizing best practices and resources for paired remote and fully remote assignments and supporting IPs to increase the volunteering culture in the countries where local volunteers are recruited. Volunteers sometimes struggle to adjust to paired remote and fully remote volunteering or prefer the in-person experience. Remote volunteers experienced challenges building relationships and establishing trust with hosts and gathering information. Remote volunteers can find it difficult to dedicate time to the assignment around other commitments, experience technology and network-related challenges, or feel disconnected from the work, unaware of the assignment outcomes. The paired remote and fully remote models of volunteering are new to F2F and the F2F volunteer base. Volunteers require additional support from IPs to adjust to paired remote and fully remote volunteering models. The program is still developing best practices for these new modes of volunteering. IPs should provide additional support to volunteers to help them succeed in fully remote and paired remote assignments. Examples include providing volunteers with data-collection and relationship-building support, addressing technical difficulties and network issues, supporting work planning and coordination between paired volunteers, facilitating improved communication between remote volunteers and hosts, and improving follow-up with volunteers about assignment outcomes. The use of paired remote and fully remote volunteer models appears to have contributed to increased participation by women and under-represented groups for whom international travel was a barrier to volunteering. This finding, however, could not be validated through data on volunteer assignments, because F2F only requires IPs to report on in-person assignments, either a U.S. in-person volunteer or a local in-person volunteer paired with a U.S. remote one. The paired remote and fully remote volunteering models have the potential to reduce barriers to volunteering among women and under-represented groups. USAID should support IPs to continue to use paired remote and fully remote volunteering to increase the diversity of under-represented groups in the U.S. volunteer demographic. vii Findings Conclusions Recommendations In FY 2019 and FY 2020, women comprised 32 percent of volunteers, while black, indigenous, or people of color comprised 31 and 35 percent of volunteers, respectively . In FY 2021, the respective percentages had fallen to 27 percent in the first case and risen to 85 percent in the second case. However, demographic data reported by IPs is incomplete in that it excludes U.S. volunteers who worked fully remotely or in paired remote assignments. USAID does not currently require IPs to report age-range data for volunteers. It is not possible to confirm trends in age, gender, or ethnic and racial demographics of volunteers with the data currently available. More complete demographic data on volunteers across all volunteering mechanisms is needed to reflect the program’s diversity and inclusion achievements. USAID should request IPs report demographic data for all modes of volunteering, including in-person, paired remote, and fully remote. USAID should also request that IPs collect data on volunteer age ranges. IPs face considerable difficulties securing buy-ins related to the lack of financial resources and the COVID-19 pandemic. Leader With Associates (LWAs) Awards are relevant to USAID and its partners since they help mitigate the risks associated with vetting new proposals from new potential partners. USAID should increase its support to IPs to help them secure additional buy-ins and Associate Awards by informing them about topics to focus on and promoting their work within the local context. 1 1.0 EVALUATION OVERVIEW 1.1 EVALUATION PURPOSE AND AUDIENCE This report describes the results of the midterm evaluation of the United States Agency for International Development (USAID) John Ogonowski and Doug Bereuter Farmer-to-Farmer (F2F) Program, conducted between July and December 2021. The F2F program was founded in 1985 and signed into law by the United States (U.S.) Congress. It has since been re-authorized in every U.S. Farm Bill since then, which speaks to its popularity and longevity.3 The fact that the program falls within the Farm Bill makes it unique and an outlier among international development programs. This F2F midterm performance evaluation had two main objectives: 1) to provide guidance for planning future F2F program activities and procurement; and 2) to identify best practices and areas where the program can make mid-course corrections. To fulfill these objectives, the evaluation team (ET) assessed the efficiency and effectiveness of program implementation in meeting the program’s goals and worked to identify ways current processes could be improved. The ET also sought to identify good practices for agricultural volunteer program planning and implementation, identify limitations of current practices, and suggest approaches to address these limitations. The Coronavirus Disease 2019 (COVID-19) pandemic presented challenges to program implementation. The ET sought to capture its impact on program implementation and discern how F2F implementing partners (IPs) and F2F stakeholders worked to address the challenges posed by the pandemic. The primary audiences for this evaluation are the USAID F2F team at the USAID Bureau for Resilience and Food Security (RFS)/Center for Agriculture-Led Growth/Production Systems Division and F2F IPs. The evaluation results will be used to support program improvement and develop future strategic directions for the program or USAID at large. A secondary audience is U.S. policy and decision makers responsible for programming U.S. Government (USG)-funded initiatives. 1.2 EVALUATION QUESTIONS All planning, data collection, analysis, and reporting activities in this evaluation center on three evaluation questions (EQs) developed by USAID. ● EQ 1: How well do F2F standard indicators capture the effects of F2F activities and provide useful information for program management? ● EQ 2: How well does the F2F program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs? ● EQ 3: What progress has the Fiscal Year (FY) 2019 to FY 2023 F2F program made toward achieving its objectives? 2.0 ACTIVITY BACKGROUND 2.1 ACTIVITY DESCRIPTION The USAID John Ogonowski and Doug Bereuter F2F Program was first authorized by the U.S. Congress in 1985 and has been re-authorized in every subsequent Farm Bill since then. The program aims to provide transfer of knowledge and expertise of U.S. agricultural producers and businesses to lower- and middle￾income countries and emerging democracies. The program’s primary goal is to generate rapid, sustainable, 3 2014 U.S. Farm Bill, the Agricultural Act; Accessed from U.S. Senate Committee on Agriculture, Nutrition, and Forestry https://www.agriculture.senate.gov/issues/farm-bill. The U.S. Farm Bill is a bundle of legislation Congress passes that sets the U.S. agenda and policies for domestic and international agriculture, nutrition, conservation, and forestry. 2 and broad-based food security and economic growth in the agricultural sector. A secondary goal is to increase the American public’s understanding of international development issues and programs and advance international understanding of the United States and U.S. development programs. To further F2F’s secondary goal, F2F designs volunteer assignments to maximize opportunities for people-to-people interactions in the host countries where they are volunteering. This centrally funded program is managed by USAID from its Washington, D.C., Headquarters (HQ) and works alongside external IPs to facilitate the exchange of experiences between U.S. experts (e.g., producers, agribusiness entrepreneurs, academics, university professors, etc.) and counterpart host￾country agricultural producers, entrepreneurs, and farmer-serving institutions in targeted countries. To date, F2F has served 117 countries worldwide. In the FY 2019 to FY 2023 program cycle, F2F awards programs through eight Leader with Associates Awards (LWAs) and the F2F Agricultural Volunteer Opportunity Program (AVOP), which is an Associate Award. Under the LWA mechanism, USAID Missions or other offices can fund additional buy-in and Associate Award activities that provide agricultural volunteer services and/or additional assistance to complement the work of volunteers. Associate Award activities, however, are funded and managed separately and are not considered a part of the F2F program managed by HQ. The AVOP award, which is currently in its third phase and third implementer, is designed to be a knowledge management service provider to F2F IPs. It manages the Small Grants Program in which it funds sub-awards and provides capacity support. The grants fund innovative approaches to use U.S.-based volunteers, draw from non-traditional sources of volunteers, and increase the capacity of organizations to effectively implement international agricultural development volunteer programs. Combined, the two types of F2F programs aim to take volunteer technical assistance inputs, including training and recommendations made by volunteers or co-designed with partners, and turn those recommendations into behavior, production, and marketing changes that lead to: 1) better farm and labor productivity; and 2) increased income and nonfinancial and intangible assets, such as social, human, and environmental forms of capital. 3.0 EVALUATION METHODS AND LIMITATIONS 3.1 DATA COLLECTION METHODS Data collection methods included online surveys with volunteers and host institutions and key informant interviews (KIIs) with volunteers, hosts, IPs, and USAID (Table 1). Owing to budget constraints and COVID-19 travel restrictions, all KIIs were done remotely using Microsoft Teams. Data collection methods are summarized in Table 1. Table 1: Data Sources for EQs EQ Data Source EQ 1: How well do F2F standard indicators capture the effects of F2F activities and provide useful information for program management? • KIIs with IPs and USAID highlighting opinions on indicators’ usefulness for program management, monitoring, and evaluation. • Indicators that are difficult to collect and need to be simplified or eliminated. EQ 2: How well does the F2F program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs? • KIIs with IPs and USAID about approaches to learning and innovation at the program and project levels. • Online surveys with volunteers and hosts. 3 EQ Data Source EQ 3: What progress has the FY 2019 to FY 2023 F2F program made toward achieving its objectives? • Online surveys with volunteers and hosts on progress made toward achieving objectives, current achievements, challenges, the impact of COVID-19, and gender and diversity. • KIIs with IPs, volunteers, and hosts on progress made toward achieving objectives, current achievements, challenges, the impact of COVID￾19, gender, diversity, and LWA awards. Document Review: Before beginning data collection, the ET conducted a thorough literature review of documents shared by USAID and IPs on the F2F implementation. The literature review enabled the ET to gain an understanding of the processes of implementation, results, and challenges that F2F faced and helped it design the qualitative and quantitative data-collection tools. Online Surveys: The ET used the insights from the literature review to create English-, Spanish-, and French-language versions of two online surveys, which were disseminated to hosts and volunteers (local and U.S.-based) asking them about their experiences with and perceptions of the program, its implementation, and its outcomes. (See Annex 5 for a copy of the English-language online surveys.) The ET collaborated with IPs to compile a list of contact names and email addresses for disseminating the surveys. In total, the IPs shared 928 valid and unique contacts for the online surveys. Once the ET compiled these contacts, it sent respondents an email to inform them of the survey, request their participation, and share a link to access the survey. To increase the survey response rate, the ET distributed an initial email letter of invitation from USAID F2F. Communication with participants was conducted in English, Spanish, and French, the same three languages used for administering the survey. Overall, 549 of 928 host organizations and volunteers completed the online survey for a response rate of 59 percent. This response rate compares favorably to an average of approximately 60 percent for surveys.4 KIIs: In parallel to the online surveys, the ET designed qualitative KII discussion guides to address the EQs, document key stakeholder experiences, and solicit recommendations from volunteers, hosts, IP HQ and in-country team members, and USAID F2F staff. The ET developed a purposive sample of F2F projects, which ensured representation across IPs; types of implementing mechanisms and agricultural sectors, and areas of F2F technical support; and geographic spread across F2F-supported countries. The sample was designed to focus primarily on countries excluded from the 2017 Midterm Evaluation.5 The ET took advantage of the remote approach to the evaluation to work across a larger number of countries compared to previous in-person F2F evaluations. After selecting country projects, the ET worked with the IPs to select a purposive sample of IP staff members based on their level of experience and involvement with the program. The ET selected a random sample of hosts and volunteers, stratified across country projects. In the end, 24 countries and ten IPs operating in 17 types of country projects/sectors constituted the qualitative sample (Table 2) from which the ET interviewed 119 key informants spread across six stakeholder groups (Table 3). 4 https://www.ncbi.nlm.nih.gov/pubmed/?term=Fincham%20JE%5BAuthor%5D&cauthor=true&cauthor_uid=18483608. 5 Midterm Theory of Change and Impact Learning Evaluation of the Farmer-to-Farmer Program (2017) Mudloon, et al., Program Evaluation for Effectiveness and Learning (PEEL). 4 Table 2: Countries, IPs, and Sectors Included in Qualitative Data Collection Region Country IP Sector Africa Rwanda Catholic Relief Services (CRS) Horticulture Uganda CRS Agribusiness Mozambique Cultivating New Frontiers in Agriculture (CNFA) Horticulture Madagascar CNFA Aquaculture Zambia CNFA Legumes Zimbabwe CNFA Livestock/Dairy Tanzania International Executive Service Corps (IESC) Access to Finance Guinea Winrock Rural Livelihood Ghana Winrock Post-Harvest Nigeria Winrock Agricultural Education & Training Asia and Middle East Kyrgyzstan Agricultural Cooperative Development International/Volunteers in Overseas Cooperative Assistance (ACDI/VOCA) Rural Enterprise Development Georgia ACDI/VOCA Rural Enterprise Development Armenia ACDI/VOCA Rural Enterprise Development Nepal CRS Crops Moldova CNFA Livestock/Dairy Lebanon Venture 37/Food Safety & Quality Food Safety & Quality Egypt Venture 37/Food Safety & Quality Food Safety & Quality Philippines AVOP Large Grants Grameen Foundation Capacity-Building of the Coconut Subsector Morocco AVOP Large Grants High Atlas Foundation Cooperatives Leading Sustainable Agriculture Caribbean, Central and South America Peru National Cooperative Business Association Cooperative League of the USA (NCBA CLUSA) Coffee Systems Ecuador NCBA CLUSA Coffee Systems Dominican Republic Partners of the Americas (POA) Rural Adaptation and Resilience Colombia POA Rural Enterprise Development Guyana POA Horticulture Production & Marketing Guatemala POA High-Value Horticulture Table 3: Distribution of KIIs Disaggregated by Type of Respondent Type of Respondent Number of KIIs USAID F2F staff 2 HQ IP staff 6 IP field staff 24 IP field staff and monitoring and evaluation (M&E) staff 33 Volunteers 30 Host organizations 24 Total 119 5 3.2 DATA ANALYSIS The purpose of this evaluation was to evaluate the F2F program overall rather than any individual IP or country program. Data analysis and reporting were thus approached in the aggregate. For the qualitative data, the ET digitally recorded each KII and automatically transcribed it using Microsoft Teams. After the interview, the ET cleaned the interview transcripts to correct errors in the automatic transcription. Where recordings were not possible, the ET took notes during the conversation. Once cleaned, the ET coded the transcripts to identify key and recurring themes, which were summarized in a coding tally sheet. For the quantitative data, the ET analyzed online survey responses using SPSS statistics software and Microsoft Excel. 3.3 METHODOLOGICAL LIMITATIONS The F2F program currently operates in 43 countries through eight primary IPs plus AVOP and their sub￾awardees and a wide array of project areas of focus. For that reason, it was not possible to design a performance evaluation that could reach all respondents, nor was it possible to interview a representative sample of key informants. In lieu of a fully random sample, the ET elected to interview a purposive sample of key informants across IPs, countries, and sectors selected in consultation with RFS to ensure that key program characteristics and their diversity were reflected in the qualitative sample. Nonetheless, the non￾random nature of the sample may have introduced some unknown level of bias to the sample and thus the evaluation results. The possibility of sampling bias is also a limitation related to the online survey. Although a 59 percent response rate is considered good for an online survey, the responses were, nonetheless, not randomly distributed across the target population. Therefore, it is possible that responses are correlated with certain characteristics of the target population and not fully representative of that population. For example, hosts with limited access to internet or limited technical capabilities may have been less likely to respond to the survey. Also, while the ET worked to make the survey available in French, Spanish, and English, it was not possible to provide surveys to all hosts and volunteers in their native languages, which may have affected the number of responses received. Because the ET conducted KIIs virtually, it selected individuals purposively based on access to the internet or cellular data. Poor internet connectivity posed a challenge to the ET, particularly when speaking with key informants outside the U.S. Several interviews were abruptly cut short by poor connectivity, and ET members had to rely on other communication tools like WhatsApp and landlines. Also, while the ET has language capabilities in Spanish, French, and Arabic, additional translation support was not possible during the KIIs. As a result, the ET also selected respondents purposively for language capabilities. In some cases, lack of translation support hindered the ET’s ability to thoroughly capture the respondents’ perspectives. These two pieces of selection criteria and related challenges in interview implementation may also have impacted the evaluation results. A final limitation was the presence of several IPs with different institutional missions and visions implementing F2F projects in several countries, each with unique socioeconomic and agricultural conditions. This wide heterogeneity across the F2F IPs makes generalizing from qualitative findings more difficult than for the more homogenous IPs and projects. 6 4.0 FINDINGS & CONCLUSIONS 4.1 EQ 1: HOW WELL DO F2F STANDARD INDICATORS CAPTURE THE EFFECTS OF F2F ACTIVITIES AND PROVIDE USEFUL INFORMATION FOR PROGRAM MANAGEMENT? 4.1.1 The Use of F2F Indicators for Monitoring Implementation Progress Table 4 lists the 26 standard performance and impact indicators used in the FY 2019 to FY 2023 program. Table 4: F2F Program Standard Performance and Impact Indicators Input Indicators Output Indicators Outcome Indicators Impact Indicators • Volunteer assignments • Number of days and value of volunteer service on assignment • Type of volunteer assistance • Type of value chain activity • Value of host contribution • Number of persons trained • Number of volunteer recommendations made • Number of host organizations assisted • Publication, radio, or TV broadcast • Social media publication by implementers, volunteers, hosts, and partners (worldwide) • Presentations made to groups by implementers, volunteers, and partners • Value of resources mobilized by host • Volunteer recommendations applied • Continued volunteer/host connection • Value of resources leveraged by implementers and volunteers in the U.S. • Number of days and value of volunteer technical assistance from the U.S. • Number of beneficiaries • Value of annual gross sales (revenue) • Annual net income • Simplified organizational development index (ODI) • Value of rural/agricultural lending expended by host • Number of rural/agricultural loans issued by host • Actual area under improved production or natural resource management (NRM) practices • Number of new or improved products and services • Area of potential direct management influence6 • Number of potential beneficiaries7 FINDINGS IPs periodically report on progress toward indicators using three tables included in the F2F indicator tables template provided by USAID: 1) volunteer and assignment data; 2) host descriptors, outcomes, and impacts; and 3) outreach and leverage. The current standard indicators list in Table 4 meets the core needs of the F2F program. Standard indicators provide responsive information internally to USAID and the U.S. Congress. Tracking indicators enables F2F to monitor performance and make decisions on implementation in collaboration with IPs. 6 Measures impact potential. 7 Ibid. 7 Key informants at USAID and IPs noted that IPs are comfortable with and accustomed to reporting on the standard set of indicators. Notwithstanding, IPs noted that more frequent communication to and training of their staff is needed regarding indicator definitions, data collection templates, and reporting. When asked about the relative importance of indicators in driving management decisions, IPs generally agreed that most or all indicators are useful. The relative importance of indicators for decision-making varies across different management levels within IP teams and between HQ and in-country offices based on their respective responsibilities and project scope. While respondents generally noted that all indicators help them make decisions and guide work planning, several indicators were frequently highlighted as being particularly important. The indicator “Volunteer Assignments” was the most frequently cited as key in decision-making by F2F IPs followed, in descending order, by “Value of Annual Gross Sales (Revenue),” “Annual Net Income,” “Number of Host Organizations Assisted,” “Number of Persons Trained,” “Number of Volunteer Recommendations Made,” and “Volunteer Recommendations Applied.” One F2F IP said: “Much of what we make programmatic decisions on are based on the number of volunteers that we are mobilizing for assignments and then the number of persons trained as well. And for volunteer assignments, we are looking at working toward the targets we set out.” While the standard indicators are developed by USAID to be responsive to federal reporting requirements and the overall objectives of the F2F program, IPs uniformly said that their selection of indicators is informed by their specific needs and the scope of their country projects. IPs identified a diverse array of indicators as important, depending on the focus of their F2F programming. For example, in addition to the indicators listed above, IPs also noted the importance of indicators related to the value of loans, improved products and services, and host resource contributions. IPs collect data to support disaggregation within indicators based on key demographic information relevant to their F2F objectives. Depending on the stakeholder, the demographic data collected is different. For example, when reporting on training participants, IPs focus on gender and age; when reporting on hosts, institution type is important; and when reporting on volunteers, gender, prior service, and ethnicity are key areas of focus. Across the IPs interviewed, a commitment to diversity and inclusion was cited as an important principle that guides decisions and programming. Dedicating resources to data collection and reporting across key demographic disaggregates is important for tracking progress toward these priorities. IPs also noted that detailed data collection and reporting across demographics is key to ensuring that the program provides USAID with data to inform Congress on the scope of F2F program impact: “… But we're also looking to engage, you know, the most qualified volunteers as well as diverse volunteers too. So, we look at the demographic information that we're collecting on our volunteers to ensure that we're meeting the target percentage for women volunteers that we wanted to engage as well as first time volunteers, because we know that new volunteers to the Farmer-to-Farmer program really bring a new and exciting perspective to their assignments too.” – F2F IP IPs commonly expressed challenges related to three indicators: the two beneficiary indicators (number of beneficiaries and number of potential beneficiaries) and the simplified organizational development index (ODI). The IPs noted that the F2F community of practice (CoP) determined that the beneficiary indicators need additional clarity to ensure accurate and consistent reporting. According to IPs, the guidance for determining total beneficiaries is vague and is, therefore, open to different interpretations. IPs especially noted that the guidance for determining indirect beneficiaries lacks clarity and thus leads to inconsistent reporting. The difference between the number of beneficiaries and number of potential beneficiaries is also unclear in that the guidance for counting individuals is similar across both indicator definitions. Notwithstanding, IPs recognized that this is an important indicator for USAID to report to the development community and U.S. Congress on the breadth of the program’s outreach. The bottom-line consensus among IPs was that the definitions of these indicators need more clarification from USAID and 8 alignment between IPs, while acknowledging that the definitions need to remain flexible enough to be responsive to diverse projects. In contrast to what the IPs stated in the 2017 F2F Midterm Evaluation,8 they generally do not find the simplified ODI data challenging to collect. The ODI is reported by all IPs. However: “The ODI has not really kept up to pace. The ODI used to be the bible—it would give you the exact needs that the host needs to be supported with. But now, it has really changed to the extent that the ODI is a tool that we administer so that we are able to meet our obligation and to report upon Impact Indicator 16. Some of the IPs now have a different tool that they [can] use to collect their needs assessments. So, the ODI really needs to be looked at, and it needs to evolve.” – F2F IP While the ODI is appropriate for measuring the capacity of relatively new organizations, it is not appropriate for measuring the capacity of advanced organizations. The ODI was described as “rigid” and not as relevant to current F2F operations given how the program has evolved to work with higher capacity hosts. Many IPs now use custom needs assessments, which provide information that is more relevant to understanding hosts’ needs and designing assignments. IPs have developed custom indicators designed to their specific needs and scope, which are also reported to USAID. No feedback is provided from USAID on results reported toward custom indicators. Due to the uniqueness of the custom indicators reported by IPs, they are not included in the F2F Annual Report Consolidated Indicator Tables. IPs indicated that custom indicators could be a good source for identifying potential new standard indicators, especially for women and youth leadership and business empowerment. These are two areas IPs identified where there is a clear need to expand the current set of F2F standard performance indicators. IPs also suggested creating indicators to track market and financial linkages, communication between hosts and volunteers, host resilience to external shocks like COVID-19, and job creation. When discussing developing new standard indicators, IPs also mentioned the potential to align F2F indicators with the Feed the Future indicator set, for example Feed the Future indicators on improved management practices or technologies and increased productivity or yields. IPs identified several data-quality challenges that arise when collecting data. The main one is poor host recordkeeping, particularly among farmers, which has a direct impact on the IPs’ ability to report accurately on host sales and income. Although IPs work to build host recordkeeping capacity, which supports strong business practices and mitigates monitoring and evaluation (M&E) challenges, it remains a persistent challenge. Another challenge is collecting data along the value chain for women and youth. As a result, data is often collected only in the aggregate. Collecting gender or youth disaggregated data can be burdensome for hosts, especially if information on gender or age has no clear demonstrated value to their business model. The IPs also noted that hosts are sometimes reluctant to share sensitive information about their finances. To encourage hosts to share data openly, IPs work to establish trusting relationships with hosts and keep channels of communication open to demonstrate and share program impact. This is not always sufficient, however, to overcome hosts’ reluctance to share sensitive financial information. From the hosts’ perspective, while they are not familiar with F2F indicator names, they agreed that data collection and reporting requirements from IPs are not challenging and that they have adequate data collection tools. According to hosts, the most common data requests are for membership and training 8 “The ODI was identified by 28 percent of IP respondents—more than any other indicator—as needing simplification through the evaluation team’s web survey. However, there was a significant difference of opinions from respondents concerning this indicator, as found during the evaluation’s field visits. One F2F IP stated: ‘The Organizational Development Index is pointless.’ However, another interviewee lauded the “ODI as a good tool, especially for AET [Agricultural Education and Training]. Most IPs, however, felt this indicator was too subjective and cumbersome.” – 2017 F2F Midterm Evaluation 9 and success stories. The hosts’ experience in providing data was highly variable. Some hosts described reporting as mostly anecdotal and frequently stated that templates are not provided by F2F IPs to aid in data collection or reporting. Other hosts described rigorous, periodic reporting of quantitative data. The variation between the hosts’ experience with data reporting for F2F is based on the nature of their business. For example, data requested from financial institutions tends to require more robust reporting involving quantitative data on the value of loans and demographics of recipients, whereas farmers report mostly on individuals trained and anecdotal success stories. CONCLUSIONS Overall, the current set of standard indicators is useful for informing implementation progress at the USAID and IP levels, although there is a defined hierarchy for the indicators that IPs see as the most useful for informing management decisions. Definitions for several indicators (e.g., beneficiaries, potential beneficiaries, and simplified ODI) could be further improved to better guide IPs on data collection and ensure greater reporting consistency. IPs are comfortable reporting on the F2F standard indicators, although they place different degrees of value on indicators based on their respective project scopes. The current set of standard indicators does not cover all results important to IPs, so they created custom indicators. The current indicators do not address women, youth leadership, or business empowerment, which several IPs identified as important indicators of project success. The data that hosts provide is guided by requests from IPs, which varies between hosts and assignment types. In some cases, the data provided is mainly anecdotal, while for others, it is quantitative and detailed. Hosts are not familiar with indicator names or definitions but are focused on specific data or information requested. Some IPs provide standard templates for data reporting, while others use their own for data collection and reporting. 4.1.2 The Extent to Which F2F Indicators Reflect the Program’s Theory of Change FINDINGS IPs generally agreed that the current set of standard indicators adequately (although not perfectly) captures program benefits and collects information from hosts and volunteers that supports decision￾making. Indicators respond to and reflect the program’s ToC, although there is particular emphasis on input and output indicators. With that said, the ET found that the level of understanding of the cause-and￾effect pathway of the program’s ToC varies across IP team members. While many IP team members interviewed clearly articulated the pathway of change from inputs to outputs, outcomes, and impacts, the connection was not as clearly articulated or understood by others. IPs consider the volunteer assignments indicator to be the most important measure of program results, and it received the most programmatic focus compared to other indicators. IPs receive the most feedback from USAID on this indicator, and country directors focus on this indicator in conversations with HQ during check-in meetings. IPs pay particular attention to the number of recommendations given by volunteers and find this indicator to be a helpful measurement of progress toward intended program results. They recommended devoting resources to conducting more in-depth analyses of recommendations applied by women and youth to be more responsive to social inclusion concerns. They also suggested that analysis comparing volunteer assignments to recommendations adopted at the program level would be beneficial to gauge program success. Further, analysis of the relationship between the number of volunteer assignments completed with a host and recommendations implemented by the host could support IPs in exploring how important recurrent assignments are in generating impact. This insight would be useful for informing future assignment design. 10 Attributing the success of program participants to the influence of F2F is an ongoing challenge for program M&E activities. IPs emphasized that it is difficult to attribute changes in economic indicators (such as sales and income) to F2F activities alone, since there are synergies and collaborations with a multitude of other actors on the ground. These include USAID activities, other donor programs, and local government initiatives. Moreover, close collaboration with other development programs is a common strategy implemented by IPs to maximize support provided to hosts and program participants. USAID acknowledges the challenges associated with attribution, as stated in the Farmer-to-Farmer Program Standard Performance and Impact Indicator Definitions: “The program recognizes that these are ‘indicators’ of impact, and that true impact cannot be rigorously quantified and attributed solely to the F2F program. In almost all cases, hosts are influenced by a variety of factors, which combine to produce impacts. The F2F program volunteers and field staff play an important role in this but are seldom the sole influence in generating impacts.”9 Key informants indicated there is a need to standardize data collection methods among IPs and devote more resources to M&E. Those additional resources should be dedicated to supporting improved data collection on outcome indicators and include more impact indicators; currently, the focus of M&E is on input and output indicators. IPs noted that important impact indicators, such as value of sales, are less useful for tracking program progress in this cycle due to the impact of COVID-19. Rather, they mentioned the need to focus on host resilience and the ability to preserve past progress made as a measure of program impact during COVID-19, rather than solely on traditional economic indicators of program success: “I think for this particular cycle, like I've mentioned, gross sales and net income are likely not going to give us too much to go from because they've been impacted quite significantly by COVID. I think we're going to look at more anecdotal and narrative ways to capture how we've supported business survival and resiliency building instead and look at how we can adapt that successfully in future program implementation.” – F2F IP IPs dedicate substantial effort to conducting needs assessments with hosts. While there is no standard F2F procedure guiding needs assessments, it is the starting point for any work with hosts and partners. The IPs use baseline assessments to design the scope of work (SOW) for volunteer assignments, making baselines a key step linking M&E with program design and implementation. They identify both needs assessments and baseline assessments as critical steps in the process of identifying and designing an effective and efficient pathway to support hosts in reaching desired impacts. CONCLUSIONS The program’s ToC is the basis for the activities that each IP implements. The specific interpretation of the ToC varies depending on each IP’s context and project focus. Each IP’s understanding of the ToC, its underlying cause-and-effect pathway, and the relative importance of indicators on this pathway thus varies considerably from IP to IP. Notwithstanding the stated importance of several standard F2F outcome and impact indicators to IPs, programmatic focus prioritizes input and output indicators and, principally, the volunteer assignments indicator. Given this limited focus, it is not surprising that the relevance of each indicator as a reflection of the program ToC, particularly those further up the causal chain, is sometimes not clearly articulated to IP team members. Along these lines, IPs stress the importance of giving greater priority to outcome and impact indicators and replacing traditional economic indicators (i.e., sales and income) with indicators that better reflect the market realities of the COVID-19 era (i.e., host resilience and ability to preserve past progress made). At the same time, however, attributing observed results at the outcome and impact levels to program activities is currently, and is likely to remain, a significant challenge. 9 Farmer-to-Farmer Program Standard Performance and Impact Indicator Definitions, February 2021 version. 11 4.1.3 Effectiveness of Outreach/Cultural Exchange, Capacity Development, and Environmental / Natural Resource Management Indicators in Capturing the Effects of F2F FINDINGS There are four indicators focused on outreach and cultural exchange reported by all IPs (Table 5). While IPs feel that outreach and cultural exchange is well captured by the indicators, they also expressed concerns that progress toward these program objectives is not well captured. The challenges arise when volunteers are inconsistent in reporting on what they do after the assignment is completed. Collecting and verifying data is also challenging. There are sometimes issues with links provided for articles or social media posts that require periodic verification. Table 5: Outreach/Cultural Exchange Indicators Indicator Type Indicator Output Publication, radio, or TV broadcast Output Social media publication by implementers, volunteers, hosts, and partners (worldwide) Output Presentations made to groups by implementers, volunteers, and partners Outcome Continued volunteer/host connection Among the IPs that consider outreach and cultural exchange indicators challenging to collect, cultural exchange is highlighted as particularly difficult. These IPs suggest that stakeholder engagement studies could be a tool for capturing cultural exchanges, but due to IPs’ limited M&E resources, it may be more realistic to capture progress via third-party evaluations directly commissioned by USAID. While IPs consider the homestay disaggregate of the volunteer assignments indicator to be a useful measurement of cultural exchange, the F2F Annual Report Consolidated Indicator Tables for the current implementation cycle reveals only 4 percent of volunteers stayed at the residence of a host country national. There are four indicators used to report on capacity development, which focus on the number of persons and host organizations trained/assisted, recommendations applied, and simplified ODI, which is used to measure strengthening of hosts’ organizational or technical capacity (Table 6). IPs identified the first three of these indicators to be important for implementation and decision-making, and IPs are experienced in collecting data for them. Table 6: Capacity Development Indicators Indicator Type Indicator Output Number of persons trained Output Number of host organizations assisted Outcome Volunteer recommendations applied Impact Simplified ODI The IPs noted challenges to collecting data for the capacity development indicators. For disaggregates based on age and gender, data collection can be challenging in the contexts in which hosts work. For example, some participants are unable to recall their birth year. Depending on the cultural context, some IPs do not feel comfortable addressing nor collecting data on gender, particularly non-binary self￾identification. Further, IPs noted that in some regions, participants feel unsafe or uncomfortable sharing information on their gender when they identify as non-binary. Regarding volunteer recommendations applied, IPs mentioned that it is critical to be present during the wrap-up session with hosts and volunteers to discuss the recommendations made, agree on which are realistic to achieve and most useful, and plan how to implement them. The IPs identified the simplified ODI as a useful indicator of a host’s organizational development and the impact of volunteer assistance for organizations at a lower baseline of capacity. However, as noted above, 12 the IPs also emphasized that the ODI needs to be updated to address its limitations in assessing organizational capacity for higher capacity hosts. The indicator, “Actual area under improved production or natural resource management (NRM) practices,” is the only one related to NRM. The IPs noted it is not sufficient to show the scope of F2F’s impact on NRM, considering one of F2F’s main objectives is to “improve conservation and sustainable use of environmental and natural resources.” This indicator is relevant for projects dealing directly with agricultural production activities but less so for projects dealing with financial services or other areas outside of production. However, IPs recognize the indicator’s value for the program overall. IPs noted it is challenging to measure area accurately when collecting data for this indicator. In some cases, there is reluctance from farmers to even provide an estimate. Several IPs argued that this impact should be measured by specific technical studies or assessments, which require specialized human resources beyond their capacity, while others noted that impacts on land management take time to emerge beyond the life of the project. The NRM indicator is one of the main indicators that IPs identified as challenging to collect, along with the area of potential direct management influence: “I'd say in my opinion that we can better connect our indicators to Feed the Future indicators and make them kind of more standardized. I know that programs vary a lot across different implementers. Like our program is focused on food safety and quality, so not all the indicators really make sense for us to be reporting on. But providing some menu of options for different implementers who are whose programs are focused on different things and making them a bit more standardized.” – F2F IP CONCLUSIONS The IPs have confidence in the capacity development indicators and report on them without major issues. They can, however, be improved by including social inclusion disaggregates. At the same time, the ODI needs to be made more adaptable to better reflect the capacity of hosts at any development stage. The cultural exchange and NRM indicators are not currently sufficient to describe the program’s performance and impact in these areas. Further, data collection and reporting for these indicators can be improved, although IPs were not able to provide specific recommendations other than looking into other indicator sources such as Feed the Future. Finally, outreach indicators vary in terms of complexity to collect and validate. 4.2 EQ 2: HOW WELL DOES THE F2F PROGRAM SYSTEMATICALLY LEARN AND INTEGRATE INNOVATION TO IMPROVE QUALITY AND PROMOTE GOOD PRACTICES IN AGRICULTURAL VOLUNTEER PROGRAMS? The ET explored how F2F systematically learns and integrates innovation across two levels of program implementation: 1) program-level learning activities; and 2) project-level learning activities. 4.2.1 Program Learning Activities FINDINGS IPs described a strong culture of collaboration and sharing between F2F IPs. According to them, there is open communication at the IP level and a willingness to share information. They are positive about the strong community of collaboration and support, especially given that they compete for funding every five years. They describe this culture of collaboration as one of F2F’s strengths compared to other development programs. Communication and collaboration between IPs occur in formal settings, such as within the CoP, meetings, and workshops. Valuable communication and collaboration also occur through informal channels, such as working relationships between F2F team members employed across IPs. For example, M&E managers who 13 work for different IPs are connected professionally and rely on each other as resources for addressing indicator and data collection system challenges. F2F IPs communicate with each other and USAID in the CoP with the goal of collectively improving the program. The CoP is made up of learning groups dedicated to specific focus areas that collaborate through virtual and in-person meetings. CoP meetings provide opportunities for IPs to engage in peer-to-peer knowledge exchanges with counterparts working in other regions and facilitate sharing of strategies and best practices. Most key informants feel that the CoP is beneficial and supports IPs to learn and adapt effectively, as shared by one F2F IP: “It's very much a community that's open and willing to share and learn together, which allows the program to operate globally on a level unlike other programming. We all are able to share best practices and grow and learn together.” The IPs noted that participation in the CoP is typically limited to HQ staff based in the U.S. and upper￾level management team members in-country. They expressed a desire for improved inclusion of in-country teams in CoP meetings to better represent the views of all F2F team members and improve the CoP’s effectiveness by incorporating a broader range of experiences and perspectives. The desire to make the CoP more inclusive with in-country team members is consistent with the perspectives shared by respondents in the 2017 F2F Midterm Evaluation. Whereas, before the COVID-19 pandemic IPs coordinated in-person learning meetings through the annual IP meeting and the CoP, during the pandemic CoP meetings moved to virtual platforms. The IPs noted that the transition to fully virtual platforms facilitated improved inclusion of in-country team members. They recommended that the CoP continue to leverage virtual meeting methods to promote inclusion and work to make inclusion of in-country team members a priority even after in-person meetings resume. While virtual meetings have many benefits, IPs emphasized the importance and effectiveness of in-person CoP meetings and hope to return to a hybrid model when travel restrictions permit. Respondents noted that while the CoP is a strong resource, there needs to be commitment and engagement from both IPs and USAID for it to be effective. According to key informants at USAID, the CoP benefits from USAID’s support and involvement, and when USAID engages, participation from IPs is stronger. During a period of reduced staffing early in the program cycle, USAID had limited involvement in the CoP. USAID has since solicited feedback from IPs to determine the optimal meeting frequency, and beginning in late 2020, USAID has attended and led most CoP meetings. Key informants highlighted the work the CoP has done to: 1) collect recommendations on modifications to key indicators; 2) design studies on topics of shared interest, such as paired and remote volunteering; and 3) produce shared promotional materials. However, such recommendations need more support from the IPs and USAID to be actualized. In KIIs, USAID noted that the one of the challenges facing the CoP is a lack of structure to guide authority for decision-making and suggested that the CoP would benefit from improving decision-making frameworks. Within their F2F programs, most IPs utilize internal workshops and meetings as platforms for program learning. The format and frequency of these meetings vary across IPs, ranging from coordination between team members on a single country project to larger meetings across all projects within an IP’s portfolio. The impact of the global pandemic on these meetings has been mixed. In some cases, planned in-person meetings were delayed or canceled. In other cases, the frequency of meetings and degree of participation increased as IPs expanded their use of virtual meeting platforms. For example, one IP implemented regular townhall meetings—bringing together team members across all country projects. The IPs acknowledged that there is a need for more data-driven approaches to learning, so they have several new innovative approaches in the planning or implementation stages. Examples include outcome mapping to discern and attribute program impact and stakeholder engagement studies aimed at gathering stakeholder perspectives to gain clearer understanding of their needs and perceptions. IPs are interested in using other qualitative data collection methods, such as case studies, to improve learning and develop 14 new outcome and impact indicators. IPs emphasized that while opportunities exist to expand the program’s focus on learning, resources are limited. To implement improved learning practices, respondents cited the need to allocate more resources to learning and M&E. However, this can be challenging for IPs within their existing budgets, where resources are largely allocated to implementation. IPs and hosts agreed F2F could do more to leverage learning between stakeholders. For example, the program might facilitate host-to-host knowledge exchanges, providing a platform for hosts to share the practices and solutions developed through volunteer assignments, thereby amplifying the program’s impact. Several respondents also suggested that F2F IPs facilitate knowledge exchanges between other program stakeholders, e.g., between volunteers or in-country development partners and practitioners. IPs and hosts also agreed that collaboration and knowledge-sharing with other USAID programs, as well as external development partners, is beneficial and supports strong project design, learning, and adaptation. Learning from other USAID projects and external partners often occurs when establishing F2F in a new country through conversations between IPs, local stakeholders, and development partners. Collaboration and learning continue during program implementation. IPs remain in touch with local stakeholders, USAID programs, and development practitioners about their areas of focus and need. Ongoing communication enables F2F to be strategic and apply a flexible, responsive approach to address key gaps in local development work while avoiding duplicating the efforts of other USAID programs. Communication and collaboration happen in both formal settings—including official partnerships, coordinated workshops, and meetings to address gaps—and in informal settings, including day-to-day working relationships with host country and regional counterparts. Feed the Future Innovation Labs offer another opportunity for collaboration and partnerships as well as a resource for best practices that volunteers can utilize for assignments. Sharing approaches and techniques happens in both directions between F2F and the Innovation Labs. The Innovation Labs have been an important partnership for some F2F IPs, and the IPs and USAID are interested in pursuing them further. CONCLUSIONS IPs sharing information and collaborating is a strength of F2F. Collaboration and communication between IPs supports program learning and avoids duplication of efforts. Valuable collaboration and learning happen in formally organized meetings and platforms between IPs and through informal channels and inter-IP working relationships. The CoP is a valuable resource for IPs that supports program learning and evolution. The CoP benefits from guidance and participation from USAID to coordinate meetings and accomplish tangible results. USAID has solicited feedback from IPs to determine the frequency of meetings, and since late 2020, USAID has attended and led all CoP meetings. Several IPs indicated that the CoP could become stronger with better representation and inclusion of in-country team members to leverage the true scope of the program’s knowledge base and expertise. Internal learning meetings held by IPs at the program and country levels are a beneficial and commonly used resource for learning. More data-driven approaches are needed to improve learning and innovation, such as the use of case studies, more qualitative data collection, and additional outcome and impact indicators. IPs have developed several strong and innovative internal learning methods, such as outcome mapping and stakeholder engagement studies. However, these are not used by all IPs. To expand the program’s focus on innovative M&E-based learning, IPs will need additional resources to build capacity in data collection and analysis. There are limited resources available to dedicate to learning, which makes the synthesis and implementation of new ideas challenging. Opportunities exist to leverage learning opportunities between stakeholders, e.g., facilitating host-to-host or volunteer-to-volunteer learning exchanges and workshops. Stakeholder learning platforms have the potential to amplify program impacts. However, establishing learning opportunities between stakeholders 15 will require additional resources, particularly to ensure inclusion of hosts located in remote areas or with limited access to virtual platforms, who will require in-person events. F2F IPs have a strong practice of learning and leveraging information from other programs to maximize the responsiveness and impact of F2F and avoid duplication of efforts. F2F is well-positioned to address gaps in needs for regional agricultural development projects due to its broad focus on high-quality technical support across many technical areas within agricultural value chains and its flexible, responsive implementation approach. As a result, collaboration and learning with other partners is common and effective. Learning with external partners happens through formal meetings and workshops as well as informal settings and routine working relationships. Opportunities exist to continue to leverage knowledge from other USAID programs, including the Feed the Future Innovation Labs. 4.2.2 Project Learning and Innovation Activities FINDINGS While internal program learning happens through planned data collection efforts, workshops, and meetings, program learning and innovation also takes place through the cycle of assignment design and implementation. Important avenues for learning include the participatory process for assignment development and the communicative relationships between 1) IPs and hosts; and 2) IPs and volunteers throughout the assignment lifecycle, which enable rapid learning-feedback loops between assignments. This type of learning is a critical driver of the program’s responsive, needs-based focus. The SOW development process is highly participatory. Sixty-eight (68) percent of hosts who responded to the online survey reported being directly involved in SOW development from the earliest stages, and 90 percent said they were satisfied with their involvement in assignment and SOW development. While SOW development is primarily driven by hosts, recommendations from previous volunteer assignments also guide SOW design. Key informants noted that, from a learning perspective, SOWs for F2F assignments could be improved by incorporating learning questions for the volunteer to address during their work with host organizations. Hosts, volunteers, and IPs each noted that there are opportunities to improve learning support for volunteers, particularly in volunteer orientation at the start of a new assignment. While U.S.-based volunteers may be experts in a specific technical area, they are sometimes lacking in contextual knowledge about the region where they assist. While IPs generally do provide relevant contextual information to volunteers, this practice is neither consistent nor systematic within and across IPs. Key informants recommended that investing in volunteer learning prior to beginning an assignment by providing information on local context and available regional studies and datasets can enable volunteers to be better prepared and assist the host in a culturally and contextually relevant way. Respondents noted that cultural sensitization, especially as it relates to inclusion, is also important. To support effective learning and orientation processes for volunteers, key informants strongly emphasized the value of robust communication between the host and volunteer before the assignment begins. Hosts commonly noted that limited time is a critical constraint on assignments. They described the process of orienting a volunteer to the local context and the host’s operations as time consuming. Once the volunteer is fully oriented, the remaining time for accomplishing assignment goals is limited (or reduced). Key informants observed that if volunteers are thoroughly briefed and oriented about the host’s operations and the country context before the assignment begins, there is more time to focus on assisting during the assignment. During the pandemic, the use of virtual platforms to orient volunteers made it clear that opportunities exist to use these platforms to undertake robust volunteer orientation before starting assignments. Key informants emphasized the importance of this lesson learned for improving F2F operations in the future. 16 Key informants generally agreed that there is strong communication between the IPs and hosts during assignments. Most hosts interviewed reported that that IPs are frequently in touch during assignments to provide support and identify and address challenges. At the close of the assignment, the IP coordinates debrief sessions between the IP, volunteer, and host organization. Among volunteers, 80 percent of online survey respondents said they met with the IP to discuss their report and recommendations. Key informants identified debrief sessions as valuable tools for learning. During debrief sessions, IPs collect feedback from the volunteer and host regarding challenges faced and strategies employed to resolve them. They work with hosts to identify which volunteer recommendations are realistic for a host to implement, given their context, and which are unrealistic without further support. Recommendations that are unrealistic to implement without further assistance are used as guidance to drive SOW development for follow-on assignments. Key informants emphasized the importance of this process as a learning tool and a way in which volunteer recommendations inform SOW development. Although communication between IPs and volunteers is strong during the assignment, several volunteers noted that they did not receive sufficient follow-up from the IPs post-assignment about assignment outcomes or whether their host organization adopted any of their recommendations. While most hosts said they received sufficient follow-up from the IPs after completing the assignment, and 85 percent of host online survey respondents were “satisfied” or “very satisfied” with their ability to follow up on volunteer recommendations, experiences among hosts varied. Especially in the context of COVID-19, some hosts did not receive strong communication from IPs tracking their progress. When hosts or volunteers do not receive strong communication, they can become frustrated with the program, feel undervalued, or feel that their needs are not being prioritized. There was strong communication during the assignment between volunteers and hosts. Ninety-nine (99) percent of host respondents to the online survey said they feel their priorities were valued and respected by volunteers. KII and online survey respondents emphasized the value of learning relationships between hosts and volunteers after the assignment ends. Online survey respondents reported that most hosts and volunteers remained in communication after the assignment ended: 69 percent of volunteers and 70 percent of hosts reported being in touch with their counterparts after the assignment was complete (Table 7). Volunteers and hosts were most commonly in contact about ongoing guidance for the assignment (Table 8). Among these, less than one-half (42 percent) of hosts indicated that they communicated with volunteers monthly or more frequently, while another 28 percent communicated with volunteers less than once a month. Table 7: Frequency of Post-Assignment Communication Respondents Daily Weekly Monthly Less Than Monthly Never Don’t Know Hosts (N=185) 1.1% 11.4% 29.7% 28.1% 25.4% 4.3% Volunteers (N=359) 1.7% 10.0% 16.4% 41.2% 24.5% 6.2% Table 8: Primary Purposes of Post-Assignment Host Communication with Volunteers10 Respondents Finalize Work or Deliverables Ongoing Guidance Related to Assignment Ongoing Guidance Unrelated to Assignment Personal Communi￾cation Other Don’t Know Hosts (n=128) 25.8% 80.5% 12.5% 14.8% 2.3% 0.8% Volunteers (n=362) 16.6% 40.9% 13.5% 26.5% 4.4% 1.1% 10 The survey allowed multiple responses, such that responses may total more than 100 percent. 17 Hosts emphasized the value of ongoing communication with volunteers and expressed the desire for more communication after the assignment ends. The KIIs also revealed that volunteers and hosts are sometimes unclear about whether and how they should communicate after the assignment ends. Key informants emphasized that close communication between volunteers, IPs, and hosts after assignment completion supports the hosts in addressing challenges faced in implementing recommendations and facilitates the development of follow-on assignments. Further, through the paired remote volunteering mechanism, the ability of local volunteers to follow up on the ground has been particularly beneficial. CONCLUSIONS Important learning in the F2F program occurs through the assignment lifecycle and is fundamentally integrated into the F2F program through the responsive, needs-focused process of assignment development. Learning and information gathering from hosts, stakeholders, and past volunteers drives assignment design and SOW development. Learning from hosts is particularly critical to ensuring program effectiveness, sustainability, and responsiveness, and the commonly used needs assessments are an important tool that drives learning from the host at the start of the assignment. The reflection and information-gathering that is done between one assignment and the next is an important learning activity. Commonly used and helpful practices include debrief sessions with hosts and volunteers and incorporation of recommendations from past volunteer assignments into follow-on SOWs. Dedicating the time and resources to do this follow-up is a critical component of program learning and important for developing assignments that deliver substantive support. The program supports highly communicative relationships between the IPs, volunteers, and hosts. Communication between the participants drives program learning between assignments and ensures that the support delivered to both hosts and volunteers is responsive and effective. While communication from the IPs to hosts and volunteers is generally strong, especially during the assignment, communication from the IPs to volunteers is not as strong after the assignment ends. Some hosts also do not receive sufficient communication from the IPs after the assignment ends. Strong communication from the IPs is critical to retaining a talent pool of volunteers, maintaining relationships with hosts, and ensuring the program’s responsiveness and impact. Providing more information on local context and location-specific culture, data, and studies to the volunteer before the assignment can support faster assignment startup, efficient use of time, and better designed assistance. Volunteers would like to have closer communication during the planning stages before assignments begin. Improved communication leading up to an assignment improves startup efficiency and allows volunteers to make the most of their time while on assignment. Strong communication before the assignment begins provides a learning opportunity for the volunteer and ensures learning is incorporated into the assignment. Remote assignment models and virtual platforms for connecting hosts and volunteers have provided F2F with innovative ways to improve communication leading up to assignments. In general, hosts and volunteers appreciate being connected and want more and extended communication. In particular, hosts want continued communication with the volunteer after the assignment ends to support implementation of recommendations. Norms and expectations for communication between hosts and volunteers after the assignment ends are not always clear, however, which can inhibit communication. Strong communication between all parties after the assignment ends can support future assignment development and ensure that learning is incorporated into future assignments. Normalizing post￾assignment communication standards between volunteers and hosts can also support IPs in conducting effective follow-up with hosts by leveraging the connection between the host and the volunteer. This is especially relevant for local volunteers who can follow up on the ground. It is important to note that not all hosts and volunteers can be expected to stay in touch following the assignment, and while continued communication can be encouraged by the program, it should by no means be mandatory. In some cases, 18 additional resources may be necessary to support on-going communication with hosts in remote areas, with limited access to virtual platforms, or in host-volunteer relationships where language barriers exist. 4.2.3 Effectiveness of the Agricultural Volunteer Opportunity Program (including sub￾awards) in Supporting F2F Learning, Innovation, and the CoP FINDINGS AVOP (previously known as the Special Program Support Project [SPSP]) is a service provider to the F2F IPs implemented by Partners of the Americas (POA). Currently in its first phase as AVOP, it was implemented in two previous phases by different implementers under SPSP. AVOP has two major roles: knowledge management and sub-awards management. Knowledge Management Under knowledge management, AVOP supports CoP learning on how it can be more efficient and effective in implementing F2F activities. It seeks to promote learning within the CoP and implement activities between IPs by sharing and discussing issues, ideas, challenges, success stories, and new implementation practices. By doing so, AVOP is flexible and responds to USAID F2F needs when requested. It facilitated CoP committee meetings, organized the FY 2021 Implementers Meeting, and hosted three F2F seminars. According to key informants, AVOP, through a small buy-in of $96,032, played an important role during the USAID F2F staff transition by managing CoP program directors’ meetings during the first two years of this program cycle, in addition to different essential activities answering the needs of IPs and USAID F2F. With the help of IPs, it also supported USAID F2F in finalizing the monitoring database by leading the work to validate the IPs’ data. It collaborated with the National Cooperative Business Association Cooperative League of the USA (NCBA CLUSA) and Winrock International to plan and execute the F2F 35th Anniversary Celebration and its associated social media campaign in December 2020. AVOP also manages the digital media platforms and the F2F website.11 POA developed the new F2F website where improvements were added based on IPs’ suggestions highlighting yearly success stories, photo and video galleries, and promotional and technical resources for F2F and external audiences. The new version of the website included a section for volunteers with information on how to volunteer and interest forms that could be filled in by potential volunteers. This section also contains success stories from volunteers disaggregated by country, IP, and other searchable key words. POA plays an important role in managing and screening the interest forms and sharing potential volunteers with IPs. Despite being active within the CoP through learning activities and being flexible in responding to different requests and needs from USAID F2F and IPs, AVOP faces challenges in managing the data and information received from IPs. Key informants indicated that sharing success stories is important; but the most important purpose for sharing success stories, as described by one of the AVOP managing staff, “[is] to further explore ideas and bring more innovative approaches to implementation, it’s all anecdotal.” The KIIs revealed that without a systematic approach to analyzing data and information shared with CoP, ideas will remain untested for implementation, thus hindering the development of new approaches to bringing innovation within F2F. Another challenge is related to staffing and availability of funds—where IPs continuously ask AVOP to support them in different studies and activities not anticipated before the start of the program, thus putting AVOP in a situation where it cannot afford to answer all requests due to staffing and financial constraints. Sub-Awards Management Like its predecessors, AVOP was designed to be a third-party entity supporting the IPs. The sub-awards program is tailored to small organizations with the goal of bringing fresh, innovative, and non-traditional 11 https://farmer-to-farmer.org. 19 models to development. Under the sub-award component, as shown in the FY 2019 to FY 2021 AVOP annual reports, AVOP delivered capacity-building activities for five sub-awardees: The High Atlas Foundation, Grameen Foundation, University of Tennessee, Purdue University, and Browse & Grass Growers Cooperative (BGGC). AVOP also completed three rounds of Requests for Applications (RFAs) and awarded eight F2F AVOP sub-awards totaling $4.85 million. The sub-awards component faces challenges in the process of receiving applications for the small awards program under AVOP. AVOP issues the RFAs, receives the applications from interested organizations, maintains communication with the potential sub-awardees, and conducts the first eligibility screening and technical review before handing over the applications to USAID F2F for its review. In turn, F2F submits the applications to the respective country Missions for their review. If USAID F2F and/or the country Missions have no objection, the application is approved. POA made numerous improvements to the RFA competition structure during the program’s first two years. AVOP manages the process but concurrence and approval from USAID/Washington and the Missions are needed to advance the application to the next phase. CONCLUSIONS Knowledge management and sub-awards management are two large components managed by a single organization. The tasks allocated under each component are sometimes burdensome when associated with challenges falling outside of the program’s scope. POA has dual roles under this component. Under knowledge management, when possible, POA delivers services requested by both USAID F2F and the IPs, thus transforming it into a third-party service provider. However, under the sub-awards management, POA is an IP managing multiple sub-awards that represent different organizations. Most of the time, discussions between the IPs are anecdotal and not well-documented, tracked, or data-driven, and following up on these discussions and ideas cannot be done if there is no systematic approach for recording, validating, and following up on the information. Despite being supported by different IPs in knowledge management, POA requires time and resources sufficient for the multiple tasks involved with managing both knowledge and sub-awards for the CoP and ad hoc studies requested by USAID F2F. The lack of financial resources and high staff turnover hinder innovation in both components. Moreover, POA is managing different sub-awards for the five organizations responsible for following up with them on their activities and continuously building their capacity, which is a lot for a single organization to manage. Given all this, POA shoulders a considerable workload, potentially affecting the quality of its work and limiting the innovation and testing of implementation models, which could benefit other IPs and F2F in future program phases. The decision-making process of approving the sub-awardee applications, moreover, is bureaucratic, involving multiple levels of review with final approval outside POA’s remit, which limits AVOP’s responsiveness in supporting potential sub-awardees. 4.3 EQ 3: WHAT PROGRESS HAS THE FY 2019 TO FY 2023 F2F PROGRAM MADE TOWARD ACHIEVING ITS OBJECTIVES? 4.3.1 Progress Made Toward Objectives FINDINGS The ET addressed the progress F2F made toward achieving its objectives, as well as the challenges that hindered program implementation in three categories: 1) program management and implementation, 2) volunteer experience, and 3) host achievements. The most significant challenge currently confronting F2F is the COVID-19 pandemic. The impact of COVID-19 and F2F’s response are explored in Section 4.3.2. 20 Program Management and Implementation In addition to COVID-19, country projects faced numerous challenges related to conditions within host countries that adversely affected project implementation, such as civil unrest in Colombia and Armenia and economic and financial crises in Lebanon. However, key informants were largely satisfied with F2F’s adaptive approach as a response to the pandemic and other crises facing F2F countries. Program management (HQ and in-country project management) and field-based IP staff were very complimentary about F2F’s flexible approach, which enabled it to modify program implementation to meet the evolving needs of program stakeholders and respond to country-related crises. As noted by management staff at one IP: “There is … room for flexibility on how to plan and implement, especially the scope of work in terms of duration and content.” The online surveys revealed that 92 percent of the hosts were satisfied with their involvement in F2F and the services they received. F2F aims to build and develop local capacities to be more productive and profitable. To do so, the program takes a bottom-up approach that prioritizes the needs of local host organizations in developing SOWs. Many respondents, including a prevailing number of management and field staff, attributed F2F’s success to its market-driven and needs-based approach. Management staff at one IP said: “The beauty of this program is demand-driven [and] that’s why it worked well, so we are not coming with an idea that we are suggesting to our partner that we think it will work and we are trying to convince them with it.” Host organizations also appreciated that F2F seeks to address their essential needs. One host organization director shared: “The country staff worked with us to better identify our needs and plan the work together as if they were part of our organization.” Key informants generally agreed that F2F is highly responsive and focused on host needs. The participatory process for assignment development is a key component driving program responsiveness. For most IPs, work with a new host begins with an assessment of a host’s needs, which the IP conducts in collaboration with the host. Key informants emphasized that gathering a clear understanding of the capacity, needs, and priorities of a host is very important in developing assignments. Following the initial needs assessment, the IPs begin the process of developing an assignment SOW. Host involvement in assignment and SOW development is strong. Approximately 90 percent of hosts who responded to the online survey said they were “satisfied” or “very satisfied” with their involvement in key assignment phases—including their overall role in the assignment, their involvement in the assignment planning process, and their role in developing the assignment SOW (Table 9). Table 9: Host Satisfaction in Its Involvement in the F2F Volunteer Assignment (N=186) Assignment Phase Very Dissatisfied Dissatisfied Neither Satisfied nor Dissatisfied Satisfied Very Satisfied Don’t Know Overall 7.0% 0.5% 0.5% 32.6% 59.4% 0.0% Assignment planning 5.4% 0.5% 2.8% 38.9% 51.9% 0.5% SOW development 5.9% 0.0% 1.6% 37.6% 52.2% 2.7% Just under 70 percent of hosts responding to the online survey were involved in or led the SOW development from its early stages (Figure 2), including 23.6 percent who developed the first version of the SOW and 45.6 percent who developed the first version of the SOW in collaboration with the IP. 21 Figure 2: Role of Hosts in SOW Development (N=182) Volunteer Experience Volunteers generally reported positive experiences working with the IPs, with more than 87 percent of online respondents indicating that they were “satisfied” or “very satisfied” with the competency of the IP (Figure 3), while nearly 80 percent of the same volunteers said they were very likely to volunteer again. Figure 3: Satisfaction of Volunteers with IP Competency (N=364) Despite the onset of the COVID-19 pandemic and the subsequent shift from in-person to remote support, key informants were highly laudatory of the F2F volunteers, both U.S.-based and local. A prevailing number of hosts appreciated how well the volunteers were chosen and matched with their needs and the SOW. Many hosts were actively involved in volunteer selection and spoke positively about the IPs’ processes for engaging them in the volunteer recruitment process. For many of them, this translated directly to increased sales and income, improved quality, and, in at least one case among interviewed hosts, starting an export line. One host and owner of a swine farm explained: “Our operations in our swine farm were halted by a deadly virus inside the farm, and the volunteer was able to identify it remotely and saved the swine and piglets. Also, he changed the feed ratios and the feeding system, and our piglet grew by weight, all this being done remotely.” Among hosts responding to the online survey, 96 percent stated that the work supported by volunteers significantly improved because of their contributions. According to the IPs, the major volunteer successes were mainly due to well-crafted SOWs, which they described as strategic, considering the present challenges and most pressing needs and what would benefit 3.3% 1.6% 2.2% 6.0% 17.6% 45.6% 23.6% 0% 10% 20% 30% 40% 50% Don't know/Prefer not to say Other Never received SOW Received finalized SOW w/o input IP created first version of SOW Created first version of SOW with IP Created first version SOW 1.9% 3% 8% 30.8% 56.3% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Don't know/prefer not to respond Dissatisfied Neither satisfied nor dissatisfied Satisfied Very satisfied 22 the host organizations in the medium and long term through defined milestones. From the volunteers’ perspectives, maintaining a strong and lasting relationship with the host organizations is a key element in helping them apply the proposed recommendations. As discussed in EQ 2, many volunteers continued communicating with the hosts after the end of their assignments. Approximately 90 percent of hosts responding to the online survey expressed satisfaction with both their U.S.-based and local volunteers, while 99 percent of the hosts felt that their opinions, perspectives, and experiences were valued and respected by the volunteers. The online survey asked volunteers to identify the main challenges they faced completing the deliverables specified in their SOWs. The most frequently cited challenge (by 29 percent of respondents) was technological, e.g., poor internet or mobile phone connectivity, followed by diverse other challenges (21 percent), language barriers (16 percent), poor host responsiveness and communication (9 percent), and several additional challenges mentioned by fewer than 10 percent of respondents (Figure 4). Figure 4: Main Challenges Faced by Volunteers (N=358)12 According to the online survey results, volunteers were tasked with completing, on average, 3.2 deliverables per assignment. Two-thirds of online survey respondents said they completed 100 percent of their deliverables, and 17.8 percent said they completed 76 percent to 99 percent of their deliverables. For those who failed to complete all their deliverables, the primary reason given was that the length of time for the assignment was too short. This was followed, at a distance, by a lack of program or IP follow￾up, activities completed that were not in the SOW, deliverables that were not clearly defined in the SOW, project activities that were delayed, and a lack of program or IP support (Figure 5). 12 The survey allowed multiple responses, such that responses may total more than 100 percent. 0% 0% 3% 3% 7% 8% 8% 9% 10% 16% 21% 29% 0% 5% 10% 15% 20% 25% 30% 35% Cultural barrier Project outside expertise Prefer not to say Recruitment poorly organized Expectations unclear Poor communication from IP Difficulty finding solutions Poor communication from host Lack responsiveness host Language barrier Other challenges Technology barriers 23 Figure 5: Main Reasons for Not Completing All SOW Deliverables (N=102)13 In support of the online survey findings, both volunteers and hosts indicated the assignment duration— and the time required to plan, prepare, implement, and follow up—can be short compared to the SOW’s specified tasks and deliverables. Many volunteers expressed the need to expand the volunteering assignment time. According to these volunteers, a longer assignment duration would provide better insights for the volunteer to prepare the assignment, get to know and understand the SOW, get to know the host, and do a better follow-up on the recommendations. One U.S.-based volunteer said: “It [the duration of the assignment] was always short and tight, volunteers should spend more time with the hosts to get to know them and better understand the assignment for a better impact, in two weeks you just work for five days.” Host Achievements F2F hosts generally had high capacity and were well organized, contributing to the successes they achieved. IPs worked on profiling the hosts and choosing those with the most pressing needs yet good potential. The initial assessment was done either using the ODI alone or the ODI coupled with other IP-related needs assessment forms, depending on the type of country project. Host technical expertise and know￾how proved to be essential, beginning with SOW design and extending through implementation of project activities. Selecting hosts with sufficient capacity was even more important given that volunteer assignments are relatively short and there is limited time to effect meaningful changes. Repeat volunteers who have worked with F2F over time noted that the program used to focus mostly on production-related assignments, targeting hosts with a lower level of capacity and limited access to resources. Volunteers note that the program’s shift to more rigorous vetting of hosts with a higher capacity level allowed the program to operate successfully across a broad range of technical support categories, such as business development and strategic planning. To a lesser extent, several KII respondents—mostly IP management staff—indicated that working with the same host more than once is beneficial and could help them to better implement the recommendations from previous assignments and build on them “since they already know our systems, [and] it would be much easier and efficient to support them.” 13 The survey allowed multiple responses, such that responses may total more than 100 percent. 24.5% 19.6% 7.8% 8.8% 11.8% 13.7% 17.6% 30.4% 0% 5% 10% 15% 20% 25% 30% 35% No factors Other Lack of program/IP support Project ativities delayed Deliverables not clearly defined Activities not in SOW Lack of program/IP follow-up Assignment length too short 24 Overall, hosts were satisfied with the volunteers’ recommendations, a finding confirmed in interviews with host organizations and in the online survey where 95 percent of the hosts expressed satisfaction with volunteer recommendations. The online survey revealed that 92 percent of hosts were satisfied with their involvement in all the activities and services received through F2F, and 94 percent were willing to host volunteers again. In particular, the online survey indicated that 91.4 percent of hosts were satisfied with their involvement in the process of developing the plan of support, including identifying goals and expected results. According to hosts responding to the online survey, the most common recommendations (84.2 percent) were technical, followed at a distance by economic-based recommendations at 25.0 percent (Figure 6). (Note that hosts might receive different types of recommendations from a single assignment.) Figure 6: Type of Recommendations Received from Volunteer (N=184)14 The online survey also revealed that 95 percent of the hosts were satisfied with the recommendations made by the volunteers, whereas almost 85 percent were satisfied with their ability to follow up with the volunteers after the end of the assignment. When asked if they implemented the proposed recommendations, 67.8 percent of the hosts declared that they were applied to a good or great extent. In response to an online survey asking the hosts to indicate the factors limiting their implementation of recommendations, lack of financial resources was the most-cited inhibiting factor at 48.3 percent, followed by the negative impact of COVID-19 at 35 percent (Figure 7). Overall, hosts indicated that their work was significantly improved by the work of the volunteers, with a 95 percent approval ratio in the online survey. 14 The survey allowed multiple responses, such that responses may total more than 100 percent. 84.2% 25.0% 19.6% 17.4% 2.2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Technical recommendation Economic recommendation Financial recommendation Environmental recommendation Other recommendation 25 Figure 7: Factors Limiting Implementation of Recommendations Received from Volunteer (N=180)15 When asked in the online survey what challenges hosts faced working with their F2F volunteers, 49.5 percent said they experienced no challenges at all, 21.7 percent cited technological problems related to poor internet or mobile phone connectivity, 11.1 percent cited time differences, and 10.6 percent cited language barriers (Figure 8). As noted by one IP: “The priority was to move to remote assignments, and we already knew that the internet connection is already weak, so we diversified the communication platforms to have multiple options to keep up the communication.” In many countries like Morocco, Nigeria, Philippines, Guatemala, Peru, Ecuador, and Georgia, it was particularly difficult to communicate with hosts who live in mountainous and remote regions. Figure 8: Main Challenges Faced by Hosts in Working with Volunteers (N=198)16 15 The survey allowed multiple responses, such that responses may total more than 100 percent. 16 Figure 8 shows the main challenges identified by hosts not all challenges, such that the responses do not add up to 100 percent. 0.6% 2.2% 2.2% 8.9% 9.4% 12.8% 13.9% 17.2% 35.0% 48.3% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% Legal Restrictions Low priority Other limitations Lack of time Volunteers recently concluded, not enough… Local tech, financial, market Lack Local Access Goods/Services None, no difficulties COVID-19 Lack Financial Resources 49.5% 21.7% 11.1% 10.6% 0% 10% 20% 30% 40% 50% 60% No challenges Technology barriers Language barriers Time differences 26 Another challenge noted in KIIs, particularly among volunteers working in Central America and Africa, was the low literacy of certain hosts, which adversely affected the level of interaction and the transfer of information between the volunteers and hosts. The major issue was that it made it more difficult for volunteers and IPs to convince the farmers to adopt the volunteer’s recommendations. Hosts indicated that lack of working capital and limited access to financial resources were the main challenges they faced in adopting recommendations that specifically required them to purchase new equipment, adopt new technologies, or start a new production line. According to one host working in food processing: “We were able to apply most of the recommendations not requiring purchasing or financing, and unable to apply the recommendations that required purchasing new equipment for laboratory tests since they are expensive, and the program can’t help.” CONCLUSIONS The core of F2F’s success as a needs-based, host-driven program is its highly participatory approach to assignment development. IPs conduct thorough needs assessments with new hosts to identify technical and capacity gaps and define areas for F2F support. Hosts are involved in assignment design from the earliest stages of SOW development. F2F’s commitment to careful and participatory assignment design and tailor-made support is central to the program’s value and sets it apart among development programs. F2F assignments are often constrained by time limitations, notwithstanding hosts’ desires for longer assignments to maximize the program’s impact. F2F’s approach to assignment design also makes the program highly flexible, responsive, and adaptive, allowing for rapid adjustments in scope of support to meet the changing needs of hosts or respond to local challenges and crises. Careful and strategic host selection is critical to program success. IPs focus on selecting hosts with sufficient existing capacity to benefit from the program, allowing the program to successfully provide support across a broader range of technical areas related to business development and strategic planning. Working with the same host over multiple assignments maximizes the impact of the program and leverages the strength of relationships built between hosts, IPs, and volunteers. Hosts are highly satisfied with volunteers, and volunteers are well matched to assignments. In some cases, hosts were actively involved in the volunteer recruitment process. The recommendations generated by F2F assignments are well received by hosts, who find them valuable, relevant, and realistic to implement. Challenges facing hosts include low literacy, lack of working capital and access to financing to implement recommendations, and technology and network-related issues that make remote communication difficult. Sufficient resources must be provided to IPs to allow for: 1) strong investment in needs assessments driving assignment design; and 2) debrief sessions to facilitate learning from hosts and volunteers. IPs, in turn, must allocate sufficient time and resources to needs assessments and debrief sessions to fully maximize program potential and responsiveness. 4.3.2 Impact of the COVID-19 Pandemic on Implementation and Achieving Objectives FINDINGS COVID-19 Challenges The most significant challenge confronting F2F in its present program cycle is the COVID-19 pandemic. F2F’s core activity of fielding on-site volunteers was postponed in most country projects. The travel restrictions, internal country curfews, and other types of restrictions limited person-to-person interactions and imposed new norms of remote communication. Key informants uniformly agreed that COVID-19 delayed program implementation, while international and internal travel restrictions limited the mobility of volunteers and local in-country staff. The IPs’ FY 2020 and FY 2021 annual reports likewise 27 noted that delays in fielding volunteer assignments made them unable to achieve their annual volunteer assignment targets. In the online survey, almost 81 percent of hosts indicated that SOWs were affected by the pandemic, with 41 percent of hosts indicating that SOWs were significantly affected. The IPs’ M&E staff further reported that it was almost impossible in certain cases to collect data as was previously done on-site and in-person. As a result, they had to move to new types of communication and data collection methods with the hosts: “We customized our data collection tools to be more user-friendly with the remote nature of data collection, while preserving the quality of the collected data.” Several IPs faced pandemic-imposed difficulties in conducting daily remote work tasks where in-country staff numbers were already limited. With the start of the pandemic and the transition to remote work, additional responsibilities were handed to the staff like finding and recruiting local volunteers, maintaining communication with hosts, and following up with the U.S.-based volunteers, sometimes outside working hours. As noted by a manager at an IP: “We were short on staff before the pandemic, and afterwards we had additional responsibilities, so the staff felt overwhelmed, and we couldn’t bring more people on board.” Several key informants pointed out that the pandemic negatively impacted the daily business and supply chain operations of host organizations. Travel restrictions imposed by local authorities disrupted transport and working capital access, leading to lower sales and exports. IPs indicated that small-farmer hosts were worried about the remote approach adopted by F2F, principally whether they would receive the support they required during the pandemic. But those who received remote volunteers were happy to receive at least some assistance, as IP management staff acknowledged: “Though they understand the major concerns related to the pandemic, many of the hosts were reluctant to proceed with the activities and preferred in-person assignments, whereas others who received remote assignments were happy about them.” Implementation Response to COVID-19 Despite the pause in implementation at the start of the pandemic, respondents noted how program management in Washington, D.C. and at the IPs’ HQs worked quickly to modify their initial approach by shifting toward more remote support. According to management staff at one IP: “After the two months break, we came up with a new structure composed of paired assignments and remote assignments, and so … it really started to evolve, and we got USAID/Washington approval for paired assignments.” Notwithstanding COVID-related program disruptions, F2F was effective in shifting its implementation approach to address the needs of host organizations. Most of those interviewed felt that the program was successful during the pandemic in maintaining person￾to-person interactions and the solidarity between the different program actors through the cultural exchange between U.S.-based volunteers, local volunteers, and in-country staff, in addition to building local partnerships with supporting institutions and other actors within the specific value chain. One IP manager said: “I think the biggest success is the reputation, the linkages we have succeeded to set up with many markets and sub sectors, now we have relationships and linkages with all of those sectors like seafood, meat, dairy, chicken slaughterhouses, in addition to the governmental organizations.” Among interviewees, several noted that it was initially challenging to build rapport between host organizations and volunteers, but the commitment shown by the different program actors helped alleviate challenges the pandemic imposed on the program. Field staff at one IP noted: “During the pandemic, we knew that the remote communication would be challenging, but the efforts of the hosts and volunteers, together, helped us a lot in restarting our activities.” Despite the multiple challenges that the COVID-19 pandemic imposed on F2F, the IPs were proactive in finding solutions to respond to these challenges and implementing mitigation measures and alternative tools to ensure full and satisfactory service delivery. The online survey indicated that 62 percent of the volunteers and 75 percent of the hosts were satisfied with the program’s COVID-19 mitigation measures. New approaches for implementation were discussed between the IPs and submitted to 28 USAID/Washington for approval. As a result, remote assignments and paired volunteering mechanisms were initiated and new methods for implementation were put into action. This change in the program implementation approach and new system of work increased the IPs’ internal organizational capacities. It was a learning phase for each IP in that the remote implementation approach had to be launched without pilot testing. Many IPs adopted a learning and adaptive management approach to facilitate implementation and test different hypotheses. Many IPs revealed that it was very challenging at first, but they adapted after making adjustments. According to M&E staff at one IP, “We had like a plan of what will do, what are the signs that are coming to key vision, but everything changed. It was hard at first, but you learn how to adapt quickly and how to find solutions.” It was imperative for the in-country staff to take over the role of the HQ staff in recruiting local volunteers, and, as a result, they were trained on the recruitment process. Support was also given to hosts on how to use remote communication applications like Zoom, Skype, WhatsApp, and others. As a result, the follow-up process during SOW preparation, remote training, and the post-training phases were facilitated with the communication software. As many hosts and IP staff indicated, there was a stronger sense of ownership and belonging to the F2F program because of the communication that was built between the different parties. A Central Asia host said: “Compared to the previous years, the pandemic made us closer, it is a global threat, so we were checking on the staff, reaching out to them whenever needed, we felt close even we were not seeing each other in person.” CONCLUSIONS F2F’s ability to survive during the pandemic was largely due to its core strategy of being a needs-based and flexible program. F2F started its FY 2019 to FY 2023 implementation phase with most IPs meeting the initial input and output indicators for the first year. Then the COVID-19 pandemic started in mid-FY 2020, requiring adaptations to program delivery plans and implementation by moving to remote activities and assignments. With the support of USAID/Washington, the IPs were proactive in shifting from fielding in￾person volunteers to remote assignments and paired volunteering with local volunteers. IPs played a significant role in following up with hosts during the pandemic. This shift proved to be difficult for most IPs since new responsibilities emerged that were not envisaged, like finding and recruiting local volunteers, working remotely made more challenging at times by poor internet connectivity, working under a crisis mode, and working remotely with hosts with a limited number of staff. For FYs 2020 and 2021, most IPs did not achieve their initial targets of fielding volunteers, yet they improved their implementation activities by enhancing the paired assignments, working closely with selected hosts, monitoring hosts’ most pressing needs, and communicating with other IPs. Overall, F2F stakeholders were satisfied with the flexible and adaptive approaches taken, enabling them to modify their programming to meet the needs of the COVID-19 crisis. Although the strain from COVID￾19 continues, the program was effective in shifting its implementation and delivery program strategy to address the changing needs of hosts, thus keeping the program on track to achieve its objectives. One of the program’s biggest achievements for this implementation phase was its ability to resume core project activities and launch the remote and paired assignments quickly on the heels of the COVID-19 breakout. This achievement was largely due to the IPs’ adaptive management approach facilitated by USAID/Washington. F2F’s market demand-driven nature contributed to overcoming different hurdles arising from COVID-19. IPs maintained a close relationship with the hosts to monitor their needs, design well-crafted, strategic SOWs, and follow up on volunteers’ recommendations. This participatory approach created a sense of ownership and belonging by both hosts and volunteers (U.S. and local) and increased human-to-human interaction and communication during the challenging times of the pandemic, despite being mostly remote. Such a participatory approach would not have succeeded if the hosts and volunteers were not chosen based on detailed profiling. 29 4.3.3 Alternative Volunteering Mechanisms FINDINGS When the COVID-19 pandemic shut down most of the world in March 2020, the F2F program came to a near standstill. With restrictions on international travel, it became impossible for U.S.-based volunteers to travel to conduct assignments. In response, the IPs began to conduct assignments remotely, connecting U.S.-based volunteers to host organizations through virtual platforms. These fully remote assignments, while providing important support to hosts, did not count toward volunteer assignment targets per USAID’s guidance. In August 2020, IPs requested and received permission from USAID to conduct paired remote assignments. Paired remote assignments leveraged the dual support of a local expert providing in￾person volunteer assistance to hosts and a U.S.-based volunteer providing remote support. Besides remote U.S. volunteer assignments and paired remote assignments, the program also implemented local volunteer assignments, where non-U.S. local experts direct supported host organizations. Currently, only volunteer assignments delivered in person (whether by a local volunteer paired with a remote U.S. volunteer or an in-person U.S. volunteer) are counted toward IP volunteer targets (Table 10). Table 10: F2F Volunteering Models Assignment Model Mode of Assistance Assignment Counted Toward Volunteer Targets In-person U.S. volunteer Assistance delivered in person in the host country by U.S. volunteer Yes In-person local volunteer Assistance delivered by local volunteer No Paired remote Assistance delivered collaboratively—in person in the host country by local volunteer and with remote support by U.S. volunteer The local volunteer providing in￾person assistance counts toward volunteer target; the remote U.S. volunteer does not Remote U.S. volunteer Assistance delivered remotely by U.S. volunteer No The response to alternative volunteering models among key informants was strongly positive, particularly among IPs and host organizations. Before the innovation of paired remote volunteering, the IPs recalled a challenging time when the operations were paralyzed, and they worried that the program might not be able to continue. Alternative modes of volunteering allowed IPs to continue to provide much-needed support to the hosts despite the difficult pandemic conditions. IP respondents noted that paired volunteering allowed IPs to get back on track to meet targets for volunteer assignments. Most of the IPs, hosts, and local volunteers interviewed recommended continuing with the hybrid approach of paired remote, remote, and in-person assistance once the pandemic ends. The IPs see great value in investing in local expertise. Local volunteers are highly valued by IPs, hosts, and their paired U.S. counterparts. Hosts and IPs noted that local volunteers have a stronger understanding of the in-country context, which is beneficial to their assignment and contributes to developing relevant and achievable recommendations. They were also often able to complete the orientation process more efficiently, uninhibited by the barriers of jet lag, and had a strong base understanding of the local context and challenges. They were thus able to dedicate more of the time allocated for the assignment to supporting the host. According to IPs and hosts, hosts generally have a high level of comfort with local volunteers, while local volunteers reduce the challenges associated with language barriers. The advantage of leveraging the expertise of local volunteers extends beyond the assignment; local volunteers can maintain relationships and communication with hosts after the assignment is completed and follow up in person to provide ongoing support as hosts work to implement recommendations. Also, local volunteers are better able to 30 replicate the assistance provided to hosts with other organizations in the region. Therefore, investing in a robust network of local expertise has the potential to expand the program’s impact. Hosts were highly satisfied with the support provided by both local and U.S. volunteers. Among online survey respondents, 89 percent of hosts reported being “satisfied” or “very satisfied” with the U.S. volunteers, with 53 percent being “very satisfied,” while 90 percent of hosts reported being “satisfied” or “very satisfied” with the local volunteers, with 59 percent being “very satisfied” (Figure 9). Figure 9: Host Satisfaction with Volunteers As a result of widespread use of virtual platforms to connect hosts and volunteers operating under the remote and paired remote models, respondents found that communication and coordination between the hosts and volunteers leading up to the start of the assignment improved. Volunteers emphasized the value of receiving as much information as possible from hosts before the assignment began to optimize the design of their support and maximize assignment efficiency and effectiveness. The paired volunteering mechanism posed challenges and opportunities for volunteer recruitment. The IPs sometimes experienced challenges recruiting local experts, particularly in countries where there is not a strong established culture of volunteerism. In-country teams received support from their HQ counterparts on volunteer recruitment and worked to 1) introduce and promote a culture of volunteerism in countries where volunteerism is not prevalent; and 2) recruit local experts specializing in areas relevant to F2F SOWs. The IPs reported variable experiences regarding local volunteer recruitment. While recruitment was not a notable challenge for some, IPs in countries without an established culture of volunteerism faced difficulties identifying local experts with an interest in volunteering pro bono. Similarly, while for some IPs identifying local subject matter experts was not a notable challenge, others struggled to identify local experts specializing in relevant technical areas. As one management team member explained: “It is difficult to find local experts here, many thought it is a paid job and they could not understand that we only want volunteers.” IPs also noted that it was challenging to recruit and retain repeat U.S. volunteers for remote assignments who had previously provided in-person assistance. A minority of IPs stressed the recruitment burden of paired assignments, which require double the recruitment effort. However, these were offset by several clear recruitment benefits to the paired and remote assistance models. Reported benefits of remote assignments were: 1) increased diversity of the U.S. volunteer demographic; 2) increased participation of first-time, youth, and women volunteers because travel and time off was a 0% 20% 40% 60% 80% 100% US Volunteer (n=162) Local Volunteer (n=108) Percentage of Hosts Don't know/prefer not to respond Very dissatisfied Dissatisfied Neither satisfied nor dissatisfied Satisfied Very satisfied 31 barrier to participation for volunteers in the past;17 and 3) simplified recruiting by sidestepping the challenges posed by international travel logistics and the volunteer’s availability to dedicate time for international travel. The 2017 F2F Midterm Evaluation also highlighted this challenge, noting that it could be difficult to recruit highly qualified volunteer consultants who were also able to set aside time to travel. “Whatever you set your mind to, no matter the difficulties, no matter the obstacles, thousands of miles and hours of differences, you can get it done if you commit to it, and if you have a group of people who have that same vision.” – U.S. volunteer, on remote volunteering IPs and volunteers observed that some repeat U.S. volunteers who delivered in-person assistance in the past either struggled to adjust to remote volunteering or expressed strong preference for the in-person experience. For example, a manager at one IP remarked: “They [volunteers] got used [to] the F2F system and [having] suddenly to change it felt like it is a new program for them, but they managed it at the end.” Volunteers participating in remote assistance highlighted several related challenges. To begin with, relationship￾building and establishing trust with hosts was more difficult through remote platforms. In some cases, being unable to collect data in person posed a challenge. Remote volunteers sometimes found it difficult to dedicate sufficient time to the assignment around other commitments. Some volunteers experienced technology- and network-related challenges, while others described feeling disconnected from the work, unaware of outcomes, or feeling like "information machines" responsible for producing materials or composing reports rather than being active participants. Nonetheless, U.S. volunteers overall reported a positive experience with remote volunteering. Among online survey respondents, local and U.S.-based volunteers reported satisfaction with different aspects of their paired remote assignment; 74 percent of U.S. and local volunteers were “satisfied” or “very satisfied” with the paired remote volunteering mechanism, 88 percent of U.S. volunteers were “satisfied” or “very satisfied” with the competency of their paired counterpart, and 90 percent of U.S. volunteers were “satisfied” or “very satisfied” with the quality of their paired counterpart’s deliverables (Table 11). Table 11: Volunteer Satisfaction with Paired Remote Volunteer Mechanism Aspect of Paired Remote Assignment Very Dissatisfied Dissatisfied Neither Satisfied nor Dissatisfied Satisfied Very Satisfied Don't Know Paired remote mechanism (N=333) 0.0% 14.2% 11.9% 29.0% 44.9% 0.0% Competency of paired volunteer (N=175) 0.0% 3.4% 6.9% 28.6% 58.9% 2.3% Quality of paired assignment deliverables (N=164) 0.0% 0.6% 9.1% 38.4% 51.9% 0.0% When asked what challenges they faced working with their paired local counterparts, U.S.-based volunteers listed, in order of importance, technological difficulties (e.g., poor internet or mobile phone connectivity or bandwidth), time zone differences, work planning difficulties, poor counterpart communications, language barriers, and “other” challenges (Figure 10). 17 This hypothesis could not be verified against IP volunteer records because IPs are not currently required to report on the U.S. volunteers who participate in remote or paired remote volunteer assignments. This is discussed further in Section 4.3.4. 32 Figure 10: Volunteer Challenges Working with Paired Counterpart (N=171)18 The paired remote volunteering mechanism provided new opportunities for cultural exchanges between local and U.S.-based volunteers. Key informants noted that professional and cultural exchanges between volunteers were mutually beneficial. Communication between paired volunteers was strong, and volunteers felt that their inputs were valued by their counterparts. Many respondents further noted that the opportunity to volunteer yielded positive benefits for local volunteers and that remote assistance has an important role to play in the program and should be promoted as an alternative method of providing support in certain situations. Moreover, fully virtual trainings are successful, can extend the reach of assignments to more individuals, and can be leveraged into documented/recorded training materials which can be disseminated to larger audiences. Certain areas of assistance are well suited to remote and virtual platforms. and in these cases, remote assistance is a strong option and should be utilized when appropriate. Examples include assignments where volunteers do virtual tours of a facility with the host and make recommendations about machinery to purchase, assignments where hosts can access trainings on technical subjects from highly skilled professionals who are unable to deliver trainings in person, and assignments in which remote training of local trainers is followed by in-person training delivered by the local trainees. At the same time, key informants added that other areas of assistance are better suited to in-person support, e.g., pest management trainings and assignments focused on food safety. One U.S.-based volunteer said: “Some activities could be done remotely like a marketing plan or a business plan, but to check the food processing unit and come up with recommendations, you definitely need to be on site.” Several volunteers observed that in-person assignments can be more effective in transferring knowledge and skills and building host capacity than the remote assignments, but the ability to incorporate support from local volunteers in paired remote assignments mitigated this challenge. Finally, several IPs emphasized that building program capacity in remote assistance supports program resilience by allowing assistance to continue when travel to a region is impossible due to difficult conditions. Respondents felt that this was of great importance. “One of the greatest lessons [of the COVID-19 pandemic] is that building resilience also is a critical aspect of programming. We cannot talk about development if we do not have strategies for coping in case something like this happens.” – F2F IP19 CONCLUSIONS There is high value in alternative volunteering models developed in response to the pandemic. Alternative models should be retained in some form after travel restrictions are reduced. Paired remote and remote volunteering improves the program by making it possible to continue assisting host organizations even 18 The survey allowed multiple responses, such that responses may total more than 100 percent. 19 In this context, “resilience” refers to the ability to continue to assist when travel is restricted. 0.6% 1.2% 9.4% 11.1% 21.1% 33.3% 36.8% 45.6% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% No challenges Cultural barriers Language barriers Poor communication from counterpart Workplanning Other challenge Time zone differences Technical difficulties 33 when travel from the U.S. is not possible. Continued investment in the remote, paired remote, and local in-person volunteering models improves the ability of the program to cope with future challenges that may inhibit travel. Hosts are well supported by the paired remote volunteering mechanism and highly value both local and U.S. volunteers. Hosts also will be well served by continuing alternative volunteering models. The use of virtual platforms for communication improved coordination between hosts and volunteers in the preparation stages of the assignment. The ability to complete a more thorough orientation and planning process before the assignment began allowed hosts and volunteers to use the limited time available for the assignment more effectively. Going forward, the use of virtual platforms to complete orientation and any aspects of the assignment that can be completed remotely will provide an opportunity to shorten the duration of in-person assistance, thereby reducing costs and reserving resources for additional assignments. Alternatively, the duration of assignments could remain the same, with volunteers able to accomplish more in the time allocated for in-person assistance. In this way, the use of hybrid volunteer models that incorporate remote and in-person assistance will address the issue of limited time, a challenge commonly cited by volunteers and hosts alike. Remote assistance will also expand the pool of support available to hosts to include experts who are unable or unwilling to dedicate the time to travel. In some cases, this will mean that hosts can access a higher level of expertise. The transition to paired remote and remote assignments came with challenges and opportunities for IPs regarding recruitment. They sometimes struggled to recruit local experts, especially in countries where there is not a well-established culture of volunteerism. It can also be challenging to recruit and retain U.S. volunteers for remote assignments. Repeat recruitment of U.S. volunteers who had previously provided in-person assistance was sometimes difficult because of lower interest or difficulty adapting to remote assignments. However, the use of remote assignments increased participation of first-time U.S.-based volunteers for whom travel may have previously been a barrier. Remote volunteering opportunities also supported increased diversity in U.S. volunteer demographics, including increasing participation by women, youth, and racial and ethnic minorities. Local volunteers support hosts with strong understanding of local context, which gives hosts a high level of comfort with the volunteer and contributes to relevant and achievable recommendations. Local volunteers also reduce the challenges associated with language barriers. Local volunteers are often easier for hosts to maintain contact with after the assignment ends, and they are available to follow up in person, which facilitates support to hosts to implement recommendations and, ultimately, sustain program results by building a network of in-country support. Local volunteers can be more efficient in assignment start￾up because there is no adjustment required after international travel, and an understanding of the local context speeds up the orientation process. Also, the opportunity to volunteer can yield positive benefits/impacts for local volunteers. Remote U.S. volunteers experienced challenges adapting to remote assistance, including feeling disconnected from the work or unaware of the outcomes. Remote assistance also presented challenges to collecting data or building relationships with hosts from a distance. Remote U.S. volunteers also found it challenging to dedicate sufficient time to assignments while still in the U.S. due to work distractions and other commitments. However, they still felt that the assignments were beneficial and would continue to volunteer remotely in the future. Valuable cultural and professional exchanges took place between local and U.S. volunteers in the paired remote assignment model. This is of mutual benefit to both U.S. and local volunteers and serves to support F2F's mission of cultural exchange. Communication between volunteers and their paired counterparts tends to be strong, while volunteers generally feel valued by their counterparts. 34 Fully remote assistance where a volunteer engages remotely with the host in the absence of in-person support from F2F is a valuable mode of service delivery for the IPs that contributes to program resilience. While remote assistance may not be suited to all areas of support, it can be very effective for many. Remote and virtual modes of assistance can expand the program’s reach to more individuals and support resilience during crises that make travel difficult. Remote and virtual trainings can also be easily leveraged as learning materials for broader dissemination. Alternative volunteering models are a key innovation by the F2F program and have been a driver of program success over the past two years. Despite the demonstrated value and success of local experts in volunteer positions and the success of fully remote assignments, to date F2F has not: 1) counted assistance provided independently by a local expert toward assignment targets; nor 2) counted fully remote assistance toward assignment targets. To continue to encourage and support the use of innovative assignment models, it will be necessary for F2F to develop clear criteria and pathways through which IPs may count assignments completed under alternative models toward volunteer targets. 4.3.4 Diversity and Inclusion FINDINGS The ET examined two aspects of diversity and inclusion in the F2F program: 1) the extent to which the program achieved its volunteer recruitment diversity objectives; and 2) progress toward objectives of greater inclusion of women and youth in F2F-supported agricultural sectors. Volunteer Recruitment The primary sources of findings related to volunteer recruitment diversity objectives were F2F’s FY 2019 through FY 2021 Annual Report Consolidated Indicator Tables, the online survey of volunteers, and insights from KIIs with IPs and volunteers. Unfortunately for this evaluation, data on U.S. remote or U.S. paired remote assignments was not requested. As a result, it is not possible to obtain a complete picture of progress toward volunteer diversity recruitment objectives with the available data. While the number of women volunteers increased from 97 in 2019 to 106 in 2020 and to 124 in 2021, the percentage of female volunteers fell from 32 percent in both FY 2019 and FY 2020 to 27 percent in FY 2021 (Table 12). It is important to note, however, that this data only includes volunteers who provided in-person assistance. Without a full dataset that includes demographic data on volunteers across remote and paired remote assignments in addition to in-person assignments, it was not possible to ascertain the impact of the transition to remote and paired remote volunteering on volunteer demographics. As F2F transitioned to fully remote and paired-remote volunteer models, the percentage of first-time volunteers rose from 27 percent in FY 2019 to 44 percent in FY 2020 and 69 percent in FY 2021. The percentage of local volunteers surged from 0 percent in FY 2019 to 15 percent in FY 2020 and 86 percent in FY 2021. The rise of first-time F2F volunteers in FY 2021 and, to a lesser extent, FY 2020 can likely be explained by the inclusion of local volunteers in the program. However, without a complete dataset accounting for U.S. volunteers participating in remote and paired remote assignments, it was not possible to draw conclusions about changes in first-time volunteering among U.S. volunteers. Individuals who self-identify as black, indigenous, or people of color comprised 31 percent of volunteers in FY 2019, 35 percent in FY 2020 (like the demographics of volunteers who participated in the previous program cycle), and 82 percent in FY 2021. Black volunteers made up the largest racial group in the FY 2021 volunteer population, comprising 42 percent of the volunteer base. The rapid rise in racial and ethnic diversity of F2F volunteers in FY 2021 and, to a lesser extent, FY 2020 can likely be explained by the inclusion of local volunteers in the program. However, without a complete dataset accounting for U.S. volunteers participating in remote and paired remote assignments, it is not possible to draw conclusions about changes in the demographics of U.S. volunteer populations. F2F IPs do not report on the age of volunteers, making it hard to quantitatively assess the inclusion of youth in the F2F volunteer program. 35 Table 12: Demographics of Volunteers for FY 2019, FY 2020, and FY 2021 Demographic Characteristic FY 2019 FY 2020 FY 2021 # % # % # % Total volunteers 301 - 335 - 463 - Sex Female 97 32% 106 32% 124 27% Male 204 68% 229 68% 339 73% Race White/Not Hispanic or Latino 206 68% 209 62% 60 13% White/Hispanic or Latino 32 11% 29 9% 14 3% Black or African American/Not Hispanic or Latino 26 9% 37 11% 193 42% Black or African American/Hispanic or Latino 4 1% 2 1% 37 8% Asian 18 6% 32 10% 64 14% Any other race or multi-race 9 3% 15 4% 70 15% American Indian or Alaskan Native 2 1% 0 0% 0 0% Native Hawaiian or Other Pacific Islander 0 0% 0 0% 1 0% Decline to share race/ethnicity 4 1% 11 3% 24 5% Prior F2F Service Prior F2F 220 73% 187 56% 145 31% First time 81 27% 148 44% 318 69% Type of Volunteer Local volunteer 0 0% 49 15% 397 86% U.S. volunteer 301 100% 286 85% 66 14% Source: F2F FY 2019, FY 2020, and FY 2021 Annual Report Consolidated Indicator Tables The indicator tables have both gender and occupational distribution for F2F volunteers, if requested. This disaggregation was not done for the evaluation, but according to the online survey results (which are not necessarily representative of the volunteer population), volunteers were drawn predominantly from private enterprise, cooperatives, or associations, nonprofit or non-governmental organizations (NGOs), education, and retirees (Figure 11). The results held largely true for both male and female volunteers, although a higher share of the former were drawn from education and retirement, and a higher share of the latter were drawn from private enterprise, cooperatives/associations, or NGOs. Private enterprise employment has enabled volunteers to assist hosts in organizational development, business planning, and operations. Cooperatives and associations were more common in F2F host countries engaged in agricultural production and marketing and may reflect local volunteers' occupations. NGOs and other nonprofits are found both in the U.S. and host countries and are frequent implementers and advocates of international development programs. 36 Figure 11: Primary Place of Employment During Volunteer Experience When the online survey asked volunteers why they volunteered for an F2F assignment, 50 percent or more of respondents cited (in descending order of importance) an interest in supporting host organizations and countries, a desire to improve their personal knowledge and skills, and an interest in giving back. One-third cited an interest in cultural exchanges, about one-quarter cited a desire to travel, and another one-fifth said they had previously either been an F2F volunteer or done something similar (Figure 12). Reasons for volunteering were similar for male and female respondents. Eighty (80) percent of both male and female volunteer respondents said they were “very likely” to volunteer again, and another 11 to 12 percent, respectively, said they were “likely” to volunteer again. 1% 1% 4% 7% 5% 6% 17% 1% 15% 10% 15% 17% 0% 0% 2% 3% 4% 5% 7% 9% 10% 13% 22% 25% 0% 10% 20% 30% Don't know/Prefer not to say Rural financial institution Financial Institution Individual private farmer Other Public sector/Government Retired Student Public/private education Non-profit/NGO Cooperative/Association Other private enterprise Women (n=117) Men (n=247) 37 Figure 12: Reasons for Volunteering with F2F20 According to online survey results, F2F volunteers first heard about F2F from a variety of sources, the largest of which was an F2F IP followed by a former F2F volunteer, the F2F website, a program partner or stakeholder, social media, listserve or email, or an F2F event (Figure 13). The responses were broadly similar among male and female respondents with males being marginally more likely to have heard about F2F from its website, program partners, stakeholders, social media, or other sources and less likely to have heard about F2F from IPs and former volunteers. Figure 13: How First Heard About F2F Women and Youth Inclusion in F2F The second portion of the ET’s diversity and inclusion investigation focused on progress toward inclusion of women and youth in F2F-supported agricultural sectors. The ET identified key findings from KIIs with the IPs, volunteers, and hosts about IPs’ strategic targeting of hosts, the focus on youth hosts to support youth inclusion, and particular challenges facing women and youth in the countries where F2F operates. 20 The survey allowed multiple responses, such that responses may total more than 100 percent. 2% 11% 11% 23% 20% 21% 33% 59% 64% 66% 6% 6% 9% 17% 21% 26% 34% 52% 61% 63% 0% 20% 40% 60% 80% Other reasons Personal knew other volunteers Personal relationships with F2F IPs Previous non-F2F experience Volunteered previously Desire to travel Interested in Cultural exchange Interested giving back Desire to improve personal knowledge/skills Interested in supporting host & countries Women (n=117) Men (n=247) 3% 12% 4% 4% 11% 9% 13% 18% 26% 3% 10% 3% 4% 7% 7% 11% 24% 31% 0% 5% 10% 15% 20% 25% 30% 35% Prefer not to respond Other F2F event Listserve or email Social media Program partner or stakeholder Website Former F2F volunteer F2F IP Women (n=117) Men (n=247) 38 The ET found that IPs target hosts with a focus on women and/or youth. It is included as a strategic focus in IP work plans to ensure beneficiary equity or as an indirect target from selecting a value chain where women or youth are prevalent. For women’s inclusion, the focus is on existing women’s associations and cooperatives, many of which have received previous or ongoing support from other development partners, which signals stronger capacity and makes them stronger candidates as F2F hosts. Membership in women’s groups consists primarily of married women and older single women who may be the head of a household. Management staff at one IP said: “Women are (our) power of development and youth the future … and when they work as a group, they have more leverage.” This captures the essence of inclusion and diversity, a core value and mission within F2F IPs and their partners who make inclusion a development priority. When IPs target women, they focus on value chains where women already predominate in production, value addition, or local marketing such as horticulture, poultry, small ruminants (e.g., goats and sheep), post harvesting, aquaculture, and rice. One IP coordinator stated: “Women and youth cooperatives or associations may already be well established within a value chain but need further capacity-building to prosper.” Emerging youth associations and agricultural educational development institutions are targeted by many IPs to better prepare the next generation of agricultural stakeholders and to stem the flow of out￾migration. KIIs with country-level IPs found that youth are leaving rural areas and farm production across F2F regions (i.e., Africa, Asia, Middle East, Central and South America, and the Caribbean) for urban centers within and outside of their birth country. Guinea and Nigeria are two examples where public and civil society agencies have made it a priority to attract (or retain) youth to the agricultural sector. The IPs with women and youth foci may partner with other agricultural development programs as local￾level public extension services, helping to ensure sustainability after the F2F project has completed a host assignment. F2F partner NGOs provide women and youth with empowerment programs, such as leadership, communication skills, and basic financial literacy. KIIs with IPs identified the following ways in which partnerships strengthen the F2F program. • They help the IP identify hosts that are thriving or not, and why. • They leverage human and financial resources of other programs with a women/youth focus. • They help ensure sustainability after F2F has completed assignment(s). • They work in complementary capacity-building—such as leadership, entrepreneurship, and financial literacy—that deepen the impact of F2F activities and their partners. Challenges Women and youth challenges revolve around access to finance to expand their operations in production or in providing services downstream along the value chains. The ET desk study and KIIs support this finding, particularly that the main barrier to financial resources by women and youth is the lack of collateral, such as land or other tangible assets. Women’s cooperatives also are limited by the low literacy rates of their members, according to KIIs with IPs. Secondary barriers mentioned include the lack of access to tools, equipment, and quality inputs (such as seeds and fertilizer) for weed control, post harvesting, and value addition, which are common in agricultural development programs. Youth are further constrained by the lack of funds for agricultural￾related education programs and certification. Women’s double work burden in agricultural-related work and domestic responsibilities, and their limited flexibility in daily schedules, can make it difficult to participate in trainings and hinder their ability to earn more from production and service activities. F2F and its IPs are aware of these challenges to women and youth inclusion and say they are focusing more resources, including the timing of trainings, to address these systemic barriers. According to IPs, there is strong demand for financial literacy training, particularly among women’s groups, and business planning among youth-focused groups and at agricultural educational development organizations. An 39 example is the newly formed Network for Agricultural Technical Education of Nigeria, which is heavily focused on bringing youth into agricultural-related opportunities that do not require land. Guinea has a similar network that was founded earlier. Grant writing is also provided to F2F women and youth hosts, primarily NGOs, to improve their funding options. CONCLUSIONS Based on available datasets, F2F has seen a large increase in the racial and ethnic diversity of volunteers since the incorporation of the remote and paired remote volunteering models. There has also been a large increase in first-time volunteers over the same period. Demographic data on F2F volunteers, however, is incomplete in that it excludes U.S. volunteers who worked remotely or in paired remote assignments. USAID does not currently require IPs to report age range data for volunteers. It is, therefore, not possible to confirm trends in age, gender, or ethnic and racial demographics of volunteers with the available data. More complete demographic data on volunteers across all volunteering mechanisms is needed to reflect the true diversity and inclusion achievements of the program. Women volunteers who completed the online survey are similar to their male counterparts. In both groups, their desire to help others was the primary reason for volunteering with F2F. Women and men also learned about F2F mostly from the IPs and others who have volunteered. Social media and the F2F website are about equal as sources of program information but secondary to in-person networking by volunteers and IPs, which are the primary communications channels for volunteer recruitment. F2F focused more on women and youth inclusion in this program cycle, and they are also a strategic focus within country work plans and at the IP HQ level. The targeting and selection of women- and youth￾focused or led hosts (especially associations and cooperatives) are notable examples of more program focus on inclusion. The selection of value chains and partners with inclusion at the forefront is another example of the program’s emphasis on reaching more women and youth. Public agencies that focus on women and youth capacity-building are assets to the country staff and help ensure sustainability and outreach through ongoing extension services to women and youth groups. While there are challenges, such as access to finance, F2F’s design enables flexibility and leverages new opportunities, such as stakeholder collaboration and partnerships ranging from global to local financial institutions. F2F’s support of agricultural educational development organizations is a foundation for supporting the next generation of men and women who will work toward innovation, sustainability, and adaptation of the agriculture sector into the 21st century. 4.3.5 Effectiveness of the LWA Award Mechanism FINDINGS F2F awards to core IPs are LWA Cooperative Agreements. The IPs use the LWA award funding to run their primary country programs designed by the IPs and approved by USAID/Washington. Each LWA also has additional options for adding funding that includes a buy-in component up to $1 million and an additional ceiling for Associate Awards. Both Missions and USAID/HQ can fund additional F2F projects through these two options. The F2F LWAs enable IPs to provide volunteers or undertake buy-ins and Associate Awards in any F2F-eligible country for agriculture program-related activities. This enables F2F to have a greater outreach and respond to more opportunities with fewer costs and turnaround time. In addition to the LWAs held by implementers, AVOP has a buy-in mechanism that is designed to meet many of the same goals. In the hope of obtaining country buy-in or Associate Awards, IPs kept promoting the LWA with USAID Missions and other donors to increase F2F’s impact. As a result, in FY 2019, CNFA was awarded a buy-in of $1 million in organic agriculture. POA’s F2F Guatemala field office collaborated with the Fundación 40 Mario Lopez Estrada and Grupo Onyx to implement a two-year project focusing on livestock production, with a $1 million buy-in and $1 million in matching funds. In FY 2021, the West Africa F2F program received $150,000 in buy-in funds from USAID/RFS to implement a diversity and inclusion pilot, and AVOP was awarded a small buy-in of $96,032 to assist USAID F2F during its staffing transition. Two other IPs are working on potential buy-ins for FY 2022. Some IPs received new Associate Awards during the FY 2019 to FY 2023 round of F2F, and several are continuing to implement Associate Awards funded under the previous round of F2F. The IPs faced considerable difficulties in securing buy-ins and Associate Awards from USAID Missions and other sources. The reasons were often related to the lack of financial resources, the COVID-19 pandemic, and funding priorities falling outside of F2F’s scope. CONCLUSIONS The availability of USAID Mission funds was a major factor in their decision to buy into F2F. In certain countries, the key F2F priorities were not complementary to those of the USAID Missions, which adversely affected their demand for buy-ins. LWA is a good implementing mechanism for USAID since it helps mitigate the risks associated with vetting new proposals from new potential partners. Buy-ins allow F2F IPs to increase activities and outreach, add new hosts, involve more volunteers, and provide more technical expertise. Nonetheless, during the COVID-19 pandemic, few buy-ins were secured by IPs. 41 5.0 RECOMMENDATIONS EQ 1: How well do F2F standard indicators capture the effects of F2F activities and provide useful information for program management? • USAID, jointly with the CoP, should provide further guidance on defining indicators and best practices to strengthen M&E capacity and standardize data collection strategies among all IPs. Current standard indicators are useful tools to inform program progress, but definitions can be further improved to guide IPs and hosts on data collection and reporting. • USAID should explore how to make the ODI more adaptable to different types of hosts so it can be used across the entire range of organization levels. While the ODI is appropriate for organizations that are just forming or being established, it is not appropriate for describing the capacity of advanced organizations. There is, therefore, a need for the ODI to be more adaptable to be able to accurately reflect the capacity of hosts at any development stage. • USAID, through the CoP, should assess the relevance of IPs’ custom indicators and recommendations to 1) create others on specific topics related to leadership, business empowerment, and resilience; 2) share that list as a menu of options; and 3) validate their potential to become standard indicators, if applicable, to tell the program’s story. IPs have developed custom indicators to respond to specific needs and project SOWs, and these could become part of the standard indicators list. • IPs should provide hosts with more recordkeeping guidance and tools for standardizing data collection to strengthen hosts' decision-making processes. • USAID should further analyze IPs’ recommendations regarding indicator disaggregates for gender, youth, and women-led organizations to respond to the program's social-inclusion focus and capture its impacts. The IPs identified challenges collecting data along the value chain for women and youth, as hosts often present aggregated data and indicated that there is no guidance within indicator definitions about how to classify or distinguish women and youth-led organizations. • USAID should collaborate with the CoP on developing meaningful indicators on NRM and cultural exchange impacts that represent the work being done. As part of this, the CoP should review the relevant Feed the Future NRM indicators to evaluate their suitability for F2F and assess the feasibility of adding cultural exchange activities, aside from home stay. Cultural exchanges and NRM indicators are not currently sufficient in describing the program’s performance and impact. EQ 2: How well does the F2F program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs? • USAID should maintain strong involvement and participation in the CoP, providing support on its direction and objectives. USAID should encourage IPs to increase inclusion of in-country team members and develop a systematic approach for addressing data and ideas generated by the CoP along with a mechanism for follow-up and accountability. • USAID and IPs should consider investing additional resources to develop data-driven approaches to learning, including outcome mapping, stakeholder engagement studies, case studies, qualitative data collection, and new outcome and impact indicators. The CoP should leverage the support of IPs that are already piloting these methods. As resources available for M&E are limited, USAID should work with IPs to strategically identify opportunities where trade-offs between investment in program implementation and additional investment in M&E are worthwhile. • USAID and IPs should invest in meetings and virtual platforms to leverage learning and knowledge￾sharing between stakeholders within F2F countries or regions. Opportunities exist to leverage F2F to more organizations, facilitate learning from past assignments and recommendations, and foster sustainability of program outcomes in the regions where F2F operates. Examples include 42 host-to-host knowledge exchanges, in-person meetings, or virtual forums for stakeholder communication. In cases where stakeholders are in remote areas or have limited access to virtual platforms, additional resources will be necessary to facilitate in-person meetings. • There is a need for improved communication from IPs to volunteers after the assignment ends. Volunteers desire more information on host outcomes. After the assignment, strong communication enhances the volunteer's sense of involvement, ensures that they feel valued in their contributions, and supports the retention and expansion of the talent network for future assignments. USAID should encourage the CoP to synthesize best practices for communication from IPs to volunteers after the assignment ends. Improved communication should focus on providing information to volunteers on host outcomes. • USAID should encourage the CoP to develop best practices for communication between hosts and volunteers before the assignment begins. The IPs should facilitate more extensive communication between hosts and volunteers before the assignment begins to orient the volunteer to the local context and the host's operating environment. Improved communication between the host and volunteer prior to the assignment can provide the volunteer with a strong orientation to the local context, challenges, and operating environment before the assignment begins, maximizing the effectiveness of time spent on assignment. • IPs should facilitate conversations between hosts and volunteers to clarify the norms and expectations around post-assignment communication, supporting continued relationships between volunteers and hosts after the assignment ends. In general, most hosts and volunteers appreciate being connected and desire extended communication; however, norms and expectations for communication between hosts and volunteers are not always clear, which can inhibit continued contact where feasible and desired by both parties. In cases where language barriers or limited access to virtual platforms inhibit communication with hosts, additional resources may be needed to support post-assignment communication. • In the next F2F phase, USAID should seek either to split AVOP into two projects, preferably with two different partners, or have IPs with extensive experience in F2F manage and guide one to two sub-awardees each. • The IP managing the next AVOP award should have clear-cut responsibilities in the decision￾making process for choosing the sub-awardees based on their initial applications. Concurrence from USAID and the Missions should come at an early stage to better enhance the process of choosing sub-awardees. EQ 3: What progress has the F2F program in FY 2019 to FY 2023 made toward achieving its objectives? • F2F’s adaptive approach is strong and supports program responsiveness to shocks and stressors. USAID should work with the IPs to continue to identify innovative approaches to improve resilience in program implementation and build local networks of support. • F2F practices for needs assessments and SOW development are generally strong, but not standardized with diverse approaches and methods across IPs and projects. USAID should encourage the CoP to develop guidance around best practices for needs assessments and SOW development to leverage the richness and diversity of the strong practices that exist and ensure the program’s continued responsiveness. USAID should ensure that sufficient resources are provided to IPs to allow for investment in rigorous needs assessments. • Remote and paired remote volunteering models have enabled assistance to continue when international travel is restricted, while building local networks of support to host organizations and adding new aspects of cultural exchange to the F2F program through volunteer-to-volunteer relationships. After travel restrictions have lifted, F2F should retain fully remote and paired remote models of volunteering, in addition to resuming in-person assignments with U.S. volunteers when conditions permit. After travel restrictions have lifted, the program should also consider 43 coordinating paired in-person assignments with U.S. and local volunteers to further facilitate cultural exchange while continuing to leverage the benefits of the paired model. • To continue to leverage the innovation and successes realized through fully remote and paired remote assistance, USAID should work with the CoP to develop updated guidance ensuring assignments conducted under alternative models are counted toward volunteer targets. Specifically, F2F should develop clear and accessible approval pathways to count: 1) assignments conducted independently by a local expert toward assignment targets; and 2) fully remote assignments toward assignment targets. For example, the CoP might define criteria for counting fully remote assignments toward targets if local expertise cannot be identified, travel to the host country by the identified expert is not possible, and the assignment SOW is well-suited to remote work. The success of alternative assignment models demonstrates that innovation should always be a part of programming and encouraged. Therefore, it is critical that alternative assignment models count toward volunteer targets. • After travel restrictions have lifted, USAID should support IPs to adopt a hybrid assignment model that incorporates remote and in-person support. In this model, IPs would guide volunteers to prepare their assignment remotely with the host, deliver it in person, and follow up remotely. This hybrid model would maximize the effectiveness of time spent in person and address one of the commonly noted constraints of limited time. • USAID should support IPs to address challenges related to volunteer recruitment for paired remote and fully remote assignments. Support should include tasking the CoP with synthesizing best practices and resources for paired remote and fully remote assignments and supporting IPs to increase the volunteering culture in the countries where local volunteers are recruited. • The IPs should provide additional support to volunteers to help them succeed in fully remote and paired remote assignments. Examples include: − Providing volunteers with data collection and relationship-building support. − Addressing technical difficulties and network issues and supporting work planning and coordination between paired volunteers. − Facilitating improved communication between remote volunteers and hosts to cultivate a sense of direct involvement. − Improving follow-up with volunteers about assignment outcomes to increase their sense of involvement and accomplishment. Volunteer Demographic Diversity • USAID should support IPs to continue to use paired remote and fully remote volunteering to increase the diversity of under-represented groups in the U.S. volunteer demographic. USAID should request that IPs report on data for all assignment modes to verify the impact of remote volunteering on volunteer demographics. • USAID should request IPs report demographic data for all modes of volunteering, including in￾person, paired remote, and fully remote. USAID should also request that IPs collect data on volunteer age ranges. LWAs • USAID should increase its support to IPs to help them secure additional buy-ins and Associate Awards by informing them about topics to focus on and promoting their work within the local context. 44 6.0 ANNEXES 45 ANNEX 1: PEEL TASK ORDER EXPRESSION OF INTEREST PEEL TASK EOI EXPRESSION OF INTEREST – PERFORMANCE EVALUATION I. BACKGROUND INFORMATION A. Identifying Information 1. Project/Activity Title: John Ogonowski and Doug Bereuter Farmer-to-Farmer Program 2. Award Number: 7200AA18LA00001; 7200AA18LE00002, 5, 6, 7, 8, 9, 10, 11 3. Award Dates: FY19-23 4. Project/Activity Funding: LWAs, ~$6-12M each; 1 AVOP Cooperative Agreement, ~$7M 5. Implementing Organization(s): LWAs: ACDI/VOCA, CNFA, CRS, Land O’Lakes, Partners of the Americas, Winrock, IESC, NCBA, AVOP: Partners of the Americas 6. Project/Activity COR/AOR: Peggy Carlson and Evania Robles B. Development Context 1. Problem or Opportunity Addressed by the Project/Activity being Evaluated The Farmer-to-Farmer (F2F) Program’s goal is to generate rapid, sustainable, and broad-based food security and economic growth in the agricultural sector. A secondary goal is to increase the American public’s understanding of international development issues and programs and international understanding of the US and US development programs. 2. Intended Results of the Project/Activity being Evaluated See Attachment A: F2F Description and Theory of Change 3. Target Areas and Groups F2F Program awards are global in nature, although most awards have Core Country Projects within a single region as follows: ACDI-VOCA 7200AA18LE00002 Catholic Relief Services (CRS) 7200AA18LE00006 Tajikistan: Rural Enterprise Development Kyrgyzstan: Rural Enterprise Development, Ag Education and Training Georgia: Food Safety, Rural Enterprise Development Armenia: Rural Enterprise Development Benin: Cashew, Soybeans Ethiopia: Livestock, Crops Nepal: Livestock, Crops Rwanda: Horticulture, Maize Uganda: Livestock, Agribusiness Development Timor-Leste: Modernizing Agriculture CNFA 7200AA18LE00005 IESC 7200AA18LE00010 Madagascar: Horticulture, Rice, Livestock, Aquaculture Malawi: Horticulture, Aquaculture, Legumes Mozambique: Horticulture, Poultry, Services Zambia: Horticulture, Legumes, Aquaculture Zimbabwe: Horticulture, Livestock/Dairy, Legumes Moldova: Organic Agriculture, Livestock/Dairy Kenya: Access to Finance Tanzania: Access to Finance Sri Lanka: Access to Finance Winrock 7200AA18LE00009 Partners of the Americas 7200AA18LE00007 Ghana: Postharvest Strengthening Guinea: Ag Education & Training, Rural Livelihood Development Mali: Rural Livelihood Development Nigeria: Ag Education & Training Colombia: Rural Enterprise Development Dominican Republic: Youth Development in Agriculture, Rural Adaptation and Resilience Guatemala: Rural Enterprise Development, Horticulture Guyana: Livestock, Horticulture 46 Senegal: Ag Education & Training, Postharvest Jamaica: Resilience, Livestock & Aquaculture Myanmar: Agroforestry Land O’Lakes Venture 37 7200AA18LE00011 NCBA CLUSA 7200AA18LE00008 Bangladesh: Food Safety and Quality Egypt: Food Safety and Quality Lebanon: Food Safety and Quality Peru: Coffee, Cacao Ecuador: Coffee, Cacao Honduras: Coffee Partners of the Americas: Agricultural Volunteer Opportunity Program (AVOP) 7200AA18LA00001 High Atlas Foundation Morocco: Sustainable Agriculture Grameen Foundation Philippines: Coconut Browse and Grass Growers Cooperative Burkina Faso: Food security, nutrition, income generation University of Tennessee Cambodia: Sustainable intensification Purdue University Trinidad & Tobago: Agricultural education In furthering F2F’s secondary goal of increasing public understanding and involvement in international development efforts and improved international understanding, F2F Programs design volunteer assignments and scopes of work to maximize opportunities for volunteer work and interactions with rural people in the host countries. 4. Approach and Implementation See Attachment A: F2F Description and Theory of Change C. Existing Data Key reference materials for F2F Program evaluation: • Midterm Theory of Change and Impact Learning Evaluation of the USAID Farmer-to-Farmer (F2F) Program FY 2014 – FY 2018 • Farmer-to-Farmer Program Annual Report FY 2018 • Farmer-to-Farmer Program Annual Report FY 2019 • Farmer-to-Farmer Program Annual Report FY 2020 • GAO review of Farmer-to-Farmer Program (2015) • Cooperative Agreements, Work Plans, Semi-Annual and Annual Reports, Country Project Descriptions, and Indicator Data available for FY 2019 – FY 2023 F2F Programs • Cooperative Agreement, Work Plans, and Progress and Special Reports for Agricultural Volunteer Opportunity Program (AVOP) and associated grants • Final Reports for F2F Programs FY 2009 – FY 2013 and FY 2014 – FY 2018 • Lessons Learned from F2F Programs FY 2009 – FY 2013 and FY 2014 – FY 2018 • USAID F2F special studies, ongoing analysis, and official letters to the implementing partners concerning their reports and plans • US Farm Bill PL 480 legislation governing the F2F Program • 2018 RFAs for Farmer-to-Farmer Program Cooperative Agreements • USAID and USG Relevant Food Security and Agricultural Strategies • Reports on F2F Implementers Meetings for FY 2019 – FY 2023 • John Ogonowski and Doug Bereuter Farmer-to-Farmer Program Evaluation (2012) 47 II. EVALUATION RATIONALE A. Evaluation Purpose This Midterm Evaluation of the F2F Program has two main objectives. First, it will provide guidance for planning future F2F Program activities and procurement. The second objective is to identify best practices and areas in which the program can make mid-course corrections. This Midterm Evaluation will review the goals and implementation of the F2F Program and ensure that the program operations are efficient and effective in meeting the Program’s goals. Information from the evaluation will be used in shaping future F2F activities. Evaluation recommendations will be shared with current IPs and applicable adjustments in implementation may be made to the extent possible. Documentation on the recommendations will be provided to applicants for implementation of future F2F activities. This will enable IPs to take stock of implementation approaches, strategies, organization, and results and make any changes that are indicated to improve performance. In addition to the traditional IPs, the AVOP sub-award activity will be included in the evaluation. The project is different as it is intended to strengthen the overall F2F Program by involving new IPs, testing new approaches to use of volunteers, supporting knowledge management in implementing agricultural volunteer programs, and facilitating special studies and activities on behalf of the F2F Program. The Midterm Evaluation is expected to answer the following evaluation questions, providing findings, conclusions, and recommendations. The ET should attempt to identify ways in which current processes can be improved. The team should discuss good practices for agricultural volunteer program planning and implementation, discuss any limitations of current practices, and suggest methods to address any limitations. The COVID-19 pandemic has presented several challenges to implementation of the Program. Although many of these challenges have been mitigated by using paired remote U.S. volunteers (see Attachment A), the Midterm Evaluation will enable USAID to capture the impact of the global challenges of the pandemic on implementation. This SOW has specific evaluation questions for the ET to provide findings, conclusions, or recommendations in current implementation and to help inform future F2F programs. For example, challenges, emerging lessons, potential good practices, and models of intervention or elements of the current F2F Program that could be strengthened or modified to better reach activity goals and objectives. The team is also encouraged to include findings, conclusions, or recommendations on other aspects and unintended impacts that may arise during data collection relevant to improving the Program’s efficiency, effectiveness, and sustainability. B. Audience and Intended Uses The USAID F2F team (RFS/CA/Production System Division) and the F2F IPs will use the evaluation to identify potential for changes to program strategies, implementation arrangements, and approaches as appropriate to achieve F2F objectives. USAID also will use the evaluation to inform design of future F2F activities. C. Evaluation Questions The ET will evaluate progress with the implementation of the current F2F Cooperative Agreements against the stated objectives of the F2F Program. The ET will need to be creative in using existing data and reports to assess expected effectiveness and efficiency of the Program. The report will assess F2F Program experience and strategies and provide empirical evidence to support conclusions and recommendations, with focus on the following questions. Evaluation Questions Draft 1. How valid is the F2F Program Theory of Change? 48 • Attachment A describes the Theory of Change. 2. How well do F2F standard indicators capture the effects of F2F activities and provide useful information for Program management? • How well do F2F indicators serve for monitoring of implementation progress? • Looking particularly at outreach/cultural exchange, capacity development, and environmental/ natural resource management, how effectively do these indicators capture effects of F2F? 3. How well does the F2F Program systematically learn and integrate innovation to improve quality and promote good practices in agricultural volunteer programs? • In what ways could the Program improve learning to become more efficient and effective? • What learning and innovation activities or COP support are particularly effective or needed and what may be of lesser effectiveness or utility? • How effectively has AVOP (including sub-awards) supported F2F learning, innovation, and the COP? 4. What progress has the FY 2019 through FY 2023 F2F Program made toward achieving its objectives? • How well are the F2F IPs monitoring their progress toward the stated objectives of the F2F Program, including how they are using this information to adjust and improve performance? • How has the COVID-19 global pandemic impacted F2F implementation and to what extent will that impact achieving Program objectives? • To what extent has the program achieved its volunteer recruitment diversity objectives - ethnicity, occupation, sex? • How effective has the LWA Award mechanism been in promoting F2F objectives through associate awards and buy-ins? III. TIMEFRAME AND VIRTUAL FIELD WORK A. Timeframe The Evaluation should start around May 2021 and be completed no later than March 2022. Note that the midterm data from F2F IPs will be submitted in the October 30, 2020, FY 2021 Annual Reports. This is important data to review but should not delay other components of the evaluation. Work of the ET members does not have to be continuous and may be phased as appropriate to the most efficient, effective, and convenient completion of the required Evaluation. The Concept Note should further define the timeframe for each of the deliverables detailed below. B. Virtual Field Work As travel will be unlikely, the team should plan on remote data collection and field work in a selection of countries. The team would work with the IPs in each selected country to receive additional documents to review and, also, discuss a plan for virtual interviews with select hosts, local stakeholders, and others as needed. IV. DELIVERABLES AND DESIGN A. Deliverables Deliverables shall be as follows. Due dates to be defined in the concept note. 49 • Concept Note: This will be a high-level description of the performance evaluation, including the finalized evaluation questions, team description, and updated timeline. • Performance Evaluation Protocol, including description of the evaluation methodology, indicators/variables to be measured, detailed data collection and analysis methods, data quality assessment methods, work plan and timeline, questionnaire/guidelines for conducting key informant interviews/focus group discussion guides (data collection instruments). These documents will be prepared and submitted to USAID for review prior to commencement of data collection activities. • Work Plan Presentation: A team member should present the inception report and work plan for the evaluation to the F2F IPs. This will provide an opportunity to solicit information and coordinate ET activities with all F2F implementers. • Presentation of Findings: The team shall be available for one to two seminars or presentations of findings for USAID, estimated at one to 1.5 hours each. These will have the presentation and an interactive Q&A portion. • Draft Final Report: The team shall submit a draft final report for review by USAID and the IPs and make a presentation of the findings. USAID and F2F IPs will have two weeks from the presentation to provide the team with written comments, corrections, and other input to the Evaluation. The team members shall be available to USAID and IP staff for reasonable amounts of time for verbal inputs and discussion of the draft report. The team shall schedule the debriefing on the draft report at least three weeks in advance to allow all relevant parties to attend. • Final Evaluation Report: The team shall submit a Final Report, which must contain an Executive Summary and relevant annexes. USAID may provide an additional annex responding to the evaluation and any recommendations, which will be incorporated into the final Evaluation Report. The Evaluation Report shall be submitted electronically to the USAID F2F staff. • Infographic: The team shall develop an infographic(s) to summarize the key findings of the report, methodology used, highlight cross-cutting themes, and recommendations to the program. B. Evaluation Design The Program operates in about 42 countries under multiple IPs and project types. The ET will draw from multiple sources for data and insights in carrying out the Midterm Evaluation. An approach and methodology are proposed below. The team is encouraged to modify this as appropriate within the constraints of the resources available to carry out a robust and thorough Midterm Evaluation as objectively as possible. A mixed-methods or process evaluation approach to collecting and analyzing qualitative and quantitative data is recommended to arrive at robust findings and conclusions and fully address all the evaluation questions, using four to six country programs for case studies to dig deeper into and obtain insights to program operating procedures, reporting, and activities. Qualitative and quantitative data should be used to provide objective evidence to answer the evaluation questions, Qualitative questions will address the experiences, perceptions, opinions, motivations, and knowledge of the stakeholders interviewed; the context within which the activity is operating; and the factors (causal mechanisms) leading to observed results (or non-results). In-depth conversations guided by structured and semi-structured key informant interview instruments provide the opportunity for a flow of information that is not constrained by predetermined response categories. This allows for exploration of subjects that surface during the interview and elicits information that provides nuance and insight for addressing the evaluation questions. 50 Qualitative data will be synthesized and analyzed to provide robust evidence of findings. The ET will more fully elaborate data collection methods in the evaluation plan. The evaluation should review the experience and impacts of the FY 2014 through FY 2018 F2F Program (core programs and SPSP) as reflected in final reports, as well as the experience to date in the current FY 2019 through FY 2023 program. The programs are largely similar, such that the FY 2014 through FY 2018 Program may suggest results that can be expected from the current Program. Assessment of potential impact from any changes between the programs, if any, would be of interest. 1. Document Review: The team shall review available documentation relating to the F2F Program outlined in Section I Part C above. 2. Focus Group Discussions: Virtual focus group discussions may be carried out with key stakeholders, and direct and indirect beneficiaries. Focus groups could allow the ET to collect information about more people in a shorter amount of time and provide valid ways of identifying trends or conclusions with respect to counterfactuals or external factors. The virtual environment can be limiting but may also allow discussions with a wider variety of stakeholders and beneficiaries if organized that way. 3. Key Informant Interviews: Virtual key informant interviews will be used to collect information from stakeholders. Interviews will be employed in answering all evaluation questions. The ETs will use structured or semi-structured interview guides for interviews to ensure key questions are systematically answered. These guides also will allow interviewers the freedom to ask spontaneous questions to uncover important and unanticipated information. The ET will design a separate key informant interview guide for each stakeholder group. The ET shall interview key informants familiar with F2F program operations. This should include USAID F2F team; staff of F2F IPs (recruiters, coordinators, program directors, technical support staff, field staff), AVOP staff, and AVOP sub-awardees. The ET may also interview other USAID staff if desired. 4. Stakeholder Surveys: The team may choose to conduct surveys of hosts and volunteers, although not required. If used, the surveys should ensure perspectives from multiple countries, country projects, types of hosts, and participants new to and with extensive experience with F2F. Surveys may also be conducted of other stakeholder groups, including IPs. The ET can select questions for surveys as appropriate to inform evaluation assessment of the Program. The decision as to the need for surveys, approach, target stakeholders and final questions shall be determined by the ET in consultations with USAID. 5. Existing Data: The team shall draw on the M&E data available from the F2F progress reports, as well as compilations of F2F data, for analysis of program implementation performance. 6. Other Surveys. The team may complement this with surveys of returned volunteers, country staff, or others and/or with independent ratings of program reports, capabilities, implementation, or other program aspects. Quantitative data will be useful to support evaluation conclusions and to provide insights on program operations but will not be available or relevant to all issues to be assessed by the team. 7. Case Studies: Four to six country programs will be selected. Selection should attempt a balance among different implementers, types of country projects, history with F2F and FTF/non-FTF countries. Consider a mix of the “star” performers as well as the average ones to really tease out what is working. Including at least one country with a returning F2F activity may be of particular interest. The team would work with the IPs in each selected country to receive additional documents to review and, also, discuss a plan for virtual interviews with select hosts, local stakeholders, and others as needed. 51 The sampling plan for selection of key informants, focus groups, and survey participants shall be included in the evaluation plan and finalized in consultation with USAID and the F2F implementers to minimize potential bias and taking into consideration the limitations on Level of Effort (LOE) and travel. The ET is responsible for its own logistical arrangements but supported by introductions to implementing partners. The ET will keep USAID updated on an ongoing basis as agreed upon during the kickoff call. V. TEAM COMPOSITION USAID envisions a team of approximately four Key Persons, although the exact configuration will be determined later. It is envisioned the team would include an Evaluation Specialist/Team Lead, a Research Analyst, and technical specialists. Taken together, the team must have familiarity/experience in five areas: a) agricultural development, technology transfer, organizational and capacity development; b) gender analysis; c) quantitative and qualitative evaluation; d) USAID-agricultural projects; and e) agricultural volunteer program management, implementation, or service work. All team members must also be able to analyze issues and formulate concrete recommendations orally and in writing. The team composition and individual skill/experience mix may vary, as determined by the Team Leader and in collaboration with USAID. Below are descriptions of the qualifications of the ET Leader and the Research Analyst and an illustrative estimate of LOE. Evaluation Team Leader – Estimated LOE: 65 days A senior-level evaluator with a minimum of 10 years of experience evaluating multifaceted international development activities ideally in the agriculture sector. S/he will provide leadership in the conceptual design of the evaluation and will oversee the ET. The Team Leader must have experience designing and implementing rigorous evaluations, collecting and/or analyzing data, and conducting quantitative and/or qualitative research, as well as strong management skills. S/he will design and/or oversee the development tools, survey instruments, and other evaluation materials. The Team Leader also will oversee and coordinate activities as needed, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling into a high￾quality document the final well-grounded findings. It is preferred that s/he have an understanding or familiarity with agricultural development, economic growth, local capacity building activities, and/or international volunteer programming. The candidate should also have a demonstrated capacity to conduct independent program evaluation and understand USAID’s foreign assistance goals and its specific objectives for agricultural development and food security. Research Analyst – Estimated LOE: 50 days Must be at least a mid-level analyst with five or more years of experience carrying out research and/or socioeconomic analysis ideally related to international agricultural programs. S/he will take the lead in data collection and analysis as well as conducting surveys. This team member will have an excellent understanding of the tools and research methodologies for monitoring and evaluating development programs. S/he should have experience assessing the impacts of technology transfer and capacity development on various types of beneficiary organizations, as well as in assessing the adequacy of monitoring and evaluation indicators in reflecting the accomplishments and impacts of development and capacity building activities. The qualified candidate must also demonstrate experience recommending practical and pragmatic development indicators. 52 Other Technical Specialist and Team Positions The ET is anticipated to have other positions as well, which will be determined by the Team Leader in collaboration with USAID. Below are some suggestions and examples that may be useful—although again, the profiles and details will be determined. Development Specialist An experienced expert (eight years or more) in international development related to agriculture. S/he must have experience developing and implementing technical approaches to agricultural development and capacity building activities, particularly related to technology transfer, capacity development, value chain development. S/he should understand production systems, producer and community organizations, the dynamics of institution building, human capacity building, and agricultural research/extension/education institutions. Experience with agribusiness, agricultural value chains, and working with volunteers and consultants is desirable. Agriculture Specialist An agricultural professional (five years or more) with knowledge of the U.S. agricultural sector. S/he must have some familiarity with the U.S. agricultural community, NGO programs, and volunteer programs. Experience with international agriculture is desirable. S/he should understand institutional and human capacity needs for an effective agricultural sector. S/he should understand the science and technology behind current agricultural systems and have knowledge of international agricultural standards. Ideally have experience in agricultural volunteer program management, implementation, or service work. The LOE and professional qualifications of the specialists may be interchanged. The LOE also may be allocated differently—with, for example, a fifth Specialist included with the same total LOE (205 workdays). Specialists should have university degrees in fields relevant to evaluation, agriculture, and/or rural development. No specific degree qualification is indicated as this requirement will be subsumed by the requirement for experience in international development. A table of expected LOE should be included in the Concept Note with the proposed Team Description. Attachment A: F2F Description and Theory of Change GENERAL DESCRIPTION OF FARMER-TO-FARMER PROGRAM The F2F Program goal is to generate rapid, sustainable, and broad-based food security and economic growth in the agricultural sector. A secondary goal is to increase the American public’s understanding of international development issues and programs and international understanding of the U.S. and U.S. development programs. In furthering the secondary goal of increasing public understanding and involvement in international development efforts and improved international understanding, F2F Programs design volunteer assignments and scopes of work to maximize opportunities for volunteer work and interactions with rural people in the host countries. 53 FARMER-TO-FARMER THEORY OF CHANGE Theory of Change Inputs: Volunteer technical assistance with some additional staff support ↓ Outputs: Volunteer recommendations and training ↓ Outcomes: Adoption of recommendations or host change in behavior, management practices, and technology (innovation) ↓ Impacts: Productivity and income increases. Improved social, human, and environmental capital FARMER-TO-FARMER PROGRAM APPROACH The F2F Program facilitates delivery of a broad range of U.S. agricultural expertise, using American volunteers who work with farmers, agricultural support systems and agribusinesses in developing countries. F2F has been an effective, low-cost program that blends two goals—providing technical services to enhance sustainable economic development and opportunities for good people-to-people level exchanges. The F2F volunteer program by its nature involves several fundamental trade-offs that must be balanced in implementation. These generally do not require either/or decisions, but present challenges to planning and implementing the F2F programs to maximize impact and respond to the multiple demands on the volunteer program. These trade-offs include: a. Development impact vs. the volunteer experience: The key objective in the program is developmental impact. At the same time, an important subsidiary objective is to provide the volunteer with meaningful cross-cultural experience and understanding of development and foreign policy issues. The balance in this affects such decisions as: how often to use repeat volunteers, as opposed to maximizing numbers of discrete individuals placed as volunteers; how much emphasis to put on outreach activities by returned volunteers; how much to spend on support costs to make volunteer services more effective; and what types of volunteer assignments to develop. b. Poverty reduction vs. economic growth: These two objectives can be quite compatible. Volunteers and implementing partners often prefer to work directly with the poor rather than with larger firms and farmers, and USAID strategy supports broad-based economic growth and poverty reduction. Still, larger farmers and businesses may be better able to use volunteer services and might ultimately have broader, but indirect, impacts on the poor. Often the need is to enhance the spread of innovations and benefits from volunteer assignments to prevent “capture” by elite hosts. c. Poorer vs. wealthier countries: Like the issue with type of hosts, countries with more dynamic economies and better regulatory and governance systems are best able to use services and benefit from volunteers. However, needs may be greater in countries with less developed private sectors and market economies. While a bad policy environment may limit the ability of volunteers to influence change, volunteer assistance can benefit the immediate host organizations and can help establish a base for future policy reform that improves a country’s prospects for development. d. Immediate vs. long term impacts: As a development program, F2F must recognize that many activities may have broad impacts, but only in the long term. It may be more effective to work on regulatory or extension programs that ultimately will affect many farmers than to work with a few individual agribusiness firms. Still, volunteers and funding agencies want to see and be able to report on immediate impacts that come from activities with short term objectives. 54 e. Demand vs. supply driven: Programs typically should seek volunteers to meet needs of hosts rather than seeking host assignment opportunities for pre-identified potential volunteers. Still, volunteer supply considerations are important to providing volunteer opportunities to qualified people and to being able to provide volunteers to fill specific host requests. f. Focused activities vs. targets of opportunity: Volunteers can work with host institutions throughout the agricultural sector, and good assignments often come up in diverse sub-sectors. Experience has shown limiting work to one sub-sector allows for better planning, synergistic assignments, and greater impact. The concept of a “demand-driven” program that responds only to needs of hosts argues for providing volunteers exclusively in response to host requests. Such an approach, however, need not imply a commitment to responding to all sub-sectors, as responsiveness to hosts can be limited to pre-identified sub-sectors. Even with the most extreme demand-driven approach, requests must be for activities that are legal, viable and in the public interest. Program parameters can impose appropriate restrictions on host eligibility and program promotion to focus work on industries or sub-sectors expected to produce high impact, while still responding to host demand within the defined sub-sector. g. Maximizing number of volunteers vs. ensuring adequate support: Programs must seek to hold costs down and make volunteer services cost effective. Adequate logistical and planning support is essential to volunteer safety, morale, and effectiveness. Qualified program staff are needed to plan, support, follow up and report on volunteer assignments. h. Longer vs. shorter assignments: Longer assignments appear more cost efficient in that they lower cost per volunteer-day of services. Many potential volunteers are unwilling or unable to serve for longer than two to three weeks and some hosts do not want to spend more time with volunteers. The two-to-three-week assignment has been the standard for F2F programs and has proven successful. Ultimately, the scope of work and length of assignment must be defined by the requirements of the hosts, country sector needs, and ability to recruit qualified volunteers. Alternative assignment options may provide volunteer opportunities to other populations of potential volunteers and offer new services to hosts. This might also allow for use of intern￾volunteers or volunteer consultants to support implementation of the volunteer program itself. i. Supporting partner programs vs. achieving independent results: Volunteer programs generally work best when volunteers work with an ongoing program that provides a framework for their assistance and follow up as necessary. F2F program effectiveness is enhanced when it draws on other resources to support its work. USAID and other donor projects are often good partners but use of volunteers to substitute for or duplicate work of consultants available from other programs is questionable. F2F volunteer assignments should add value, not simply replace other donor or private sector resources. j. Volunteer services vs. complementary support: Complementary support (training, additional technical assistance, travel by hosts, demonstration grants and pilot programs, etc.) can often facilitate adoption of volunteer recommendations and expand impact from volunteer assignments. F2F Program funding is limited and funds only volunteer services. Other USAID funding, if available, does not have the same restrictions and can fund needed complementary support, either through Associate Awards or through parallel-financed projects. FARMER-TO-FARMER PROGRAM STRUCTURE F2F programs vary depending on country and sector needs and conditions, but all follow generally similar approaches for planning and fielding volunteers under Country F2F Projects and as flexible volunteer assignments. Definitions used in the F2F Program are defined below: 55 • Core Country: A country with one or more Country F2F Projects • Country F2F Program: The F2F activities in a specific country • Country F2F Project: The F2F activities in a country focused on a specific sub-sector objective • Country Partner: An institution, project, or program with which F2F works in country to plan and implement Country F2F Projects • Farmer-to-Farmer Program: The over-all F2F program administered by USAID • Flexible assignment: A volunteer assignment undertaken outside of a Country F2F Project • Host project: May be used to refer to all F2F activities with one host • Host: A local institution or person receiving assistance from one or more F2F volunteers • Implementing Partner: The Cooperative Agreement organization implementing an F2F Program • F2F Program: The program administered by an individual with funding from this RFA Country F2F Project Descriptions are used to summarize the problem to be addressed, important implementation issues, how volunteer services and other resources will be used, and results expected from work on the defined country subsector. The following is an outline for a proposed Country F2F Project. Outline for Country F2F Project Descriptions 1. Objective (approx. one sentence) 2. General description of sub-sector being targeted for assistance and why it is targeted 3. Description of the key problems/constraints to be addressed or opportunities to be exploited 4. Proposed F2F activities a. How will volunteers be used? (Numbers and types of volunteer assignments planned) b. What are the key country partners (if any) and their roles? c. What are likely target hosts? (Number/type of expected hosts; criteria for identifying them) d. How can volunteer recommendations be shared broadly to maximize impacts? 5. Any key feasibility issues (economic/financial, institutional, environmental, social, technical), including some economic assessment relating costs to benefits. 6. Other resources available/needed to accomplish desired objectives (partner projects, loans, etc.) 7. Logframe or similar results framework summarizing project design with quantified targets for numbers of volunteers, hosts, beneficiaries, and impacts. Note: The Country F2F Project Description should provide evidence of understanding the sub-sector for the Project and planning for expected activities along with clear targets for performance and impact indicators. It is understood that experience during implementation may lead to substantial changes in activities, hosts, targets, and other aspects of the Project. Flexible assignments are any F2F volunteer assignments programmed outside of Country F2F Projects. Flexible assignments: respond to targets of opportunity for substantial developmental impact, carry out exploratory activities in new areas or sectors, and respond to unforeseen needs and opportunities of USAID Mission programs and partners. Up to 15 percent of volunteer assignments may be used for flexible assignments in eligible countries and sectors outside of Country F2F Projects. They generally require little additional management or staff costs and are approved in Annual Work Plans. Flexible assignments may be in any country, within or outside the geographic region for the core country F2F programs. Flexible assignments are generally not undertaken in countries in which another implementer has an ongoing Country F2F Program. F2F Programs retain flexibility to adjust activities based on experience and changing conditions. Most Country F2F Projects are five-year activities, but some may be shorter or shortened if experience so indicates. New Country F2F Projects can be initiated during the five-year program. F2F PROGRAM IMPLEMENTATION 56 Implementing partners will establish institutional capacity and management systems necessary to implement F2F volunteer assignments effectively and efficiently to support the objectives of the F2F Program. While other functions and activities may also be required, they will establish systems required to carry out the functions, as described below. a. Sector Analysis and Planning: Earlier F2F programs assigned volunteers wherever good hosts were found, but evaluations indicated that greater impact was achieved when activities were focused on targeted commodity and service sub-sectors. Past F2F Program Assessment noted the need for a balanced assessment of the agricultural sector and potential sub-sectors in selecting and designing country projects. Analyses include, as appropriate, economic, social, environmental, and institutional issues—with emphasis on development of private sector market chains for agricultural products. Various methodologies can be used in analyzing agricultural sector needs and opportunities. Some of these include sub-sector analysis, supply chain analysis, value chain analysis, industrial organization, food systems framework, or SWOT (Strength-Weakness￾Opportunity-Threat) analysis. Because of limitations on funding for F2F programs, they will make maximum use of sector analytical work available from other sources, such as USAID Mission projects, other donors, host country governments, private institutions, and others. b. Selecting Country Partners: Country Partners may assist in development and implementation of the country F2F program and project strategies and plans. Such partners can be public or private agencies and can include trade associations, NGOs, commercial firms, projects, government departments and programs (e.g., Peace Corps) or educational institutions. Partners are commonly involved in providing goods and/or services to agricultural producers or agribusinesses. The clients of such partners are often the public and private sector organizations that are potential hosts for volunteers. Roles of partners vary, but commonly include serving as: a) local representative for the F2F program implementer; b) retailer of volunteer services; c) link to clients (hosts); or d) technical backstop. In some cases, volunteers may work with partners to strengthen the partner’s own capacity to provide goods and services to their clients (in which case the partner is also the “host” organization). Generally, good partners will help to see that the specific knowledge or skills that volunteers bring to a host organization are spread more widely to benefit the sector. Working with and possibly embedding volunteer technical assistance and Country F2F Project implementation within a local Country Partner organization may serve to advance the potential for sustaining development activities in line with the objectives of the USAID Forward policies. c. Selecting hosts: Hosts are organizations or individuals that request volunteer services and work directly with the volunteers. Host organizations commonly include private businesses, community groups, trade or business associations, local government offices, cooperatives, or public or private service providers (inputs, credit, information, marketing, processing, storage, etc.). Hosts may be the ultimate beneficiaries (typically small farmers) that the F2F Program is designed to assist, but may also be intermediaries (service providers, agribusinesses, etc.) whose improved operations benefit the target populations. The ability of host organizations to absorb, utilize, and disseminate information provided by volunteers is a critical consideration in determining the impact of a volunteer’s assignment. Host organizations should be involved early on in developing the F2F program strategies and plans. They should also, where possible, contribute to the costs of fielding volunteers and assist in assessment of the impact of the volunteer’s efforts. The impacts of a volunteer program are primarily a function of what host organizations can do with the information and training provided by volunteers. Recruiting from active farmers is encouraged. 57 d. Development of assignments: Implementing partners will establish systems to develop SOWs that describe services required by host organizations and identify the skills and experience volunteers will need for the assignment. Scopes of work may also include background on the sub￾sector and host project, relating the specific volunteer assignment to the Country F2F Project strategy and plan. The SOW identifies the results anticipated from the assignment. The SOW may also identify complementary inputs that are expected to be forthcoming from the host and other partners. Scopes of work translate program and project plans into specific tasks for volunteers. Ideally, there should be a high degree of participation by host organizations in preparing them. The SOWs document the understandings reached among the parties involved, namely the host organization, any partners, and the F2F program implementer. The clearer the SOW can be in specifying who, what, where, when, why, and how, the easier it will be to recruit for the volunteer assignment, guide the volunteer once on the job and, finally, assess the success of the volunteer assignment. The process of approving a SOW also clarifies the roles of the implementing agency, the volunteer(s), any partners, and the hosts in making the assignment a success. Ideally, individual volunteers work with farmers and target beneficiaries to develop personal relationships and mutual understanding. e. Volunteer recruitment: The F2F Program requires systems for identifying and recruiting volunteers to provide services as per defined SOWs, including consider options for recruiting from new or non-traditional sources of volunteers to maximize the pool of volunteer resources available to the F2F program and to maximize the potential for public education and outreach. The F2F Program will support services of U.S. volunteers (U.S. citizens and permanent residents); non-U.S. volunteers will not be used, unless prior approval is obtained from the USAID AOR for any unique situation requiring such use of non-U.S. volunteers. f. Volunteer orientation: The F2F Program requires systems for appropriate volunteer orientation, including information about travel, health and security issues, other logistics and support, the country, and the organizations and individuals with whom the volunteer will work, as well as the specific tasks of the assignment. The orientation process begins as part of volunteer recruitment and extends through the initial days of their arrival in the host country. The more informed volunteers are about what they are getting themselves into, the better equipped they will be to make their assignment successful both for the host as well as for themselves. g. Volunteer logistics and support: The F2F Program requires systems to ensure volunteers are provided with adequate logistical and technical support to provide efficient and effective services to hosts. The logistics and support services greatly impact the effectiveness of volunteers. Logistical support includes transport, housing, translators, secretarial assistance, compensation arrangements, and other services. Technical support includes information, links to local technology programs, teaching aids, and equipment. Volunteers that are appropriately supported will have more time to focus on their assignments and will develop a better understanding of the host’s operating environment and constraints. Technical support can help volunteers adapt their technical knowledge and skills to local conditions and develop appropriate linkages to local technical support institutions. Such support is particularly important given the relatively short duration of many volunteer assignments. They will provide adequate medical and evacuation insurance, vaccination and health briefings, in￾country security, and other support to ensure safety and wellbeing of volunteers. h. Complementary support to hosts: The F2F Program must consider needs and make appropriate arrangements, as far as is possible, to ensure that complementary services (e.g., financial services, equipment and infrastructure, input supply, marketing, and processing arrangements, etc.) are available to enable hosts to make use of volunteers’ recommendations. 58 Complementary support services represent the range of resources available to enable host organizations to operate effectively, including such things as financial services (especially credit), equipment and infrastructure, input supply, database services (e.g., marketing and pricing information), marketing, and processing arrangements. These services are generally not within the scope of an individual volunteer assignment, but volunteers’ recommendations often implicitly or explicitly require a host to draw on such resources to implement recommendation. Thus, while a volunteer may complete his/her assignment satisfactorily without such services, service availability varies according to the environment in which the host operates, and they are often critical determinants of the level of impact from a volunteer assignment. F2F programs with limited budgets are not able to provide complementary support to hosts but may be able to facilitate access to such services from other programs. i. Project staffing: The F2F Program requires adequate staff and staff support resources for efficient and effective implementation of the volunteer program, including the planning, implementation, and monitoring and evaluation of the F2F program. j. Volunteer outreach activities: The F2F Program requires systems to encourage and maximize the amount and impact of outreach activities carried out by returned volunteers. Outreach encompasses the full set of activities designed to inform both the public and specific interested parties of the character and accomplishments of the F2F program. Outreach has three principal benefits: 1) educating the public on global development issues and market opportunities; 2) involving the public in international development; and 3) recruiting new volunteers. Past F2F program assessments noted the need to strengthen outreach activity by returned volunteers, perhaps requiring an increase in funding support for such activities. k. Associate Awards: The F2F Program requires capability to respond, as appropriate, to requests from USAID Missions or offices with proposals for Associate Awards under the Leader Cooperative Agreement. Associate Awards may provide for: a) additional volunteer services in a core F2F country or other country; b) complementary support for Country F2F Project sub￾sectors (additional technical assistance, grants, training, equipment, and facilities, and/or other inputs) that can improve the F2F Program outreach and impact; or c) volunteer services and complementary support for agricultural projects addressing a specific development objective in an F2F core country or other country. Associate awards may expand F2F program activities into additional countries within or outside of the given region. l. Buy-Ins: F2F awards require capability to respond, as appropriate, to requests from USAID Missions or offices with proposals for buy-ins. Buy-ins may provide for: a) additional volunteer services in a core F2F country or other country; b) complementary support for Country F2F Project sub-sectors (grants, training, additional technical assistance, equipment and facilities, and/or other inputs) that can improve F2F Program outreach and impact; or c) volunteer services and complementary support for agricultural projects addressing specific development objectives in an F2F core country or other country. Buy-ins or associate awards may expand F2F program activities into additional countries within or outside of the given region. Buy-ins are available through transfer of funds to an F2F LWA or to AVOP. Buy-ins to AVOP are subject to a competitive process and can usually be completed within 12 weeks. Any U.S. organization (except LWA holders listed above) is eligible as an AVOP buy-in implementer. OTHER CRITICAL IMPLEMENTATION REQUIREMENTS The F2F Program requires attention to planned and unplanned results and impacts, including especially as may relate to USAID policy issues of importance to the F2F programs, as discussed below. 59 a. Gender: The F2F Program requires systematic consideration of gender issues and impacts in the F2F program and will seek to promote gender equity. Each Country F2F Project description will assess gender issues relevant to the sub-sector and measures the project may take to enhance positive gender impacts. Although gender analysis considers impacts on both men and women, in practice, the concern is nearly always ensuring equitable participation and access to benefits by women. Biases against women are pervasive in much of the world, as are differentiated roles between the sexes. Programs can have unintended negative impacts on women. For example, economic development has occasionally increased the gap between the status of women and men by introducing technologies and crops that require more labor (e.g., weeding, watering) from women while men control the profits from the increased yields. Benefit distribution is a key consideration for ensuring gender equity in development. Furthermore, it is not uncommon for development programs to fall short of their goals (e.g., improvements in health and education) specifically because women have not been permitted to participate fully in program design and implementation. The implementing partner will assess gender impacts and gender equity issues in all Country F2F Programs. b. Environment: The F2F Program requires systematic consideration of the implications of the F2F Program activities on environmental and natural resources and should seek to mitigate any adverse impacts. Economic activities impact on environmental and natural resource resources. In the case of agriculture, development may be completely dependent on natural resources and potentially deplete them or degrade environmental resources. A healthy environment and the sustainable use of natural resources are essential to long-term economic growth. All U.S. government financed programs, including the F2F program, are obligated to comply with U.S. environmental regulations stipulating that the environment is to be safeguarded from adverse consequences in the implementation of all program activities. USAID's environmental guidelines require programs that it funds to: • Ensure that the environmental consequences of USAID-financed activities are identified and considered by USAID and the host country prior to a final decision to proceed and that appropriate environmental safeguards are adopted. • Assist developing countries to strengthen their capabilities to appreciate and effectively evaluate the potential environmental effects of proposed development strategies and projects, and to select, implement, and manage effective environmental programs. • Identify impacts resulting from USAID's actions upon the environment, including those aspects of the biosphere which are the common and cultural heritage of all mankind. • Define environmental limiting factors that constrain development and identify and carry out activities that assist in restoring the renewable resource base on which sustained development depends. The Initial Environmental Evaluation (IEE) completed by USAID for the F2F Program made a Recommended Threshold Determination of a Negative Determination with Conditions. Most F2F Program activities qualify for Categorical Exclusion under 22CFR216.2I: • Per 22 CFR 216.2(c)(2)(i) for activities limited to education, technical assistance, or training programs except to the extent such programs include activities directly affecting the environment (such as construction of facilities, etc.). • Per 22 CFR 216.2(c)(2)(iii) for activities limited to the performance of analyses, studies, academic or research workshops and meetings. • Per 22 CFR 216.2(c)(2)(v) for activities limited to document and information transfers. 60 • Per 22 CFR 216.2(c)(2)(xiv) for activities limited to studies, projects or programs intended to develop the capability of countries to engage in development planning, except to the extent designed to result in activities directly affecting the environment. However, even though the F2F program will be essentially training and advisory activities with no direct effect on the environment, the subject of the training or advice may involve use of pesticides. While there will be no procurement or physical use of pesticides, the training in how to decide when to use them and how to use them safely can be seen as constituting a “use.” This requires that some environmental safeguards be put in place on any involvement with pesticide use. The same holds true for any involvement with Genetically Modified Organisms (GMOs), which are politically sensitive in some countries and present some environmental dangers. The F2F Program must ensure that no pesticides are provided, used, or sold under the program. Any pesticide-related training will be in an IPM context and emphasize training on safe use of pesticides. Training and recommendations for use will be in an IPM context where pesticide use is the last resort. The focus of the training will be on IPM and the environmentally responsible and safe use of pesticides when they are needed. They will ensure that this caveat is included as a standard element in briefing materials provided to all volunteers prior to their traveling. No GMOs will be provided, used, or sold. Only the benefits and disadvantages of GMOs will be discussed during any training. If research, provision, use or recommendation for use of a GMO is subsequently planned, it will first be submitted to USAID for clearance by the USAID bio-safety committee and subsequently the information flowing from the USAID bio-safety committee will be used as the basis for an amendment to this IEE to make a final determination whether such GMO may be researched, provided, used, or recommended for use. This IEE amendment must be approved in writing by the BFS Bureau Environmental Officer prior to a final decision or expenditure of funds to research, provide, use, or recommend the use of a GMO. Given the sensitivity of both the use of pesticides and the use of GMOs, the implementing partner will ensure that the following specific conditions are applied in its F2F Program: • The syllabus for each training event will be reviewed and approved by the Mission environmental officer and/or the USAID AOR. • A representative from USAID (preferably the Mission) should attend the training sessions to the extent possible. • If any specific pesticides are to be used, procured, or recommended for use, the USAID Pesticide Procedures (§216.3(b) must be followed. An amended IEE will be prepared prior to the training and must include a separate section evaluating the risks and benefits of the proposed use. The 12 factors outlined in Section 216.3(b) will be analyzed in that evaluation. This evaluation is known as a Pesticide Evaluation Report and Safe Use Action Plan (PERSUAP). • If the training includes recommendation for use of GMOs, a Biosafety Analysis will be required and an amendment to this IEE will be prepared. The extent of such an analysis will depend on the type of activity proposed and the status of acceptance of the GMO. • If the activity will have a significant adverse effect on the environment, based on the analysis in an amended IEE, a full Environmental Assessment will be required. • They will familiarize all staff with the summary “F2F Program Environmental and Natural Resources Management Guidelines for Implementing Partners” (Annex E) and with F2F guidelines for volunteers. 61 The F2F Program must ensure that volunteers consider potential environmental consequences of their work and promote active environmental stewardship. All local laws regarding pesticide or GMO regulation will be respected. Any variation to these principles must be preceded by USAID approval of an amended IEE. If, during implementation, activities are considered other than those described in this RFA as the basis for the categorical exclusions, the implementing partner will inform the USAID AOR and request approval for an amended IEE before proceeding with such activities. The implementing partner may use funds from the Cooperative Agreement to pay for the cost of undertaking any required IEE amendments. The training of applicators, vendors, decision makers, and farmers will improve IPM decision making and reduce pesticide misuse, thus benefiting the agricultural areas. The eventual effect on the environment is expected to be highly beneficial as trained host institutions and individuals are likely to make better decisions regarding pest management technologies and, when pesticides are used, they will be used in a safe and environmentally responsible manner. Given the wide use and importance of pesticides in agricultural production systems, the F2F Program should plan to collaborate in preparation of a programmatic PERSUAP within the first six months of award. A PERSUAP basically consists of two parts, a “PER” and a “SUAP.” The Pesticide Evaluation Report (PER) section performs the systems analysis of the country’s pesticide system from import to ultimate disposal and addresses the 12 informational elements required in the Agency’s Pesticide Procedures. The Safe Use Action Plan (SUAP) puts the conclusions and recommendations reached in the PER into a plan of action, including assignment of responsibility to appropriate parties connected with the pesticide program. c. Restricted sub-sectors and activities: The F2F Program does not support F2F Program activities that are restricted by USG policies and restrictions. USAID policies and regulations restrict support for certain activities involving: tobacco production and processing; African oil palm production or processing; degradation or conversion of tropical forests; agricultural products in surplus on world markets; assistance to activities or projects of Communist countries; assistance to military, police or prisons; luxury goods; alcoholic beverage production or marketing; export of U.S. jobs, relocation of U.S. businesses or restrictions on internationally recognized workers’ rights; or production of agricultural products that compete with U.S. products in international markets. Activities involving support for any host projects that might involve support to such sub-sectors, products, or activities will be discussed with USAID before initiating or making any commitments to work with hosts on such activities. The implementing partner will obtain prior approval from the USAID AOR in Annual Work Plans or activity-specific approval prior to undertaking work on any activities involving these products and sub-sectors. USAID will not support the growth of tobacco as a cash crop, nor will it support agribusiness activities contributing to tobacco production, promotion, and use. In countries in which tobacco is a major cash crop and an important source of income for low-income farmers, USAID may work with local agricultural interests to identify crops that are economic alternatives to tobacco and may support the introduction or expansion of these alternative crops. Program Implementation Response to COVID-19 The COVID-19 pandemic has posed significant challenges to the F2F Program implementation, most notably travel restrictions for volunteers and host country and organization necessary social distancing measures. In response while international travel restrictions are in place, implementing partners received approval to recruit and place F2F volunteers from the host country or country within the surrounding region on a temporary basis as domestic and regional travel restrictions are lifted. 62 Implementing partners are to retain all usual procedures and standards for recruitment, candidate vetting, orientation, and volunteer management for these volunteers. Non-U.S. volunteers are to also conform to qualifications, reference checks, and implementing partner volunteer final selection procedures. IPs are required to, whenever possible, couple remote U.S. F2F volunteers with core-country or third country national volunteers for collaboration, support, and assistance in virtual continuation of assignments as well as carrying out activities related to U.S.-generated indicators for F2F implementation. COVID-19 social distancing procedures are to be adhered to according to USAID and implementing partner guidance to ensure the safety of both volunteers, F2F hosts, other technical assistance recipients, staff and other people who may be in contact with volunteers during their assignment. The use of paired remote U.S. volunteers hopes to mitigate some of the effects of the pandemic and allow implementing partners to continue making progress toward Life of Project and annual volunteer targets. 63 ANNEX 2: BIBLIOGRAPHY 2014 US Farm Bill, the Agricultural Act; Accessed from United States Senate Committee on Agriculture, Nutrition and Forestry https://www.agriculture.senate.gov/issues/farm-bill. ACDIVOCA, Farmer-to-Farmer Europe, Caucasus, and Central Asia (ECCA) F2F ECCA: Annual Report Final, USAID, October 2020. ACDIVOCA, Farmer-to-Farmer Europe, Caucasus, and Central Asia (ECCA) F2F ECCA: Annual Report Final, USAID, October 2021. Catholic Relief Services, CRS Farmer-to-Farmer Digital Library (accessed January 3, 2022). https://f2flibrary.crs.org. CNFA, Farmer-to-Farmer Southern Africa Year 3: Annual Report, USAID, October 2021. CRS, FY20 Annual Performance Report, Annual Report, USAID, October 2020. CRS, FY21 Annual Performance Report, Annual Report, USAID, October 2021. IESC, Farmer-to-Farmer Access to Finance Program: Annual Report, USAID, October 2019. IESC, Farmer-to-Farmer Access to Finance Program: Annual Report, USAID, October 2020. IESC, Farmer-to-Farmer Access to Finance Program: Annual Report, USAID, October 2021. IESC, Farmer-to-Farmer Access to Finance Program: Performance Management Plan, USAID, August 2020. ME&A, Midterm Theory of Change and Impact Learning Evaluation of The Farmer-to-Farmer: Evaluation Report, USAID, August 2017. NCBA CLUSA, Farmer-to-Farmer Year One Annual Report: USAID, October 2019. NCBA CLUSA, Farmer-to-Farmer Year Three Annual Report: USAID, October 2021. NCBA CLUSA, Farmer-to-Farmer Year Two Annual Report: USAID, October 2020. Partners of the Americas, Farmer-to-Farmer Agricultural Volunteer Opportunity Program: Annual Report, USAID, October 2019. Partners of the Americas, Farmer-to-Farmer Agricultural Volunteer Opportunity Program: Annual Report, USAID, October 2020. Partners of the Americas, Farmer-to-Farmer Agricultural Volunteer Opportunity Program: Annual Report, USAID, October 2021. Partners of the Americas, Farmer-to-Farmer Program: Annual Report, USAID, October 2019. Partners of the Americas, Farmer-to-Farmer Program: Annual Report, USAID, October 2020. Partners of the Americas, Farmer-to-Farmer Program: Annual Report, USAID, October 2021. Partners of the Americas, Farmer-to-Farmer website (accessed January 3, 2022). https://farmer-to￾farmer.org. USAID, Farmer-to-Farmer Program Standard Performance and Impact Indicator Definitions, February 2019. USAID, John Ogonowski and Doug Bereuter Farmer-to-Farmer Volunteer Program Implementation: FY 2018 Annual Report, December 2018. USAID, John Ogonowski and Doug Bereuter Farmer-to-Farmer Volunteer Program Implementation: FY 2019 Annual Report, December 2019. 64 USAID, John Ogonowski and Doug Bereuter Farmer-to-Farmer Volunteer Program Implementation: FY 2020 Annual Report, April 2021. Venture 37, Farmer-to-Farmer Food Safety & Quality: Annual Report, USAID, October 2019. Venture 37, Farmer-to-Farmer Food Safety & Quality: Annual Report, USAID, October 2020. Venture 37, Farmer-to-Farmer Food Safety & Quality: Annual Report, USAID, October 2021. Winrock, FY 2019 Annual Report: USAID, October 2019. Winrock, FY 2020 Annual Report: USAID, October 2020. Winrock, FY 2021 Annual Report: USAID, October 2021. 65 ANNEX 3: LIST OF PERSONS INTERVIEWED Code Type Date M F Total M01 HQ September 23, 2021 1 2 3 M02 HQ September 28, 2021 1 1 2 M03 HQ September 29, 2021 0 2 2 M04 HQ October 4, 2021 0 1 1 M05 HQ October 6, 2021 1 2 3 M06 HQ October 5, 2021 1 0 1 M07 IP October 14, 2021 1 0 1 M08 IP October 8, 2021 0 1 1 M09 IP October 15, 2021 1 0 1 M10 IP October 6, 2021 1 0 1 M11 IP October 11, 2021 0 1 1 M12 IP October 13, 2021 1 0 1 M13 IP October 4, 2021 1 0 1 M14 IP October 5, 2021 1 0 1 M15 IP October 6, 2021 0 1 1 M16 IP October 5, 2021 1 0 1 M17 IP October 5, 2021 0 1 1 M18 US October 5, 2021 0 1 1 M19 US October 8, 2021 0 1 1 M20 IP October 6, 2021 1 0 1 M21 HQ October 6, 2021 0 1 1 M22 IP October 7, 2021 0 1 1 M23 HQ October 7, 2021 1 0 1 M24 IP October 7, 2021 0 1 1 M25 IP October 8, 2021 0 1 1 M26 IP October 7, 2021 1 0 1 M27 IP October 12, 2021 0 1 1 M28 IP October 10, 2021 1 0 1 M29 IP October 13, 2021 1 0 1 M30 IP October 13, 2021 1 0 1 M31 IP October 12, 2021 1 0 1 M32 IP October 11, 2021 1 0 1 M33 IP October 12, 2021 0 1 1 M34 IP October 13, 2021 0 1 1 M35 IP October 12, 2021 0 1 1 66 Code Type Date M F Total M36 IP October 13, 2021 1 0 1 M37 IP October 13, 2021 0 1 1 M38 IP October 14, 2021 1 1 2 M39 IP October 14, 2021 0 1 1 M40 IP October 18, 2021 1 1 2 M41 IP October 18, 2021 1 0 1 M42 IP October 19, 2021 0 1 1 M43 IP October 21, 2021 1 0 1 M44 IP October 15, 2021 1 1 2 M45 IP October 18, 2021 0 1 1 M46 IP October 20, 2021 1 0 1 M47 IP October 18, 2021 0 1 1 M48 IP October 19, 2021 1 0 1 M49 IP October 22, 2021 1 0 1 M50 IP October 22, 2021 1 0 1 M51 IP October 20, 2021 1 0 1 M52 IP October 20, 2021 0 2 2 M53 IP October 18, 2021 1 1 2 M54 IP October 25, 2021 1 0 1 M55 IP October 26, 2021 0 1 1 M56 IP October 27, 2021 0 1 1 M57 IP October 27, 2021 1 0 1 M58 IP October 27, 2021 1 0 1 M59 IP October 28, 2021 1 0 1 M60 IP October 28, 2021 0 1 1 M61 IP October 29, 2021 1 0 1 M62 IP November 1, 2021 1 0 1 M63 IP November 3, 2021 0 2 2 M64 IP November 1, 2021 1 0 1 M65 IP November 3, 2021 0 1 1 V01 U.S. -Based October 25, 2021 1 0 1 V02 U.S. -Based October 26, 2021 1 0 1 V03 U.S. -Based October 27, 2021 1 0 1 V04 U.S. -Based October 28, 2021 1 0 1 V05 Local October 28, 2021 1 0 1 V06 U.S. -Based October 19, 2021 0 1 1 V07 U.S. -Based October 20, 2021 0 1 1 67 Code Type Date M F Total V08 U.S.-Based October 20, 2021 1 0 1 V09 U.S.-Based October 25, 2021 0 1 1 V10 U.S.-Based October 26, 2021 1 0 1 V11 U.S.-Based October 27, 2021 0 1 1 V12 U.S.-Based October 28, 2021 1 0 1 V13 U.S.-Based October 29, 2021 0 1 1 V14 Local October 26, 2021 1 0 1 V15 U.S.-Based November 3, 2021 0 1 1 V16 U.S.-Based October 26, 2021 1 0 1 V17 Local October 21, 2021 0 1 1 V18 U.S.-Based October 20, 2021 1 0 1 V19 Local October 16, 2021 0 1 1 V20 U.S.-Based October 25, 2021 1 0 1 V21 Local October 25, 2021 1 0 1 V22 Local October 28, 2021 1 0 1 V23 Local October 29, 2021 1 0 1 V24 U.S.-Based October 29, 2021 0 1 1 V25 U.S.-Based November 1, 2021 0 1 1 V26 Local November 3, 2021 0 1 1 V27 Local November 5, 2021 0 1 1 V28 Local November 5, 2021 0 1 1 V29 Local November 3, 2021 1 0 1 V30 U.S.-Based November 3, 2021 1 0 1 H01 Zimbabwe _host November 5, 2021 0 1 1 H02 Nigeria _host November 1, 2021 1 0 1 H03 Ghana _host November 4, 2021 0 1 1 H04 Mozambique _host November 3, 2021 1 0 1 H05 Nigeria _host November 2, 2021 0 1 1 H06 Philippines _host November 2, 2021 1 0 1 H07 Philippines _host November 3, 2021 0 1 1 H08 Georgia _host October 25, 2021 1 0 1 H09 Nepal _host November 3, 2021 0 1 1 H10 Rwanda _host October 28, 2021 1 0 1 H11 Rwanda _host October 28, 2021 1 0 1 H12 Uganda _host October 13, 2021 1 0 1 H13 Tanzania _host October 21, 2021 0 1 1 H14 Tanzania _host October 15, 2021 0 1 1 68 Code Type Date M F Total H15 Tanzania _host November 2, 2021 0 1 1 H16 Guatemala _host November 1, 2021 1 0 1 H17 Ecuador _host November 5, 2021 0 1 1 H18 Guatemala _host November 2, 2021 1 0 1 H19 Lebanon _host October 29, 2021 1 0 1 H20 Lebanon _host November 2, 2021 0 1 1 H21 Guyana _host November 3, 2021 1 0 1 H22 Armenia _host November 4, 2021 0 1 1 H23 Peru _host November 5, 2021 0 1 1 H24 Colombia _host November 5, 2021 0 1 1 H25 Lebanon _host October 27, 2021 0 1 1 66 66 132 69 ANNEX 4: INTERVIEW GUIDES INTERVIEW WITH USAID HQ STAFF Researcher Name: 70 5. [EQ2] From your perspective, what have F2F’s biggest successes/impacts been? 6. [EQ2] Which activity or component stands out for you as particularly successful? How so? Why do you think this was the case? 7. How do the F2F activities intersect with other USAID country programs to respond to the recent crises? [probe on COVID-19 pandemic, and specific major crisis in regions] 8. [EQ2] Looking back on your work thus far, is there anything you think the program should have done differently regarding any of its core activities? 9. [EQ3] To the extent visible at the midpoint of the program, where do you think F2F’s outcomes and impacts will be most achievable? Why? 10. [EQ3] Were there any buy-ins with other projects or associate awards within the past three years of the F2F program? To what extent were these buy-ins effective and efficient in helping to implement the activities and achieving initial results? What were the major successes of such approach? Monitoring of Indicators 11. [EQ1] Is the current execution of the M&E plan by IPs efficient enough to inform decision-making at the USAID level? How so? 12. [EQ1] Does the F2F program get reported in the Mission’s annual report? If so, do you use the standard F2F indicators, or do they feed into other indicators used by the Mission? If you use F2F indicators, do you feel that they are comprehensive and accurately capture the impact of the F2F program? 13. [EQ1] Does each indicator have a clear description and definition to ensure data quality and consistency? If so, is this description and definition consistent across all IPs collecting and reporting data for the same indicator? 14. [EQ3] Are the lessons learned and/or innovations introduced by the F2F program integrated into other USAID Mission projects? 15. [EQ2] Which perceptions or ideas could improve F2F’s learning to become more efficient and effective? Gender and Youth Inclusion 16. [EQ3] What are the F2F program goals and objectives for greater emphasis on women and youth inclusion? What impacts and outcomes are of the greatest priority? What have been the results, successes, and failures? What are the major constraints and opportunities? 17. [EQ3] Thirty-five (35) percent is the target many IPs have selected for recruitment of women F2F volunteers. Did USAID suggest this target? Why? Where are women volunteers most effective in reaching program diversity and empowerment goals? In retrospect is the 35 percent target still valid? 18. [EQ3] With respect to M&E, how are women’s and youth’s advancement along a value chain, commercial enterprise, or agricultural education development organization being captured? What improvements do you suggest? AVOP-related 19. [EQ2] What is the mission of the AVOP program, as you understand it? How does the AVOP program accomplish this mission? What other ways—outside of the AVOP program—could F2F use to accomplish this mission? What do you think can be done to improve the AVOP program? 71 COVID-19 Pandemic 20. [EQ3] To what extent has the COVID-19 global pandemic impacted F2F implementation? Which activities were affected? What were the strategies that USAID asked IPs to implement to support project activities during the pandemic? How effective were the alternative tools and approaches used? 21. [EQ3] To what extent has the COVID-19 global pandemic affected achieving the intended results? To what extent do you think the pandemic affected the achievement of the program outcomes and objectives? Will USAID continue to encourage the use of strategies implemented in response to the pandemic beyond the pandemic? For example, the model of remote US-based volunteers and local paired volunteers? 22. [EQ3] What are the lessons learned that you take from the period of COVID-19 global pandemic? INTERVIEW WITH IMPLEMENTING PARTNER (MANAGEMENT) Researcher Name: Notetaker Name: Interview Date: Respondent ID: Respondent Partner: Region/Country: Respondent Position: Years in Current Business: Respondent Gender: Start Time: End Time: Consent Statement Hello, I am _______________ and I work with ME&A, an American company conducting research for the United States Agency for International Development (USAID). We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful F2F has been in meeting its objectives and how its work over the past two years can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. There are no expected direct benefits to you beyond the possibility of helping to improve the design of USAID programs that you may implement in the future. But nor are there expected to be any significant risks to your participation. The report will not connect your name with any of the specific information you provide in the interview. On rare occasions, USAID may ask to access the notes from this interview, which we would provide them upon request. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. 72 Are you willing to participate? Yes___ No___ (if respondent says no, stop the interview immediately) Do you allow voice recording of the interview? Yes___ No___ Introduction/Warm-Up 1. What is your role in the implementation of F2F within your organization? 2. What type of funding has your organization received to implement the F2F program? Let’s talk about your organization’s involvement with F2F program and the effects of those partnerships. 3. [EQ3] Overall, how satisfied have you been with F2F program? Why? 4. [EQ3] What has worked well during your organization's implementation of the F2F program? What have the challenges been? 5. [EQ3] What has the F2F program enabled your organization to do that it wouldn’t have been able to do otherwise? 6. [EQ3] What do you think have been the biggest successes resulting from the F2F program? What factors assisted these successes? 7. [EQ3] To what extent has the program been successful in meeting its objectives for: Increasing gross value of sales? [Probes: biggest successes, biggest challenges, progress on targets] Achievement of development objectives? [Probes: biggest successes, biggest challenges, progress on targets] Improvement of products and services? [Probes: biggest successes, biggest challenges, progress on targets] Gender and youth inclusion? [Probes: biggest successes, biggest challenges, progress on targets] 8. [EQ3] What were the major factors influencing the achievement or non achievement of reaching the intended results? Did these factors fall under your control or outside? What were the steps that you took to overcome the non-achievement of these results? 9. [EQ3] To the extent visible at the midpoint of the program, where do you think F2F’s outcomes and impacts will be most achievable? Why? 10. [EQ3] Were there any buy-ins with other projects or associate awards within the past three years of the F2F program? To what extent were this buy-ins effective and efficient in helping to implement the activities and achieving initial results? What were the major successes of such approach? Monitoring of Indicators 11. [EQ1] To what extent do the indicators you collect drive your management plan? Which indicators that you collect are most important for the management plan? Which indicators that you collect do you feel do not enhance your management decisions? How are these data used for designing future programs? Are these data shared with volunteers after they have left? 12. [EQ1] Based on your experience, do you have any recommendations going forward for linking M&E with implementation? Program Learning and Innovation 73 13. [EQ2] Which perceptions or ideas could improve F2F’s learning to become more efficient and effective? To what extent would your organization be able to implement these ideas? [probe on challenges, opportunities] 14. [EQ2] To what extent did the stakeholders (hosts, volunteers, other local partners) influence the program activities? In which way? 15. [EQ2] To what extent can you negotiate with the stakeholders (hosts, volunteers, other local partners) in sensitive matters related to the program’s activities? [probe on business records, income, illiteracy…] 16. [EQ2] For AVOP IPs: What is the mission of the AVOP program, as you understand it? How does the AVOP program accomplish this mission? What other ways—outside of the AVOP program— could F2F use to accomplish this mission? What do you think can be done to improve the AVOP program? Gender and Youth Inclusion 17. [EQ3] What are the key factors in program design (including selection of hosts, sectors, and volunteers) to empower women and youth in their F2F supported activities? What are the major constraints that you encounter in women and youth empowerment? 18. [EQ3] Do the existing monitoring mechanisms and indicators capture an improvement in women’s and youth’s roles, productivity, and leadership? If not, what would you recommend? May I please get the contact details for the senior level diversity specialist (women and youth) in your office if follow up is necessary? 19. [EQ3] In the current context, what, if anything, could be done to help improve gender, youth, and Person with Disability (PWD) inclusion under F2F program? COVID-19 Pandemic 20. [EQ3] To what extent has the COVID-19 global pandemic impacted F2F implementation? Which activities were affected? What were the alternatives that you adopted to maintain the project activities in response to the pandemic? How effective were the alternative tools and approaches used? 21. [EQ3] To what extent has the COVID-19 global pandemic affected achieving the intended results? To what extent do you think the pandemic affected the achievement of the program outcomes and objectives? 22. [EQ3] What are the lessons learned that you take from the COVID-19 global pandemic? INTERVIEW WITH IMPLEMENTING PARTNER (FIELD OFFICERS) Researcher Name: Notetaker Name: Interview Date: Respondent ID: Respondent Partner: Region/Country: Respondent Position: Years in Current Business: 74 Respondent Gender: Start Time: End Time: Consent Statement Hello, I am _______________ and I work with ME&A, an American company conducting research for the United States Agency for International Development (USAID). We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful F2F has been in meeting its objectives and how its work over the past two years can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary, and you are free to decline to respond to any question for any reason and to end the interview at any time. There are no expected direct benefits to you beyond the possibility of helping to improve the design of USAID programs that you may implement in the future. But nor are there expected to be any significant risks to your participation. The report will not connect your name with any of the specific information you provide in the interview. On rare occasions, USAID may ask to access the notes from this interview, which we would provide them upon request. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Are you willing to participate? Yes___ No___ (if respondent says no, stop the interview immediately) Do you allow voice recording of the interview? Yes___ No___ Introduction/Warm-Up 1. What is your role in the implementation of F2F within your organization? 2. What type of involvement has your organization had with the F2F program? Let’s talk about your organization’s involvement with F2F program and the effects of those partnerships. 3. [EQ3] Overall, how satisfied have you been with F2F program? Why? 4. [EQ3] What has worked well during your organization's implementation of the F2F program? What have the challenges been? 5. [EQ3] What has the F2F program enabled your organization to do that it wouldn’t have been able to do otherwise? 6. [EQ3] What do you think have been the biggest successes resulting from the F2F program? What factors assisted these successes? 7. [EQ3] To what extent has the program been successful in meeting its objectives for: Increasing gross value of sales? [Probes: biggest successes, biggest challenges, progress on targets] Achievement of development objectives? [Probes: biggest successes, biggest challenges, progress on targets] 75 Improvement of products and services? [Probes: biggest successes, biggest challenges, progress on targets] Gender and youth inclusion? [Probes: biggest successes, biggest challenges, progress on targets] Program Learning and Innovation 8. [EQ2] To what extent did the stakeholders (hosts, volunteers, other local partners) influence the program activities? In which way? 9. [EQ3] What has worked well during your organization's implementation of the F2F program? What have the challenges been? Gender and Youth Inclusion 10. [EQ3] In the current context, what, if anything, could be done to help improve gender, youth, and PWD inclusion in financial services? 11. [EQ3] To what extent do you think the F2F program was able to help address the primary support needs of gender and youth at host organizations? COVID-19 Pandemic 12. [EQ3] To what extent has the COVID-19 global pandemic impacted F2F implementation? Which activities were affected? What were the alternatives that you adopted to maintain the project activities in response to the pandemic? How effective were the alternative tools and approaches used? 13. [EQ3] To what extent has the COVID-19 global pandemic affected achieving the intended results? To what extent do you think the pandemic affected the achievement of the program outcomes and objectives? 14. [EQ3] What are the lessons learned that you take from the COVID-19 global pandemic? INTERVIEW WITH IMPLEMENTING PARTNER (M&E STAFF) Researcher Name: Note taker Name: Interview Date: Respondent ID: Respondent Partner: Region/Country: Respondent Position: Years in Current Business: Respondent Gender: Start Time: End Time: Consent Statement 76 Hello, I am _______________ and I work with ME&A, an American company conducting research for the United States Agency for International Development (USAID). We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful F2F has been in meeting its objectives and how its work over the past two years can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. There are no expected direct benefits to you beyond the possibility of helping to improve the design of USAID programs that you may implement in the future. But nor are there expected to be any significant risks to your participation. The report will not connect your name with any of the specific information you provide in the interview. On rare occasions, USAID may ask to access the notes from this interview, which we would provide them upon request. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Are you willing to participate? Yes___ No___ (if respondent says no, stop the interview immediately) Do you allow voice recording of the interview? Yes___ No___ Introduction/Warm-Up 1. What is your role in the implementation of F2F within your organization? 2. What type of involvement has your organization had with the F2F program? Monitoring of Indicators 3. [EQ1] Based on your experience in collecting and analyzing data, could please inform us on the extent of which you are able to collect data related to the following topics: − Outreach/cultural exchange? − Capacity development? − Gender and youth inclusion? − Environmental and natural resources management? Which ones are easy/difficult to collect? Why? Which ones are easy/difficult to analyze? Why? 4. [EQ1] To what extent do the indicators you collect drive your management plan? Which indicators that you collect are most important for the management plan? Which indicators that you collect do you feel do not enhance your management decisions? How are these data used for designing future programs? Are these data shared with volunteers after they have left? 5. [EQ1] From the standard indicators list, could you please indicate which of the output indicators you face challenges in collecting them? Are these indicators useful for your work? Are these indicators important in leading to the desired outcomes? In which ways? 6. [EQ1] How were the indicators selected? Was the selection of indicators a part of the requirements from F2F? Was it a result of something else? 7. [EQ1] Do you have custom indicators to track progress within your organization and/or this program? 77 8. [EQ1] From the standard indicators list, could you please indicate which of the output indicators you don’t face challenges in collecting them? Are these indicators useful for your work? Are these indicators important in leading to the desired outcomes? In which ways? 9. [EQ1] What was the original vision for linking M&E with design and implementation of this program? How has that vision been realized? 10. [EQ1] Based on your experience, do you have any recommendations going forward for linking M&E with implementation? 11. [EQ3] To what extent have the monitoring mechanisms been focused on the results, outcomes and objectives and helped to improve them? 12. [EQ2] Which perceptions or ideas could improve F2F’s learning to become more efficient and effective? Did you implement these perceptions/ideas? If yes, what were their results on the program? If not, why? 13. [EQ3] Do the existing monitoring mechanisms and indicators capture an improvement in women’s and youth’s roles, productivity, and leadership? If not, what would you recommend? COVID-19 Pandemic 14. [EQ3] To what extent has the COVID-19 global pandemic impacted F2F implementation? Which M&E activities were affected? What were the alternatives that you adopted to maintain the M&E activities in response to the pandemic? How effective were the alternative tools and approaches used? 15. [EQ3] To what extent has the COVID-19 global pandemic affected achieving the intended results? To what extent do you think the pandemic affected the achievement of the program outcomes and objectives? 16. [EQ3] What are the lessons learned that you take from the COVID-19 global pandemic? INTERVIEW WITH HOSTS Researcher Name: Notetaker Name: Interview Date: Respondent ID: Respondent Partner: Region/Country: Respondent Position: Years in Current Business: Respondent Gender: Start Time: End Time: Consent Statement Hello, I am _______________ and I work with ME&A, an American company conducting research for the United States Agency for International Development (USAID). We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful 78 F2F has been in meeting its objectives and how its work over the past two years can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. There are no expected direct benefits to you beyond the possibility of helping to improve USAID programming in your country. But nor are there expected to be any significant risks to your participation. The report will not connect your name with any of the specific information you provide in the interview. On rare occasions, USAID may ask to access the notes from this interview, which we would provide them upon request. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Are you willing to participate? Yes___ No___ (if respondent says no, stop the interview immediately) Do you allow voice recording of the interview? Yes___ No___ Introduction/Warm-Up 1. What type of host organization do you belong to? 2. What is your role with your organization? 3. What type of assistance has your organization received through the F2F program? 4. How did your organization use this assistance through the F2F program? Let’s talk about your organization’s involvement with F2F program and the effects of those partnerships. 5. [EQ3] Have you participated/benefited from any other USAID programs or projects? 6. [EQ3] How did you hear about F2F work/activities? 7. [EQ3] How satisfied have you been with your involvement in the F2F program overall? Why? 8. [EQ3] What has worked well during the partnership with F2F program? What have the challenges been? 9. [EQ3] What has the assistance you received from F2F program enabled your organization to do that it wouldn’t have been able to do otherwise? 10. [EQ3] What do you think have been the biggest successes resulting from the support you received from F2F program? What factors assisted these successes? 11. [EQ3] To what extent has the program been successful in meeting its objectives for: Increasing gross value of sales? [Probes: biggest successes, biggest challenges, progress on targets] Achievement of development objectives? [Probes: biggest successes, biggest challenges, progress on targets] Improvement of products and services? [Probes: biggest successes, biggest challenges, progress on targets] 79 Gender and youth inclusion? [Probes: biggest successes, biggest challenges, progress on targets] Other, please specify: [Probes: biggest successes, biggest challenges, progress on targets] 12. [EQ3] What has been your experience with how volunteers are selected and matched to the scope of work? Have you had any issues with volunteers not completing their deliverables? 13. [EQ1] To what extent was the development of the scope of work challenging? Were you confident that the scope of work was clear and stood a good chance of success? Do you feel that volunteers do well in terms of being able to tailor training to your needs? Gender and Youth Inclusion 14. [EQ3] How were women and youth considered in your initial discussions with the F2F Program team? Were there any criteria or other commitment that you had to meet? If yes, what? Innovation and Learning 15. [EQ2] What types of logistical challenges have you faced as a host organization? What was the cause of these challenges? How did you solve them? How can logistics be improved for host organizations? Besides logistics, what other challenges have you faced? 16. [EQ2] What types of language challenges have you faced as a host organization? What was the cause of these challenges? How can these be improved? 17. [EQ1] The F2F program is interested in how you collect and report on required indicators. Please tell me about your reporting processes and how you report on indicators. What tools for data and knowledge management do you use? What is missing? What would make your work easier (besides staff and time and money)? Do you feel these indicators capture the impact of the F2F program? 18. [EQ2] Upon completion of the assignment/SOW, has follow up by the volunteer been adequate? If not, what would you recommend? What about follow up by the IP staff? Any recommendations? 19. Would you participate again in the F2F program? What lessons would you take away for future work with F2F? COVID-19 Pandemic 20. [EQ3] To what extent COVID-19 global pandemic impacted F2F activities? Which activities were affected? 21. [EQ3] Were the alternatives for training and assistance adopted by the implementer and the volunteer able to maintain the project activities during the pandemic? How effective were the alternative tools and approaches used? 22. [EQ3] To what extent did the COVID-19 global pandemic impact the activities included in the F2F SOW? Which activities were affected? To what extent do you think the pandemic affected the achievement of proposed recommendations by the volunteers? 23. [EQ3] Given the current context related to the pandemic, what approaches would you recommend for future hosts and volunteers? INTERVIEW WITH VOLUNTEERS Researcher Name: 80 Notetaker Name: Interview Date: Respondent ID: Respondent Partner: Region/Country: Respondent Position: Years in Current Business: Respondent Gender: Start Time: End Time: Consent Statement Hello, I am _______________ and I work with ME&A, an American company conducting research for the United States Agency for International Development (USAID). We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful F2F has been in meeting its objectives and how its work over the past two years can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. There are no expected direct benefits to you beyond the possibility of helping to improve the design of USAID programs that you may participate in going forward. But nor are there expected to be any significant risks to your participation. The report will not connect your name with any of the specific information you provide in the interview. On rare occasions, USAID may ask to access the notes from this interview, which we would provide them upon request. To ensure all your responses are captured accurately and can be reviewed during data analysis and report writing, we would like to record the interview. The recordings will be kept in a secure data storage and will be destroyed after submission of the report. The report may be made public upon finalization. This interview will last approximately one hour. Are you willing to participate? Yes___ No___ (if respondent says no, stop the interview immediately) Do you allow voice recording of the interview? Yes___ No___ Introduction/Warm-Up 1. How long ago were you an F2F volunteer? 2. How many times have you volunteered with the F2F program? 3. What was the focus of your activities in your most recent assignment? Effectiveness Let’s talk about your involvement with the F2F program and the related effects. 4. [EQ1] For volunteers who volunteered for the first time: When you first volunteered, how did you hear about F2F? Was the experience what you expected? 5. [EQ1] For volunteers who volunteered more than once with F2F: What improvements have you noticed from your older assignments to your more recent ones? 81 6. [EQ3] Overall, how satisfied have you been with your involvement with the F2F program? Why? 7. [EQ3] What has worked well during your involvement with the F2F program? What have the challenges been? 8. [EQ3] What has the support you provided through the F2F program enabled the host organization to do that it wouldn’t have been able to do otherwise? 9. [EQ3] What do you think have been the biggest successes resulting from the support you provided through F2F program? What factors assisted these successes? 10. [EQ3] Is there anything you had hoped to achieve but were unable to achieve? If so, what? And what factors contributed to this? Learning and Innovation 11. [EQ1] When you think about the technical support you provided and your experience overall, do you feel confident that the host you worked with can be effective drivers of change? How so? 12. [EQ1] To what extent were you aware about the outcomes of your assignment in relation to applying the proposed recommendations? From whom did you get the related information? What kind of information was shared with you? 13. [EQ2] Can you say a bit about what happens after your assignment is complete? 14. [EQ2] Was there strong communication with the IP about what worked and what didn't? 15. [EQ2] Was there follow up on any deliverables not completed during the assignment? 16. [EQ2] Would you volunteer again? And if so, how would you integrate any lessons learned into a new assignment? Gender and Youth Inclusion 17. [EQ3] The F2F program has, in the past few years, become more focused on women and youth inclusion. Was this focus discussed in any of the program materials or interviews? How so? Were you instructed or encouraged to focus on women and/or youth in the assignment? If so, were you provided recommendations or other support in term of best practices for gender and/or youth inclusion? 18. [EQ3] Do you prefer working with hosts where women and/or youth inclusion and empowerment is a focus? Why? 19. [EQ3] Do you believe that you were able to be responsive and positively benefit women and/or youth? How so? In you were to volunteer again what are the primary recommendations and actions that you would implement for empowering women and youth? 20. [EQ3] Aside from the COVID-19 pandemic, have you faced logistical challenges as a volunteer? What was the cause of these challenges and was the challenge resolved? 21. [EQ3] Did you experience any language barriers and how you mitigate these challenges? Did this experience affect the results of your assignment? COVID-19 Pandemic 22. [EQ3] We know most volunteers operated remotely during this period, with limited exceptions. Did you travel for your assignment or operate remotely? 23. [EQ3] If you did a paired assignment, to what extent was the paired volunteers’ mechanism effective in assisting host organizations? Please describe the process. What were the associated challenges and how you resolved them? What was the role of IP in resolving them? What was the role of the host organizations in resolving them? 82 24. [E3] If the assignment was done remotely, could please tell us how effective were the alternative tools and approaches used? What kind of logistics measures were taken to help the service delivery? 25. [EQ3] To what extent has the COVID-19 global pandemic affected achieving the intended results from your assignment? To what extent do you think the pandemic affected the achievement of the recommendations you provided to the hosts? 26. [EQ3] What are the lessons learned that you take from the COVID-19 global pandemic? 83 ANNEX 5: VOLUNTEER AND HOST ONLINE SURVEY QUESTIONNAIRES VOLUNTEER SURVEY QUESTIONNAIRE Consent We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) program. We want to understand how successful F2F has been in meeting its objectives and how its work can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in the agriculture sector. Your participation in this interview is completely voluntary, and you are free to decline to respond to any question for any reason and to end the interview at any time. The data that you will provide will be kept in a secure and encrypted data storage and your identity will be kept anonymous. Question Answer Choice How long ago were you a Farmer-to￾Farmer volunteer? 1=Previous 0-2 years 2=Previous 3-5 years 3=6 or more years ago What is your sex? 1=Male 2=Female 3=non-binary 99=Prefer not to answer How did you first hear about the USAID Farmer-to-Farmer program? 1=List serve/email 2=Social media (Facebook, twitter other) 3=Website 4=F2F Implementer 5=Former F2F volunteer 6=Program partner/stakeholder (not a F2F Implementer) 7=Event 96=Other: Specify [OPEN-ENDED] What was your primary employment before your most recent volunteer opportunity with Farmer-to-Farmer? 1=Cooperative/Association 2=Public and Private Education Institution 3=Individual Private Farmer 4=Public Sector/Government Agency 5=Non-Profit, Public Interest NGO 6=Other Private Enterprise 7=Rural Financial Institution 8=Retired 9=Student In which regions have you served as a volunteer for Farmer-to-Farmer? [Please select all that apply.] 1=Asia 2=East Africa 3=Former Soviet Union 4=Latin America and Caribbean 5=Middle East and North Africa 6=Southern Africa 7=West Africa 8=Europe (including Balkan states) 96=Other: Specify [OPEN-ENDED] How many times have you volunteered with Farmer-to-Farmer? Range 1-10 84 Question Answer Choice What were the reasons why you wanted to volunteer with the Farmer-to-Farmer program? [Choose up to 3 most important reasons] 1=Had volunteered with F2F previously 2=Personally know others who volunteered with F2F 3=Interest in cultural exchange 4=Interest in supporting host organizations and countries 5=Previous (non-F2F) experience in developing countries 6=Personal relationship to NGOs who are implementing F2F activities 7=Desire to travel 8=Interest in “giving back” 96=Other: Specify [OPEN-ENDED] What were the main challenges you faced as a volunteer? [Please select all that apply.] 1=Cultural barriers 2=Language barriers 3=Recruitment process was unclear or poorly organized 4=Poor communication from host organization 5=Poor communication from program/implementing agency 6=Assigned to project outside of area of interest/expertise 7=Expectations for assignment not clearly explained 8=No challenges How likely are you to volunteer again with Farmer-to-Farmer? 1=Not at all likely 2=Somewhat unlikely 3=Unsure 4=Somewhat likely 5=Very likely For the following questions in this survey, please think only about your most recent assignment with the Farmer-to-Farmer Program. How long ago did you finish your most recent Farmer-to-Farmer assignment? 1=0-6 months 2=7-12 months 3=More than 1 year-3 years 4=More than 3 years-5 years 5=More than 5 years What was the focus of your activities in your most recent assignment? 1=Increase agricultural sector productivity and profitability 2=Improve conservation and sustainable use of environmental and natural resources 3=Expand agricultural sector access to financial services 4=Strengthen agricultural sector institutions 5=Administrative/logistic development 6=Gender or Youth focused 96=Other: Specify [OPEN-ENDED] How would you classify the main recommendations that you provided to the host for your most recent assignment? 1=Economic 2=Organizational 3=Environmental 4=Financial 96=Other: Specify [OPEN-ENDED] Did you volunteer during the pandemic? 1= No 2=Yes Did you do a remote or a paired remote assignment? 1= No 2=Yes If you did a paired assignment, how satisfied were you with the paired remote US volunteer mechanism? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied 85 Question Answer Choice How satisfied were you with the competency shown by the counterpart volunteer? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied What were some of the challenges you faced working together with your paired counterpart? 1=Cultural barriers 2=Language barriers 3=Work planning challenges 4=Time zone difference 96=Other: Specify [OPEN-ENDED] Do you feel working with your paired counterpart added value to the outcomes of your assignment? 1=Added immense value 2=Added value 3=Neutral 4=Did not add value 5=Detracted value How effective were the alternative tools and approaches used during the COVID￾19 pandemic? 1=Very Effective 2=Neutral 3=Not effective How many times did you speak with the Implementing Partner’s Head Office in the United States for your most recent assignment? 1=0 2=1-3 3=4-6 4=More than 7 times How satisfied were you with the overall competency shown by the implementing partner? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied What assistance/deliverables were due as part of your most recent assignment? [Please select all that apply.] 1=Training/mentoring 2=Demonstrations 3=Educational materials/manuals 4=Business strategy plans 5=Business assessment 6=Equipment assembling/calibration/fix 7=Creation of a video, media, or a website 96=Other: Specify [OPEN-ENDED] How many deliverables were due? Range 1-20 What percentage of these deliverables did you complete? 1=Less than 25% 2=25-50% 3=51-75% 4=76-99% 5=100% What were the inhibiting factors that prevented you from completing the deliverables? [Please select all that apply.] 1=Length of assignment was too short 2=Project activities were delayed 3=Deliverables were not clearly stated 4=Lack of support from host organization 5=Lack of support from program/implementing agency 6=Activities competed were different than those listed in Statement of Work 96=Other: Specify [OPEN-ENDED] If your counterpart volunteer had separate deliverables, how satisfied were you with the quality of deliverables? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 86 Question Answer Choice 4=Satisfied 5=Very Satisfied How often have you communicated with your host since you left? 1=Daily 2=Weekly 3=Monthly 4=Less than 1 time a month 5=Never You mentioned that you communicated with your host since you left. What was the primary purpose of this communication? 1=Finalizing deliverables 2=Providing ongoing guidance related to your assignment 3=Providing ongoing guidance unrelated to your assignment 4=Personal communications 96=Other: Specify [OPEN-ENDED] Did you have a meeting with the implementing partner to discuss your reports and recommendations at the conclusion of your most recent volunteer assignment? 1=Yes 2=No 98=Don’t know HOST SURVEY QUESTIONNAIRE Consent We are conducting a midterm performance evaluation of the Farmer-to-Farmer (F2F) Program. We want to understand how successful F2F has been in meeting its objectives and how its work can inform future USAID work. You have been selected to participate in this evaluation because we believe you can provide a unique perspective on F2F and its potential contributions in agriculture sector. Your participation in this interview is completely voluntary and you are free to decline to respond to any question for any reason and to end the interview at any time. Question Answer Choice What type of host organization do you belong to? 1=Farmer cooperative or association 2=Individual private farmer 3=Other private enterprise 4=NGO 5=Educational institution 6=Public sector agency 7=Rural financial institution 98=Don't know What is your sex? 1=Male 2=Female 3=Non-binary 99=Prefer not to answer How many fulltime, contract staff does your organization employ? 1=Less than 10 2=10-20 3=20-40 4=40-60 5=More than 60 98=Don't know 87 Question Answer Choice What type of assistance has your organization received as part of the F2F program? [Please select all that apply.] 1=Technology transfer 2=Organizational development 3=Business/Enterprise development 4=Environmental conservation 5=Financial services 6=Administrative 7=Gender and/or youth inclusion 96=Other: Specify [OPEN-ENDED] How satisfied have you been with your involvement in the F2F program overall? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied were you with your organization's involvement in developing a plan for support from the F2F program, including identification of your organizations goals and expected results from participating in the program? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied were you with your organization’s involvement in developing the Scopes of Work (SOWs) for the volunteers, which detailed the volunteer’s responsibilities and activities? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied For the most recent assignment, at what stage was your organization involved in developing the SOW for the volunteer? 1=Your organization created first version of the SOW 2=Your organization and the implementing partner created first version of the SOW together 3=Implementing partner created first version of the SOW, which your organization made changes to 4=Your organization received finalized SOW without the opportunity to provide ideas 5=Your organization never received SOW for volunteer 96=Other: Specify [OPEN-ENDED] 98=Don't know To what extent was the proposed SOW for the volunteer affected by the COVID-19 pandemic? 1=Not affected 2=Little 3=Normal 4=Affected considerably 5=Very Affected How satisfied were you with the mitigation measures that the implementer takes as a response to the COVID-19 pandemic? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied Did you receive remote U.S. assistance? 1=No 2=Yes Did you receive paired remote assistance? 1=No 2=Yes How satisfied were you with the U.S. volunteer? 1=Very Dissatisfied 2=Dissatisfied 88 Question Answer Choice 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied were you with the local volunteer? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied Please answer the following questions related to your satisfaction at your organization. How satisfied were you with the recommendations made by the volunteers? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied were you with the quality of the remote training implemented by the U.S. volunteer? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied were you with the quality of the activities implemented by the local volunteer? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How satisfied are you with your ability to follow up on the work of the volunteers? 1=Very Dissatisfied 2=Dissatisfied 3=Neither Satisfied nor Dissatisfied 4=Satisfied 5=Very Satisfied How often have you communicated with this/these volunteers since they left? 1=Daily 2=Weekly 3=Monthly 4=Less than 1 time a month 5=Never You mentioned you communicated with these volunteers since they left. With whom did you communicate? 1=US Volunteer 2=Local Volunteer 3=Both Volunteers You mentioned you communicated with the U.S. Volunteer since he/she left. What was the primary purpose of this communication? 1=Finalizing deliverables 2=Provided ongoing guidance related to their assignment 3=Provided ongoing guidance unrelated to their assignment 4=Personal communications 96=Other: Specify [OPEN-ENDED] You mentioned you communicated with the Local Volunteer since he/she left. What was the primary purpose of this communication? 1=Finalizing deliverables 2=Provided ongoing guidance related to their assignment 3=Provided ongoing guidance unrelated to their assignment 4=Personal communications 96=Other: Specify [OPEN-ENDED] 89 Question Answer Choice You mentioned you communicated with both volunteers. What was the primary purpose of this communication? 1=Finalizing deliverables 2=Provided ongoing guidance related to their assignment 3=Provided ongoing guidance unrelated to their assignment 4=Personal communications 96=Other: Specify [OPEN-ENDED] What types of recommendations did you receive from the volunteers? 1=Economic/Technical 2=Organizational 3=Environmental 4=Financial 96=Other: Specify [OPEN-ENDED] Approximately what percentage of the volunteer’s recommendations have you already implemented? What factors have limited your ability to implement all the recommendations? [Please select all that apply.] 1=Volunteer left recently, so have not had time to implement all recommendations 2=Lack of financial resources 3=Lack of local access to goods/services mentioned in recommendations 4=Legal restrictions on enacting recommendations 5=COVID-19 Restrictions 6=Did not find recommendations practical or helpful 96=Other: Specify [OPEN-ENDED] Please read the following statements and indicate the extent to which you agree or disagree. The work supported by the volunteer was significantly improved because of their contributions. 1=Strongly disagree 2=Disagree 3=Neither disagree nor agree 4=Agree 5=Strongly agree 97=Not applicable 98=Don't know The opinions of my organization were valued and respected by the volunteer(s). 1=Strongly disagree 2=Disagree 3=Neither disagree nor agree 4=Agree 5=Strongly agree 97=Not applicable 98=Don’t know My discussions with the volunteer included careful consideration of how potential benefits might be distributed differently between men and women. 1=Strongly disagree 2=Disagree 3=Neither disagree nor agree 4=Agree 5=Strongly agree 97=Not applicable 98=Don’t know Women and men are equal participants in discussions related to assignment activities 1=Strongly disagree 2=Disagree 3=Neither disagree nor agree 4=Agree 5=Strongly agree 97=Not applicable 98=Don't know Did you face any challenges with your F2F volunteers? 1=No 2=Yes 90 Question Answer Choice You mentioned you faced challenges with your F2F volunteers. What were the main one(s)? 1=Cultural barriers 2=Language barriers 3=Recruitment process of volunteer was unclear or poorly organized 4=Poor communication from volunteer 96= Other. Specify: [OPEN-ENDED] How do you share data with the local F2F staff? 1=Using my own templates 2=Using F2F provided templates 96=Other. Specify: [OPEN-ENDED] Which of the following systems do you use to verify the data that is reported? [Please select all that apply.] 1=The same data collected by 2 or more staff 2=Data collected by someone with no association to the F2F program 3=No verification system is used [END SURVEY] 96=Other: Specify [OPEN-ENDED] Would you like to host a F2F volunteer again? 1=No 2=Yes 91 ANNEX 6: SIGNED CONFLICT OF INTEREST FORMS 92 93 94 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523