MID-TERM PERFORMANCE EVALUATION OF THE REGIONAL AGRICULTURE DEVELOPMENT PROGRAM IN AFGHANISTAN February 2019 Prepared for the United States Agency for International Development Contract No.: AID-OAA-A-13-00040 Submitted by: Abigail Conrad, Project Director/Qualitative Analysis Lead, Abt Associates Charlotte Maxwell-Jones, Data Collection Manager Marius Meijerink, Analyst, Abt Associates Betsy Ness-Edelstein, Analyst, Abt Associates Phomdaen Souvanna, Analyst, Abt Associates Russ Jones, GIS/Remote Sensing Analysis Lead, Abt Associates Tulika Narayan, Project Quality Advisor, Abt Associates Contractor: ME&A, Inc. (PEEL) 1020 19th Street NW, Suite 875 Washington, DC 20036 Tel: 240-762-6296 www.meandahq.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS ABSTRACT.......................................................................................................................................................................... I EXECUTIVE SUMMARY.................................................................................................................................................. II 1. EVALUATION PURPOSE AND QUESTIONS ...................................................................................................8 Evaluation Purpose .............................................................................................................................................8 Evaluation Audience...........................................................................................................................................8 Evaluation Questions.........................................................................................................................................8 2. PROJECT BACKGROUND.....................................................................................................................................9 3. EVALUATION METHODS AND LIMITATIONS ............................................................................................12 Qualitative Approach.........................................................................................................................................12 Data Sources........................................................................................................................................................12 Data Analysis........................................................................................................................................................15 Limitations............................................................................................................................................................15 4. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ......................................................................16 EQ1: Is the agribusiness support model likely to be sustainable?...........................................................16 EQ2: Is the training model design and implementation effective in contributing to beneficiary adoption of improved agricultural technologies and licit crops? .....................................................25 EQ3: Based on the implementation approach, status of Implementation, and milestones achieved at the time of mid-term evaluation, is there evidence that the program is likely to achieve its results by the end date?.............................................................................................................................33 EQ4: To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals?..............42 EQ5: Is the regional approach to the RADP programming likely most effective for achieving program goals? For example, how might a regional approach compare to a commodity-specific approach? ......................................................................................................................................................47 EQ6: To what extent will institutional and informational structures be sustainable after project implementation?...........................................................................................................................................48 Overall summary and Conclusions.................................................................................................................50 Recommendations..............................................................................................................................................51 Annex A. Expression of Interest.....................................................................................................................53 Annex B. Travel Itinerary .................................................................................................................................61 Annex C. References Cited .............................................................................................................................62 Annex D. Data Collection Instruments ........................................................................................................64 LIST OF TABLES Table 1. Evaluation Questions........................................................................................................................................8 Table 2. RADP Characteristics.....................................................................................................................................10 Table 3. Data Sources and Data Analysis Method by Evaluation Question ......................................................12 Table 4. Key Informant Interview Sample..................................................................................................................13 Table 5. Beneficiary Farmer Focus Group Discussion Sample .............................................................................14 Table 6. Reporting Conventions by Unit of Analysis..............................................................................................15 Table 7. M&E Indicators Related to Implementation of the Agribusiness Support Model ............................18 Table 8. Agribusiness Support Model in Each Region.............................................................................................19 Table 9. M&E Indicators Related to Engagement and Results of the Agribusiness Support Model.............23 Table 10. Farmer Training Activity Types by Region..............................................................................................27 Table 11. Challenges Reported By Beneficiary Farmers ........................................................................................28 Table 12. M&E Indicators Related to Implementation of the Farmer Training Model....................................31 Table 13 M&E Indicators Related to Results of Agribusiness Support Model ..................................................35 Table 14 M&E Indicators Related to Results of the Farmer Training Model ....................................................36 Table 15. Quantification of Land Cover Types across RADP Regions...............................................................40 Table 16. M&E Indicators Related to Results of Female Integration...................................................................46 LIST OF FIGURES Figure 1 Summary findings, conclusions, and recommendations........................................................................... iii Figure 2. Map of RADP Districts .................................................................................................................................10 Figure 3. RADP Results Framework ...........................................................................................................................10 Figure 4. RADP-East Supported Agribusinesses Processing Nuts and Dried Fruits (left) and Yogurt (right)..................................................................................................................................................................................20 Figure 5. Dairy Processing Equipment at Supported Agribusiness......................................................................24 Figure 6. Desert Area in RADP North Region Showing Expansion of Poppy Crop.......................................42 Figure 7. Visiting Project Site After Female FGD.....................................................................................................43 ACRONYMS APPRO Afghanistan Public Policy Research Organization B2B Business-to-Business BFS Bureau for Food Security CSO Central Statistical Office DAI Development Alternatives, Inc. DAIL Department of Agriculture, Irrigation, and Livestock DO Development Objective DOWA Department of Women’s Affairs EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion FY Fiscal year GIS Geographic Information System HVC High Value Crop IR Intermediate Result KII Key Informant Interview LOP Life of Project MAIL Ministry of Agriculture, Irrigation, and Livestock M&E Monitoring and Evaluation PAIL Provincial Directorate of Agriculture, Irrigation, and Livestock PEEL Program Evaluation for Effectiveness and Learning RADP Regional Agriculture Development Program USAID United States Agency for International Development USG United States Government VFU Veterinary Field Unit i ABSTRACT The purpose of this mid-term performance evaluation was to provide an independent assessment of the overall progress and accomplishments to date of the umbrella Regional Agriculture Development Program (RADP) in four regions of Afghanistan. USAID funds the umbrella RADP, which has a budget of just over $241.7 million. The program is run as four separate regional projects—RADP-North, RADP￾West, RADP-South, and RADP-East—across 21 provinces and 116 districts in Afghanistan. RADP aims to improve the food and economic security of rural Afghans. This evaluation was undertaken during implementation of RADP-North and RADP-East, and at the end of RADP-West and RADP-South. Using a cross-sectional design, the Evaluation Team (ET) used qualitative methods to answer all but one evaluation question, where geographic information system (GIS) and remote sensing methods were used by the ET. The results demonstrate mixed performance among the RADPs for key indicators. In addition, the projects did not always reach the intended scale. However, the ET found that project beneficiaries often experienced benefits from engaging with the projects. Many businesses reported improved operations with RADP support, and many farmers reported improved yields and incomes because of program participation. However, beneficiaries still faced significant challenges that were not addressed by the projects, including limited access to finance, a weak enabling environment, and particularly for farmers, water shortages and limited access to markets. The ET recommends that RADP strengthen linkages between farmers and the private sector, focus support on low input technologies, and increase collaboration with the government. ii EXECUTIVE SUMMARY Evaluation Purpose The primary purpose of this mid-term performance evaluation was to provide an independent assessment of the overall progress and accomplishments to date of the umbrella Regional Agriculture Development Program (RADP) in four regions of Afghanistan (North, East, South, and West). The evaluation occurred approximately halfway through RADP-North and RADP-East, and at the end of RADP-West and RADP-South. The evaluation identified achievements, performance issues, and constraints related to program implementation. It also identified results and lessons learned from implementation, and provides recommendations to scale up or modify activities. The primary audience for the evaluation is the USAID/Afghanistan Office of Agriculture, which may use the results to guide the management and future orientation of the program. The performance evaluation addressed the following six primary evaluation questions (EQs): 1. Is the agribusiness support model likely to be sustainable? 2. Is the training model design and implementation effective in contributing to beneficiary adoption of improved agricultural technologies and licit crops? 3. Based on the implementation approach, status of implementation, and milestones achieved at the time of the mid-term evaluation, is there evidence that the program is likely to achieve its results by the end date? 4. To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals? 5. Is the regional approach to the RADP programming likely most effective for achieving program goals? For example, how might a regional approach compare to a commodity-specific approach? 6. To what extent will institutional and informational structures be sustainable after project implementation? Evaluation Design and Methods The Evaluation Team (ET) used a qualitative approach and geographic information system (GIS) and remote sensing methods to answer the EQs. The results for all EQs are presented in this report. However, the ET only summarizes the GIS results here with the detailed results presented in a separate report (Jones, Narayan, and Conrad 2018b). For RADP-North and RADP-East, the ET answered the EQs based on implementation to date. RADP-West and RADP-South ended before data collection commenced, and therefore the ET answered the EQs for the full period of performance. For the qualitative approach, the ET used a cross-sectional design. The aim of this design was to collect qualitative data from a range of respondents at one point in time to assess activity performance to date. The ET asked respondents to describe their exposure to RADP, to characterize current conditions, and to describe the extent to which conditions had or had not changed since RADP began. The ET collected primary data through 135 semi-structured interviews with key informants (i.e., key informant interviews or KIIs) and 58 focus group discussions (FGDs) with beneficiary farmers. The ET obtained secondary data primarily through a review of project documents. Project Background The umbrella RADP is funded by USAID with a budget of just over $241.7 million and operations in four regions across 21 provinces and 116 districts in Afghanistan. RADP aims to improve the food and economic security of rural Afghans. This is in alignment with the goals of the Government of Afghanistan iii and the Ministry of Agriculture, Irrigation, and Livestock, as well as the Departments of Agriculture, Irrigation, and Livestock (DAILs) at the sub-national level. The program’s development hypothesis is that if all value-chain actors have access to appropriate training, information, inputs, and technology within a more competitive, efficient, and profitable agricultural value chain, as well as a conducive policy and regulatory environment, then they will increase their incomes and food security. Summary of Findings, Conclusions, and Recommendations The ET findings, conclusions, and recommendations are summarized by EQ in Figure 1 below. Figure 1 Summary findings, conclusions, and recommendations Findings Conclusions EQ1. Is the agribusiness support model likely to be sustainable? • About two-thirds of supported agribusinesses expressed some confidence in the growth potential of their firm, particularly in the East and South. While some enterprises were informal and small, they still expressed confidence in their growth potential. • Two-thirds of supported agribusinesses across the RADPs reported changing or adapting during the project period. • KII findings supported contractors’ M&E data that RADP assistance resulted in over half of private enterprises and organizations applying improved organization-level technologies or management practices. Businesses reported that technical assistance helped increase staff capacity and market linkages, while in-kind support helped improve efficiency, product quality, and production capacity. • Two-thirds of businesses reported that RADP support had a positive effect on their business, including improved efficiency, staff capacity, and market linkages. • Nearly half of business respondents identified their business models as sustainable. • Over three-quarters of businesses reported key challenges that they still face. Insecurity was the most commonly reported challenge (nearly half of businesses), a lack of equipment and machinery (a quarter of businesses), and limited profits, cash flow, or access to finance (a quarter of businesses). • Findings suggest that the agribusiness support model is more likely to be sustainable for businesses with high growth potential. • A majority of RADP supported businesses demonstrated some level of ability to change and adapt during the project period. Businesses that were able to adapt to changing conditions are more likely to be sustainable. • Issues stemming from the external environment pose risks to businesses’ future growth and sustainability. • Limited access to finance limits businesses’ ability to grow and negatively affects sustainability. • Organizational changes made from RADP assistance are likely to be sustained, although changes that were dependent on RADP non￾durable in-kind support or grant assistance may not be sustained after the project. Recommendations • Because we found that some businesses were constrained by lack of access to finance, which we conclude will negatively affect the model’s sustainability, we recommend RADP-East and future programs support alternative development financing by connecting businesses with sources of finance and build their capacity to obtain loans successfully. • Future programs should continue to provide a mix of technical assistance and in-kind support to help businesses improve capacity and production quality and efficiency. EQ2. Is the training model design and implementation effective in contributing to beneficiary adoption of improved agricultural technologies and licit crops? iv Findings Conclusions • RADP did not employ detailed selection criteria for farmer beneficiaries, although training was relatively well suited to those farmers that were selected. In a few cases, there were issues with fairness and the appropriateness of the participants that were selected, particularly in RADP-West. • Farmers in about three-quarters of FGDs found that RADP assistance was effective and trainings included relevant content. Male and female farmers most commonly learned about vegetable production, followed by wheat and fruit production among men, and livestock production among women. Male and female farmers most frequently reported learning about the use of improved inputs, irrigation, planting techniques, and land preparation. • Farmers in nearly a quarter of FGDs reported that the training content was not useful because they did not learn new information, the information was not effective when applied, or they were unable to use the advice provided due to the resources required for implementation. • Although farmers in about three-quarters of FGDs found the trainings useful, farmers in over three-quarters of FGDs reported challenges that were not sufficiently addressed by the project, namely access to markets and lack of water. • Overall the training model was effective in contributing to beneficiary adoption of improved agricultural technologies by building farmers’ knowledge and skills. In particular, training activities were effective that used practical demonstrations and those that taught technologies that did not require access to large equipment or machinery. • Training was not effective for beneficiaries who were not producers of targeted value chain products. • RADP training did not adequately address one or more challenges farmers faced, which may limit initial adoption and sustained adoption of improved agricultural technologies or licit crops. Challenges like limited access to markets and lack of water reduce farmers’ ability and willingness to use improved agricultural technologies and produce licit crops. Therefore, they present a barrier to their initial and sustained adoption. • RADP training was not responsive to beneficiary needs in all cases. For example, the curriculum included some methods that farmers did not have the resources required to implement. In some cases, the training did not include information on livestock production, which is an important agricultural activity for some households. Recommendations: Future programs and RADP-East should: • Strengthen and monitor quality of beneficiary selection so that training is provided to beneficiaries for who it will have the most value; ensuring that beneficiaries are farmers who engage in the targeted value chains will increase their likelihood of adopting new technologies and licit crops. • Further tailor the content of training to address the key constraints faced by farmers in different regions and to ensure that techniques and technologies promoted are accessible and feasible for farmers to implement. Improved practices and new crops need to be profitable for farmers to incentivize and sustain adoption. o In drought-prone areas, provide more training to farmers and provide demonstrations on water conservation technologies and the production of drought resistant crops. Farmers may be more likely to take up new technologies that do not have high water needs in areas where access to water is limited. • Include more market related training modules to male and female farmers to build their skills on how to access markets for relevant crops and livestock products, build relationships with buyers, negotiate with buyers, and access market information (e.g., on prices, demand, quality requirements). Provide training to farmers that includes practical demonstrations and training at demonstration plots to improve the effectiveness of training on farmer learning and skill building. EQ3. Based on the implementation approach, status of implementation, and milestones achieved at the time of mid-term evaluation, is there evidence that the program is likely to achieve its results by the end date? • About half of agribusinesses reported improved market linkages because of RADP support, • RADP is likely to achieve its results related to outcomes on increased agriculture sector v Findings Conclusions particularly in the East and South. However, farmers in less than a quarter of FGDs overall cited market linkage or access improvements because of project assistance (larger portion in the South). As a result, it was not clear whether the RADPs were able to successfully match-make between farmers and agribusinesses. • About two-thirds of businesses reported improvements that were likely to help them respond to market demand (the portion was smaller in RADP-West), such as increased staff capacity and efficiency. However, businesses that were unable to make improvements with RADP support were unlikely to be more responsive to market demand. • Farmers in three-quarters of FGDs reported adopting changes they learned from the project, primarily to wheat (for men) and vegetable (for women) production. However, farmers in less than a quarter of FGDs reported making improvements in livestock production, most of who were women. • Farmers in over three-quarters of FGDs reported that their yields increased due to RADP assistance, including trainings and inputs. Farmers in three-quarters of FGDs reported that their incomes improved due to increased yields and high-quality products. Slightly fewer FGD participants reported improved incomes in RADP￾North and RADP-West. • While not a counter narcotics program, RADP expected that supporting improvements to licit agricultural production and incomes would encourage farmers to increase production of licit rather than illicit crops. GIS analysis showed mixed results on changes in illicit crop production. Poppy production fell in the RADP￾East and RADP-West regions. Total hectares allocated to poppy production increased in RADP-South, although the proportion of total agricultural land allocated to poppy production lowered due to an increase in other agricultural production. In RADP-North, poppy production rose substantially in terms of its absolute land area and its proportion of total agriculture. productivity and profitability, agribusiness profitability, and women’s participation in agriculture value-chain activities. • About half of supported agribusinesses are likely to be more responsive to market demand due to increased market linkages and improvements in business capacity and operations. • Farmers supported by the program largely reported adopting improved practices resulting in increased yields and incomes. However, results related to improved market linkages for farmers are less likely to be met. • Although not an explicit program objective, the RADP training and agribusiness support models had mixed results on changes in illicit crop production, particularly in the North. Recommendations: Future programs and RADP-East should: • Strengthen efforts to help farmers access markets for high value crops by facilitating direct linkages between farmers and buyers to help increase farmers’ revenue from licit crops, because farmers may increase adoption licit crops if they can provide an economic substitute for poppy. o Strengthen agribusiness capacity to create direct linkages with farmers and to provide extension services to farmers to improve quality of production to improve farmer sales and agribusiness profits. vi Findings Conclusions EQ4. To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals? • Women were successfully engaged in training and input distribution in most cases, particularly when training was offered in culturally appropriate locations. This is a model that can be replicated in other programs. As an exception, RADP-East grantees offered some mixed sex trainings, which women reported as an issue. • Women largely found the trainings useful, and like men, were able to make changes to their agricultural production. A slightly smaller proportion of women compared to men reported improved yields and incomes because of RADP assistance. • Inclusion of women in agribusiness support interventions was more difficult than inclusion in training due to cultural norms (particularly in the South and East) and other barriers to female entrepreneurship, such as access to finance. • While youth were not an explicit target group for RADP, some beneficiaries were youth (according to the Afghan government definition). In addition, RADP-South specifically included a small training component for high school girls. Youth beneficiaries experienced the same benefits reported by farmers in general, and parents reported that adolescents indirectly benefited from their involvement in RADP. • Women were relatively well integrated into RADP training interventions, particularly considering the cultural context in rural Afghanistan. However, women were not well involved in agribusiness interventions due to cultural and material barriers that limit female entrepreneurship. RADPs in regions with greater gender equality (e.g., the North), were able to increase female participation in agribusiness interventions. • Youth were not explicitly targeted by RADP in general. However, implementing partners used opportunities to engage youth in a targeted manner. Recommendations: • RADP-East should encourage their agribusiness grantees to provide female-only trainings to improve the accessibility, acceptability, and effectiveness of training for women. • Future programs and RADP-East should work with women to obtain financial assistance with the backing of male relatives if necessary, to improve their ability to start and grow businesses. • Future programs should tailor agribusiness support interventions to small women-owned enterprises and develop criteria for grants that are possible for female entrepreneurs and women-owned businesses to meet. RADP-East should continue to evaluate its grant criteria to maximize participation by women-owned businesses. EQ5. Is the regional approach to the RADP programming likely most effective for achieving program goals? For example, how might a regional approach compare to a commodity-specific approach? • The regional approach provided support for agribusinesses and farmer production across a number of value chains, but had mixed results on illicit crop production. • Respondents did not report achievement of synergies in activities across specific commodities or value chains that contributed to market systems development. However, contractor M&E systems did not track common indicators across RADPs that allowed assessment of market development outcomes that could result from a • RADP’s regional approach was fairly effective at improving household food security and income. However, RADP’s regional approach was less effective at shifting production from illicit to licit crops. • Evidence did not exist to confirm that the regional approach was effective at fostering broader market systems development, in part due to limitations in M&E data. vii Findings Conclusions regional approach. Recommendations: • Future programs should choose whether to use a regional approach or a commodity-specific approach based on specific, core goals of the program. • If RADP continues to operate as multiple regional projects, harmonize M&E systems across projects where applicable to facilitate and strengthen performance monitoring and assessment. EQ6. To what extent will institutional and informational structures be sustainable after project implementation? • Project staff and agribusinesses were reasonably confident about the sustainability of interventions. However, government stakeholders had mixed opinions. For example, some felt there was insufficient government engagement and that specific actors or organizations did not have responsibility for sustaining outcomes, which lowers the chances of sustainability. • RADP provided policy support to government and helped draft several policies. However, government stakeholders shared mixed opinions about the government initiatives that were supported by RADP and who was responsible for sustaining them. • Some institutional and informational structures are likely to be sustained after RADP ends, such as agribusiness organizational changes and improved farmer capacity. However, the degree to which interventions were institutionalized and whether business and farmer improvements will sustain without support is an open question. Recommendations: • Future programs and RADP-East should, increase coordination and collaboration with government to increase stakeholder buy-in and alignment with government policies and priorities to improve sustainability. 8 1. EVALUATION PURPOSE AND QUESTIONS EVALUATION PURPOSE The primary purpose of this performance evaluation was to provide an independent assessment of the overall progress and accomplishments of the umbrella RADP activity to date in four regions of Afghanistan. The performance evaluation came in Year 3 for RADP-East and Year 5 for RADP-North, and after the termination of convenience for RADP-West and RADP-South. The performance evaluation was designed to identify achievements, performance issues, and constraints related to implementation and effectiveness of the program. It was also designed to identify results and lessons learned from implementation, and to provide recommendations to scale up or modify activities. EVALUATION AUDIENCE The USAID/Afghanistan Office of Agriculture planned to use the evaluation to guide the management and future orientation of the program. In addition, USAID/Afghanistan, USAID/Bureau of Food Security, and the Feed the Future Initiative intended to consider the evaluation findings and recommendations when designing follow-on activities. Supporting offices, such as the Office of Acquisition and Assistance, Office of Financial Management, Office of Program and Project Development, and implementing partners were identified as recipients of the evaluation report. EVALUATION QUESTIONS As outlined in Table 1 below, the performance evaluation answered six EQs. In this report, the ET presents the results for all EQs, but only summaries EQ3d, which used GIS methods and is presented in detail separately (Jones, Narayan, and Conrad 2018b). For RADP-North and RADP-East, the ET answered the EQs based on implementation to date. For RADP-West and RADP-South, which ended before data collection, the ET answered the EQs for the life of project (LOP). Table 1. Evaluation Questions Evaluation Questions A. Implementation 1. Is the agribusiness support model likely to be sustainable? a. Are we supporting businesses that largely have good potential for growth? b. Are beneficiary enterprises flexible or adaptable enough to be responsive to market shifts? 2. Is the training model design and implementation effective in contributing to beneficiary adoption of improved agricultural technologies and licit crops? a. How did selection criteria, curricula, and pedagogy address farmer constraints? b. To what level are the communities being reached (saturation)? c. To what extent does the approach include considerations for varying socioeconomic status among beneficiaries (e.g., security of land tenure, income)? B. Results 3. Based on the implementation approach, status of implementation, and milestones achieved at the time of mid-term evaluation, is there evidence that the program is likely to achieve its results by the end date? Agribusiness model results: a. Are activities creating more market linkages? Are those market linkages more efficient in terms of maximizing returns to the smallholders while meeting the market needs? b. Are beneficiary enterprises more responsive to market demand? 9 2. PROJECT BACKGROUND USAID funds the umbrella RADP in Afghanistan with a budget of just over $241.7 million and operations in four regions across 21 provinces and 116 districts. The program is run as four separate regional projects, RADP-North, RADP-West, RADP-South, and RADP-East. Implementation areas are shown in Figure 2. Working in alignment with the Government of Afghanistan goals, and with the Ministry of Agriculture, Irrigation, and Livestock (MAIL) and the Departments of Agriculture, Irrigation, and Livestock (DAILs) at the sub-national level, RADP aims to improve the food and economic security of rural Afghans (see Figure 3). The program’s development hypothesis is that if all value-chain actors have access to appropriate training, information, inputs, and technology within a more competitive, efficient, and profitable agricultural value chain, as well as a conducive policy and regulatory environment, then they will increase their incomes and food security. Each RADP activity employs a value chain facilitation methodology to address bottlenecks in targeted value chains. The methodology works in all segments of agriculture supply chains—from the farmers (producers) to the consumers. RADP activities are designed to strengthen relationships and trust among private sector entities, and focus on: improving productivity of wheat; investment in and production of high-value horticultural crops (e.g., grapes, raisins, pomegranates, stone fruits, nuts, and vegetables); and production of livestock (see Table 2 below). Each RADP also focuses on critical cross-cutting themes, such as on-farm water management, nutrition, alternative development, finance, and support for DAILs. Increasing women’s participation in and benefits from the agricultural sector are critical components of each activity. RADP activities also include an enabling environment component to strengthen the private sector. This is through a mix of direct implementation and a $10-million grant fund for each RADP to promote an improved legal and regulatory framework and for policy advocacy and dialogue. Enabling environment activities were led by RADP-South until its termination, and subsequently by RADP-North. Training model results: c. To what degree did beneficiary farmers adopt improved agricultural technologies? d. To what degree did beneficiary farmers adopt licit crops? What was the percentage change in land area under poppy and under non-poppy crops in the target areas? C. Cross-cutting 4. To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals? a. Which approaches have been most effective in reaching women and youth respectively? b. How have women and youth respectively benefited from RADP implementation, and how are these benefits measured? c. Are there any missed opportunities with incorporating women and youth respectively into RADP activities? d. Are there any successes from RADP that can be adapted to specifically integrate women and youth respectively? D. Thematic Questions 5. Is the regional approach to the RADP programming likely most effective for achieving program goals? For example, how might a regional approach compare to a commodity-specific approach? 6. To what extent will institutional and informational structures be sustainable after project implementation? 10 Figure 2. Map of RADP Districts Figure 3. RADP Results Framework DO 1: Sustainable, Agriculture-led Economic Growth Expanded IR 1.2: Vibrant & prosperous agriculture sector developed Sub-IR 1.2.1: Productivity of key agriculture crops increased Sub-IR 1.2.2: Commercial viability of agribusinesses increased Outcome 1: Increased agriculture sector productivity and profitability in target region Outcome 4: Increased and sustained adoption of licit crops Outcome 2: Increased profitability of small, medium, and large agribusinesses Cross-cutting Outcome 3: Increased women’s participation in agriculture value-chain activities Cross-cutting Outcome 5: Improved enabling environment for farmers and agribusiness Table 2. RADP Characteristics RADP-North RADP-West RADP-South RADP-East Award number 306-C-14-00002 306-C-14-00007 306-C-13-00018 306-C-16-00011 Award date May 2014 August 2014 October 2013 July 2016 End date Anticipated May 2019 September 27, 2016* October 2017* Anticipated July 11 RADP-North RADP-West RADP-South RADP-East 2021 Funding amount $78.4 million $27.6 million $107.6 million $28 million Activity funding AG 75%, AD 25% Implementing organization Development Alternatives, Inc. (DAI) Chemonics International Inc. Chemonics International Inc. DAI Sub￾contractors/ grantees ACDI/VOCA, Alcis, Development and Training Services, Inc., Dutch Committee for Afghanistan, Joint Development Associates, Pax Mondial Risk Management Company, , Afghan Public Protection Force Agency for Rehabilitation and Energy, Conservation in Afghanistan, Coordination of Humanitarian Assistance, Dutch Committee for Afghanistan, Equal Access, Services International LLC, Afghan Public Protection Force Afghan Development Association, Dutch Committee for Afghanistan, Equal Access Organization for Sustainable Development and Research, Relief International, Crimson Capital and Mondial Risk Management Company Province coverage Badakhshan, Baghlan, Balkh, Jawzjan, Kunduz, and Samangan Herat, Farah, and Badghis Kandahar, Helmand, Uruzgan, and Zabul Kabul, Nangarhar, Laghman, Ghazni, Logar, Wardak, Parwan, Kapisa Targeted value chains Wheat, livestock, high-value crops Livestock, high￾value crops Note: * Terminated for convenience Each regional project blended direct technical assistance and training, in-kind support, and grants with strengthening value chain linkages. However, variations in climate, capacity, and security, and shifting government priorities, have led to some differences in activity implementation. For example, RADP￾South, RADP-West, and RADP-North provide technical assistance to wheat producers and value chain actors, whereas RADP-East does not focus any of its activities on the wheat sector. All RADPs support producers and value chain actors of high value crops (HVCs), but specific crops vary widely by regional climate. Likewise, all RADPs support livestock value chains, but the challenges vary widely by species. RADP-South put more resources towards veterinary units than the other regions. Providing opportunities for licit farming and employment became a government and donor priority over the course of implementation. While not a counter narcotics program, RADP expected that supporting improvements to licit agricultural production and incomes would encourage farmers to increase production of licit rather than illicit crops. Although not explicit in the design of all RADPs, this impacted RADP-South, which covered the regions traditionally most active in opium poppy production. The highly volatile environment of RADP-South led to more limitations on implementation locations than other regions, though RADP-East could potentially face similar problems with the growing presence of armed opposition groups in the eastern provinces. 12 3. EVALUATION METHODS AND LIMITATIONS The ET conducted an evaluation using qualitative and GIS methods to assess the performance of the RADP. The protocols that detail the evaluation design and methods are provided in separate documents (see Conrad and Narayan 2018; Jones, Narayan, and Conrad 2018a). The GIS findings are presented in detail in a separate document (Jones, Narayan, and Conrad 2018b) and are only summarized here. This section describes the purpose of the evaluation, its intended audience, and the EQs. QUALITATIVE APPROACH Using a cross-sectional design, the ET used qualitative methods to answer all questions, except EQ3d (EQ1-3c and EQ4-6). The aim of the design was to collect data from a range of respondents at one point in time. The ET used qualitative instruments to assess activity performance to date by asking respondents to describe their exposure to RADP and to characterize current conditions and how they had or had not changed since RADP began. The ET further asked respondents to explain how they thought RADP activities had influenced those changes and why. DATA SOURCES The ET collected both primary and secondary data (see Table 3). Primary data were collected through semi-structured key informant interviews (KIIs) and focus group discussions (FGDs) with beneficiary farmers. The ET primarily reviewed project documents for secondary data. These data sources are described in more detail below. Table 3. Data Sources and Data Analysis Method by Evaluation Question Evaluation question Data source Data analysis method EQ1: Agribusiness support model KIIs with value-chain actors Project documents Inductive thematic analysis EQ2: Training model FGDs with beneficiary farmers KIIs with activity staff and key stakeholders Project documents Inductive thematic analysis EQa-c: Agribusiness and training model results FGDs with beneficiary farmers KIIs with activity staff, key stakeholders, and value chain actors Project documents Inductive thematic analysis EQ3d: Change in licit crop production GIS/remote sensing data High resolution images GIS/remote sensing analysis to classify field types and assess change using pre–post comparison EQ4: Gender and youth integration FGDs with beneficiary farmers KIIs with activity staff and key stakeholders Project documents Inductive thematic analysis EQ5: Regional approach KIIs with value chain actors and key stakeholders Project documents Inductive thematic analysis Data from EQ1 and EQ3 Synthesis 13 Evaluation question Data source Data analysis method EQ6: Sustainability KIIs with value chain actors, activity staff, and key stakeholders FGDs with beneficiary farmers Project documents Inductive thematic analysis Data from EQ1-3c, EQ4, and EQ5 Synthesis Key informant interviews. The ET interviewed a purposive sample of implementing partner staff, value-chain actors, and other stakeholders. The ET selected respondents from a range of project roles and sectors to capture salient variation across activity areas and value chains. Implementing partner staff included the Chief of Party, Deputy Chief of Party, and the Monitoring and Evaluation (M&E) Lead. The interviews explored details about RADP interventions and factors that affected activity implementation. KIIs with a purposive sample of value-chain actors covered agribusiness engagement with RADP and perceptions of the impact of this involvement. Finally, KIIs with government stakeholders (e.g., DAIL staff) gathered perceptions about activity implementation and factors affecting implementation (see Annex D). The ET conducted the KIIs in English, Dari, or Pashto as appropriate. The KIIs were audio recorded when consent was provided and documented through verbatim notes in English. The notes were reviewed for quality assurance by a senior member of the ET who did not conduct the interview. The ET conducted interviews in Kabul, Mazar-e-Sharif, Herat, Kandahar, Jalalabad, and by phone when needed. Overall, the ET completed 135 interviews as shown in Table 4. The team reached the target sample for all categories, with the exception of stakeholders for RADP-North and project staff in RADP-West (target was six KIIs each for these two categories), due to difficulties contacting and scheduling interviews with respondents due to holidays in the North and project closure for RADP￾West, respectively. Nearly all value chain actors interviewed were male due to the limited number of female value chain actors engaged and limitations on the ET’s ability to travel to conduct interviews. Table 4. Key Informant Interview Sample Respondent type Number of respondents RADP-North RADP-West RADP-South RADP-East National Project staff 8 3 12 17 0 Value chain actor 16 11 21 17 0 Stakeholder 3 6 8 8 5 Total 27 20 41 42 5 Focus Group Discussions. The Afghanistan Public Policy Research Organization (APPRO) conducted the FGDs with female and male beneficiary farmers in activity districts in Dari or Pashto. The FGDs were completed with a purposive sample of beneficiary farmers selected from beneficiary lists provided by implementing partners. The FGDs explored key agricultural challenges that beneficiary farmers faced and their engagement with RADP. Questions covered how farmers were exposed to the program, how they became engaged with it, what types of agricultural practices or technologies they adopted since engaging with the program, and what they perceived to be the effects of those changes (see Annex D). The FGDs were audio recorded with consent. The recordings were first transcribed in the language they were conducted in and then translated into English. The ET reviewed all transcripts for quality control. As shown in Table 5, APPRO completed 58 FGDs with a total of 475 respondents (221 female, 254 14 male). APPRO conducted FGDs in two districts per province in four of six provinces for RADP-North, 1 all three provinces for RADP-West, 2 three of four provinces for RADP-South, 3 and four of eight provinces for RADP-East. 4 APPRO conducted FGDs in a smaller portion of provinces for RADP-East than the other regions due to resource constraints. The team reached the target sample for all categories for each region as a whole; however, in three provinces (one each in the West, South, and East) the ET had to do one less female FGD due to limited numbers and accessibility of female beneficiaries. Table 5. Beneficiary Farmer Focus Group Discussion Sample Respondent type RADP-North RADP-West RADP-South RADP-East No. FGDs No. respondents No. FGDs No. respondents No. FGDs No. respondents No. FGDs No. respondents Female beneficiary farmers, all 9 77 6 48 6 49 6 47 Female beneficiary farmers, youth 2 17 4 16 6 32 6 30 Male beneficiary farmers, all 7 58 7 53 7 58 10 85 Male beneficiary farmers, youth 2 4 3 7 5 24 10 48 Total 16 135 13 101 13 107 16 132 Secondary data. The ET collected and reviewed secondary data to inform the responses to the EQs. Sources included over 100 RADP activity documents provided by USAID and implementing partners, including activity work plans, performance management plans, M&E data, annual reports, and requests for proposals. The ET examined M&E data for the life of project (LOP) for RADP-West5 as that project had closed, and for fiscal year (FY) 2018 for RADP-North and RADP-East as they were still active. The ET used M&E data for FY16 for RADP-South as the M&E data for FY17 was not yet approved by USAID. As a result, the M&E data reported for RADP-South does not reflect progress made in the last year of the project. In addition, the ET reviewed relevant sector reports and academic literature sourced online to inform instrument design and data analysis. Reporting The ET followed a common convention for describing the magnitude of findings across KIIs and FGDs— Table 6 outlines how this convention applies across data collection strategies and respondent groups. The ET used the same convention when disaggregating by RADP region and beneficiary sex. The aim of this convention was to show what proportion of each KII respondent group and in what proportion of 1 Baghlan, Balkh, Jawzjan, and Samangan provinces. 2 Badghis, Farah, and Herat provinces. 3 Helmand, Kandahar, and Uruzgan provinces. 4 Kabul, Laghman, Nangarhar, and Parwan provinces. 5 RADP-West LOP targets reflect targets set for the original five-year period rather than the actual project period, which biases progress shown towards targets. 15 FGDs6 specific themes and findings were found. The ET used the total number of FGDs and KIIs for each respondent group as the denominator. This was because interviewers did not always ask the same questions during KIIs and FGDs, and because respondents often raised similar themes in response to multiple questions. The ET provided the number or fraction of KIIs or FGDs in which a finding was reported in some instances in this report. Table 6. Reporting Conventions by Unit of Analysis Represents FGDs (n=58) KIIs (n=135) Agribusinesses (n=65) Project staff (n=40) Stakeholders (n=30) All/almost all >76% >76% >76% >76% Most/majority/many 56-75% 56-75% 56-75% 56-75% About half 45-55% 45-55% 45-55% 45-55% Some/minority 26-44% 26-44% 26-44% 26-44% Several/few <25% <25% <25% <25% DATA ANALYSIS The ET imported data into NVivo 11, a qualitative data analysis software package, to organize and code the data. Analysis of qualitative data began with coding text (i.e., flagging pieces of data) that related to a theme or concept of interest (thematic codes), or to a specific research question, sub-question, or objective (structural codes). A priori codes (those identified prior to data analysis) were informed by the EQs, and inductive codes (those that emerge from the data) were determined based on the data. To ensure inter-coder reliability, the ET trained coders on guidelines and held regular coder team meetings to discuss questions. In addition, the qualitative lead reviewed coding for quality and consistency. The ET analyzed the coded data in NVivo. First, the ET conducted exploratory data analysis using inductive thematic analysis to identify common patterns, cultural categories, themes, and outliers that emerged from the data related to the EQs and other topics of interest. Next, the ET used structured, variable-oriented analysis strategies, including unit-by-variable matrices in NVivo (where units are respondents or FGDs and variables are themes and topics of interest), to determine whether reported changes in outcomes aligned with expected patterns, and which factors influenced reported changes. The unit-by-variable analysis enabled the ET to compare and disaggregate responses by sex, age, value chain, and region. In particular, data was disaggregated to determine the differential experiences of female and youth (male and female) beneficiaries to answer EQ4. The ET used project documents to synthesize information on activity design, goals, implementation, and reported challenges and adjustments. The team used the findings from primary data to understand more about how the activities were implemented, how beneficiaries experienced participation in the projects, and challenges and successes. The team also drew M&E data from project documents to provide information about activity reach and outcomes. LIMITATIONS There were several contextual constraints that limited data collection and evaluation design. First, security conditions limited data collection in certain areas, which impacted the sample size and respondent representation. There were several districts where the ET could not conduct FGDs, and KIIs were limited to provincial cities. The team worked closely with the data collection firm and a private security firm to continually assess security conditions and respond as feasible (e.g., by adjusting 6 We use the number of FGDs to determine proportion rather than the number of individual respondents that report something in a FGD because not all respondents answer each question in a FGD. 16 the timing of data collection in regions and/or adjusting sampling). In addition, the ET was not able to ask farmers about opium poppy production due to security and ethical risks. Second, the ET was unable to complete the target number of FGDs with women because there were few female beneficiaries in some districts, or it was difficult to reach them (e.g., few women had working mobile numbers). The third limitation was recall and response bias. For RADP-South and RADP-West, the ET asked respondents to recall the project period, which may introduce bias in memory or interpretation. Response bias was also a limitation. Some households were exposed to RADP, which meant they may have had an idea of what the ET wanted them to say, or they had been exposed to other donor initiatives, which may have biased their responses. The ET assessed whether response bias was a significant issue during piloting, which it did not seem to be. In addition, the ET mitigated the risk by ensuring a clear consent statement was included at the beginning of the protocol, and by triangulating data where possible (e.g., gathering data on some questions from both FGDs and activity documents). Nevertheless, there was apparent bias in response to questions in some FGDs where respondents seemed reluctant to share challenges they faced or seemed to contradict themselves when discussing the impact of the project. The ET took this into account during the analysis and how the findings were reported for these questions. 4. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS The findings are presented below by EQ. EQ1 and EQ2 cover implementation of the agribusiness support model and the farmer training model, including participant selection and support provided. EQ3 covers the results of the agribusiness support model and the farmer training model, including progress achieved towards performance indicators to date and outcomes reported by beneficiaries. EQ3 includes the results of the GIS/remote sensing analysis on the change in land area for licit production. EQ4 focuses on the integration of women and youth in program activities and the outcomes of that integration. EQ5 and EQ6 focus on the effectiveness of the regional approach and the sustainability of program activities respectively. This section also includes overall conclusions and recommendations for future programming. EQ1: IS THE AGRIBUSINESS SUPPORT MODEL LIKELY TO BE SUSTAINABLE? The ET first examined the reach of agribusiness support provided and the agribusiness support model employed in each RADP region to provide context for EQ1a and EQ1b. The ET then examined the sustainability of supported agribusinesses by discussing their growth potential (EQ1a) and their likely flexibility and adaptability in the face of market shifts (EQ1b). Both of these factors were key components in the sustainability of supported agribusinesses, their potential to grow in the future, and thus whether RADP’s support had the potential to contribute to sustainable agribusiness-led growth. Agribusiness support model in four regions With the exception of RADP-West, the projects made strong progress towards or exceeded targets for engagement of agribusinesses according to program M&E data (see Table 7).7 The RADPs used different 7 Throughout the report, M&E data is drawn from FY18 annual reporting for RADP-North and RADP-East, annual reporting for RADP-South (FY16), and the final report for RADP-West (FY16). Note that USAID has not approved the M&E data for FY17 reported in RADP-South’s final report, therefore the data we present from FY16 does not reflect the LOP achievements. We only present key indicators, with a focus on those that allow for comparison across the regions. 17 indicators to track the number of agribusinesses and actors engaged or supported. RADP-North has nearly met its LOP target, benefitting 912 agribusinesses, and RADP-East has met nearly half of its LOP target at the project midpoint having benefited 232 agribusinesses. RADP-South provided assistance to 217 agribusinesses as of FY16. RADP-West set an ambitious target to train 8,000 public-private agricultural change agents, but only trained 472 during the LOP. 18 Table 7. M&E Indicators Related to Implementation of the Agribusiness Support Model Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Number of beneficiaries from program interventions – Businesses N/A N/A No target 217 N/A N/A Number of agro￾enterprises and new businesses created and/or receiving benefits from the project 970 912 94% 550 232 42% N/A N/A Number of public-private agriculture change agents trained on new techniques 1,920 2,003 104% N/A N/A 8,000 472 6% Note: *N/A means it was not an indicator for that RADP 19 Table 8. Agribusiness Support Model in Each Region RADP-North RADP-East RADP-South RADP-West Grants and in-kind support. Grantees provide training to farmers. Technical assistance and capacity building training on management, processes, and access to finance. Participation in trade fairs. Supported enterprises, cooperatives, and veterinary field units (VFUs) in target value chains. Grants and in-kind support. Grantees provide training to farmers. Technical assistance and capacity building training on management, processes, and access to finance. Business-to-business and business-to-farmer events. Participation in trade fairs. Supported formal enterprises, cooperatives, and VFUs in target value chains. Grants and in-kind support. Technical assistance and capacity building training on management, processes, and access to finance. Participation in trade fairs. Supported enterprises, cooperatives, and VFUs in target value chains. Grants and in-kind support. Technical assistance and capacity building training on management, processes, and access to finance. Business-to-business events. Participation in trade fairs. Supported enterprises, cooperatives, and VFUs in target value chains. All RADPs incorporated agribusiness support as a key component of their implementation, with variation between projects on approach and integration with farmer training models (see Table 8). They used a modified value chains and market facilitative approach focusing on technical support, small grants, and market linkages. The agribusinesses supported through these models were envisioned as contributing to improved farmer productivity by providing farmers with improved access to inputs and services. However, direct linkages between beneficiary farmers and beneficiary agribusinesses were not a stated goal for RADP-North, RADP-South, and RADP-West. Three regions, RADP-North, RADP￾South, and RADP-West, focused on wheat, HVCs, and livestock value chains—HVCs and livestock encompass a wider range of products than typically fall under a single value chain. RADP-East focused only on HVCs and livestock. Grant programs were used in all four regions, with differing beneficiary criteria for identifying businesses or microbusinesses with potential for growth and sustainability (growth potential varied widely). To improve the enabling environment for agribusinesses and farmers, RADP￾South, and following its termination, part of RADP-North, provided technical assistance to MAIL to strengthen agricultural policies, laws, and regulations. For example, RADP supported the review and drafting of documents for policies and bills, drafted implementation plans, and advised MAIL on identification of obstacles to reform by conducting constraint analysis to identify obstacles to reform. This included identification of key constraints to animal movement controls, border controls for vaccine imports, and gender issues. For RADP-West and RADP-South, this component contributed to creating government and business connections, and bringing government stakeholders on board with agribusiness development goals. RADP-North largely engaged agribusinesses providing services and inputs to farmers as opposed to businesses that process and sell products. The former agribusinesses included contract harvesters or business providing capacity building to farmers through training or radio broadcasts, and the latter included agricultural input suppliers, farmer cooperatives, and veterinary clinics. RADP-North’s agribusiness component was smaller in scale than the other regions. However, it tried to connect business and farmer beneficiaries, and engage women’s businesses and farming service needs to a greater 20 degree than other regions, taking advantage of the greater freedoms typically afforded to women in northern Afghanistan. RADP-North selected veterinary field units (VFUs) 8 to participate that expressed interest in coordination with input from local shuras (council) and DAIL extension officers, with approval from district governors. RADP-North also provided technical assistance to farmer organizations to improve management and processes, and to develop business plans and register as agricultural cooperatives using their ‘Sell More for More’ and ‘Start and Improve Your Business’ tools. RADP-West, similar to RADP-North, worked with agribusinesses along single value chains, from inputs to post-harvest production. At the time of its termination, it was increasing its grants program and facilitating market connections through trade fairs. However, it was still in the early stages of implementation and had not made significant progress towards achieving its targets. In addition, the goal of providing technical assistance in operations, quality control, supply management, and access to financing had not yet come into effect. RADP-South was market-led and engaged both large and small agribusinesses. Value chain assessments were conducted in each province during the first year, and different support models were deployed based on agribusiness size and capacity. In the second year, RADP-South focused first on medium and large businesses, fostering national and international market connections, and then focused on much smaller businesses. Capacity building and training were undertaken for groups of small businesses, often focusing on business plans, budgets, cash flow management, and low-cost adoptable technologies. For larger businesses, the focus was often on larger scale change, such as improving cold chain storage or specific forms of packaging, and working one-on-one with businesses to create tailored problem solving plans. RADP-South also helped businesses promote themselves through activities like radio advertisements. Trade fairs were used to increase market connections for businesses and farmer beneficiaries, and in the final year of the project, direct connections were made between agribusiness beneficiaries and farmers whose capacity had increased due to project activities. The focus on licit alternative crops as a substitute for opium poppy became greater throughout implementation. RADP-East used a market facilitation or market systems approach, which did not work directly with farmers. Instead, RADP-East provided grants to agribusinesses, including input suppliers and buyers who 8 The Dutch Committee for Afghanistan started VFUs with paravets to diagnose and treat animals and medicines available in the field. The VFUs “provide primarily clinical veterinary services (diagnosis, treatment and surgery), preventive vaccinations or herd health and production services such as deworming, control of external parasites, advice on nutrition, breeding or animal management and receive fees from animal owners for the services and remedies provided” (Mehraban, Ward, and Khaliqyar 2013:5). Figure 4. RADP-East Supported Agribusinesses Processing Nuts and Dried Fruits (left) and Yogurt (right). 21 worked with farmers. The market systems approach aimed to use a demand-pull strategy rather than a supply-push strategy. At the outset, RADP-East conducted an analysis to identify the four most promising value chains in the region—these were dairy, poultry, tomatoes, and apricots. RADP-East staff saw the value chain analysis as an important component of program design. Revolving primarily around grants programs, RADP-East required more significant buy-ins than other regions, with the aim of increasing sustainability and ownership. Those agribusinesses that received grants were involved in one of the focal value chains, and often significant capacity was built prior to award. Capacity was built typically by local DAILs/ Provincial Directorates of Agriculture, Irrigation, and Livestock (PAILs) following introductions and needs identification by RADP-East implementers. In order to receive grants, agribusinesses were asked to make their own investments through either capital or, more frequently, through securing financing. RADP-East applied its agribusiness selection criteria less rigorously to women-owned businesses because they found that their criteria, including formal registration and willingness to pay 50 percent of cost share, excluded almost all women-owned businesses. The depth of support provided by the RADPs varied across agribusinesses and regions. Just under half of the agribusinesses interviewed across the regions reported receiving technical assistance only, a quarter received in-kind support, and one fifth received a combination of technical assistance, in-kind support, and/or financial assistance (most of which were located in the North or East). Very few businesses reported receiving only financial assistance or not receiving any assistance. Agribusinesses reported receiving technical assistance on a range of topics. Agribusinesses most frequently reported receiving technical assistance in the form of training, such as on marketing, accounting, and general business management. A couple of agribusinesses reported that they received on-going training either every three or six months; however, this was not the typical experience reported across the agribusinesses interviewed. Additionally, a couple of RADP-East agribusinesses received assistance with setting up a text-messaging system that they could use to communicate pertinent information to farmers. Lastly, more RADP-North and RADP-South agribusinesses reported receiving technical assistance than RADP-East agribusinesses. Those who received financial assistance received it in the form of grants. The agribusinesses that received in-kind support received a range of equipment, such as refrigerators, computers, milk cans, and machinery for processing and packaging products. More RADP-East agribusinesses reported receiving in-kind support than agribusinesses in other regions. EQ1a: Are we supporting businesses that largely have good potential for growth? To assess growth potential, the ET asked business respondents for their perspectives on their businesses’ growth potential and reviewed M&E data on the profitability of businesses. As evidence of growth during the project period, increased profitability of businesses is an indication of potential for growth potential. About two-thirds of the agribusinesses that responded to the question whether they expect to see a growth of their business expressed confidence in the growth potential of their firm. Especially in RADP￾East and RADP-South, businesses stated that their business was growing and/or provided growth projections for the next three to ten years. They expressed the aim to expand into other provinces in Afghanistan, or increase their international export. A comparatively larger share of businesses for RADP-West and RADP-North9 seemed less confident about their growth potential. They were trying to expand their business while some emphasized the necessity of continued support from the program, such as in the form of technical and financial assistance. Some businesses highlighted the security situation, which inhibits them from growing their business. Some firms also shared the expectation that they will create more job opportunities in the future, adopt more advanced production techniques, 9 Note however that there was a lower response rate for this question among business respondents interviewed for RADP-North than in the other regions. 22 expand their product range, or standardize their products and become ISO certified. A few agribusinesses stated that they would want to create job opportunities especially for youth and women, and raise awareness and more intensively engage women in farming by establishing farming centers or mass media. While many enterprises were optimistic about future growth potential, some enterprises were informal, small, or only earned a few thousand dollars in profit. Therefore, they may not have strong potential for growth. For example, in RADP-East, some female-owned businesses were quite small, like dairy product producers, and had very weak linkages to the dairy value chain. A few others noted that their growth potential depends on factors like security and government policy to support business growth. Furthermore, M&E data shows that only firms supported by RADP-North, RADP-South, and RADP-East reported operating more profitably because of assistance to date (see Table 9). As a result of these three RADPs, 906 firms reported operating more profitably. RADP-West support did not result in improved profitability for firms during the project period, although it did not set this as a significant goal. RADP-North and RADP-South exceeded their LOP targets. RADP-East 19 supported firms are operating more profitably (24 percent of LOP target), but was only halfway through project implementation at the time of assessment. It is likely that firms that did not become more profitable with RADP support do not have good potential for growth. EQ1b: Are beneficiary enterprises flexible or adaptable enough to be responsive to market shifts? To assess whether beneficiary enterprises were more responsive to market demand, the ET looked at factors related to the health of enterprises and how they changed as a result of project engagement. Improvements in business practices and management demonstrate some flexibility and ability to change on the part of the enterprises, and the changes are likely to improve businesses’ capacity to adapt to future market shifts. According to program M&E data, 1,047 private enterprises and organizations applied improved organization-level technologies or management practices across the RADPs (see Table 9), with the strongest performance among RADP-North. With USG-assistance, 790 enterprises in RADP-North (113 percent of LOP target), 149 enterprises in RADP-South as of FY16 (44 percent of LOP target), and 79 enterprises in RADP-East (40 percent of LOP target) applied improved practices. As a result of RADP-West assistance, 29 enterprises applied improved practices, which represented 26 percent of their LOP target. In interviews, business representatives reported benefits their businesses experienced because of RADP support, such as improved management practices or capacity. Two-thirds of agribusiness representatives interviewed for RADP-North, RADP-South, and RADP-East felt that project assistance had positive effects on their businesses. In comparison, only about a quarter of RADP-West businesses reported experiencing benefits from project engagement, in large part due to receiving limited support over a short period of time. Agribusinesses that received more intensive support (about two-thirds of agribusinesses), such as in-kind support and one-on-one technical assistance, typically reported more significant benefits than agribusinesses that did not receive in-kind support, or that only participated in group trainings, or that received less one-on-one support (about a third of agribusinesses). Agribusinesses that received both technical assistance (in the form of training or direct one-on-one support) and either in-kind support or financial assistance reported that the project assistance was beneficial and perceived there were positive changes to their business. The most commonly perceived changes were related to increased staff capacity and market linkages. Agribusinesses reported that they were able to strengthen their connections with farmers through their participation in the projects. For example, one agribusiness reported that they were able to meet “face-to-face” with more farmers throughout the district. 23 Table 9. M&E Indicators Related to Engagement and Results of the Agribusiness Support Model Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Number of private enterprises, producers’ organizations, water users’ associations, women’s groups, trade and business associations and CBOs that applied improved organization-level technologies or management practices with USG assistance 701 790 113% 200 79 40% 340 149 44% 110 29 26% Number of firms (excluding farms) or civil society orgs engaged in agricultural or food security-related manufacturing and services operating more profitably because of USG assistance 370 480 130% 80 19 24% 290 79 27% 8 0 0% 24 Agribusinesses that reported receiving only technical assistance most commonly reported improvements in staff capacity and connections with farmers. For example, a RADP-East agribusiness that received support in setting up text-based messaging system reported that their relationships with input suppliers and farmers are stronger than before. A RADP-North agribusiness felt they received good support in finance, which enabled them to hire 20 new employees. Agribusinesses that received only in-kind support, such as refrigerators, milk cans, or other types of machinery (see Figure 5), were more likely to report that they experienced one or more of the following positive changes: improved efficiency in their business process, increased production capacity, and enhanced product quality. For instance, one agribusiness reported that the machinery they received from RADP helped to increase their products, which led them to export more products and enabled them to hire more employees. A couple of RADP-East agribusinesses felt they had not received the equipment they were promised but believed that once they received the equipment, their processes would become more efficient. Nearly half of business respondents reported that they saw their business models as sustainable and able to adapt to future conditions. The positive changes that respondents noted can help them to adapt to market shifts. In particular, improved staff capacity, market linkages, efficiency, and growth reported by businesses were likely to help enterprises to be flexible and adaptable in the future and their ability to change shows some level of ability to adapt. Despite project assistance, about a third of business respondents reported that their businesses have low sustainability related to their business health or the enabling environment that may constrain their ability to adapt and respond to market shifts. A few businesses, like dairy companies and VFUs that were not profitable and did not think their business model was sustainable, are unlikely to be able to adapt to market shifts as they are unable to operate profitably in current market conditions. Respondents reported not being profitable and sustainable because they are not able to process products and only sell raw goods, have limited staff capacity, did not have permanent office, and cannot recover costs for animal health services provided. A few businesses reported being dependent on donor-funds, therefore they are unlikely to be adaptable enterprises without donor assistance. For example, one business in the North that imports farm equipment noted, “We don’t know when or if we can sell [the equipment]. We need donor contracts to maintain our organization and teach farmers the benefits of using modern farm equipment.” Figure 5. Dairy Processing Equipment at Supported Agribusiness 25 Further, over three-quarters of businesses reported that they faced key challenges that were not addressed by RADP support. The most common ongoing challenge and risk reported by businesses was a lack of security. Nearly half of businesses reported this. Of the challenges within the scope of RADP programming, a quarter of businesses reported interconnected challenges of a lack of access to equipment and machinery (e.g., veterinary instruments, refrigerators), low profits, and lack of access to finance. Some businesses in the East also reported a lack of electricity and limited ability to afford solar panels. EQ2: IS THE TRAINING MODEL DESIGN AND IMPLEMENTATION EFFECTIVE IN CONTRIBUTING TO BENEFICIARY ADOPTION OF IMPROVED AGRICULTURAL TECHNOLOGIES AND LICIT CROPS? The ET identified three sub-questions under EQ2 on the design and implementation of the farmer training model—EQ2a examined how the selection criteria, curricula, and pedagogy addressed farmer constraints; EQ2b examined saturation in communities; and EQ2c focused on the extent to which the approach considered the varying socioeconomic status of beneficiaries. EQ2a: How did selection criteria, curricula, and pedagogy address farmer constraints? Farmer beneficiary selection for RADP-West, RADP-South, and RADP-North was based on a simple set of criteria. In the districts that were deemed safe enough for implementation, selected farmers were: aged over 18 years (some programs were geared towards adolescents but these differed from the normal programming); at least 15 percent female; able to access land and water with sufficient control of these resources to manage them; and producers of the target crop or livestock. Some farmers were also selected to participate due to membership in farmer cooperatives. In practice, because village elders and maleks (village headman) were usually gatekeepers, they frequently assisted in identifying beneficiaries. This sometimes contributed to beneficiaries being selected for training who did not fit the criteria and could not benefit fully from activities because they were not farmers (e.g., relatives of government officials). RADP-East's process for farmer selection differed from the other regions, in part because training was conducted almost entirely through agribusiness beneficiaries. RADP-East identified farmers related to each of the four identified value chains: poultry, dairy, tomatoes, and apricots. Project staff developed a database of farmers during their value chain analysis. They also worked with MAIL, PAIL, and DAIL to identify farmers in each value chain to participate in trainings, and with the agribusinesses that identified their farmer suppliers. There were a few cases of farmers participating in trainings that were not relevant to them, such as one training on dairy production where women attended that did not own cows. RADP-West faced the most challenges with farmer selection (although they improved the selection criteria and its consistent use in the second year). Some project staff reported that inconsistent selection criteria were used across provinces, and subcontractors used their own selection criteria. For instance, one project employee noted that clear documentation was lacking about beneficiary selection, and subcontractors selected farmers near the city of Herat due to convenience. GPS locations of beneficiary villages for RADP-West corroborate this. In addition, a few FGD respondents noted that beneficiaries were included who did not engage in agricultural production. Furthermore, some FGD respondents reported that the selection process created conflict in communities. One respondent said, “It created problems in our community and village. They select a number of farmers from five to ten to participate in the project. The rest quarrel and criticize that why do they not include all farmers? This causes a problem among people.” The RADPs used specific processes to identify female beneficiaries. RADP-North selected female beneficiaries in coordination with Department of Women’s Affairs (DOWA), and interviewed women to identify suitable beneficiaries. The project also worked with village leaders to ensure that the village 26 was accepting of the activities. For RADP-East, in addition to women identified by agribusiness grantees, the project tried to select women who were engaged in entrepreneurial activities or were interested in doing so. This included women with ongoing businesses, women with money and a plan to start a business, and women who wanted to start a new business but did not have the resources to do so. The largest share of farmer beneficiaries in the FGDs (about two-thirds) learned about RADP through neighbors, other farmers in their community, local leaders, and in some cases, village councils. Some farmers also heard about the project through project staff, particularly in the North, and through advertisements. Although some were beneficiaries of previous projects, this was not necessarily how they became involved in RADP. Each project sought to address the agricultural production constraints faced by selected beneficiary farmers primarily through training and distributing agricultural inputs. Farmer training was the cornerstone of the initial RADP program design and was key to its theory of change. Working with farmers in the wheat, HVC, and livestock sectors, RADP-West, RADP-South, and RADP-North conducted farmer training directly using approved training models. Implementers tailored training for female farmers (see section 4.3 on gender), but otherwise did not significantly tailor trainings based on the socio-economic characteristics of participants (see section on EQ2c). Implementers also adapted and improved training models during implementation (e.g., RADP-South redesigned HVC training to focus on one topic in detail instead of covering all six key topics). RADP-North provided training to some agribusiness grantees, such as seed companies, which in turn provided training to farmers. To varying degrees, the RADPs also tried to improve farmer linkages to markets (see section on EQ3b). To enable and encourage farmers to attend trainings, the RADPs provided stipends to cover farmers’ transportation costs (e.g., AFN 200 or USD 2.63). RADP-East relied almost entirely on its agribusiness grantees to conduct trainings with farmers that were integral to sustaining their businesses, thereby incentivizing them to conduct effective trainings. This approach also lowered activity costs, which was necessary in the region with the largest geographic coverage and lowest overall budget. However, there was significant variety in the content and method of these trainings because they were not conducted by the implementer, and the efficacy of individual models was more difficult to determine. Training sessions were often provided at demonstration plots run by a lead farmer,10 which other beneficiary farmers could visit for hands-on training and see the results of improved practices (see Table 10).11 Ideally farmers were able to visit these farms at points along the entire growth cycle, including field preparation, planting, fertilizing and pest management, irrigation, harvesting, and post-harvest processing. However, farmers did not necessarily visit demonstration plots or receive training on the whole growth cycle, which affected the benefits the farmers received. In addition to hands-on training, farmers attended classroom training. In some cases, farmers attended an initial one or two-day classroom training, which was followed with what farmers called “practical” training at demonstration plots. For instance, HVC farmers in RADP-South attended a day of classroom training that included lectures and flip charts, and a second day of practical training at a demonstration plot. There was variation in how many training sessions an individual farmer attended. Some farmers reported only attending a single training session, while others attended several training sessions. Some project staff, particularly in RADP-North, reported that high beneficiary targets constrained their ability to provide repeated training support to farmers. 10 A lead farmer was a farmer selected from the community who ran a demonstration plot that showcased improved agricultural technologies and mentored neighboring farmers on improved practices. The lead farmer to beneficiary farmer ratio was relatively high in the North, South, and West, with a ratio of approximately one to 100 for RADP-West and RADP-North and one to about 900 for RADP-South. The ET does not have this ratio for RADP-East. 11 RADP-South made a distinction between use of model farms that showed production of one crop versus demonstration plots that showed multiple crops. 27 Some farmers reported that training that included practical instruction and demonstration plots were particularly useful (e.g., greenhouses, fields, vineyards) compared to only receiving lecture or classroom based learning. For example, a female farmer in the East noted that dairy training did not include practical demonstrations, but instead “the trainer showed [techniques] from a book and nobody understood.” Table 10. Farmer Training Activity Types by Region Region Wheat High-value crops Livestock RADP￾North Demo plots Market linkage activities Production training Sample seed packs Seed business and market development Technologies like laser land leveling and backpack sprayers Demo plots Grants for greenhouses Market linkage activities Post-harvest management training, including food safety Production training, including pest management and improved seeds Solar water pump installation Butcher training Field veterinary capacity building Livestock health training Small ruminant rearing training Wool and cashmere production and processing RADP￾South Demo plots Nutrition training Post-harvest management training Production training, including water management and improved seed Demo plots Greenhouse production training Market linkage activities Post-harvest management training Production training, including pest and water management Livestock model farms Dairy and cashmere processing Field veterinary capacity building Livestock disease prevention and control Poultry, small and large ruminant rearing training RADP￾East N/A Demo plots Market linkage activities Post-harvest management, including solar driers Production training Dairy and poultry production and processing training Market linkage activities, including to veterinary services RADP￾West Demo plots Market linkage activities Purdue Improved Crop storage bag distribution Production training Seed multiplication Demo plots Market linkage activities Post-harvest management training Production training Cashmere production and processing Farmer field days Veterinary field capacity building Market linkages for cashmere Trainings covered wheat production, over 30 types of HVCs, and several types of livestock and livestock products. Farming techniques and technologies were introduced, including seed selection, water management, land leveling, post-harvest management, drip irrigation, raised bedding, and raised trellising. More financially demanding technologies were also introduced, such as use of tractors and product processing equipment. Trainings in RADP-South, RADP-West, and RADP-North often addressed multiple crops, with multiple vegetable crops or fruit trees being addressed in the same set of trainings. Wheat was the exception, as this is a staple crop throughout the country and often the largest scale crop any farmer grew, and trainings focused only on wheat production. RADP-East had a much narrower value chain focus than the other regions, focusing only on dairy, poultry, tomato, and apricot. However, although agribusiness-run trainings focused on one of these areas, the trainings were often not limited to them as technologies were useful for multiple crops. Such carryover trainings made it 28 difficult to determine the full range of training benefits in that region. Training in livestock was the most diverse, differing in terms of the species covered, ruminant or poultry focus, diseases, and health topics, as well as the products made from livestock—from highly perishable dairy products like milk and yogurt to longer-lived kurut and cheese, as well as meat, eggs, and a variety of wool products. Livestock, especially ruminants, are particularly vulnerable to drought, and much of the variation in training content was tailored to drought levels throughout the country. Even if they found the trainings and inputs useful, farmers in over three-quarters of FGDs across regions reported challenges that were not sufficiently addressed through project support, as shown in Table 11. Of these, inadequate access to markets access (about half of FGDs) and a lack of water (about a third of FGDs; particularly in the North and West) were mentioned most often. Across the RADPs, beneficiaries felt the projects did little to improve their access to markets, linkages with buyers and traders, demand for their products, or ability to obtain higher prices for products (see EQ 3a for more detail). There was little indication by farmers that the project tried to address water availability. For example, one farmer in RADP-West said, “The big challenge we face is lack of water. If we have water we would have all things. This project could not help us. They did not dig wells and run solar power systems for our village.” Pests and crop diseases were also raised as an issue across the RADPs. Some farmers felt the project focused on crop production and should have provided more support on livestock production. A few farmers also felt more in-kind support was needed because farmers had limited resources to implement the practices taught. One male farmer in RADP-South explained, “Only trainings have been provided. There has been no kind of support or assistance. So practical works couldn’t be carried out due to our poor economy and these were the shortcomings of the project.” Table 11. Challenges Reported By Beneficiary Farmers The early terminations of RADP-West and RADP-South were reported negatively by some farmer respondents. One farmer said, “They have not kept their promises to support the farmers,” which was a frequently expressed sentiment. Many farmers in these regions had not been informed of project RADP-North RADP-South RADP-West RADP-East Crop production – pests, diseases Crop storage – lack of cold storage, lack of proper storage Inputs – lack of access to improved seeds, lack of machinery Markets – poor market access, low demand for products, low prices Water – lack of water, lack of fuel access for irrigation Crop production – pests, diseases Livestock production – livestock diseases, lack of access to medication, lack of knowledge about animal husbandry Markets – poor market access, low prices Water – lack of water Crop production – pests, crop diseases Inputs – lack of access to improved seeds, lack of access to fertilizer Livestock production – livestock diseases, lack of veterinary services Markets – lack of buyers, low demand for products Water – lack of water, lack of fuel access for irrigation Crop production – pests, diseases Crop storage – lack of cold storage, lack of storage facility Infrastructure – lack of electricity Inputs – lack of machinery Livestock production – livestock diseases Markets – poor market access, low prices Sales – low selling prices Water – water shortages, low water levels 29 closure and were still expecting assistance. For example, a farmer’s group was promised assistance that never materialized and it caused suspicious behavior among them, with rumors spread about who may have received what support causing controversy. EQ2b: To what level are the communities being reached (saturation)? The RADPs trained 284,180 individuals in agricultural sector productivity or food security. This includes training provided to farmers as well as government officials (e.g., DAIL officials) and private sector actors (see Table 12). RADP-North and RADP-South trained the most individuals (191,652 and 77,883 individuals, respectively). RADP-East trained 5,616 individuals, exceeding their LOP target of 5,000 individuals. RADP-West trained 9,029 individuals but did not make significant progress towards its LOP target of 80,000. In total, 158,977 beneficiary households (number of people in direct farmer beneficiary households) were reached across the RADPs (see Table 12). RADP-North and RADP-South reached the largest number of households, reaching 97,472 and 37,346 households respectively. RADP-West fell below its LOP target of 80,000 households, reaching 12,911 beneficiary households, although this was expected with early termination. RADP-East had lower household targets than the other regions with a LOP target of 51,550, and reached 11,248 households by FY18. The RADPs reached a range of beneficiaries at the community level, with less than 10 beneficiaries in some villages and over 100 in others. GPS locations of beneficiary villages for RADP-South and RADP-West, and training locations for RADP￾North12 showed that beneficiary villages tended to be clustered around provincial capitals. The ET was not able to determine what level of saturation the project reached at the community level due to a lack of population data at the community level. Based on provincial population estimates, the project reached only a fraction of the rural population. Comparing 2018-19 population estimates from the Central Statistical Office (CSO) with RADP beneficiary numbers, RADP did not reach more than 1 percent of the rural population in most provinces, except RADP-North which reached up to 5 percent of the rural population in the provinces Balkh and Jawzjan. EQ2c: To what extent does the approach include considerations for varying socioeconomic status among beneficiaries (e.g., security of land tenure, income)? The RADPs included some basic considerations of socioeconomic status in designing and implementing farmer support interventions. In particular, the RADPs often adapted the content and location of trainings to the needs of female beneficiaries (see EQ4a for details). While training was not explicitly adapted to farmers’ income levels, RADPs tried to align training with farmers’ access to inputs, particularly agricultural machinery. The RADPs also provided small transportation stipends to farmers to enable their travel to training locations. In a few cases, they made specific adaptations in certain localities. For example, RADP-South created two demonstration farms in one village due to tribal divisions in the village. Beneficiary farmers in about three-quarters of FGDs across the RADPs reported that the training content was relevant to them and provided an important source of agricultural information. Farmers’ positive assessments implied that the trainings included considerations for their socioeconomic conditions. A few farmers made particularly strong statements about the usefulness of trainings, indicating farmers had limited access to agricultural information. One male farmer in RADP-South stated that, “The training helped us learn new things, otherwise we had no idea of these things and learned nothing in the past 50 years.” Farmers most commonly discussed learning about vegetable production, followed by wheat, fruit, and livestock production. Women more commonly learned about vegetable and livestock production, compared to men who more commonly learnt about wheat and fruit production. This aligns with gendered divisions of agricultural labor in rural Afghanistan (USAID 2016a). 12 RADP-North did not have GPS locations of beneficiary villages available to provide the ET. RADP-East did not have GPS locations for beneficiary villages or trainings. 30 Beneficiaries most commonly reported that they learned about the use of fertilizers, pesticides and herbicides, irrigation, planting techniques, land preparation techniques, and seeds. Other topics included 31 Table 12. M&E Indicators Related to Implementation of the Farmer Training Model Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Households that benefited from agriculture and alternative development interventions 96,800 97,472 100% 51,550 11,248 22% 79,510 37,346 47% 80,000 12,911 16% Individuals that received US Government￾supported short-term agricultural sector productivity or food security training 140,000 191,652 137% 5,000 5,616 112% 124,400 77,883 63% 80,000 9,029 11% 32 harvesting techniques, equipment or tool use, crop diversification, and crop diseases. Both men and women felt they learnt about these topics, although men primarily discussed land preparation, which is typically a male task (USAID 2016a). Farmers reported learning concrete skills from the trainings. As one male RADP-South beneficiary explained, “The useful thing in the project was that they taught us practical activities. They would practically teach us the cultivation methods in the demonstration plots. For example, they taught us how to grow wheat and not to leave rye plants next to the wheat crops.” This respondent also explained that trainings earned more credibility over time as farmers saw the advice resulted in production improvements. He said, “At the beginning, people were not attentive to the training, but later on they got to know that the program was useful,” and once they saw that tree planting advice resulted in higher production, “We wanted to participate and learn something from the project.” In addition to training, the RADPs also provided inputs, such as improved seeds, fertilizer, pesticide, and equipment like buckets or small-scale greenhouses. This was both to address material constraints to adoption of agricultural technologies, and to incentivize technology adoption. In some cases, the projects distributed packages of inputs to lead farmers and other beneficiaries. For example, RADP-North provided melon seedling input packages to 83 female farmers that included a small-scale greenhouse, peat trays, peat moss soil, a sprinkler, a shovel, a thermometer, disposable gloves, and disposable masks. In a few FGDs, farmers reported that some of the saplings and seeds they received from the project and planted did not provide the yield improvements expected by farmers. In about two-thirds of RADP-West FGDs, all respondents reported receiving inputs, compared to respondents in about a third of RADP-North FGDs and less than quarter of RADP-South FGDs. All respondents in only one FGD for RADP-East reported that they received inputs, however some respondents received inputs in about two-thirds of RADP-East FGDs. No respondents reported receiving inputs in about two-thirds of RADP-South FGDs, which was the highest portion of no respondents in FGDs receiving inputs across the regions. Some respondents reported that previously they did not have access to the inputs provided by the project and it was not clear whether they would be able to access these inputs on their own when RADP ends. In some areas, farmers faced barriers to participating in the project. The key constraints reported by participants were distance and security. Distance was particularly a challenge for women who were unable to attend trainings far from home at places such as DAIL. As one woman said in RADP-West, “If a seminar is conducted in our area, we can attend. If the seminar is conducted out of our area, we cannot participate because we are women and it is difficult for us.” Another woman noted that their husbands would only allow them to participate in trainings in their village. She said, “The seminars should take place in our village and there should be an incentive. This way our husbands would allow us to participate in the seminar.” RADP-South particularly faced constraints due to instability and changing security conditions, which reduced the project’s ability to maintain engagement with beneficiary farmers. Farmers in the RADP-West gave mixed responses about whether or not security was a challenge, and farmers in the RADP-North and RADP-East did not raise security as a common constraint. At least one farmer in nearly a quarter of FGDs reported that they did not find the trainings useful because they did not provide new information, the training was of poor quality, the information did not provide the expected benefits, or because they were unable to implement the advice provided. Farmers reported this in all regions, but more commonly in RADP-East and RADP-North. For example, one male farmer in the North reported, “We did not learn any new skills. I was part of the training and am doing the same things as before.” Some farmers found the project’s advice too hard or expensive to implement because it relied on the use of agricultural machinery or resources that farmers could not access. A male youth farmer in the West said, “The project encouraged us to use agricultural technologies but did not provide us the technologies. We do not have the financial means to purchase 33 these machineries. They told us to use tractors but we could not afford that. They told us to use cows to plow if we do not have tractors. We also do not have cows.” Farmers who only received inputs, typically reported that only receiving inputs was not sufficient support. In three FDGs for RADP-West, farmers reported that they only received inputs but not training. Farmers in RADP-West more often felt they did not receive support or assistance from the project compared to farmers in other regions. For example, one RADP-West participant said, “We didn’t receive training and we were not encouraged. They only gave us five maan13 fertilizer and wheat.” EQ3: BASED ON THE IMPLEMENTATION APPROACH, STATUS OF IMPLEMENTATION, AND MILESTONES ACHIEVED AT THE TIME OF MID￾TERM EVALUATION, IS THERE EVIDENCE THAT THE PROGRAM IS LIKELY TO ACHIEVE ITS RESULTS BY THE END DATE? In order to address EQ3, the ET analyzed results relating to the agribusiness model (EQ3a-b) and the farmer training model (EQ3c-d). Sub-questions relating to the agribusiness model addressed whether the activities created more market linkages (EQ3a) and if beneficiary enterprises were more responsive to market demand (EQ3b). Sub-questions relating to the farmer training model addressed the degree to which beneficiary farmers adopted improved agricultural technologies (EQ3c) and licit crops (EQ3d). EQ3c also covered the outcomes that beneficiary farmers reported experiencing as a result of agricultural technology adoption to help determine whether the project was likely to achieve its results. EQ3d also examined the change in land area under poppy and non-poppy crops using GIS/remote sensing analysis. EQ3a: Are activities creating more market linkages? Are those market linkages more efficient in terms of maximizing returns to the smallholders while meeting the market needs? Farmers rarely reported an improvement in market access or market linkages. While the project tried to help foster market linkages between enterprises through trade fairs and other activities, beneficiary farmers in less than a quarter of FGDs overall cited market linkage or access improvements as a result of project assistance, most commonly in the South (nearly a third of RADP-South FGDs). Farmers in about half of FGDs (and higher portions in North and West) discussed market access and linkages as an area that RADP did not provide adequate assistance. As a woman in RADP-North explained, “The only problem the project didn’t influence was that the project didn’t introduce any buyer of our products and most of the crops got spoiled.” It is possible that there were indirect beneficiaries who experienced improved engagement with supported agribusinesses. Farmers in nearly a third of FGDs in the West noted that the project said they would introduce them buyers or purchase their products, but this did not happen before the project ended. For example, a woman in RADP-West said, “There were no changes” regarding market access. “They just told us that they would introduce us to the shopkeepers and they have not done that because the project got terminated and we got separated. They have not introduced us to anyone.” About half of agribusinesses reported improved market linkages, primarily those in the East and South. Most project staff reported that one of the main goals was to establish and consolidate market linkages to help ensure project sustainability. Staff referred to different strategies to achieve this: business-to￾business (B2B) events, business-to-farmer events, and trade shows and agricultural exhibitions. B2B events were organized in primarily RADP-East and RADP-North to help agribusinesses connect with input suppliers, wholesalers, buyers, importers and exporters, and to help improve understanding of market requirements. As many of the agribusinesses were small producers, they often did not have the experience to market and export their products. At B2B events, the RADPs aimed to link agribusinesses 13 One maan is equal to five kilograms. 34 with larger firms that often had experience with exporting their produce nationally and internationally. Besides B2B events, three project staff from RADP-East also mentioned that they organized business-to￾farmer events to link farmers with inputs suppliers and processors, and helped identify suppliers of high quality products. For example, RADP-East project staff connected women who dried tomatoes to spice companies and exporters of dried tomatoes. They also reported teaching women how to collect and market their eggs and linked them with larger marketers. Project staff from RADP-East, RADP-North, and RADP-South said they used trade shows and agricultural exhibitions to display new agricultural technologies and connected agribusiness with national and international buyers and suppliers. In addition, RADPs provided some support that resulted in contracts for purchases or exports for agribusinesses. Several RADP staff emphasized the success of these initiatives in forging market linkages, and considered B2B events and trade shows particularly effective in generating higher sales for agribusiness. Staff from RADP-East gave anecdotal evidence of the success of linking producers to markets, including an example of a woman who participated in a trade show and won a contract to export apricots, and a feed-miller who decided to organize its own B2B events in other Afghan provinces. EQ3b: Are beneficiary enterprises more responsive to market demand? To assess whether beneficiary enterprises were more responsive to market demand, the ET reviewed M&E data on increased sales from the enterprises, which indicates that they have been able to respond to market demand and increase the sale of their products. The ET also reports qualitative data on respondents’ perceptions of their responses to market demand. Assisted enterprises and farmers generated incremental sales valuing $55.4 million (see Table 13). RADP-North assistance resulted in the highest value of incremental sales, at over $49 million (141 percent of LOP target). RADP-South resulted in $3.4 million in incremental sales as of FY16. RADP￾West resulted in just under $500,000 in incremental sales, which was only one percent of the LOP target. RADP-East resulted in over $2.4 million in incremental sales, meeting their LOP target in Year 3. RADP-South and RADP-West performed better in relation to increased sale of commodities and livestock (from producers and agribusiness), with a value of $101.2 million and $1.3 million respectively. RADP assistance also generated exports, with exports of targeted commodities valued at $1.2 million for RADP-East and $138,368 for RADP-South (RADP-North and RADP-West did not have this as an indicator). As discussed under EQ 1b, the positive changes that business respondents noted can help them to respond to market demand. In particular, improved staff capacity, efficiency, marketing, and market linkages can help businesses respond to market demand in the future. Although RADP has only been able to create market linkages to a limited extent for beneficiary farmers, improved market linkages was one of the most common improvements reported by businesses. A few business respondents also discussed working in areas that have significant market demand and implicitly responding to that demand. For example, a business in the South noted that they import hybrid seeds, which have high demand from farmers and need to be replaced every year. A few businesses discussed plans to expand their business operations and trade in response to market demand. For example, one RADP-East business said that they plan to target portions of the Indian market where there is high demand for Afghan dry fruits. A RADP-South business discussed trying to increase partnerships to import more improved seeds, pesticides, and fertilizers to meet farmer demand. EQ3c: To what degree did beneficiary farmers adopt improved agricultural technologies? In this question, the ET discusses beneficiary farmer technology adoption and the benefits they reported from those technology changes to help answer EQ 3: whether the project will achieve its intended results. 35 Table 13 M&E Indicators Related to Results of Agribusiness Support Model Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Value of sales of targeted commodities as a result of USG assistance; livestock sales (producers and agribusinesses) increased $92,300,000 $164,915,919 179% $8,000,000 $1,213,326 15% $95,000,000 $101,207,185 106% $4,000,000 $1,325,824 33% Value of incremental sales by agribusinesses and farmers as a result of USG assistance $34,800,000 $ 49,071,924 141% $2,425,382 $2,413,654 100% No target $3,387,235 433.6% $58,000,000 $487,103 1% Value of targeted agricultural commodities exported with USG assistance N/A $1,191,196 $1,213,326 102% No target 138,368 N/A *N/A means it was not an indicator for that RADP 36 Table 14 M&E Indicators Related to Results of the Farmer Training Model Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Number of farmers and others who have applied new technologies and/or management practices as a result of USG assistance 90,375 65,255 72% 40,000 5,388 13% 83,880 41,971 50% 60,000 11,130 19% Number of hectares under improved technologies or management practices as a result of USG assistance 60,923 91,956 151% 500 280 56% 183,172 59,725 33% 12,000 1,753 15% Percentage change in annual production (S, W) /productivity (E) of key agricultural crops by target farmers, as compared to non￾beneficiary farmers N/A Increase by 30% from baseline Not yet reporting Not yet reporting 20% 30.5% 153% 23% 8.4% 37% Percentage increase in household income from licit agriculture in targeted areas 25% over base￾line 68% 172% 40% increase over base-line Not yet reporting Not yet reporting 186% 196% 20% 2.3% 12% Note: N/A means it was not an indicator for that RADP. 37 Due to RADP assistance, 123,744 farmers and other producers applied new technologies or management practices, and 153,714 hectares were under improved technologies or management practices (see Table 14). RADP-North and RADP-South reported the strongest results for producers. RADP-West only achieved a small portion of LOP targets (19 percent of target for number of farmers and others who applied new technologies or management practices, and 15 percent of target for production of key agricultural crops by target farmers compared to non-beneficiary farmers). RADP￾South exceeded its target (220 percent of LOP target) with a 30.5 percent change in annual production (as of FY1614). RADP-West did not meet its LOP target before termination, with only 8.4 percent change in annual production achieved. RADP-East set a target of 30 percent from baseline but had not reported on their achievement to date. RADP-South, RADP-North, and RADP-West all reported an increase in household income from licit agriculture in target areas (RADP-East had set a target but not reported their achievement on this indicator as of FY18). RADP-South reported the largest percentage increase of 196 percent, followed by RADP-North at 68 percent, and RADP-West at 2.3 percent. Agricultural technology and crop adoption. The majority of farmer respondents reported that they applied the information they learned. Respondents primarily reported making changes to wheat production (primarily male beneficiaries) and vegetables (primarily female beneficiaries). Respondents in less than a quarter of FGDs reported that they made changes to livestock production. Almost half of respondents reported that they made changes to how they plant, care for, harvest, and store crops as a result of project training. Most commonly, farmers reported that they changed their use of agricultural inputs, namely fertilizer, pesticides, and improved seeds. The use of improved inputs largely was because farmers used the inputs provided to them by the RADPs, although some farmers reported purchasing these improved inputs after learning about them from the project. For example, RADP-East did not provide inputs to farmers as frequently and created more direct linkages between agribusinesses and farmers. One farmer in the East reported “We buy the fertilizer from the market and buy seeds from the companies which sell improved seeds. They have not provided us improved seeds.” Changes to how they plant crops was the most common agricultural technology change reported by farmers, followed by improvements in irrigation, land preparation and land use, and harvesting techniques. The majority of the respondents reported that they used the inputs received from the project and they found the inputs beneficial. In particular, farmers reported positive benefits from the green houses. In a few cases, respondents said that they received agricultural inputs even though they were not involved in production. For example, a respondent said they received wheat seeds, but they do not produce wheat so they took the seeds to a mill to grind and then used the flour for consumption. In a few cases, farmers reported problems with quality of the seed provided or its suitability for the local area. Farmers reported several motivations for the changes they made to their agricultural production. They made changes to their agricultural production to improve yields and income, with broader goals to improve their family livelihood and self-sufficiency, improve their lives, and contribute to Afghanistan’s development in some cases. Because of the benefits they experienced from initial application, most farmers reported that they planned to continue the production changes they adopted, although some farmers noted that input access would be a challenge in the future without project support. Outcomes reported by respondents. An overwhelming majority of respondents reported their yields improved as a result of project participation. All respondents in all FGDs for RADP-East and in about 80 percent of the FGDs for RADP-South, RADP-West, and RADP-North reported that their yields increased. Female respondents in one FGD for RADP-South reported that their yields did not improve as a result of program participation. In the remaining FGDs, a mix of respondents said their yields improved and others said their yields did not improve. Farmers did not necessarily report the specific crops or livestock products for which their yields improved, but they most commonly reported 14 FY16 was the last year that the M&E household survey was conducted due to early termination, so FY16 is the last year that RADP-South collected data on some indicators. 38 their yields increased for vegetables followed by wheat and fruit. While many respondents reported yield improvements, the ET was not able to determine how much their yields increased. Farmers reported that their yields increased as a result of applying what they learned during trainings (e.g., how to apply fertilizers and use improved planting techniques), and because of inputs received from the project (e.g., improved seeds). For example, a male farmer in the North explained, “our yields have increased comparing to the past, because in the past we did not know the modes of cultivation, irrigation and fertilizer application, but now we have learnt how to cultivate, how to irrigate, and how to apply fertilizers.” A few farmers explicitly said the improved yields helped improve their access to food and reduced their need to buy food from the market. For instance, a female RADP-North beneficiary said, “We don’t purchase spinach from the market and we collect from our home garden, and we don’t need to buy from the market as we can address our problems with our products, which is the best help for us.” In addition to increased yields, some farmers also reported that their production was of higher quality due to production, storage, and processing changes. Beneficiary farmers reported yield improvements in a context where there were some weather stresses (particularly in the West), but normal wheat production during most of project implementation until 2017/2018 when drought and low rainfed wheat production caused widespread food insecurity in 2018. Based on the 2013 to 2018 FEWS NET Food Security Outlooks and remote sensing data, most regions experienced near average weather conditions until 2017/2018, with the exception of moderate drought in the South in 2013 and severe drought in the West in 2014 during the main growing season (FEWS NET 2018; Van der Schrier et al. 2013; Osborn et al. 2016; Osborn et al. 2017; Osborn et al. 2018; Jones et al. 2012). According to FEWS NET, wheat production was at or above the 10-year average between 2013 and 2016 nationally. The start of the wet season in 2013 was slow with below average amounts of rain and snow, the second half of the wet season received ample snow leading to normal harvests of staple crops and above average national grain production. However, the South experienced drought in 2013, with some portions having minor drought and others having moderate to severe. In 2014, the wet season started earlier than usual with sufficient rains resulting in adequate rainfed irrigation, with the exception of the West, which experienced severe drought. Despite a temporary dry spell, growth of staple crops was normal in most regions. Agricultural seasons in 2015 and 2016 witnessed near average precipitation and wheat production, although there was a decrease in rainfed wheat production in 2015 and 2016 and yields below the 5-year average in parts of the North. Parts of the North and nearly all of the RADP regions in the South and West had dry spells and below average cumulative precipitation in 2017 main growing season, but higher than average temperatures, leading to rainfed wheat production only slightly below the 10-year average. The situation worsened in 2018 when most of the country faced drought conditions, in particular in the West and North regions where the drought, in combination with increased conflict, reduced agricultural production, increased the wheat deficit, considerably reduced household food stocks, and led to an atypically high number of households in need of emergency assistance. Despite drought conditions and a deficit in rainfed wheat production in 2018, fruit, vegetable and irrigated crop yields were not significantly impacted, in part due to above￾average temperatures. A majority of respondents reported their incomes improved as a result of project participation. Respondents primarily attributed higher incomes to increased crop sales due to increased yields, and some reported it was due to higher-quality products (e.g., from using improved drying techniques). Few reported increased market access or linkages helped improve income. All respondents reported their incomes increased in all FGDs for RADP-East, in about three-quarters of FGDs for RADP-South, and in about two-thirds of FGDs for RADP-North and RADP-West. In two FGDs with female beneficiary farmers, one for RADP-South and one for RADP-West, the women all reported their incomes did not improve. In the remaining FGDs, some respondents said that their incomes improved, while others said they did not. It is important to note the ET could not determine the degree to which beneficiaries’ 39 incomes improved through the FGDs. Increased incomes were reported during a period of rising national poverty. The 2016-17 Afghanistan Living Conditions Survey found poverty increased from 34 percent in 2007-08 to 55 percent in 2016-17, mostly since the 2013-14 survey round (CSO 2017). Farmers reported that improved income benefited their families, including improving their ability to purchase food and school supplies. Improved yields and incomes are critical to household food security and nutrition where 45 percent of the population is food-insecure (CSO 2017). EQ3d. To what degree did beneficiary farmers adopt licit crops? What was the percentage change in land area under poppy and under non-poppy crops in the target areas? The ET quantified licit and illicit crop production using GIS/remote sensing analysis with a GIS to compare cropping patterns before RADP implementation and during the evaluation period for the participating districts in each of the four RADP regions.15 The results of this analysis were mixed with regard to changes in licit and illicit crop production. In the RADP-East and RADP-West regions, the amount of land allocated to poppy production fell both in absolute terms and in terms of the proportion of total hectares dedicated to agricultural production. In the RADP-South region, total hectares allocated to poppy production increased, but the amount of land allocated to other agriculture also increased. Consequently, the proportion of total agricultural land allocated to poppy was actually lower during the evaluation period than it was before RADP-South implementation began. In the RADP-North region, results were less encouraging—poppy production rose substantially in terms of its absolute level and its proportion of total agriculture. Table 15 below summarizes these results. In the RADP-South region, the ET estimated 23,600 hectares of poppy in the pre-program period (2013) and approximately 28,400 hectares in the program evaluation period (2017)—an increase of about 20 percent. For other agricultural lands, the ET saw a similar increase: from about 284,000 ha to about 353,500 ha—a roughly 24 percent increase in area. However, the area of poppy as a percentage of total agriculture decreased slightly from approximately 7.7 percent in 2013 to 7.4 percent in 2017. It is interesting to note that the amount of water/snow was estimated to be 120 percent higher in 2017 than 2013. Therefore, it could be that the increase in the area under poppy cultivation was due to increased water availability. This argument is supported by the fact that the other crops also increased as well, but the reduction in poppy as a percentage of total agriculture is encouraging. In the RADP-North region, the ET found a large increase in poppy production between the two periods. In 2013, the ET estimated about 15,000 ha of poppy, which increased to almost 81,000 ha by 2017—an increase of approximately 440 percent. In contrast the RADP-South region, other agriculture decreased about 22 percent—from about 700,000 ha in 2013 to about 545,000 ha in 2017. The ET also saw an increase of about 67 percent in land cover classified as water/snow. Therefore, the increase in poppy production could be in part due to increased water availability. However, as the ET saw areas of poppy production in the desert areas in the evaluation period but not in the pre-program, it is also possible that the increase could be due to a shift from licit to illicit agriculture, especially as the decrease in absolute amount of other agriculture (-151,673 ha) was roughly twice as much as the increase in poppy area (65,827 ha). As a percentage of total agriculture, poppy production increased from 2.1 percent in 2013 to almost 13 percent by 2017. Alternatively, the anomalously low amount of production in 2013 could be due to environmental conditions. 15 Jones et al. 2018. Mid-Term Performance Evaluation of the Regional Agriculture Development Program in Afghanistan: Estimating Percentage Change in Land Area under Licit Agriculture in Program Areas. Washington, DC: Program Evaluation for Effectiveness and Learning, ME&A. 40 In the RADP-East region, there was a large decrease in poppy production between the post-program (2018) and pre-program (2015) periods. The ET estimated about 12,000 ha in 2015, which decreased to approximately 2,600 ha in 2018—the reduction of about 9,300 ha represented about 78 percent decrease in crop area. The ET saw a large decrease in other agricultural production between the evaluation and pre-program periods as well, with over one million ha estimated in 2015 and about 786,000 ha in 2018—a reduction of about 23 percent. As a percentage of total agriculture, poppy represented a very small amount, but decreased from 1.14 percent to 0.33 percent between the time Table 15. Quantification of Land Cover Types across RADP Regions Classification Hectares Change 2013 2017 (2018 for RADP￾East) Change in hectares % change in hectares Poppy as % of total agriculture 2013 2017 (2018 for RADP￾East) RADP-South Poppy 23,663 28,401 4,738 +20.0% 7.7% 7.4% Other agriculture 284,285 353,499 69,214 +24.3% N/A Water/Snow/Ice 9,155 20,201 11,046 +120.7% N/A RADP-North Poppy 14,967 80,794 65,827 +439.8% 2.1% 12.9% Other agriculture 699,315 547,642 (151,673) -21.7% N/A Water/Snow/Ice 9,909 16,581 6,672 +67.3% N/A RADP-East Poppy 11,873 2,593 (9,280) -78.2% 1.1% 0.3% Other agriculture 1,026,206 785,874 (240,331) -23.4% N/A Water/Snow/Ice 164,726 496,768 332,043 +201.6% N/A RADP-West Poppy 31,518 13,490 (18,028) -57.2% 10.9% 2.1% Other agriculture 257,369 640,210 382,841 148.8% N/A Water/Snow/Ice 1,660 5,284 3,624 +218.4% N/A periods. Similar to the North and South regions, the ET saw an increase in land cover classified as water/snow—with roughly a doubling (202 percent) of area between the two periods. There are a number of potential reasons for the large reduction in illicit crop production between the program periods. However, environmental conditions could have been less favorable for agriculture in the latter time period, given that other agriculture decreased by about 240,000 ha while water increase by 200 percent increase (mainly in the form of snow) in 2018 compared to 2015. The provinces in Eastern Afghanistan are also located in a mountainous region of the country. Lastly, the RADP-West region (Table 15) saw a large reduction in poppy production between the 2014 pre-program period and the 2017 program evaluation period. In 2014, the ET estimated approximately 31,500 ha of poppy, while in 2017 our estimate was 13,500—a decrease of about 57 percent. Other agricultural area increased by approximately 150 percent, from 257,000 ha in 2014 to more than 640,000 ha in 2017. The percentage of poppy to total agriculture also dropped from about 11 percent to 2 percent between the time periods. This, along with the fact that total licit agriculture increased by 41 about 383,000 ha, could plausibly be explained as a shift from illicit to licit crops, although other environmental factors could be at play. The increase in area classified as snow/water (about 3,600 ha) between the time periods could have made conditions more favorable for licit agriculture. GIS results lead us to conclude that poppy production is still a fairly significant component of local economic activity in most RADP regions, even where it has decreased since RADP started. Mansfield (2018) discusses several key drivers that explain the persistence of opium poppy production in Afghanistan. One is its obvious appeal as a high-value/low-weight commodity, and it is a drought tolerant crop. Another is the practice of sharecropping, in which landowning households give land-poor ones access to land that they can then leverage to make advance purchases backed by future opium production; otherwise they would not be able to access credit. More directly, hundreds of thousands of Afghans have resettled to desert areas since the fall of the Taliban, looking for economic opportunities or for new land to cultivate following opium poppy bans in other parts of the country. Because these desert lands require significant capital and labor to become productive, necessitating deep wells that cost thousands of dollars, opium poppy is the only viable crop. It stands to reason then that much of the expansion in poppy production in recent years has occurred in desert lands that were previously un￾cultivated. Figure 6 below shows an example of the expansion of poppy production into the desert in the RADP-North region. The recent proliferation of solar-powered water pumps in recent years may spur further expansion of desert poppy production, as Mansfield (2018) points out these systems are more reliable and have lower recurrent costs than diesel-powered deep wells. Because of the particular agricultural economics of crop production in the desert, it is unlikely RADP could have significantly reduced opium poppy production in those areas. At most, RADP could have mitigated further expansion of opium poppy into already-fertile riverbed areas where cultivation of licit crops is economically viable. In those areas, there is a risk that farmers can use the agricultural inputs provided by RADP for opium poppy production and some agricultural knowledge and skills may be applicable to poppy production. In particular, inputs like fertilizer and pesticides can be used on illicit rather than licit crops, though respondents did not report re-purposing RADP-supplied inputs or training toward illicit crops. The ET was not able to ask respondents directly about opium poppy production due to ethical Institutional Review board concerns; however, a few respondents in FGDs for RADP-South and RADP￾North brought up opium poppy production. Farmers in a few FGDs reported that they, or people in their area, had stopped growing opium poppy, despite an overall trend of continuing opium poppy production. In a few cases, this was due to advice from elders or warnings from the government. For example, one female youth respondent said that, “We plant wheat and cotton. Previously people used to plant opium and this opium destroyed the land and the people got exhausted. Elderly people told us not to cultivate opium and now we don’t plant opium anymore. Instead of opium now we plant cotton, wheat and melons and this is for the land owners and those without lands are workers and just harvest the crops.” A male respondent in the North said, “Last year, poppy cultivation was growing in our area but the village leader announced in the mosque that whoever cultivates poppy will be punished by the government. The people heard the village leader and stopped cultivating poppy.” In three FGDs for RADP-South and two FGDs for RADP-North, respondents said that the reduction in opium poppy production was related to project participation, including from using improved seeds, earning a good income from Halal crops, or diversifying to high value crops like saffron. A female beneficiary from RADP-South explained, “It was useful for the men. Earlier, they would grow poppy which could not generate good revenue for them, but now, they changed the crops as a result of knowledge and skills they got from the project.” 42 Figure 6. Desert Area in RADP North Region Showing Expansion of Poppy Crop EQ4: To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals? To answer EQ4 on the integration of women and youth in RADP and their contribution to program goals, the ET addressed four sub-questions related to women and youth separately. The ET discussed which approaches were most effective in reaching women and youth (EQ4a), how women and youth benefited from RADP (EQ4b), missed opportunities for integrating women and youth (EQ4c), and successes that could be adapted to integrate women and youth (EQ4d). EQ4a: Which approaches have been most effective in reaching women and youth respectively? Integration of women. The integration of women in RADP activities needs to be understood within the broader context of gender equality in rural Afghanistan. Overall, women’s labor in the agricultural sector remains largely invisible. Although 70 percent of rural women are involved in farming, and women make up a large share of the livestock and poultry sector, their work is typically unpaid and lumped into “domestic” labor. This makes their contribution difficult to quantify (Ganesh 2017). Women are often not allowed in public spaces, even farms outside their houses, without mahrams (a husband or close male relative who serves as a chaperone). As a result, most of women’s agricultural work takes place in their home garden, contributing to household food security and nutrition, and HVC processing. Most agricultural development projects, including RADP, include women-focused components. However, skills-building interventions for women often fail to address the market linkages needed for Pre-program (left) and evaluation period (right). Red = poppy; Green = other agriculture; beige = other land cover (e.g., desert). 43 home production, and overestimate women’s connections with the local economy. In addition, there is a persistent government policy deficiency that allows women’s work to remain undervalued and unpaid (Ganesh 2017). According to Ganesh, there is insufficient evidence on women’s participation in agriculture and other sectors and their income generation, which means that their “contribution is undervalued, while their potential remains underutilized” (2017:19). Afghan government stakeholders recognize the role of women in the agricultural sector, but progress in implementing gender mainstreaming in agricultural policy has been slow, although it has been stronger than in other ministries (USAID 2016a). The RADPs engaged women in varying capacities between regions, working within the diverse constraints present in each area. Women in the North, West, and Eastern regions were able to participate more fully in farming, including activities that occurred in relatively public places. Women in the South, as well as certain areas of the East, were more constrained from participating in these activities due to greater gender inequality. For example, a few women in the South said that they could not attend trainings because they did not have a mahram available to accompany them. Gender was not well mainstreamed in RADP-South programming, as the gender program operated independently from other activities completed with male beneficiaries. In addition, RADP-South training for women relied heavily on social media and PowerPoint presentations. This may have limited the transfer of knowledge and skills, as well as the ability to improve farming practices, as these are unfamiliar training methods to Afghan women. Furthermore, some trainings were located outside villages, which women from traditional Pashto households could not access. While RADP-North mainstreamed gender into its program activities, integration was slow and could have received stronger staff support. The goal of ensuring at least 15 percent of project beneficiaries were women was met or exceeded in every region (see Table 16). RADP-West and RADP-North were more easily able to integrate women’s agribusiness, whereas RADP-South faced greater challenges, as highlighted above. In all regions, it was less common for women to have tazkeras, bank accounts, or assets. Without these, formal financing can prove difficult. However, this was mitigated in part by encouraging male family members to secure funding on behalf of businesswomen. RADP-North was able to help develop a few female-owned businesses, including a noodle factory and two bakeries. However, RADP-North found that they could not include women in some agribusiness activities, such as butcher shops, because there were no women-owned butcher shops. RADP-East changed their grant criteria so that the project was able to work with female-owned businesses without the capital to co-invest. Women’s theoretical classroom training, training in demonstration farms, and agribusiness support was typically conducted in a single-sex environment. This was critical to enable women to attend trainings and visit demonstration plots (see Figure 7). For example, RADP-South established female-only demonstration farms with the DOWA. However, occasionally RADP-East trainings— conducted by agribusiness beneficiaries, not the prime implementer or its subcontractors—were conducted in a mixed-sex environment. Figure 7. Visiting Project Site After Female FGD 44 Integration of youth. The RADP design did not explicitly target youth as a beneficiary group. However, the Government of Afghanistan defines youth as 18-35 years of age: as 37 percent of FGD respondents fell into this age range, this suggests that a sizable proportion of RADP beneficiaries were youth. RADP-South conducted some trainings in high schools as a safe training location for women. While training youth in schools was not a formal programmatic goal, training participants may have been younger than if the training had been conducted in a different location. In addition, RADP-South directly targeted female adolescents in high school (under18 years of age) for capacity building around entrepreneurship. RADP-South also conducted a limited number of mobile theater performances targeting high school attendees. FGD respondents reported that other adolescents benefited indirectly from the program, because their parents shared what they had learned from training and household income improved. In some cases, improved income directly benefited adolescents as their school expenses were covered. A few respondents also highlighted the importance of training and job opportunities offered by the project to some youth. Farmer beneficiaries recognized the importance of transferring knowledge and skills gained during training to youth, and of encouraging youth to become more involved in farming. Farmers underlined the significance of training youth to improve their farming skills, not only because farmers often relied on their children for labor, but also because farming was seen as a tradition and the only livelihood opportunity for youth. EQ4b: How have women and youth respectively benefited from RADP implementation, and how are these benefits measured? Female beneficiaries reported adopting agricultural practices they had learnt and using the inputs provided. This included adopting new production and post-harvest practices, and starting gardens in their compounds. For example, a woman in the West said: “The project staff gave us seeds for cultivation and we cultivated. Before, we did not have anything to cultivate and our lands were unused, but now we yield crops such as kale and cauliflower and they make profit for us.” In a few cases, women did not directly use the information they learned about improved inputs because their husbands were in charge of purchasing inputs. Females reported the same benefits of project participation as males. The majority of female FGD participants reported improved yields due to project assistance (reported by women in three-quarters of female FGDs for RADP-North, RADP-East, and RADP-West, and all women in two-thirds of female FGDs for RADP-West). In some FGDs, both men and women provided mixed responses about whether their yields and incomes improved in all regions. However, it was only women in one FGD from RADP￾East and one FGD from RADP-South who unanimously agreed their yields did not improve. Women often could not directly sell agricultural products on their own due to gender norms and constraints on female mobility; they sold goods through their husbands, sons, or other male relatives. Despite this, the majority of women reported that their income improved as a result of program participation (reported by all women in two-thirds of female FGDs for RADP-West and RADP-South, and all women in three-quarters of female FGDs for RADP-North and RADP-East). However, women may not have maintained control of their income due to patriarchal norms in extended family households. Some women reported that the project benefited their husbands as they shared the knowledge they learned with them, and the results were beneficial for their households as a whole. For example, according to a woman in the North: “We received the training and implemented what we learned. Our men supported us. When we collected the crops and sold them, we were able to buy stuff for our house, clothes, and other necessary things. This is a help to our men. In the past, we did not have income; our men could not afford to buy everything. Therefore, they would feel uncomfortable. Now that we can purchase things, the men also benefit.” 45 Women did not report any reduction in their labor burden. However, a few men felt that women benefited from reduced workloads due to men’s adoption of machinery (e.g., wheat threshers). No conflicts were reported by women as a result of project participation or benefits accrued. This may be related to the perceived benefit of the project for the household, and that the project largely did not challenge existing gendered divisions of labor. In a few cases, men reported that they were not aware of women participating in or benefiting from the project. One woman in the South described how the project was beneficial to men and women while maintaining gendered divisions of labor: “The project benefitted both men and women. We grow crops and practice agriculture while our men go and sell products in markets. So, both women and men found work, and as a result our men were prevented from wrongdoings and all the problems were resolved.” Another woman in the West explained that women’s improved income was beneficial for men and women: “It is helpful for men in a sense that women are enabled to help them in regard to income. If women earn, they can take care of their own expenses, which helps men and they can both live happily.” A few female FGD participants said that the project increased their self-sufficiency and decreased their reliance on their husbands for access to agricultural knowledge and farming. According to project documents, at least half of female beneficiaries reported increased self-efficacy at the end of training/programming (except RADP-East, which did not have this indicator) (see Table 16). RADP￾North and RADP-South nearly achieved LOP targets with 75 percent and 63 percent of females respectively reporting an increase in self-efficacy. RADP-West achieved 72 percent of its LOP target, with 54 percent of females reporting an increase in self-efficacy. Despite this, there were still significant constraints that limited the benefits women experienced from RADP participation. Project staff from all regions acknowledged the systemic and policy framework changes needed to enable fuller engagement of women in the agricultural sector in formal and quantifiable ways. EQ 4c. Are there any missed opportunities with incorporating women and youth respectively into RADP activities? While the RADPs met their targets for engagement of female beneficiaries, there may have been some missed opportunities around engaging women in agribusiness activities. RADP-East made adjustments to its selection process to increase the participation of female-owned businesses. However, the project still struggled to engage female-owned businesses that met its criteria. In addition, RADP-East grantees did not implement trainings in a gender sensitive manner, because they conducted them in a mixed-sex environment. Women faced some barriers to engaging with the project. For example, female respondents were unable to participate in project trainings and demonstrations that took place outside of their village if they did not have a mahram who could accompany them. Women also faced barriers to directly selling agricultural products due to gender norms that constrained their mobility. Not all women were able to travel to markets to sell their products, so some discussed selling through male relatives. The RADPs did not explicitly target youth or adolescents. According to the 2016-17 Afghanistan Living Conditions Survey, the country has one of the highest shares of young people in the world, with almost half of its population aged under 15 years (CSO 2017). Some respondents suggested providing farming equipment to motivate youth to participate in trainings and take up farming, in addition to directly transferring knowledge. Farmers frequently expressed the view that more youth should be encouraged to become involved in agriculture by their families, project staff, or village elders, because the agriculture sector provides important job opportunities for them. This view was shared by one of the project staff who thought youth should be involved in trainings, particularly paravet trainings. According to this interviewee, para-veterinary training provides a good basis for youth employment. Some respondents highlighted the need for more local income generating opportunities for youth, particularly male youth. 46 Table 16. M&E Indicators Related to Results of Female Integration Indicator RADP-North (as of FY18) RADP-East (as of FY18) RADP-South (as of FY16) RADP-West (LOP) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) LOP target LOP achieved LOP achieved (%) Percentage of female participants in USG assisted programs designed to increase access to productive economic resources 15% 16% 107% 15% 22% 146% 15% 11% 73% 15% 18% 120% Proportion of females who reported increased self-efficacy at the conclusion of USG supported training/prog ramming 80% 75% 94% N/A 70% 63% 90% 75% 54% 72% Note: N/A means it was not an indicator for that RADP 47 EQ 4d. Are there any successes from RADP that can be adapted to specifically integrate women and youth respectively? Overall, RADP demonstrated successful integration of female farmers in training activities, although they were less successful integrating female business owners in agribusiness activities. There is potential to adapt the success of providing both grant and technical assistance to primarily male-owned agribusinesses to meet the needs of women-owned businesses using lessons from the successful female farmer training. However, projects would need to create criteria for assistance that women-owned businesses are able to meet. Projects can draw from factors that influenced the success of female farmers training. These include holding trainings in accessible environments for women, such as near their homes, and obtaining buy-in from “gatekeepers,” such as fathers or elders, who constrain women’s mobility. Agribusiness support can also focus on areas where women traditionally have more control, such as enterprises related to homestead garden products and livestock production, which women typically have more control over [USAID 2016a]). In addition, a few women farmers found the practical training and demonstrations particularly useful. Therefore, agribusiness support can include practical, hands-on methods, such as on-the-job coaching and visits to successful businesses, rather than only lecture-based methods. EQ5: IS THE REGIONAL APPROACH TO THE RADP PROGRAMMING LIKELY MOST EFFECTIVE FOR ACHIEVING PROGRAM GOALS? FOR EXAMPLE, HOW MIGHT A REGIONAL APPROACH COMPARE TO A COMMODITY-SPECIFIC APPROACH? From its earliest phases, RADP was designed with a regional approach focused around supporting wheat, livestock, and/or HVC value chains within clearly defined Regional Economic Zones (Eastern, Western, Southern, and Northern) (USAID 2017). RADP-South and RADP-West (both now ended), as well as RADP-North and RADP-East (on-going), were implemented using this regional approach. Reporting on differing sets of indicators limits the comparability of effects of this regional approach across RADPs.16 The RADP regional approach aligns with the USAID 2015-2018 Transition Plan and emphasizes countrywide and rural development (USAID 2016b). However, USAID notes in its 2017 Project Appraisal Document that this regional approach might be shifted to a commodity-specific approach, targeting value chains countrywide to “leverage on efficiencies gained from scale, coordination or harmonized approach, and implementers’ technical niche” (USAID 2017:11). Recent USAID priorities have focused on shifting funding and activities to the five major urban areas. However, it is unclear how these priorities could align with the shifting focus of agricultural development to a commodity-specific approach. The regional approach taken by RADP supported improvements in agribusiness functioning and farmer production across a number of licit commodities and value chains. However, respondents did not report that RADP helped achieve synergies across specific commodities or value chains. In addition, contractor M&E systems did not track common indicators across RADPs that allowed assessment of market development outcomes that could result from a regional approach. While the RADPs in the West, South, and North ostensibly targeted three value chains—wheat, HVC, and livestock—they covered over 30 distinct crops and products. Although wheat was targeted as a single value chain, other RADP 16 As raised in the 2018 Special Inspector General for Afghanistan Reconstruction audit of RADP, changing performance indicators and targets hindered the ability to measure regional and RADP-wide progress. It also had the trickle-down effect of necessitating changes in M&E completed by the contractors, which frequently delayed evaluation and monitoring. Indicators and targets are understood to have been frequently changed to better align with Afghan governmental goals or account for security changes. Yet this ad hoc manner of changing indicators contributed to a lack of continuity in measurable indicators (SIGAR 2018). 48 activities concentrated effort in varying degrees on other products from production to marketing. As a result, they were often unable to influence key points or improve market linkages along the value chain in any one locality. RADP-East took a more targeted approach and focused on four single crop or product value chains. These might yield more improved market linkages within the value chain, but as yet insufficient information has been reported for decisive evaluation. Activities that focused on improving the dynamics and functioning of market systems (e.g., resources, rules, relationships, and roles) leveraged existing capabilities and incentives. While this is an attractive outcome, significant market constraints exist that limit the potential of market development activities in rural Afghanistan (Minoia and Pan 2017). Recommendations from businesses suggested that RADPs could provide more support along key points in value chains. About half of business respondents would value more training to improve the quality of production and packaging, and more support with linking their business to markets, as well as marketing their products domestically and internationally. A few value chain actors voiced interest in and a need for broader support for their businesses beyond a particular product, including grants that would allow them to pay salaries, build staff capacity, develop product certification, and facilitate linkages with international markets. Furthermore, a few business respondents wanted more support with accessing financing. Others suggested that RADP should conduct more careful assessments of business needs, including the range of problems they faced. The ET did not find strong evidence that the RADP interventions across value chains were able to impact significantly on the value chain for specific commodities. The majority of farmers did not report improved market linkages or access to markets. So while about half of businesses did report improved market linkages, farmers and businesses operating in the same commodity value chains did not report improvements in the value chains. It is still too soon to tell if RADP-East’s more targeted value chain and facilitative market systems approach better accelerates regional agricultural development compared to approaches used by the other RADPs. The ET did not find evidence that RADP had a regional influence on agricultural markets or a significant effect on specific commodity value chains. This may be in part due to RADP’s multiple goals of improving food security, incomes, and profitability. However, the ET had limited ability to assess the regional influence of the program through qualitative data collection. EQ6: TO WHAT EXTENT WILL INSTITUTIONAL AND INFORMATIONAL STRUCTURES BE SUSTAINABLE AFTER PROJECT IMPLEMENTATION? The RADPs all strived for sustainable results by transferring skills and knowledge to farmers and agribusinesses to increase business profitability, and by building the capacity of DAILs/PAILs with an enabling environment component. Responses on sustainability varied across respondent groups. Farmers and businesses will likely continue to apply knowledge and skills gained from RADP, and may pass on this knowledge to others in the future. Farmers discussed sharing information that they learned with other farmers and youth. As discussed in EQ3c, most farmers reported that they planned to continue the production changes they adopted as a result of RADP assistance, although some farmers noted that input access would be a challenge in the future without project support. Some businesses may serve as a source of information and extension for farmers going forward. However, farmers did not reference businesses as sources of agricultural information and typically did not report strong linkages to market actors. This raises questions about whether agribusinesses will be a sustainable a source of agricultural information for farmers. However, agribusinesses in the East may be more likely to be a sustainable source of information because of the key role that they are playing in farmer training during the project period. Growth potential of agribusiness and their likely flexibility and adaptability to market shifts served as indicators for the sustainability of supported agribusiness, as described under EQ1. The ET found that about two-thirds of the agribusiness expressed confidence in the growth potential of their firm referring 49 to the expansion of national supply and exports, creation of more job opportunities, investments in more advanced production techniques, or increases in their product ranges. More or less one third of the agribusiness were less confident about their growth potential, citing the necessity of sustained support or the security situation. A significant number of agribusiness in RADP-North, RADP-East, and RADP-South operated more profitably (see EQ1a for more details). Agribusiness also demonstrated some flexibility and ability to change to market shifts as a result of business capacity, and improved business practices and management. Nearly half of agribusiness reported that they saw their business models as sustainable and able to adapt to future conditions, which can help them to adapt to market shifts. About a third of agribusiness indicated that they have low sustainability, which may constrain their ability to adapt and respond to market shifts for varying reasons as discussed in EQ1b. As noted in EQ1b, a few businesses reported being dependent on donor-funds. As such, they may not be able to sustain improvements without RADP support, particularly those that were reliant on grants or in-kind support. Overall, project staff were reasonably confident about the sustainability of project activities. About half of project staff expressed confidence that results could be sustained or that the project would be sustainable conditional upon adoption of technologies by farmers, improvement in markets, or if farmers were able to make a profit. Respondents mentioned the market-based approach as the main reason that activities would continue after the project ended. They believed there was demand for farming practices and equipment that showed potential to increase farmer productivity, as long as farmers saw practical results. Although a few members of staff were concerned that training support was too short to ensure sustainable results, others saw strengthened relationships along the value chain, improved market linkages, and increased awareness of buyers leading to sustainable results. Some project staff also reported a certain degree of institutionalization that would help sustain project activities. For example, access to equipment required grantees to train their staff on equipment maintenance, and where grants were provided, grantees were required to demonstrate how they would sustain business development and reinvest operational profits. While RADP aimed to achieve sustainable outcomes through its value chain and market interventions, donor-led value chain programs may have limited ability to sustainably shift the complex structure and functioning of market systems in rural Afghanistan. This is particularly the case for programs related to the non-economic structuring of markets in rural Afghanistan (Minoia and Pain 2017). The market systems approach employed by RADP-East with buy-in required from grantees aims to improve sustainability of agribusiness interventions and farmer support. As one RADP-East staff remarked: “When development programs are value chain actors, they damage the value chain when they withdraw.” However, this perspective may overestimate the penetration of USAID programs into value chains in Afghanistan, and underestimate the risk that agribusinesses face in adopting new practices, as well as the dominance of large wholesalers. It is too soon to tell if RADP-East’s approach will be more sustainable for businesses, or the extent to which businesses will continue to provide extension services to farmers without grant support. Government stakeholders generally reported more skepticism about project sustainability than RADP staff. Coordination with the government and institutionalizing policy changes are one way that the project intended to sustain effects beyond the project period. RADP coordinated with the government, including MAIL and DOWA. However, most government stakeholders interviewed in Kabul and at the provincial level reported relatively weak coordination with RADP. Stakeholders provided mixed responses about whether RADP supported government-led initiatives or reforms. A few government stakeholders expressed disagreement with the design of RADP. The enabling environment component, implemented primarily by RADP-South, worked directly with the government to institutionalize policy reforms, including providing the government with information and policy analysis recommendations, and participating in policy drafting processes related to agriculture. RADP-South helped draft eight policies 50 on topics including livestock, horticulture, and public private partnerships. However, these policies were not ratified by the time of the evaluation. They may be de facto in use, but the use and impact of these policies is not yet clear. RADP-South provided the government with constraint analyses that reflected grassroots perspectives, and developed action matrices to promote ownership of policy reforms. In addition, RADP-South embedded a staff member at MAIL. Some of the policy support RADP provided to MAIL was constrained by uncertainty and changes in decision making by USAID. Respondents provided mixed answers about who was responsible for project sustainability. Project staff stated either that the government was responsible or that private sector grantees should have responsibility for sustaining results. Government respondents were often not sure who was responsible for sustaining interventions and had little to say about the potential for scaling or institutionalizing interventions. These mixed responses suggest that some changes spurred by the project may not be sustainable. OVERALL SUMMARY AND CONCLUSIONS RADP was implemented in a complex, volatile environment. It aimed to improve the food and economic security of rural Afghans. Each RADP activity employed a value chain facilitation methodology that worked in all segments of agriculture supply chains from the farmers (producers) to the consumers, and aimed to address bottlenecks in each targeted value chain. RADP aimed to increase adoption of improved agricultural technologies by farmers, improve licit agricultural production, and decrease illicit crop production as a result. Among businesses, RADP aimed to improve business capacity, functioning, and profitability to increase growth and sustainability in the agriculture sector. The RADPs primarily provided grants and technical assistance to agribusinesses, and training and agricultural inputs to farmers across diverse value chains. They also aimed to engage women across value chain activities. RADP is likely to achieve its results in related to: outcome 1: increased agriculture sector productivity and profitability in target region; outcome 2: increased profitability of small, medium, and large agribusinesses; and cross-cutting outcome 3: increased women’s participation in agriculture value-chain activities. RADP was successful in providing technical and material support to businesses and farmers across a range of value chains. However, the RADPs did not always meet their targets for the numbers of businesses and farmers supported. Each RADP was able to engage women, although primarily through farmer training activities rather than agribusiness support. Performance of the agribusiness support model was strongest in RADP-North and RADP-South, although implementation in RADP-East was nascent and making positive progress towards most targets. RADP-West fell short of its targets for key indicators on the implementation and performance of agribusiness support, and did not provide enough support to businesses for them to experience benefits, in part due to early closure. While most agribusinesses in RADP-North, RADP-South, and RADP-East reported improvements resulting from RADP support, most noted ongoing challenges, including insecurity and limited access to finance. Broad challenges, such as insecurity, were not addressed by RADP. Insecurity in particular was a key constraint and risk raised by businesses; however, it requires systemic changes and is outside the scope of agricultural development programming. RADP-South and RADP-North had the strongest performance along M&E indicators related to the farmer training model as of FY18 RADP-West largely fell short of farmer-related indicators prior to its termination. RADP-East was slow with start-up engaging farmers and achieving results on key farmer related performance indicators. Although there was varying performance along M&E indicators across the regions, beneficiary farmers largely reported that they experienced benefits from project support. These benefits included improved yields and agricultural incomes. More farmers in RADP-West were unsatisfied with project support and fewer farmers in RADP-West reported improvements in income. While benefits were reported, the ET was unable to determine independently the degree to which 51 agricultural productivity, incomes, or agribusiness growth or profits increased. At the same time, beneficiary farmers reported a number of challenges that RADP did not address significantly, including water access and market linkages. In addition, the ET found mixed reports with regard to changes in licit and illicit crop production. Poppy production fell in the RADP-East and RADP-West regions. However, total hectares allocated to poppy production increased in RADP-South—although the proportion of total agricultural land allocated to poppy production lowered due to an increase in other agricultural production. In the RADP-North region, results were less encouraging—poppy production rose substantially in terms of its absolute land area and its proportion of total agriculture. It is difficult to predict the sustainability of project interventions. However, project staff were often optimistic about the sustainability of interventions, and agribusinesses were similarly optimistic about their growth potential. Depending on the extent to which farmers and agribusinesses continue to apply the knowledge and skills they acquired through the project, and the durability of in-kind support, these changes have potential for sustainability. However, many beneficiary farmers expressed a need for future in-kind support to improve their agricultural production. Unsurprisingly the agribusinesses and farmers benefited from in-kind support, and grants in the case of agribusinesses, but it is unclear whether the benefits they experienced as a result of these inputs will be sustained after RADP support ends. Furthermore, government officials were often less optimistic about the sustainability of interventions and the degree to which changes were institutionalized. RECOMMENDATIONS The ET provides recommendations below based on findings for each EQ and the type of support beneficiaries reported they would like from future programs. We provide recommendations for RADP￾East and future programs. However, we do not provide recommendations for RADP-North because it is in its last year of implementation. EQ1: Is the agribusiness support model likely to be sustainable? • Because we found that some businesses were constrained by lack of access to finance, which we conclude will negatively affect the model’s sustainability, we recommend RADP-East and future programs connect businesses with sources of finance and build their capacity to obtain loans successfully. • Future programs should continue to provide a mix of technical assistance and in-kind support to help businesses both improve capacity and production quality and efficiency. EQ2: Is the training model design and implementation effective in contributing to beneficiary adoption of improved agricultural technologies and licit crops? Future programs and RADP-East should: • Strengthen and monitor quality of beneficiary selection so that training is provided to beneficiaries for who it will have the most value; ensuring that beneficiaries are farmers who engage in the targeted value chains will increase their likelihood of adopting new technologies and licit crops. • Further tailor the content of training to address the key constraints faced by farmers in different regions and to ensure that techniques and technologies promoted are accessible and feasible for farmers to implement. Improved practices and new crops need to be profitable for farmers to incentivize and sustain adoption. o In drought-prone areas, provide more training to farmers and provide demonstrations on water conservation technologies and the production of drought resistant crops. Farmers may be more likely to take up new technologies that do not have high water needs in areas where access to water is limited. 52 • Include more market related training modules to male and female farmers to build their skills on how to access markets for relevant crops and livestock products, build relationships with buyers, negotiate with buyers, and access market information (e.g., on prices, demand, quality requirements). Provide training to farmers that includes practical demonstrations and training at demonstration plots to improve the effectiveness of training on farmer learning and skill building. EQ3: Based on the implementation approach, status of implementation, and milestones achieved at the time of mid-term evaluation, is there evidence that the program is likely to achieve its results by the end date? Future programs and RADP-East should: • Strengthen efforts to help farmers access markets for high value crops by facilitating direct linkages between farmers and buyers to help increase farmers’ revenue from licit crops, because farmers may increase adoption licit crops if they can provide an economic substitute for poppy. • Strengthen agribusiness capacity to create direct linkages with farmers and to provide extension services to farmers to improve quality of production to improve farmer sales and agribusiness profits. EQ4: To what extent have women and youth been integrated into the implementation of RADP, and to what extent do women and youth respectively contribute to program goals? • RADP-East should encourage their agribusiness grantees to provide female-only trainings to improve the accessibility, acceptability, and effectiveness of training for women. • Future programs and RADP-East should work with women to obtain financial assistance with the backing of male relatives if necessary, to improve their ability to start and grow businesses. • Future programs should tailor agribusiness support interventions to small women-owned enterprises and develop criteria for grants that are possible for female entrepreneurs and women-owned businesses to meet. RADP-East should continue to evaluate its grant criteria to maximize participation by women-owned businesses. EQ5: Is the regional approach to the RADP programming likely most effective for achieving program goals? For example, how might a regional approach compare to a commodity-specific approach? • Future programs should choose whether to use a regional approach or a commodity-specific approach based on specific, core goals of the program. • In line with SIGAR recommendations (2018), RADP should harmonize M&E systems across projects where applicable to facilitate performance monitoring and assessment if it continues to operate as a regional program. EQ6: To what extent will institutional and informational structures be sustainable after project implementation? • Future programs and RADP-East should, increase coordination and collaboration with government to increase stakeholder buy-in and alignment with government policies and priorities to improve sustainability. 53 ANNEX A. EXPRESSION OF INTEREST STATEMENT OF WORK FOR EVALUATION OF REGIONAL AGRICULTURAL DEVELOPMENT PROGRAM (SOUTH, NORTH, AND WEST) I. BACKGROUND A. Project Identification Data Development Objective: Sustainable Agriculture-Led Economic Growth Expanded Project Title: Regional Agriculture Development Program Award Number: 306-C-13-00018 306-C-14-00002 306-C-14-00007 Award Dates: October 02, 2013 May 15, 2014 August 10, 2014 Funding: AG 75% AD 25% Implementing Partner (s): Chemonics International Inc. Development Alternatives Inc.(DAI) COR/Activity Manager: RADP-S (Xavier Preciado) RADP-N (Sardar Safa) RADP-W (Anafrida Bwenge) B. Development Context The agriculture sector is the main source of livelihood and subsistence for roughly 75 percent of the Afghan population and a crucial component to enhance food security and drive economic growth for the entire country. Agriculture contributes approximately 1/3 of the country’s GDP (excluding the opium economy). Only about 12% of the country’s 65 million hectares of land is arable and approximately 40% of the agricultural land is irrigated. Afghanistan’s agriculture faces a multitude of challenges including: insecurity; frequent droughts/disease; poor on-farm water management; inadequate extension services; poor agriculture practices including post-harvest handling, processing, and storage; lack of properly functioning markets; and weak agricultural associations and cooperatives. Quality input suppliers and veterinary health providers are in short supply and the market is flooded with low quality or counterfeit agro-chemicals, vaccines, and seeds. Women in agriculture are often disenfranchised due to sociocultural norms. Additionally, both the public and private sector institutions face capacity challenges including physical infrastructure, regulatory frameworks and skilled staff to build a modern and competitive agriculture sector. However, given the multitude of challenges the agriculture sector remains one of the most promising paths for economic growth in Afghanistan. Wheat is the Afghan staple crop, and efforts are underway to boost wheat productivity in the country and secure agreements with neighboring countries to help ensure food security. The high value horticulture sector (grapes, 54 raisins, pomegranates, nuts and vegetables etc.) offers nutritional benefits for the household and significant regional export potential. Finally, due to population growth (at an estimated 2.6 percent per year) and rising incomes, the domestic demand for livestock products, including meat, eggs, and dairy products, is increasing more rapidly than production. C. Project Summary The primary purpose of this mid-term performance evaluation is intended to provide an independent examination of the overall progress and accomplishments of the umbrella RADP activity to date in three regions of Afghanistan. The mid-term performance evaluation should identify achievements, performance issues, and constraints related to activity implementation and effectiveness of the program. The evaluation should provide important feedback on strengths and weaknesses of the activity for decision making by USAID, actionable items for USAID/Afghanistan, as well as lessons learned for USAID, the broader donor community, and GIRoA. This mid-term performance evaluation should also identify results and lessons learned from implementation and provide concise, recommendations to determine which component(s) of RADP to scale up, or modify. In addition, the mid-term evaluation findings and recommendations will be taken into consideration in the design of follow-on projects to increase the impact of the investment in particular regions. A secondary purpose included in this scope of work is to use the opportunity to collect useful data on two key indicators that are critical to decision making and articulating the multiplier effects of USAID’s programming on incomes. The two indicators of interest for which the awardee is expected to (i) draw a sample, and (ii) recruit enumerators to gather data are: • Prevalence of Poverty: Number of people living on less than $1.25/day • Percentage increase in household income from licit agriculture in target areas Examples of the Results Frameworks are included in Annex 1. D. Intended Results - Regional Agriculture Development Program To aid the government of Afghanistan, specifically the Ministry of Agriculture Irrigation and Livestock (MAIL) in addressing these challenges and capitalizing on opportunities, USAID is implementing the Regional Agriculture Development Programs (RADPs). The RADP program with a budget of just over $271 million operates across 13 provinces and 54 districts in the Northern, Southern and Western Afghanistan. Starting in FY 16, RADP-South will begin to add additional districts, depending on the security situation. The mandate of the RADP program is to improve economic and food security for rural Afghans. Each program employs a value chain methodology that works in all segments of agriculture supply chains from the farmers (producers) to the consumers. While there are regional differences, all programs focus on improving productivity of wheat, high value horticultural crops (grapes, raisins, pomegranates, stone fruits, nuts, vegetables etc.) and livestock. Increasing women’s participation in and benefit from the agricultural sector is a critical component of each program, and 15% of targeted beneficiaries are women. Private sector development is critical to any value chain approach, and RADP programs are working to strengthen relationships and trust between private sector entities that are involved in agriculture or those that are supporting 55 agriculture. Each RADP also focuses on critical cross cutting themes such as: on farm water management, nutrition, alternative development, finance, and supporting Departments of Agriculture, Irrigation and Livestock (DAILs) at the sub-national level. While the basic structure of all RADPs is similar, some implementation differences do exist. The paragraphs below describe some of the differences between each of the programs. This is expected, because they are implemented by different contractors and are working in different regions of the country, each with different populations, climate, and natural resources. RADP-South began implementation in October 2013 and is a $125 million project that operates in 4 provinces in Southern Afghanistan. The program has a blended approach between direct technical assistance/training and strengthening value chain linkages for input and output markets. For example, Wheat and Horticulture farmers are trained on the latest production and management practices as well as introduced to new technologies on 78 (soon to be 84) demonstration farms across their focus districts. They focus on: wheat, winter vegetables, summer vegetables, orchard crops and grapes. At the same time they are working with input suppliers and output buyers to strengthen the supply chain that farmers depend on to get high quality inputs, and then to have a market to sell their products. In year 3 the project has shifted a stronger emphasis on supporting the development of these private sector actors to provide higher quality and differentiated services to farmers. For example, it is in the interest of private sector businesses to train their own farmers/customers on production practices because they want to 1) sell fertilizer, equipment. if they are an input company; or 2) if they are a “buyer” they want to get high quality or quantities of produce to export or use in their business. They also focus on livestock, including animal health, poultry, and large/small ruminants. A similar model is used for livestock as for the other two focus sectors. Farmers are organized in extension groups or self-help groups and trained on best livestock management practices. These groups are linked to trained para-vets, supported by the project to provide animal health services to farmers. In year 3 there is a larger focus on working in the livestock supply chain and with livestock businesses including: poultry, dairy and animal feed. RADP-North began implementation in May 2014 and is a $78.4 Million project that operates in six provinces in Northern Afghanistan. RADP-North advances food and economic security for rural Afghans through a blended approach between direct technical assistance/training and strengthening value chain linkages for input and output markets. The program strengthens the capacity of farmers in improved production in the wheat, high value crop and livestock value chains. Its activities enable farmers to increase their commercial viability and competitiveness in response to market demand, thereby generating greater income opportunities and empowering families to better meet their household needs. RADP-North uses a value chain facilitation approach to address numerous bottlenecks and identify lead firms within the area capable of leveraging positive change. The final piece of RADP-North’s holistic framework is supporting a policy environment through advocacy and dialogue that strengthens the private sector. RADP-North’s implementation strategy is focused on a value chain facilitation approach that addresses bottlenecks and leverages lead organizations and farmers capable of supporting positive changes, enabling commercial stakeholders to increase viability and competitiveness to generate greater income opportunities, cross-cutting gender theme integrated throughout the program to increase women's participation and economic empowerment, promoting improved legal and regulatory framework, supporting enabling 56 environment through policy advocacy and dialogue to strengthen private sector via a mix of direct implementation and $10 million grant fund. RADP-West implemented between August 2014 and September 2017, and operated in three provinces in Western Afghanistan. RADP-West advances food and economic security for rural Afghans through a blended approach between direct technical assistance/training and strengthening value chain linkages for input and output markets. The project strengthens the capacity of farmers in improved production in the wheat, high value crop and livestock value chains. Its activities enable farmers to increase their commercial viability and competitiveness in response to market demand, thereby generating greater income opportunities and empowering families to better meet their household needs. RADP-West uses a value chain facilitation approach to address numerous bottlenecks and identify lead firms within the area capable of leveraging positive change. The final piece of RADP-West’s holistic framework is supporting a policy environment through advocacy and dialogue that strengthens the private sector. RADP-West’s implementation strategy is focused on a value chain facilitation approach that addresses bottlenecks and leverages lead organizations and farmers capable of supporting positive changes. Additionally, the project seeks to enable commercial stakeholders to increase viability and competitiveness to generate greater income opportunities, integrate gender theme throughout the program to increase women's participation and economic empowerment, promote an improved legal and regulatory framework, and support an enabling environment through policy advocacy and dialogue to strengthen private sector, and support a $10 million grant fund. II. EVALUATION RATIONALE A. Purpose This mid-term performance evaluation is intended to provide an independent examination of the overall progress and accomplishments of the RADP program in three regions of Afghanistan. The evaluation will identify achievements, performance issues, and constraints related to activity implementation and effectiveness of the program. The evaluation will provide important feedback on strengths and weaknesses of the project for USAID staff, actionable items for USAID/Afghanistan, as well as lessons learned for the USAID and the broader donor community as well as GIRoA. The evaluation will identify results and lessons learned from implementation and will provide concise, recommendations to determine which component(s) of RADP to scale up, or modify. In addition, the evaluation findings and recommendations will be taken into consideration in the design of follow-on projects to increase the impact of the investment in particular regions. A. Audience and Intended Users The result of this evaluation will be used by the USAID OAG Office to provide specific and targeted guidance on the management and future-orientation of the project. Copies of the evaluation report will be shared with relevant supporting Offices like the Office of Acquisition and Assistance, Office of Financial Management, Program Office, and implementing partners. USAID will develop a dissemination plan in accordance with the Evaluation Policy as specified in Automated Directives System (ADS) Chapter 203. 57 B. Evaluation Questions In addressing the objectives of the evaluation, the following key evaluations questions, under five broad categories, should be answered: Implementation • Agribusiness support: Is the agribusiness support model sustainable? o Are we supporting businesses that largely have good potential for growth? o Are activities creating more efficient market linkages, and responsive to market demand and shifts? • Training model: Is the training model effective? o How did selection criteria, curricula, pedagogy affect program goals? o To what level are the communities being reached (saturation)? o To what extent does the approach include considerations for varying socioeconomic statuses (land, income, etc.) Results • Measuring Success: Based on the implementation approach, status of implementation, and milestones achieved at the time of mid-term evaluation, is there evidence that the program is likely to achieve its results by end date? If so, please discuss, if not, please explain based on your findings. Cross Cutting • Gender: To what extent have women and/or youth been integrated into the implementation of RADPs, and contribute to project goals? o Which approaches have been most effective in reaching women? o How have women benefitted from RADP implementation and how are these benefits measured? o Are there any missed opportunities with incorporating women into RADP activities? o Are there any successes from RADP that can be adapted to specifically integrate women and/or youth? Thematic Questions to Consider in Conclusion: • Design: Is the regional approach to the RADP programming most efficient to achieving project goals? o For example, how does a regional approach compare to a commodity specific approach • Sustainability: To what extent will institutional and informational structures be sustainable after project implementation? The analysis and findings regarding each of the evaluation questions should be followed by specific, tailored, realistic and actionable recommendations aimed at improving outcomes. III. EVALUATION PRODUCTS 58 A. Deliverables See Section F. 7 of the Task Order contract. IV. TEAM COMPOSITION The Evaluation Team will consist of five key members, including at least one (1) international consultant (who will lead the team) and local consultants. The team members should represent a balance of several types of knowledge and expertise related to agricultural development. USAID/Afghanistan recommends the following staffing structure for the evaluation: Team Leader: An international Senior Evaluation Specialist with extensive experience in evaluating agriculture activities in developing countries. At least seven years of experience in evaluation management, and qualitative data collection and analysis. S/he must have experience in conducting evaluations and designing performance evaluations as well as the ability to produce highly quality evaluation reports in English. Strong interpersonal skills are required. Suggested 81 days Level of Effort (LOE). The Team Leader will be responsible for the overall management of the evaluation, including coordinating and packaging the deliverables. The team leader will develop the outline for the draft report, present the report, and after incorporating USAID/Afghanistan staff comments, submit the final report to USAID/Afghanistan within the prescribed timeline. In addition to the responsibilities outlined above, the Team Leader will: Preparations 1. Finalize and negotiate with USAID/Afghanistan the team’s work plan. 2. Establish assignment roles, responsibilities, and tasks for each team member. 3. Ensure that the logistics arrangements in the field are complete. Management 1. Take the lead on preparing, coordinating team member input, submitting, revising and finalizing the assignment report. 2. Manage the process of report writing. 3. Manage team coordination meetings in the field. 4. Coordinate the workflow and tasks and ensure that team members are adhering to the schedule. Communication 1. Handle conflict within the team. 2. Serve as primary interface with USAID/Afghanistan and serve as the spokesperson for the team. 3. Debrief USAID/Afghanistan as the evaluation progresses, and organize a final debriefing. 4. Keep the USAID/Afghanistan appraised of progress challenges, work changes, team travel plans in the field, and report preparation via phone conversation or email at least once a week. 5. Serve as primary interface with USAID/Afghanistan for the submission of draft and final reports and deliverables to USAID/Afghanistan. 59 6. Make decisions in conjunction with USAID/Afghanistan about the safety and security of the team. Other team members will include: A host-country national Agricultural Economist, with experience in evaluation of agricultural value chains and markets in developing countries; at least five (5) years of experience in agricultural program management and some experience managing or implementing research programs. Ability to conduct interviews and discussions in at least two major official languages, strong English language writing skills. Suggested 76 days Level of Effort (LOE). A host country or international senior or mid-level Social Scientist with strong analytical skills and gender expertise. The incumbent must be able to conduct interviews and focus group discussion, analyze the resulting data, and be familiar with qualitative and quantitative data collection methods. Strong English language speaking and writing skills essential. Suggested 76 days Level of Effort (LOE). A host country or international mid-level Private Scoter Expert, with experience in public-private partnerships; at least three (3) years of experience in private sector capacity building/assessment required. Suggested 76 days Level of Effort (LOE). A national Evaluation Specialist with at least five (5) years relevant experience and strong logistics and planning skills. Suggested 76 days Level of Effort (LOE). At least one of senior technical team member must have experience analyzing data from agricultural value-chain projects. At least one of the senior team members should have significant experience with gender integration and institutional capacity experience and expertise. USAID leaves to the contractor’s discretion other necessary team members/staff for the evaluation (e.g. logistics, scheduling and translation; data analysis). Aside from the above mentioned key personnel, the offeror must decide how the evaluation team should be structured in order to successfully address the evaluation questions. All attempts should be made for the team to be gender balanced and to include local (Afghan) experts. A statement of potential bias or conflict of interest (or lack thereof) is required from each team member. USAID will review and comment on the report for accuracy, but evaluators may accept or reject comments. The final report should reflect the opinions of the external evaluators and is the sole responsibility of the selected evaluation team. V. EVALUATION MANAGEMENT A. Logistics The Evaluation team will work under the technical direction of USAID/Afghanistan Evaluation Activity Manager, and the PEEL COR. A six-day work week is authorized for the evaluation team while in Afghanistan. The USAID/Afghanistan Activity Manager, and PEEL COR will 60 provide overall direction to the team, provide key documents, background materials, and help arrange presentation and debriefing. The contractor will be responsible for all international and in-country administrative and logistical supports, including identifying and fielding appropriate consultants (International and local). The evaluation team should be able to make all logistical arrangements including vehicle rental for travel within and outside of Afghanistan and should not expect any logistical support from the Mission. The team should also make their own arrangements for venues for team meetings, and equipment support for producing the report. USAID/Afghanistan will: • Approve country clearances for the Evaluation Team members and approve all subsequent internal travel by road and air by team members; • Provide the evaluation team with USAID essential contact information, and facilitate initial and subsequent communications and letters of introductions; • Provide the evaluation team with background documents and project documentation. C. Timeframe The expected in-country timeframe for this task is beginning on or about October 2016. Final report will be due on or about January 2017. Specific start and end-dates, travel dates, and due date for deliverables will be determined in collaboration with USAID/Afghanistan and based on the availability of the consultants. VI. CONTACT INFORMATION Mr. Evans Lartey (OAG/USAID/Afghanistan Monitoring and Evaluation Specialist - elartey@usaid.gov) will be the Activity Manager, plus COR/ACORs for the RADPs will be available for this evaluation, as needed. 61 ANNEX B. TRAVEL ITINERARY The contents of this annex have been redacted in their entirety. 62 ANNEX C. REFERENCES CITED Conrad, Abigail and Tulika Narayan. 2018. “Mid-Term Performance Evaluation for the Regional Agriculture Development Program in Afghanistan: Qualitative Evaluation Protocol.” Washington, DC: Program Evaluation for Effectiveness and Learning, ME&A. CSO (Central Statistical Organization of Afghanistan). 2017. “Afghanistan Living Conditions Survey 2016- 17: Mid-term Results-Highlights.” CSO and ICON-INSTITUT Public Sector GmbH. FEWS NET (Famine Early Warning Systems Network). 2018. “Central Asia: Afghanistan.” Accessed December 10, 2018. http://fews.net/central-asia/afghanistan/. Ganesh, Lena. 2017. “Women in Agriculture in Afghanistan. Issues Paper.” Kabul: Afghanistan Research and Evaluation Unit. Jones, PD, DH Lister, TJ Osborn, C Harpham, M Salmon, and CP Morice. 2012. “Hemispheric and large￾scale land surface air temperature variations: an extensive revision and an update to 2010.” Journal of Geophysical Research 117, D05127. Jones, Russ, Tulika Narayan, and Abigail Conrad. 2018a. “Mid-Term Performance Evaluation of the Regional Agriculture Development Program in Afghanistan: Estimating Percentage Change in Land Area under Licit Agriculture in Program Areas.” Washington, DC: Program Evaluation for Effectiveness and Learning, ME&A. ———. 2018b. “Mid-Term Performance Evaluation for the Regional Agriculture Development Program in Afghanistan: GIS Evaluation Protocol.” Washington, DC: Program Evaluation for Effectiveness and Learning, ME&A. Mansfield, David. 2018. “Still Water Runs Deep: Illicit Poppy and the Transformation of the Deserts of Southwest Afghanistan.” Kabul: Afghanistan Research and Evaluation Unit. Mehraban, Abdul Baqi, David Ward, and Sediqullah Khaliqyar. 2013. “Sustainability of Veterinary Service Delivery – Evaluation survey in Afghanistan and Tajikistan.” Ankara: Food and Agriculture Organization of the United Nations, Sub-regional Office for Eastern Europe & Central Asia. Minoia, Giulia and Adam Pain. 2017. “Understanding rural markets in Afghanistan. Working paper 58.” London: Secure Livelihoods Research Consortium. Osborn TJ, J Barichivich, I Harris, G van der Schrier, and PD Jones. 2016. “Monitoring global drought using the self-calibrating Palmer Drought Severity Index [in "State of the Climate in 2015"].” I 97, S32-S36. ——— . 2017. “Monitoring global drought using the self-calibrating Palmer Drought Severity Index [in "State of the Climate in 2016"].” Bulletin of the American Meteorological Society 98, S32-S33. ———. 2018. “Monitoring global drought using the self-calibrating Palmer Drought Severity Index [in "State of the Climate in 2017"].” Bulletin of the American Meteorological Society: In press. SIGAR (Special Inspector General for Afghanistan Reconstruction). 2018. “Regional Agricultural Development Program: Additional Evaluations and Assessments Could Improve the Performance and Sustainability of USAID/Afghanistan’s $301 Million Program.” Arlington: SIGAR. USAID. 2016a. “The Role of Women in Afghan Agriculture: A Literature Review.” Kabul: USAID/Afghanistan. ———. 2016b. “The USAID/Afghanistan Plan for Transition 2015-2018.” Kabul: USAID/Afghanistan. ———. 2017. “Agriculture Consolidated Project Appraisal Document (PAD): Intermediate Result 1.2, 63 Vibrant and Prosperous Agriculture Sector Developed.” Kabul: USAID/Afghanistan. Van der Schrier G, J. Barichivich, KR Briffa, and PD Jones. 2013. “A scPDSI-based global data set of dry and wet spells for 1901-2009.” Journal of Geophysical Research: Atmospheres. 118, 4025-4048. 64 ANNEX D. DATA COLLECTION INSTRUMENTS Focus Group Discussion Guide: Beneficiary Farmers First, we would like to learn about agriculture in this area. 1. What are your primary goals of farming? (e.g., food, income) a. What other activities do you do to earn income? 2. What are the main agricultural products that you produce? Which are the most important to your household? 3. What are you most proud of as a farmer? (e.g., knowledge, skills, yields) 4. What challenges do you face when producing crops and/or livestock? Now we would like to learn about your experiences with the RADP project, which operated in this area from [year started] to [year ended]. 5. How did you first learn about the project? (e.g., from project staff, lead farmer, business, neighbor, family member) 6. How did you first become involved with or exposed to the project? (e.g., through farmer cooperative, agricultural extension, community, seeing demonstration plot, indirectly through relatives) a. (If not specified) When did you first become involved with the project? b. (If directly involved with project) What motivated you to become involved with the project? c. (If indirectly exposed to project) To what extent were you interested in the project? Did you want to have more direct involvement with the project? 7. What kind of interactions have you had with RADP staff or implementing organizations? (e.g., in person, indirect through relatives) 8. What kind of help have you received from the project? (e.g., training, linkages with buyers, received inputs, seen demonstration plots) a. (If attended training) What topics were covered during trainings? 9. In what ways were youth exposed to or involved with the project? a. Are there ways that youth could have been better involved in the project? 10. What have you learned from the project? (e.g., skills, knowledge, technologies) a. What types of crops did you learn about from the project? b. What aspects of crop production and processing did you learn about for those crops? (e.g., which parts of the growing cycle, post-harvest management) c. How did the project encourage you to adopt new crops or agricultural technologies? 11. Did any factors constrain your ability to be involved in the project or interact with project staff? (e.g., distance, time, gender norms, security) a. (If not specified) Did security conditions affect your interactions with the project? 12. If at all, how have you changed your agricultural production as a result of engaging with the project? (e.g., adopted new technologies, changed production practices, adopted new crops, purchased new types of inputs) a. What motivated you to make these changes? b. What helped you make these changes? c. Which of these changes do you think you will continue in the future? Why? 13. What kind of changes did the project encourage that you did not make? Please explain. (e.g., technology adoption, changes to production practices) 14. If at all, how has the project changed your access to agricultural businesses or markets? (e.g., linkages with buyers, access to inputs) 15. If at all, how have you benefited from being involved with the project? (e.g., productivity, agricultural income, knowledge, empowerment, market access) 65 a. Has being involved with the project improved your yields? i. If yes, for which crops? How did involvement with the project help improve your yields? b. Has the project improved your income from agriculture? i. If yes, how did involvement with the project help improve your income from agriculture? c. (If male respondents) How have benefits from the project been different for women? How have they been different for youth? d. (If women respondents) How have benefits from the project been different for men? How have they been different for youth? e. How have benefits from the project been different for youth? How have the benefits varied for male and female youth? 16. What has been negative about being involved with the project? a. Has the project had any negative impacts for you or your community? Please explain. 17. What challenges do you face when producing crops and/or livestock that the project did not influence? 18. What other sources of support do you receive for agricultural production? (e.g., other donor or government initiatives) 19. What could future projects do to better help farmers like you improve their yields and income from agriculture? Closing questions: 20. Is there anything that we have not discussed that you would like to share about the project or being a farmer in this area? 21. Do you have any questions for us? Key Informant Interview Guide: Value Chain Actors First, we would like to learn about your business. 1. Can you give us an overview of your business? 2. How does your business work with farmers in this area? (e.g., sell inputs, purchase crops, rent equipment, provide loans, provide training) 3. What other firms do you primarily work with? What kind of relationship do you have with them? (e.g., input suppliers, processors, aggregators, distributers, financing) 4. What are the strengths of your business? 5. What are the key policy, environmental, financial or other factors that affect your ability to do business? 6. What main challenges does your business face? 7. Are there possible environmental, social, or other risks of your business activities? 8. How sustainable do you think your business model is? 9. How would you describe your firm’s potential for growth in the next few years? In 10 years? Next, we would like to learn more about the value chain or sector that your business works in. 10. How would you characterize the primary value chain/sector your business works in? (e.g., crop value chain, input supplier) 11. Who are your major competitors in the value chain/sector? a. Which are the lead firms? 12. How would you characterize competition in the value chain/sector? a. How do firms in the value chain compete? b. How do they cooperate? 13. How do national or international markets affect the value chain/sector? 66 Now, we would like to learn about your experiences with the RADP project. 14. How did you first learn about the project? 15. When did you first begin to work with the project? 16. Why did you decide to work with the project? 17. How do you work with the project? What assistance have you received from the project? (e.g., financial, management, technical) 18. Do you conduct business or partner with other businesses that the project works with? How? 19. How has working with the project affected your business? (e.g., staff capacity, efficiency) a. What kinds of changes has your business made as a result of working with the project? (e.g., product offerings, farmer engagement, operational, financial, supply chain management) 20. How have these changes affected your business? (e.g., efficiency, profitability, market linkages) a. How do you think these changes will affect your business in the future? 21. What has been negative about engaging with the project? Has the project had any negative impacts on your business? Please explain. 22. What problems does your business face that the project did not influence? 23. What could future projects do to better support agribusinesses and agricultural market development in this area? Closing questions: 24. Is there anything that we have not discussed that you would like to share about the implementation of the project? 25. Do you have any questions for us? Key Informant Interview Guide: Activity Staff First, I would like to learn about your role on the RADP project. 1. While working on the project, who did you work for? (e.g. Implementing partner (specify); USAID Mission; USAID HQ; government counterpart; other—specify) a. What was your position or title with the project? b. Did you change organizations or positions during your time on the project? Now, I would like to ask you some questions about the RADP project. 2. What problem(s) was the project trying to solve? 3. Can you briefly describe the project’s approach and activities? a. Which do you think were the most important activities to achieve results? b. How were the activities tailored to the local context? 4. How were the intervention sites and groups identified? 5. Please describe the agricultural training model implemented by the project. What are the key elements of the model? a. How did the project identify and first engage beneficiary farmers? b. How did the project support them during the project? c. What do you think the impact of this model has been? d. How could it be improved? 6. Please describe the agribusiness support model implemented by the project. What are the key elements of the model? a. How did the project identify and first engage agribusinesses? b. How did the project support them during the project? c. What do you think the impact of this model has been? d. How could it be improved? 7. How did the project specifically engage women and youth? 67 a. How did the project support them during the project? b. How could engagement of women and youth be improved? 8. How were project activities monitored and/or evaluated? 9. Can you tell me about the working relationship between the individuals and organizations working on this project? a. How did these evolve over time? 10. How did broader contextual factors affect implementation? (e.g., social, economic, security, political) a. What challenges were faced during project implementation? b. How were these mitigated? c. How could they have been better addressed during the implementation period? d. Do these challenges remain an issue today? Why? 11. What do you think the impacts of the project [to date] have been? 12. What key factors have contributed to the successes achieved by the project? 13. What has been done during the project to support the continuation of activities after the project ends? a. Who has taken responsibility for sustaining the interventions? (e.g. financial, organizational, technical responsibility) b. What are the long-term prospects of the interventions? c. What, if any, are the plans to scale-up/expand the interventions from the project? 14. Were there any project outcomes that were unexpected or unintended? Please explain. 15. What are some lessons learned from implementing this intervention that you would take forward on other projects of this nature? Closing questions: 16. Is there anything that we have not discussed that you would like to share about the implementation of the project? 17. Do you have any questions for us? Key Informant Interview Guide: Stakeholders First, I would like to learn about you and your organization’s role on the project. 1. Can you tell me about your involvement with the project? a. When were you involved with the project? 2. Who were you working for during that time? (e.g. USAID Mission; USAID HQ; government counterpart; other—specify) a. What was your position or title with the project? b. Did you change organizations or positions during your time on the project? 3. What was your organization’s role in the project? Now, I would like to ask you some questions about the RADP project. 4. What problem(s) was the project trying to solve? a. Who felt this was an issue of concern? (e.g., ministry, USAID, other stakeholders?) 5. Can you briefly describe the project’s approach and activities? a. Which do you think were the most important activities to achieve results? b. To what extent do you think the activities were appropriate for the local context? c. Was there an Afghan or other donor initiative or reform that this project was intended to support? 6. Can you tell me about the working relationship between your organization and the project? a. How did this evolve over time? 7. How did broader contextual factors affect implementation? (e.g., social, economic, security, political) 68 8. What challenges were faced during project implementation? a. How were these addressed? b. How could they have been better addressed during the implementation period? c. Do these challenges remain an issue today? Why? 9. What do you think the impacts of the project [to date] have been? 10. What key factors have contributed to the successes achieved by the project? 11. What was done during the project to support continuation of activities after the project ended? a. Who has taken responsibility for sustaining the interventions? (e.g. financial, organizational, technical responsibility) b. What are the long-term prospects of the interventions? c. What, if any, are the plans to scale-up/expand the interventions from the project? 12. Were there any project outcomes that were unexpected or unintended? Please explain. 13. What are some lessons learned from this project that you would take forward on other projects of this nature? Closing questions: 14. Is there anything that we have not discussed that you would like to share about the implementation of the project? 15. Do you have any questions for us?