PHOTO: BIBEK LUITEL FOR USAID ENDLINE EVALUATION REPORT FEED THE FUTURE KNOWLEDGE-BASED INTEGRATED SUSTAINABLE AGRICULTURE IN NEPAL (KISAN II) March 2022 ENDLINE EVALUATION REPORT FEED THE FUTURE KNOWLEDGE-BASED INTEGRATED SUSTAINABLE AGRICULTURE IN NEPAL (KISAN II) March 2022 Contract Number: AID-486-I-14-0001 Task Order: 72048619F00001 Evaluation Team David Rinck – Team Leader Gana Pati Ojha – Agriculture Expert Bibek Luitel – Business and Market Systems Expert Rita Khatiwada – Gender and Social Inclusion Expert Bandana Sinha – Field Researcher Thakur Amgai – Field Researcher Cover photo: Female farmer in Sudurpashchim sorting crops in her field. This publication was prepared independently by Social Impact, Inc. at the request of the United States Agency for International Development. i | USAID KISAN II EVALUATION REPORT USAID.GOV ABSTRACT This report is an endline performance evaluation of the Knowledge-Based Integrated Sustainable Agriculture in Nepal II (KISAN II) Activity, implemented by Winrock International and its partners under the United States Agency for International Development (USAID) Feed-the-Future (FTF) initiative. KISAN II was initially a five-year Activity but was extended by 24-months to contribute to transformational growth in fine rice and industrial maize sub-sectors, as well as to Coronavirus (COVID-19) response, extending the end date to July 2024. Social Impact, Inc. implemented the evaluation between December 2021 and March 2022 to assess the activity’s performance in the areas of: (1) Agriculture production and food/nutrition security; (2) Contribution to market systems development; (3) Potential for sustainability and scalability; (4) Enabling environment; and (5) Gender equality and social inclusion (GESI). The evaluation employed a mixed methodology, including review of KISAN II documentation and relevant literature, 85 key informant interviews (KIIs), 30 focus group discussions (FGDs) with 267 beneficiaries (200 women and 67 men), 32 site visits, and analysis of secondary quantitative data. The evaluation found that KISAN II’s private sector engagement (PSE) approaches have been effective in motivating adoption of improved agricultural practices and technologies through a mix of private sector￾led extension support through private sector extension workers, referred to as Junior Technical Assistants (JTAs), attached to agri-enterprises through a co-funded grant mechanism and information and communication technologies (ICT) learning. Likewise, many beneficiaries have adopted these practices and technologies to increase crop productivity and income. The approach also led to increased household consumption of nutritious foods and greater attention to improved diets, especially during and subsequent to pregnancies. These improvements represent the most significant ways in which the PSE models benefitted farmers and private sector actors, thereby contributing to market systems development. There is strong potential for integration of the models into Government of Nepal (GON) agriculture programming with federal and provincial government agencies already integrating them into several initiatives. KISAN II also contributed to improving the enabling environment through providing technical support to the GON and through supporting the establishment of the Joint Sector Review (JSR) Platform. However, there is significant dearth of experience within the GON with private sector￾led development models and full realization of the effectiveness of the JSR will require long-term technical support to the GON. With regard to gender equality and social inclusion (GESI), a high proportion of KISAN II beneficiary farmers were women and disadvantaged groups (DAGs) and the project supported integration of these groups, most notably through a Business Literacy Program (BLP). Nonetheless, specific barriers to participation persisted, including limited production, traditional gender roles, and limited control over household resources. Recommendations for KISAN II and USAID/Nepal include: Providing stakeholders with data required for making sound investments into their agricultural systems by adopting individualized recordkeeping systems for target crops, which can also serve to enhance Activity performance data; expanding the PSE models to include seed and input dealers to address supply chain weaknesses, especially with regard to inputs for rice production; working closer with Agricultural Knowledge Centers (AKCs) to foster linkages between the three tiers of the GON, including in the area of resource allocation; and including GESI expertise in the design phase to include GESI-tailored interventions and adopting targeted interventions that address specific barriers that inhibit the full participation of women and DAGs. USAID.GOV USAID KISAN II EVALUATION REPORT | ii TABLE OF CONTENTS Acronym List v Executive Summary i Evaluation Purpose and Research Questions i Activity Background i Evaluation Methodology and Limitations ii Conclusions ii Recommendations iv 1. Evaluation Purpose and Research Questions 1 Evaluation Purpose 1 Research Questions 1 2. Activity Background 2 Activity Description 2 Geographic Focus 2 Activity Purpose and Components 3 Theory of Change and Development Hypothesis 4 3. Evaluation Methodology and Limitations 7 Evaluation Methods 7 Document Review 7 Key Informant Interviews 7 Focus Group Discussions 8 Direct Observations 9 Geographic Sampling 9 Secondary Quantitative Analysis 9 Data Analysis 10 Gender and Social Analysis 10 Evaluation Biases and Limitations 10 4. Findings and Conclusions 12 RQ1. Agriculture Production and Food/Nutrition Security – Findings 12 RQ1. Agriculture Production and Food/Nutrition Security – Conclusions 17 RQ2. Contribution to Market Systems Development – Findings 18 RQ2. Contribution to Market Systems Development – Conclusions 24 RQ3. Potential for Scalability and Sustainability – Findings 24 RQ3. Potential for Scalability and Sustainability – Conclusions 27 RQ4. Enabling Environment – Findings 27 RQ4. Enabling Environment – Conclusions 30 iii | USAID KISAN II EVALUATION REPORT USAID.GOV RQ5. Gender Equality and Social Inclusion – Findings 30 RQ5. Gender Equality and Social Inclusion – Conclusions 33 5. Recommendations 34 Agriculture Production and Food/Nutrition Security 34 Contribution to Market Systems Development 35 Potential for Scalability and Sustainability 36 Enabling Environment 36 Gender Equality and Social Inclusion 37 Annex A: Statement of Work 39 Annex B: Evaluation Team 47 Annex C: Data Collection Tools 50 Annex D: Illustrative List of Respondents 60 Annex E: Notes on Data Interpretation 65 Annex F: Evaluation Design Matrix 66 6. Annex G: References 68 USAID.GOV USAID KISAN II EVALUATION REPORT | iv LIST OF FIGURES Figure 1: USAID Nepal Feed the Future Zone of Influence.................................................................................... 3 Figure 2: KISAN II Market Systems Development Facilitation Hypothesis......................................................... 5 Figure 3: Application of Improved Agricultural Management Practices..............................................................13 Figure 4: Actual and Target Yields (MT/HA) by Commodity ...............................................................................14 Figure 5: Actual and Target Average Sales per Household (USD) by Commodity.........................................15 Figure 6: Quantity of Nutrient-Rich Foods Set Aside for Home Consumption (MT)....................................16 Figure 7: KISAN II Private Sector Engagement Models..........................................................................................19 Figure 8: Government of Nepal - Three Functional Levels...................................................................................26 Figure 9: Gender and Vulnerability Status of Grantee Leadership ......................................................................31 Figure 10: Sample Crop Budget....................................................................................................................................34 Figure 11: Enhanced KISAN II Private Sector Engagement Models.....................................................................35 Figure 12: “Hub and Spoke” Service Delivery Model .............................................................................................36 LIST OF TABLES Table 1: KISAN II Activity Components....................................................................................................................... i Table 2: KISAN II Endline Evaluation Research Questions...................................................................................... 1 Table 3: KISAN II Activity Components...................................................................................................................... 3 Table 4: Key Informant Interviews by Stakeholder Group...................................................................................... 8 Table 5: Geographic Distribution of Qualitative Sample ......................................................................................... 9 Table 6: RQ 1: Agriculture Production and Food/Nutrition Security ................................................................12 Table 7: Increased Volumes of Selected Crops over Baseline (%)......................................................................14 Table 8: RQ 2: Contribution to Market Systems Development..........................................................................18 Table 9: RQ 3: Potential for Scalability and Sustainability......................................................................................24 Table 10: RQ 4: Enabling Environment.......................................................................................................................27 Table 11: RQ 5: Gender Equality and Social Inclusion............................................................................................30 Table 12: Change in Average Sales by Commodity Non-Adjusted and Adjusted for Inflation.....................65 v | USAID KISAN II EVALUATION REPORT USAID.GOV ACRONYM LIST ADS Agriculture Development Strategy AKC Agriculture Knowledge Center ASPT Agricultural Sector Policy Think Tank BLP Business Literacy Program CapEX Capital Expenditure CCDABC Centre for Crop Development and Agro-Biodiversity Conservation CEAPRED Center for Environmental and Agricultural Policy, Research, Extension and Development COVID-19 Coronavirus DADO District Agriculture Development Office DAG Disadvantaged group DCC District Coordination Committee DCCI District Chambers of Commerce and Industries DEC Development Experience Clearinghouse DEPROSC Development Project Service Center DFI Development finance institutions DFTQC Department of Food Technology and Quality Control ET Evaluation Team FCR Findings, Conclusions and Recommendations FGD Focus group discussion FTF Feed the Future (Initiative) GESI Gender equality and social inclusion GFSS Global Food Security Strategy GON Government of Nepal HA Hectares ICT Information and communication technology IFC International Finance Corporation IRB Institutional Review Board JNSC Joint National Steering Committee USAID.GOV USAID KISAN II EVALUATION REPORT | vi JRIP Joint Rice Improvement Program JSR Joint Sector Review JTA Junior Technical Assistant KII Key informant interview KISAN II Knowledge-Based Integrated Sustainable Agriculture in Nepal II MEL Monitoring Evaluation and Learning MOALD Ministry of Agriculture and Livestock Development MOLMAC Ministry of Land Management, Agriculture and Cooperatives MT Metric tons NPR Nepal Rupee NRB Nepal Rastra Bank OSC Overseas Consulting Ltd. PIRS Performance Indicator Reference Sheet PIU Project Implementation Units PMAMP Prime Ministers’ Agricultural Modernization Project PPP Public-private partnership PSE Private sector engagement ROI Return on investment RQ Research Question SI Social Impact, Inc. SME Small and medium-sized enterprises TA Technical assistance TC Technical Committee USAID United States Agency for International Development USD United States Dollars USG U.S. Government WASH Water, sanitation, and hygiene ZOI Zone of Influence i | USAID KISAN II EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE AND RESEARCH QUESTIONS The purpose of this endline performance evaluation was to assess the United States Agency for International Development (USAID)/Nepal’s efforts to strengthen the performance of selected agriculture market systems through the Knowledge-Based Integrated Sustainable Agriculture in Nepal II (KISAN II). The evaluation contributes to the mission’s ongoing and future Feed the Future (FTF) activities by generating evidence of good practices and learning. Findings will be shared broadly with stakeholders in Nepal that may be able to learn from and apply lessons learned. The evaluation addresses five research questions (RQs) related to KISAN II performance in the areas of: (1) Agriculture production and food/nutrition security; (2) Contribution to market systems development; (3) Potential for sustainability and scalability; (4) Enabling environment; and (5) Gender equality and social inclusion (GESI). ACTIVITY BACKGROUND USAID supports the KISAN II Activity in Nepal through the United States Government’s (USG’s) Global Food Security Strategy (GFSS), an inter-agency initiative to reduce poverty and improve food security and the successor to FTF. Winrock International is the lead contractor. Sub-contractors include the Center for Environmental and Agricultural Policy Research, Extension and Development (CEAPRED), Development Project Service Center (DEPROSC), Siddharth Inc., Digital Green and Overseas Consulting Ltd (OSC). KISAN II was initially a five-year, USD 32.7 million Activity. However, in 2020 USAID provided additional funding and a 24-month extension to contribute to transformational growth in fine rice and industrial maize sub-sectors, as well as to Coronavirus (COVID-19) response, bringing the award to United States Dollar (USD) 37.6 million, and extending the end date to July 2024. Due to the impacts of the COVID-19 Pandemic, much of KISAN II operations and outreach needed to adapt to adjust to the changed implementation scenario in 2020 and 2021. In contrast to the previous KISAN which focused on direct implementation through project staff, KISAN II focuses on market systems and private-sector actors and integration of women and marginalized groups into value chain activities. The Activity operates in FTF Zone of Influence (ZOI) in Nepal, which comprises 25 districts. The purpose of KISAN II is to increase the resilience, inclusiveness, and sustainability of income growth in the ZOI. This purpose is achieved through activities directed to specific outcomes under five components, which integrate specific cross-cutting objectives: Table 1: KISAN II Activity Components Component Description Component 1 Improve the productivity of selected agricultural market systems. Component 2 Strengthen the competitiveness, resilience, and inclusiveness of selected agricultural market systems. Component 3 Strengthen the enabling environment of selected agricultural market systems. Component 4 Increase ability of vulnerable communities to act on business opportunities within selected market systems Component 5 Apply collaboration, learning and adaptation to market systems development USAID.GOV USAID KISAN II EVALUATION REPORT | ii EVALUATION METHODOLOGY AND LIMITATIONS Social Impact (SI) undertook a final endline performance evaluation between December 2021-March 2022 deploying a six-person evaluation team (ET), supported by four home office-based management staff. The team utilized a mixed methods approach including document review, as well as qualitative data collection and analysis using key informant interviews (KIIs) and focus group discussions (FGDs), and direct observations carried out during site visits. In total, the team conducted 85 KIIs, 30 FGDs with 267 beneficiaries (200 women and 67 men), and 32 site visits. The ET conducted virtual data collection with donor, implementing partner (IP) and national-level GON officials, and in-person data collection in Baitadi, Kailali, Surkhet, Banke, and Nuwakot districts. In addition, the team undertook quantitative analysis of existing secondary data collected by the KISAN II IPs to further inform findings. USAID and SI’s Institutional Review Board reviewed and approved all data collection tools prior to use. Throughout the data collection period, the ET aggregated findings using Findings, Conclusions and Recommendations (FCR) codebook framework to generate topics for further probing questions and identify emerging findings related to each EQ. Specific data analysis methods used by the team included triangulation and trend analysis. The ET also integrated insights from a validation workshop with the IP and consultative workshop with USAID into the evaluation report. Potential biases and limitations of the methodology include response bias, selection bias, sampling limitations, and limited relevance of secondary quantitative data. CONCLUSIONS AGRICULTURE PRODUCTION AND FOOD/NUTRITION SECURITY Despite some challenges to data interpretation related to the absence of counterfactual data and an impact evaluation methodology, and despite uncertainty regarding the impact of inflation on changes in farmer income, the team concluded that KISAN II is largely on track to achieve its purpose. KISAN II has consistently met or exceeded targets between 2019 and 2021 for a range of key outcomes including productivity and aggregate sales of key commodities, application of improved agricultural practices and quantity of nutrient-rich food set aside for home consumption. In addition, the ET found: a. KISAN II motivated the adoption of improved agricultural practices and technologies through a mix of private sector-led extension support through Junior Technical Assistants (JTAs) attached to small agri-enterprises through a co-funded grant mechanism and ICT learning. Likewise, many beneficiaries have adopted these practices and technologies and their use has increased crop productivity and income. b. The Activity has increased household consumption of nutritious foods, especially vegetables, due to increases in incomes, as well as to the increased availability of nutritious foods due to expansion of production. In addition, information provided by the Activity has motivated increased consumption and greater attention to improved diets, especially during and subsequent to pregnancies. CONTRIBUTION TO MARKET SYSTEMS DEVELOPMENT The most significant ways in which KISAN II’s Private Sector Engagement (PSE) model benefitted farmers and private sector actors was through increasing access to improved agricultural practices and technologies motivating the widespread adoption of these innovations and leading to significant expansions of productivity and sales incomes. The benefits were somewhat mitigated by the limited iii | USAID KISAN II EVALUATION REPORT USAID.GOV commercial orientation and viability of targeted farmers,1 variable access to “coaching” and technical assistance (TA), and constrained access to finance, especially Capital Expenditure (CapEx) financing, and difficulty in accessing risk mitigation measures. a. Incentives: the private sector perceives expanded communication with increased numbers of upstream farmer suppliers and greater business formalization as the most significant incentives to adopting KISAN II PSE models as they have perceived higher profits and improved management as a result. Farmers perceive improvements in output, especially vegetables, as the most significant incentives for adopting the PSE models. b. Behavior of Key Actors: GON staff, especially at the federal level, recognized the effectiveness of the PSE business model, and began integrating similar public-private partnership (PPP) approaches into their agricultural development strategies. Farmer behavior change is most associated with the adoption of improved agricultural practices and technologies promoted through the model. Private sector actors have only partially adopted the model and have adapted in response to the model in several cases, such as through employment of family members in JTA-like roles, to provide potential supplier farmers with TA. POTENTIAL FOR SUSTAINABILITY AND SCALABILITY There is strong potential for integration of the PSE model into GON agriculture programming, with federal government agencies already integrating the approach into several national agricultural development initiatives including the Agriculture Development Strategy (ADS) and Prime Ministers’ Agricultural Modernization Project (PMAMP), especially the rice mill model, which aligns with GON objectives to reduce dependency on imports. Likewise, provincial government agencies have already launched several similar ad hoc initiatives aimed at strengthening the operations of commercial actors. However, scalability and sustainability are challenged by poor coordination between the three tiers of the GON and resource constraints, especially in human resources. ENABLING ENVIRONMENT KISAN II has contributed to improvements in the enabling environment for agricultural systems development through providing technical support to GON, particularly at the federal and provincial level, for policy review and formulation and practical examples of the application of private sector engagement-related approaches. The Activity also contributed to the establishment of the Joint Sector Review (JSR) Platform in support of the ADS, along with the Ministry of Agriculture and Livestock Development (MOALD) and other development partners. However, there is significant dearth of experience within the GON with private sector-led development models. Full realization of the effectiveness of the JSR will require long-term technical support to the GON and partners. GENDER EQUALITY AND SOCIAL INCLUSION A high proportion of KISAN II beneficiary farmers were women and disadvantaged groups (DAGs) because of beneficiary selection criteria that limited participation to smallholder farmers (farmers with 1 KISAN II primarily worked with smallholder farmers who often only produced at the subsistence level with an aim to help increase their production, and eventually allow farmers to be able to produce some surplus for sale. Therefore, target farmers where not intended to be highly commercially oriented. Nevertheless, our finding is that this lower level of commercial orientation is still an important factor when considering factors that mitigated KISAN II’s benefits. USAID.GOV USAID KISAN II EVALUATION REPORT | iv less than four hectares [HA] of land), and those involved in the women’s groups for the Business Literacy Program (BLP). Therefore, changes in access to improved practices and technologies, and access to finance, generally reflected changes for the overall beneficiary population. Specific barriers remained present, including limited production, traditional gender roles and limited control over household resources. However, the concrete efforts to 'increase access to market and productive resources, and improved management practices and access to finance for marginalized communities’ was confined to participation in farmers groups and BLP without targeted interventions to address their major barriers. RECOMMENDATIONS AGRICULTURE PRODUCTION AND FOOD/NUTRITION SECURITY Providing stakeholders with data required for making sound agricultural investments into their agricultural systems is a key element in agricultural market systems approaches. KISAN II and similar projects should consider adopting an individualized recordkeeping system (e.g., crop budgets) for target crops to enhance the analytical decision-making capacity of producers. In addition, when aggregated in a database, this data can provide essential investment information to agricultural enterprises on the business case for outreach strategies and technical assistance investments by agricultural enterprises along target value chains, as well as return on investment (ROI) data to GON policy makers that would inform its analysis of development strategies CONTRIBUTION TO MARKET SYSTEMS DEVELOPMENT The KISAN II Activity addresses food security objectives through the inclusion of staple crops produced by subsistence and semi-subsistence farmers for home consumption and improved nutrition in line with the goals of the USG GFSS. The KISAN II activity should consider shifting to an enhanced focus on commercial production of these crops through expanding its PSE models to include seed and input dealers that service rice, as well as vegetables and potentially other producers.2 This enhancement should also focus on addressing noted supply chain weaknesses in the seed supply and input chains for other crops. It should focus on addressing supply challenges in inputs for rice production, which impede the adoption of improved agricultural practices and technologies. In addressing these supply challenges, the activity could also encourage linkages to seed multipliers, wholesale fertilizer3 and related inputs suppliers, and wholesale suppliers of veterinary supplies. This model builds on the relative success of the agrovet model and creates a “hub and spoke” service delivery system like those employed by major commodity companies. A strength of this model is efficiency in reaching large numbers of farmers for technology transfer and TA, in addition to inputs and supplies. POTENTIAL FOR SUSTAINABILITY AND SCALABILITY A chronic and pervasive challenge to project implementation is poor coordination between the three tiers of the GON and constraints in resource flows. Given that the supply chains for the target crops are bound to cross municipal borders, especially under an enhanced model as described above, the provincial Agriculture Knowledge Centers (AKCs) are a logical nexus for connecting the federal and municipal levels of government and engaging them in PSE activities. KISAN II should consider closer collaboration with the AKCs, including creating a role for them in management of the grant fund that 2 This enhanced focus on commercial production would still enable the Activity to address outcomes related to domestic food consumption and food security, both key areas of interest for the FTF initiative. 3 As fertilizer trade is currently a public sector-controlled function, the ET understands that fostering linkages between farmers and wholesale fertilizers would likely require a different approach than fostering linkages with private sector retailers. v | USAID KISAN II EVALUATION REPORT USAID.GOV co-finances employment of the JTAs. KISAN II should also leverage the role of the AKCs in connecting the federal, provincial, and municipal governments, including engagement of the AKCs in JSR to share lessons learned and experience. KISAN II should also consider leveraging its engagement with the MOALD and ADS at the federal level through the JSR to ensure that budget allocation and revenues flow into the grant fund to facilitate continued operation and expansion of the PSE models. ENABLING ENVIRONMENT The regional transition across South Asia to PSE approaches is a long-term shift that entails building expertise and experience in a country long accustomed to state-led agricultural development approaches which faces some residual resistance. KISAN II should consider ways in which the JSR can be strengthened and better institutionalized including expanding the role of provincial and municipal stakeholders and private sector engagement in the JSR. This may include strengthening dedicated working groups for discrete value chains as is already planned in the GON’s annual JSR action plans. In addition, KISAN II should consider ways to expand engagement with development finance institutions (DFIs), such as the Asian Development Bank and the International Finance Corporation of the World Bank to engage these institutions in financing of activities. USAID/Nepal should consider a funding mechanism for long-term technical support to GON in the JSR through on-going TA on PSE approaches and related projects. There are several modalities that could be deployed to implement such a mechanism. For example, the Title VI Farmer-to-Farmer program is a useful tool to deploy volunteer consultants with specific technical expertise in selected value chains in an advisory role. Since this mechanism is already being used by KISAN II and USAID/Nepal, it could be easily expanded. USAID/Nepal could also consider a partnership for ad hoc exchange between Nepal and countries with similar bodies as planned prior to COVID-19 disruptions. GENDER EQUALITY AND SOCIAL INCLUSION KISAN II was relatively successful in reaching a large proportion of female and DAGs farmers through its beneficiary selection criteria. The Activity also strengthened its focused on female farmers through the farmer groups formed for the BLP. However, KISAN II did not implement specific activities to address inherent barriers to women and DAG’s full participation in KISAN II activities and factors that mitigate realization of outcomes. For this reason, future programming should consider more intentional GESI programming by including GESI expertise in the design phase and planning for specific GESI tailored interventions. The Activity was also less successful in reaching woman and DAG grantees (i.e., agri￾enterprises owned by women and DAGs which received grants from the Activity). KISAN II should consider adopting targeted interventions that address barriers to women and DAG’s full participation, including: ● A specific focus on connecting women and DAGs to local GON agencies which can educate them on existing provisions in access to finance ● Advocacy to GON agencies for women-friendly agricultural tools and technical expertise to design such tools drawing lessons from similar countries ● Mainstreaming gender and social norms change activities to motivate transformative change, including working with men and boys on awareness-raising activities and creating youth champions among young women and DAG farmers within farmer groups. Conduct a study with grantees, finance institutions and vulnerable groups to identify the disincentives and barriers for them to engage with these groups. Findings about the disincentives from the private sector side can inform more strategic approaches for women, minority, and vulnerable populations. 1 | USAID KISAN II EVALUATION REPORT USAID.GOV 1. EVALUATION PURPOSE AND RESEARCH QUESTIONS EVALUATION PURPOSE The purpose of this endline performance evaluation was to assess United States Agency for International Development (USAID)/Nepal efforts to strengthen the performance of selected agriculture market systems against performance indicators and targets. Specifically, this evaluation examined how the Knowledge-Based Integrated Sustainable Agriculture in Nepal II (KISAN II) activity strengthened the roles of key market actors in selected agriculture value chains to improve service delivery to farmers and how the Activity contributed to an enabling policy environment for private sector engagement (PSE) in agriculture. In addition, the evaluation examined the viability of interventions in terms of long-term sustainability and scaling up successful business models as well as how the activity contributed to increasing access to market and productive resources for marginalized communities. The findings and conclusions generated recommendations for further improvement and adaptation during remaining years. This evaluation contributes to the mission’s ongoing and future Feed the Future (FTF) activities by generating learning and evidence of good practices. Findings from the evaluation will be shared broadly with stakeholders in Nepal who may be able to learn from and apply these lessons learned (see Annex A). RESEARCH QUESTIONS This endline performance evaluation sought to answer the RQs detailed in Table 2 below. Table 2: KISAN II Endline Evaluation Research Questions AREA RESEARCH QUESTION Agriculture production and food/nutrition security 1. Is KISAN II on track to achieve its purpose with respect to: a. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth, and marginalized communities? b. The increased incomes used in the consumption of nutritious food by the household members? Contribution to market systems development 2. In what ways has KISAN II’s PSE model benefitted farmers and private sector actors? In particular: a. What do the private sector and farmers perceive to be the incentives adopting the PSE model? b. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? Potential for scalability and sustainability 3. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? Enabling environment 4. To what extent has KISAN II improved the enabling environment for agricultural systems development, specifically for private sector engagement, and institutionalizing the JSR Platform into MOALD? Gender equality and social inclusion (GESI) 5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? USAID.GOV USAID KISAN II EVALUATION REPORT | 2 2. ACTIVITY BACKGROUND Nepal was one of 12 countries chosen under the United States Government’s (USG’s) Global Food Security Strategy (GFSS). The GFSS is an inter-agency USG initiative that serves as the follow-on to the previous FTF initiative and is known as the second phase of FTF. The overall goal of GFSS is to reduce poverty and improve food security in Nepal. ACTIVITY DESCRIPTION Agriculture employs 60.4 percent of the economically active population in Nepal and contributes 25.8 percent to the gross domestic product.4 Yet Nepal is simultaneously food insecure, and the agricultural sector faces persistent challenges, including limited irrigation, increasingly extreme and erratic weather, and insufficient access to quality inputs as well as access to technical and market information, and to productive resources such as credit and insurance. Given these obstacles, there is an opportunity to support sustainable agricultural growth to support Nepal’s transition to a more food-secure country. In response to this opportunity, on July 12, 2017, USAID/Nepal, under the FTF initiative, contracted Winrock International to implement the KISAN II Activity (Contract AID-367-C-17-00001). KISAN II contributes to the Government of Nepal’s (GON) Agriculture Development Strategy (ADS). Winrock implements the Activity in collaboration with three Nepali organizations as subcontractors: Center for Environmental and Agricultural Policy, Research, Extension and Development (CEAPRED); Development Project Service Center (DEPROSC); and Siddharth Inc.; and two international subcontractors: Digital Green and Overseas Consulting Ltd. (OSC). KISAN II was initially a five-year, USD 32.7 million Activity. However, in 2021 USAID provided a USD 4.9 million and 24-month extension to contribute to COVID-19 response and recovery. With this extension, the estimated cost of the award is USD 37.6 million, and the end date of the award is July 2024. Due to the impacts of the COVID-19 Pandemic, much of KISAN II operations and outreach needed to adapt to adjust to the changed implementation scenario in 2020 and 2021. In contrast to KISAN, which focused on direct implementation through USAID implementing partners, i.e., Winrock and national firms, in coordination with GON District Agriculture Development offices, KISAN II focuses on market systems and private-sector actors, while recognizing that social and geographic contexts and barriers can create an uneven playing field for some groups. As a result, KISAN II employs a “push-pull” approach in its outreach and engagement with beneficiaries. Push strategies help to integrate women, youth, and marginalized groups into selected market systems, helping individuals build the capacity to participate in intensification, diversification, and value addition activities. Pull strategies increase demand for smallholder production, labor, and related goods and services and improve the affordability and accessibility of skills, resources, inputs, and support services needed to participate in competitive markets. KISAN II tailored its approach to empower and graduate farming households and firms into more productive, reliable, and lucrative agricultural enterprises evolving from vulnerable to developing; commercially minded; and finally, to competitive household enterprises. GEOGRAPHIC FOCUS KISAN II is implemented in the FTF Zone of Influence (ZOI) in Nepal, which comprises 25 districts, displayed as illustrated in Figure 1 below.5 The ZOI includes ten districts in Lumbini Province: Arghakhanchi, Gulmi, Kapilvastu, Palpa, Banke, Bardiya, Dang, Pyuthan, Rolpa, and East Rukum; five districts in Karnali Province: Salyan, Surkhet, Dailekh, Jajarkot, and West Rukum; and six districts in Sudurpaschim Province: Achham, Baitadi, Dadeldhura, Doti, Kailali, and Kanchanpur, containing both hill 4 “Economic Survey 2020/21.” Government of Nepal Ministry of Finance., 2022. 5 Rukum was previously one district but was split into two districts. Previous documentation referred to the Phase 1 ZOI as 20 districts and Phase 2 ZOI encompasses 24 districts. As of 2018, the Phase 1 ZOI has 21 districts and the Phase 2 ZOI has 25 districts while still covering the same geographic areas as previously noted. 3 | USAID KISAN II EVALUATION REPORT USAID.GOV and terai agro-ecological zones. Following the April 2015 earthquake in Nepal, an additional four hill districts in Bagmati Province were added to the ZOI: Kavrepalanchok, Makwanpur, Nuwakot, and Sindhupalchowk. Figure 1: USAID Nepal Feed the Future Zone of Influence ACTIVITY PURPOSE AND COMPONENTS The purpose of KISAN II is to increase the resilience, inclusiveness, and sustainability of income growth in the ZOI. This purpose is achieved through activities directed towards specific outcomes under five components, which integrate specific cross-cutting objectives: Table 3: KISAN II Activity Components Component Description Component 1 Improve the productivity of selected agricultural market systems. Component 2 Strengthen the competitiveness, resilience, and inclusiveness of selected agricultural market systems. Component 3 Strengthen the enabling environment of selected agricultural market systems. Component 4 Increase ability of vulnerable communities to act on business opportunities within selected market systems Component 5 Apply collaboration, learning and adaptation to market systems development USAID.GOV USAID KISAN II EVALUATION REPORT | 4 THEORY OF CHANGE AND DEVELOPMENT HYPOTHESIS According to a large body of research, investments in agriculture lead to reductions in poverty through a process of economic structural transformation that includes the growth of sectors other than agriculture and, within agriculture, increases in productivity and the movement of labor from lower- to higher-return activities.6 In other Asian countries, this transformation has included (1) intensification particularly of staple crops to free-up land for higher-value crops; (2) diversification into higher-value commodities; (3) diversification into higher-value non-farm and off-farm enterprises, including agro￾processing and other agricultural-related enterprises; and (4) shifts into higher-paying employment opportunities.7 In the context of Nepal, this broad theory of change is manifested in the ZOI as follows. For a large share of the population, agriculture has the potential to be a driving force for increasing incomes, and yet productivity is relatively low. Past investments in the ZOI to cost-effectively improve productivity have included: (1) strengthened market access, (2) introduction of new productivity￾enhancing technologies, (3) off-farm activities such as input supply, aggregation, and processing, and (4) coordination of the market through a structured relationship between buyers and suppliers of commodities. These investments have contributed to the intensification of rice, maize, and vegetable production, and to increased yields, quality, and scale of production. They have also led to diversification into off-season vegetables, maize for animal feed, and, to some extent, fine rice. Where the profitability of agriculture has increased, there are signs of a reduction in male out-migration and the freeing up of labor to engage in higher return activities both on and off-farm. Overall, past investments demonstrate strategies and approaches for future investments to build on and deepen to achieve increased incomes. For youth in particular, the challenge is to get them more involved in agriculture to stem out-migration. Project interventions should therefore aim to help keep youth on the farms and/or more involved in agriculture enterprises in the value chain. 6 For example, see Johnson, Bruce F., John W. Mellor “The Role of Agriculture in Economic Development” American Economic Review, vol. 51, no. 4, American Economic Association, September 4, 1961, and Johnson, B.F., P. Kilby “Agriculture and Structural Transformation, Economic Strategies in Late Developing Countries” Revue Tiers Monde, vol. 16, issue 63, 699-699, 1975. 7 Reardon, Thomas, and C. Peter Timmer. “Five Inter-Linked Transformations in the Asian Agrifood Economy: Food Security Implications.” Global Food Security 3, no. 2 (2014). 5 | USAID KISAN II EVALUATION REPORT USAID.GOV Figure 2: KISAN II Market Systems Development Facilitation Hypothesis Based on this theory of change, the development hypothesis underlying KISAN II consists of two parts: 1. If KISAN II engages in facilitating systemic changes that include: (1) greater climate-smart intensification of staple crops and diversification into higher-value commodities, (2) the strengthening of local market systems to support more competitive and resilient value chains and agricultural-related businesses, and (3) an improved enabling environment for agricultural and market systems development, then KISAN II will achieve sustainable improvements in rural (including agriculture-based) incomes. 2. If diversification of agricultural production makes available more diverse foods and increases incomes, nutrition outcomes will improve, when effectively coordinated with information, awareness, water, sanitation, and hygiene (WASH) activities, and behavior change interventions. Successful scaling of these outcomes will depend on addressing key issues of gender, youth, and social inclusion. Thus, if KISAN II increases access to markets and productive resources by women, young people, and other marginalized communities, then it will increase its impact on poverty reduction and improve nutrition outcomes. KISAN II activities are directed at the following intermediate and sub-intermediate results: INTERMEDIATE RESULTS 1. Increased yields and volumes of selected crops. 2. Increased percentage of smallholders, farmer cooperatives, and farmer associations able to meet market quality and standards requirements. 3. Increased value of new private sector investment in selected agricultural market systems. 4. Strengthened institutional and human resource capacity of the GON and private sector to support market systems development and related policy processes. USAID.GOV USAID KISAN II EVALUATION REPORT | 6 5. Platform established for collaboration and learning among KISAN II stakeholders. SUB-INTERMEDIATE RESULTS 1. Increase adoption of profitable, productivity-enhancing, and climate smart technologies. 2. Strengthen lead firms and other small and medium-sized enterprises (SMEs) to support selected market systems. 3. Enhance financial services markets and infrastructure that serve selected market systems. 4. Build capacity for GON policies, regulations, operating procedures, and guidelines to support market systems development. 5. Enhance literacy and business development skills. 7 | USAID KISAN II EVALUATION REPORT USAID.GOV 3. EVALUATION METHODOLOGY AND LIMITATIONS Social Impact (SI) was contracted by USAID to conduct a final endline performance evaluation of the KISAN II Activity (see Annex A). To implement this evaluation, SI deployed a six-person evaluation team (ET), supported by four home office-based management staff (see Annex B). Document review commenced in November 2021 and qualitative data collection took place between December 2021 and February 2022. EVALUATION METHODS To address the RQs of this endline performance evaluation, the ET utilized a mixed methods approach including document review, key informant interviews (KIIs), focus groups discussions (FGDs), and direct observations of KISAN II beneficiary farms. In addition, the team reviewed existing secondary data collected by the KISAN II implementing partners (IPs) to further inform findings derived from qualitative data collection. Due to the ongoing COVID-19 pandemic, SI implemented a hybrid approach for the qualitative data collection. KIIs with donor, IP, and national-level government stakeholders were conducted remotely, and data collection with grantees, intermediaries, beneficiaries and local GON partner staff was conducted in-person in sampled districts. Prior to start of data collection, USAID and SI’s Institutional Review board reviewed and approved all data collection instruments, all ET members engaged in data collection completed a certificate course on essential elements of ethics for engaging with human subjects, and team members were provided detailed COVID-19 risk mitigation protocols. All recordings and notes related to data collection were stored in a secure, password-protected site only accessible by the ET. DOCUMENT REVIEW Prior to commencing qualitative data collection, the ET launched a desk review of KISAN II project reporting data. This review included the KISAN II contract and subsequent modifications, the KISAN II Baseline Report, the Monitoring, Evaluation and Learning Plan, quarterly and annual progress reports, and corollary project documents such as case studies, press releases, and secondary background literature on topics related to the Activity. Findings from the document review informed probing questions included in the final data collection tools. KEY INFORMANT INTERVIEWS Following document review, the ET identified four stakeholder groups for KIIs: (1) USAID Staff, (2) IP staff, (3) GON staff, and (4) grantee firm staff and intermediaries. The KIIs consisted of in-depth facilitated discussions conducted with individuals or small functional groups of related individuals (i.e., up to four participants) using a semi-structured “evolving subject-driven” approach. This approach refers to an iterative process through which information gathered transversely across successive interviews could be aggregated and analyzed in a cohesive and consistent manner. The team purposively selected KII respondents with the selection stratified geographically to include national level respondents based in Kathmandu, and a wide sample of provincial and municipal-level respondents (see the Geographic Sampling sub-section below). The ET also selected respondents according to the likelihood of significant knowledge of KISAN II implementation and outcomes, as well as convenience of access to maximize the number of respondents reached over the course of the data collection period. Ultimately, the team implemented 85 KIIs. The distribution of KIIs across stakeholder groups is provided in Table 4 below: USAID.GOV USAID KISAN II EVALUATION REPORT | 8 Table 4: Key Informant Interviews by Stakeholder Group STAKEHOLDER GROUP KIIs (no.) USAID staff including USAID/Nepal, as well as agency staff at regional bureaus and headquarters in Washington, D.C., as deemed useful by the team. 5 IP staff including project and support staff at Winrock International, CEAPREAD, DEPROSC, Siddhartha Inc., Digital Green, and OSC. 18 GON partner staff including: a. Federal level—Ministry of Agriculture and Livestock Development, Department of Agriculture, Prime Minister Agriculture Modernization Project. b. Provincial level—Provincial Ministry of Land Management, Agriculture & Cooperative, Provincial Ministry of Agriculture, Food Technology & Land Management. c. District level—Agriculture Knowledge Center, Districts Chamber of Commerce and Industries. d. Municipality level - Rural/Municipality Chair/Mayor or Vice Chair/Mayor, Planning Officer, Technical Officers. 32 Grantee firm staff and intermediaries, including agrovets, cooperative staff, traders, rice millers, goat firms, financial intermediaries, and trainers including interviewees that did not receive direct grants from KISAN II in the sample districts. 30 TOTAL 85 Initially, the team focused on KIIs with USAID and IP staff to confirm aspects of the activity design and implementation, and identify high-level topics related to the research questions (RQs). Subsequently, the team created tailored interview guides for GON, grantees and intermediaries based to insights from the initial KIIs (see Annex C). After completion of the in-person data collection, the ET conducted follow-up KIIs with select key informants from the IP to validate and contextualize emerging findings. FOCUS GROUP DISCUSSIONS The ET implemented FGDs with a purposively selected sample of KISAN II beneficiaries (producer households) stratified to reflect geographic diversity and beneficiary social inclusion dynamics, such as gender, caste, and ethnic or religious minority status. The FGDs consisted of semi-structured and moderated discussions with small functional groups of five to ten individuals. The FGD sample was constructed using the full beneficiary list provided by the IP. As with KIIs, purposive sampling for participants constituted selection according to KISAN II activity knowledge, as well as convenience of access to facilitate inclusion of the largest number of informants possible over the course of fieldwork. Discreet groups of beneficiary farmers interviewed included women, as well as disadvantaged groups (DAGs), such as vulnerable people, youth, and marginalized communities (e.g., Dalit, disadvantaged Non￾Dalit, Terai caste groups, Muslims, and migrant returnees). To guide FGDs, the team employed a semi-structured format using a discussion guide for groups (see Annex C). The FGDs provided an opportunity to identify which sentiments were commonly held throughout the group, and where intergroup differences emerged. Ultimately, the team conducted 30 FGDs with 267 beneficiaries (200 women and 67 men). 9 | USAID KISAN II EVALUATION REPORT USAID.GOV DIRECT OBSERVATIONS The team implemented direct observations of project-related activities and operations accompanied by producers and firm staff. Visits totaled 32 producer and project-related sites. These visits helped the ET to understand context and experiences, identify unintended outcomes, assess successes and challenges in implementation, and identify lessons learned related to the RQs. These sites included farmer households and farms, including ones led by women and located in marginalized communities, as well as grantee-related sites, such as offices, worksites and workshops of project-related cooperatives, public and private sector companies, including agrovets. As with KIIs and FGDs, purposive sampling was based on likelihood of significant relation to KISAN II activities and accessibility. The results of the site visits and observations informed the team’s findings. GEOGRAPHIC SAMPLING The ET selected focus districts and municipalities for in-person data collection in consultation with the IP and based on the following criteria: ● Presence of beneficiary farmers who cultivate of each of the value chain commodities targeted by the activity. ● Presence of a variety of grantee types (cooperatives, agrovets/input supplies, traders, rice mills and goat farms) ● Representation of all ethnic/caste groups of interest amongst beneficiaries and grantees, especially DAGs ● Representation of both hill and terai regions In-person data collection at the provincial and sub-provincial level was conduct with provincial and municipal GON staff, grantees and intermediaries, and beneficiaries. Table 5 below shows the distribution of data collection activities by province (see Annex D for more details). Table 5: Geographic Distribution of Qualitative Sample Province Data Collection Event Sudur Paschim Karnali Lumbini Bagmati Provincial and municipal GON staff 8 8 5 5 Grantees/Intermediaries KII 9 9 8 4 Beneficiary FGDs 9 8 9 4 Direct observations 10 6 12 4 Note: Specific districts included Baitadi and Kailali in Sudur Paschim, Surkhet in Karnali , Banke in Lumbini, and Nuwakot in Bagmati. SECONDARY QUANTITATIVE ANALYSIS To complement qualitative data collection and analysis, the evaluation team reviewed the KISAN II monitoring and evaluation (M&E) system focused on Activity monitoring data collected throughout Wikisan 2.0, farm-level data collected via annual surveys, and statistics reported in annual reports. Upon reviewing these sources, the team determined that the quantitative component could best contribute to the evaluation by providing a review of the KISAN II M&E approach and reporting, along with presenting descriptive statistics related to RQ 1 and RQ 5 for context. USAID.GOV USAID KISAN II EVALUATION REPORT | 10 DATA ANALYSIS Following the launch of initial document review, parallel and sequential data analysis continued throughout the data collection period to identify emerging trends, aggregate findings around common themes, and generate topics for further probing questions. In addition, the team utilized a findings, conclusions, and recommendations (FCR) codebook framework to identify themes and sub-themes from qualitative data as relevant for each RQ. Specific data analysis methods used by the team included: ● Triangulation. Triangulation enabled the ET to cross-verify and cross-validate findings that emerged from distinct data sources and identify correlations between findings related to the RQs. Methodological triangulation also strengthened potential linkages and data accuracy in cases where results obtained through one method were less conclusive than another method. ● Trend Analysis. Trend analysis enabled the ET to further examine KISAN II progress over time to identify anticipated outcome convergence (or divergence) over the life of the project and explore how specific exogenous and endogenous events contribute to emerging outcomes. Analysis accelerated following data collection and included validation of initial findings with the IP as well as follow-up KIIs with key respondents to confirm conclusions. The team presented preliminary conclusions at a consultative presentation, with USAID/Nepal in February 2022. This evaluation report incorporates insights from the follow-up KIIs, validation workshop, and consultative presentation. GENDER AND SOCIAL ANALYSIS As of May 2021, KISAN II reached 199,961 farming households of which 74 percent are women and 59 percent are from vulnerable/disadvantaged groups. To assess GESI integration, the ET collected data through document review on gender, equity, and social inclusion (GESI) – through GESI focused sections in KIIs and FGDs guides. In addition, the ET followed an inclusive approach to ensure the perceptions of women and marginalized groups were incorporated throughout the evaluation process. Due to prevailing social norms, cultural barriers, and lack of confidence, women often do not express their views in front of male peers. Therefore, FGDs included both women-only groups and mixed (male and female) groups among women and DAG beneficiaries. Similarly, the team carried out KIIs with women and DAG farmers. The GESI expert led the majority of KIIs and FGDs with women and DAGs and trained other ET members on gender and social-inclusion sensitive data collection. EVALUATION BIASES AND LIMITATIONS The evaluation methodology described above had several potential biases and limitations that have implications for the types of findings and conclusions that can be drawn from this endline performance evaluation. These biases and limitations, and the steps taken to mitigate them, include: ● Response bias: Probing questions regarding personal issues and development outcomes may result in positive response bias i.e., the tendency of respondents to subjectively focus on positive outcomes. The ET mitigated this bias by probing for both successes and challenges to develop the most holistic picture possible of KISAN II achievements and challenges relative to the RQs. In addition, the team triangulated responses against documentary data, including contracts, financial analyses, and other documents produced by the IPs. ● Selection bias: Selection bias is an inherent risk when implementers help to facilitate contact with members of some stakeholder groups. The team worked closely with USAID and KISAN II IP staff to organize KIIs and FGDs. However there remains a risk that these staff selected the most active, responsive, or engaged individuals and groups. To mitigate this potential bias, the ET obtained a universal list of stakeholders and identified individuals and groups from this list to contact. 11 | USAID KISAN II EVALUATION REPORT USAID.GOV ● Sampling limitations: Due to resource limitations on time and personnel, the team strategically selected a representative sample of sites for visits. While the team worked closely with KISAN II staff to identify a reasonably representative sample set of informants from each stakeholder group, the conclusions cannot necessarily be generalized to all KISAN II sites. The design was still adequate to meet the purpose of the evaluation as the ET was able to collect in-depth experiences and perceptions of a wide range of stakeholders. To mitigate potential sampling limitations, the team strengthened the accuracy of conclusions through selecting a diverse sample and triangulating between data sources and stakeholders to the degree possible. ● Relevance of the secondary quantitative data to the RQs: The evaluation team coordinated with Winrock International and USAID on secondary project data and undertook a thorough review to assess what data was available and relevance of available data to the RQs. After this detailed review, the evaluation team identified a couple of limitations regarding usability of the quantitative data to address the RQs. First, the available data is pertinent to RQ1 and aspects of RQ5 that relate to production and nutrition outcomes only and cannot meaningfully provide insight on the remaining RQs. Secondly, the data does not enable an assessment of attribution of the KISAN II interventions to outcomes or impacts associated with RQ1. As a result, the quantitative component of the evaluation is limited to contextualizing and substantiating qualitative findings related to RQ1 and aspects of RQ 5. USAID.GOV USAID KISAN II EVALUATION REPORT | 12 4. FINDINGS AND CONCLUSIONS RQ1. AGRICULTURE PRODUCTION AND FOOD/NUTRITION SECURITY – FINDINGS This section presents the findings of the ET related to RQ 1: Table 6: RQ 1: Agriculture Production and Food/Nutrition Security AREA RESEARCH QUESTION Agriculture production and food/nutrition security 1. Is KISAN II on track to achieve its purpose with respect to: a. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth and marginalized communities? b. The increased incomes used in the consumption of nutritious food by the household members? The purpose of the KISAN II Activity is to “increase the resilience, inclusiveness, and sustainability of income growth in the ZOI”. Improvements in agriculture production and food/nutrition security were directly addressed through activities implemented under Component 1: Improve the productivity of selected agricultural market systems (see Activity Description). INCREASING PRODUCTIVITY AND INCOME THROUGH IMPROVED ACCESSIBILITY OF PRACTICES AND TECHNOLOGIES KISAN II supported adoption of new technologies and practices through Junior Technical Assistants (JTAs), i.e., private sector extension workers. JTAs supported farmers in production planning; agronomic and husbandry skills; post-harvest techniques; methods for conducting demonstrations and field days; scouting the fields during key stages of crop growth; and marketing their output.8 Notably, a significant feature of KISAN II implementation strategy is a grant mechanism designed to co-fund employment of technical assistance providers, JTAs. Grantees that employ these JTAs are agricultural entities embedded in selected target value chains, including SMEs, such as rice millers, vegetable traders and agrovets (agriculture and veterinary input suppliers), as well as cooperatives. The role of the JTAs was to provide TA to target households, including vulnerable people, women, youth, and marginalized communities, with the objective of upgrading production quality and quantity hence integrating them into grantee supply chains as suppliers. In the case of agrovet grantees, JTAs extended sales to new households thereby increasing revenues.9 To date, KISAN II has provided grants for co-financing JTAs to a total of 173 grantees. In addition to technical assistance (TA) provided by the JTAs, the KISAN II Year 4 Annual Report reported that by 2021, 62.9 percent of targeted farming households had access to information on improved technologies and practices through information communications technology (ICT) channels, especially smartphones, surpassing the Activity target of 37 percent.10 Moreover, the 2021 Annual Survey reported that 46 percent of 177,392 households adopted one or more technologies after watching a video.11 In several KIIs, farmers noted that the videos were an effective way for them to 8 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 5 Annual Work Plan - July 16, 2021 to July 15, 2022” Winrock International for USAID. 9 Social Impact KII, implementing partner staff. 10 42% of the farmers supported by KISAN II had access to smart phones in Year 4. 11 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID and “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021. Videos were not initially included in KISAN II workplan, but were a part of the remote COVID-19 activity pivots. 13 | USAID KISAN II EVALUATION REPORT USAID.GOV understand agricultural technologies because they could watch them during their free time, as opposed to attending training, which could be time consuming (though IPs also noted the cost of producing videos is high).12 On the other hand, both beneficiaries and IPs noted that timely access to some inputs, especially hybrid seeds and fertilizer, remains a constraining factor on application of the technologies learned. As beneficiaries in one FGD noted: “We saw that the hybrid seeds work, but when we go to the market, we don’t find them at planting season.” In addition, in KIIs with IP staff, one respondent stated: “The supply chain of rice seeds is very weak and unreliable. In the first phase we demonstrated the seeds and there was great demand, but there was not enough.” Staff also noted, “accessibility to improved quality seeds and accessibility to fertilizer are still major challenges even in the fourth year. We are dependent on imports for fertilizer, it is a national issue, not only for KISAN II ZOI.”13 INCREASING PRODUCTIVITY THROUGH IMPROVED USE OF AGRICULTURAL PRACTICES AND TECHNOLOGIES According to data derived from annual farm surveys between 2019-2021, the number of beneficiaries in the ZOI applying improved management practices and technologies rose from just over 50,000 to about 175,000. Likewise, the number of hectares (HA) under improved management practices and technologies rose from under 20,000 HA to just under 60,000 HA during the same period. Although these figures fall slightly short of KISAN II annual targets, this data suggests that use of these practices has significantly increased in the ZOI. Figure 3: Application of Improved Agricultural Management Practices Source: KISAN II Annual Reports. Note: Note: Black lines indicate Activity target by fiscal year. The 2018 value represents the baseline; therefore, no target was set in that year. Grey lines indicate Life of Project target defined in the November 2020 KISAN II Monitoring Evaluation and Learning (MEL) plan. A desired outcome from increased application of improved management practices and technologies is an increase in agricultural productivity. The same annual farm surveys report that the yields (in metric ton [MT]/HA) have generally increased from baseline to levels consistent with Activity targets for most target commodities except for 2019.14 12 Social Impact KIIs, beneficiaries and implementing partner staff. 13 Social Impact FGDs, beneficiaries, and KIIs, implementing partner staff. 14 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID and “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021. USAID.GOV USAID KISAN II EVALUATION REPORT | 14 Figure 4: Actual and Target Yields (MT/HA) by Commodity Source: KISAN II Annual Reports. Note: Black lines indicate Activity target by fiscal year. The 2018 value represents the baseline, therefore no target was set in that year. Data on yield for the goat value chain is only available for 2020 and 2021 and has, therefore, been omitted. Data from annual farm surveys shows increases in volumes of selected crops produced by KISAN II beneficiaries have exceeded targets for cereals, lentils and vegetables by significant amounts, as illustrated in Table 7 below.15 The Year 4 Annual Report lists contributing factors as early and uniform rainfall distribution, use of hybrid varieties, and technical support from partners.16 In FGDs and KIIs, respondents confirmed similar expansions of output, with the majority stating production increased 1.5 to 2 times. These beneficiaries described increased access to improved agricultural management practices and technologies as an important component to the growth of crops volumes, but also cited expanded areas planted due to enhanced commercial engagement. Of note, numerous beneficiaries cited the use of hybrid seeds as the most significant factor i.e., “Only the hybrid seeds have increased the production. Use of local seeds decreases production”.17 Several grantee staff confirmed this emphasis on hybrid seed varieties. For example, “Productivity from hybrids was good. The use of organic fertilizers increases the fertility of the soil. So, the product has also increased. When the soil was treated from the very first stage, then it worked well. It seemed a bit expensive, but the produce was great.”18 Table 7: Increased Volumes of Selected Crops over Baseline (%) Increased volumes of selected crops over baseline (%) Target Result Cereals 30% 52% (Rice) 129% (Maize) Lentil 50% 112% Vegetables 90% 198% 15 Ibid. 16 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID. 17 Social Impact FGDs and KIIs, beneficiaries. 18 Social Impact KII, grantee staff. 15 | USAID KISAN II EVALUATION REPORT USAID.GOV Source: KISAN II annual farms surveys and annual reports 2021. INCREASING INCOME THROUGH IMPROVED USE OF AGRICULTURAL PRACTICES AND TECHNOLOGIES Data presented in KISAN II Annual Reports demonstrates that average annual sales income growth of beneficiary farmers exceeded targets in all five target value chains in 2020 and 2021, despite the effects of COVID-19, as illustrated in Figure 5 below. Between 2018 to 2021, reported average sales income from goats per household grew from USD 141 to USD 411 over baseline, while sales of lentils went from USD 22 to USD 108, sales of maize from USD 13 to USD 100, rice sales increased from USD 111 to USD 332 and vegetable sales from USD 202 to USD 861. In KIIs, IP staff commented on the very significant expansion of sales income from vegetable sales, noting that “We have made tremendous improvements in vegetable production. Area coverage, farmers participation and increase in production and productivity is significantly increased, which creates employment as well.”19 In total, annual sales across all targeted commodities increased from USD 28.26 million at baseline to approximately 157.96 million in 2021. The Annual Farm Survey for 2021 reports that 63 percent of respondents reported increased income in 2021 compared to the previous year.20 Figure 5: Actual and Target Average Sales per Household (USD) by Commodity Source: KISAN II Annual Reports. Note: Black lines indicate Activity target by fiscal year. The 2018 value represents the baseline; therefore, no target was set in that year. In FGDs and KIIs, beneficiaries confirmed similar increases in sales income from their farming systems because of participation in KISAN II activities, especially training. In one FGD, a beneficiary stated, “Last year, I had one [greenhouse] tunnel and NPR 40-50 thousand in profits. Now I have added 2 more tunnels and more than doubled income.”21 Another farmer described sales increasing from NPR 20-30 thousand to NPR 100-150 thousand, while one in FGD, respondents stated that application of new processes led to income increasing two to four-fold.22 Finally, highlighting the potential for expansion of sales income from vegetable production, an additional FGD respondent said, "I used to work as a migrant worker in India. Once when I came back to the village, I saw my brother doing vegetable farming. I was inspired by him, and I also decided to stay back and do vegetable farming. I now plant only vegetable in my entire field. I think I make NPR 500,000 to NPR 600,000 per year after deducting the 19 Social Impact KII, implementing partner staff. 20 Respondents are asked to report on income from all income sources, not just from KISAN II target commodities. Source: “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021. 21 Social Impact FGD, beneficiaries. 22 Ibid. USAID.GOV USAID KISAN II EVALUATION REPORT | 16 input cost”.23 With regard to rice sales, one group of FGD respondents reported their income increasing five-fold. However, some FGD respondents also described challenges such as vegetable wastage during the COVID-19 lock-down, especially of tomatoes. Respondents also described marketing as a challenge because “the agreement between the farmers and the vegetable market is not formal, and it does not match with the desirable market price.” At one site, farmers stated that they had “not got the money for last year’s vegetables because the market sells the collected vegetables elsewhere at a lower rate.”24 USING INCREASED INCOMES FOR INCREASED CONSUMPTION OF NUTRITIOUS FOOD The KISAN II Activity hypothesizes that an increase in income, as well as an increase in production of nutritious foods will result in increases in home consumption of nutritious food (see Activity Description). Particularly, increased vegetable production can supplement traditional diets with greater micronutrient consumption. As noted above, beneficiaries reported significantly increased vegetable production over the course of participation in the Activity, especially during the 2020 season. As illustrated in Figure 6 below, the aggregate quantity of nutrient-rich food set aside for home consumption increased notably from 2019 to 2021, exceeding targets in both 2019 and 2020, and falling only slightly short in 2021. Figure 6: Quantity of Nutrient-Rich Foods Set Aside for Home Consumption (MT) Source: KISAN II Annual Reports. Note: Black lines indicate Activity target by fiscal year. The 2018 value represents the baseline; therefore, no target was set in that year. Grey lines indicate Life of Project target defined in the November 2020 KISAN II MEL plan. According to the Annual Farm Survey report for 2021, average annual consumption of vegetables increased significantly from 41 kg in 2018 to 230 kg in 2020, while average annual consumption of lentils almost doubled from 44 kg to 86 kg per household during the same period. Specific vegetables cited as being set aside for home consumption include bottle gourd, spinach, okra, cabbage, cauliflower, pumpkin, and carrot. In addition, the 2021 farm survey report states that about 77 percent of beneficiary household income is invested in foods that include grains, fruits, vegetables, meat, eggs, and dairy products.25 Likewise, in FGDs numerous beneficiaries described using increased incomes, as well as increased production, to increase consumption of nutritious foods, especially vegetables. Further, these 23 Ibid. 24 Ibid. 25 “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021. 17 | USAID KISAN II EVALUATION REPORT USAID.GOV respondents also reported that increased income enabled them to buy more food overall.26 One respondent stated, "These days we cook enough vegetables for everyone in the house so that the person eating last is also not deprived of it”, while another reported “we have been eating what we produce in our farms and buy what we don’t get from farms. So, there are no cases of malnutrition in the community so far. Before, we used to consume very little vegetables and all age of people including children would suffer a lot of health issues”.27 Interestingly, in FGDs several beneficiaries specifically cited topics covered in Business Literacy Program (BLP) training as motivating them to increase consumption of nutritious food, also citing media messages as strengthening their knowledge and motivation as well. In addition, many FGD respondents reported that information provided by KISAN II had motivated them to specifically focus on improving consumption of nutritious foods during pregnancy. For example, one female FGD respondent stated, “During pregnancy, we did not care about nutritious food. We even went hungry sometimes. But now we not only eat enough and on time, but we also make sure to eat meat, fish and eggs regularly,” while another stated "I am pregnant with my second child now. My husband ensures that we have at least one [serving of] meat, fish or eggs in our diet every day. That was not the case in my first pregnancy."28 LIMITATIONS OF KISAN II ACTIVITY DATA FOR IMPACT ASSESSMENT AND PROGRAMMATIC DECISION-MAKING In considering the degree to which KISAN II is “on track” to meet its purpose, it is important to note several factors that limit the extent to which the quantitative Activity data can measure the Activity’s results. Over the course of implementation, KISAN II collected a wealth of quantitative data through annual farm surveys and the Wikisan 2.0 data platform - which together provide a thorough and extensive system of tracking Activity indicators. However, there are some limitations related to the M&E plan which limit the extent to which the data can be used to determine the impact of KISAN II on project beneficiaries. Moreover, the data collected under the current M&E plan does not collect the type of information that might guide farmer, grantee and GON decisions-making on investment, which is key for a market systems approach (see Recommendations - Agriculture Production and Food/Nutrition Security). The most salient limitation is that the lack of an impact evaluation design means that the data cannot attribute changes in outcomes directly to the effects of the Activity. The M&E approach focuses on comparing before-and-after indicators of a set of outcomes relevant to the specific aspects of the activity, such as value of farm and firm level sales of commodities, inputs and services targeted by the Activity etc. However, improvements in outcomes over time are likely to depend on a combination of many factors in addition to the KISAN II interventions themselves, including market conditions, participation in or exposure to other agricultural interventions, and climatic factors. Without implementing an impact evaluation methodology that identifies and measures outcomes for a counterfactual comparison group, it is impossible to definitively measure the Activity impact on many of the key outcomes. In addition, issues with comparability of the farm survey baseline and the effects of inflation also limit the extent to which the data can measure impacts on beneficiaries (see Annex E). RQ1. AGRICULTURE PRODUCTION AND FOOD/NUTRITION SECURITY – CONCLUSIONS Despite some challenges to data interpretation related to the absence of counterfactual data and an impact evaluation methodology, and despite uncertainty regarding the impact of inflation on changes in farmer income, the team concluded that KISAN II is largely on track to achieve its purpose. KISAN II 26 Social Impact FGD, beneficiaries. 27 Ibid. 28 Ibid. USAID.GOV USAID KISAN II EVALUATION REPORT | 18 has consistently met or exceeded targets between 2019 and 2021 for a range of key outcomes including productivity and aggregate sales of key commodities, application of improved agricultural practices and quantity of nutrient-rich food set aside for home consumption. In addition, the ET found: a. KISAN II motivated the adoption of improved agricultural practices and technologies through a mix of private sector-led extension support through JTAs attached to small agri-enterprises through a co-funded grant mechanism and ICT learning. Likewise, many beneficiaries have adopted these practices and technologies and their use has increased crop productivity and income. b. The Activity has increased household consumption of nutritious foods, especially vegetables, due to increases in incomes, as well as to the increased availability of nutritious foods due to expansion of production. In addition, information provided by the Activity has motivated increased consumption and greater attention to improved diets, especially during pregnancies. RQ2. CONTRIBUTION TO MARKET SYSTEMS DEVELOPMENT – FINDINGS This section presents the findings of the ET related to RQ 2: Table 8: RQ 2: Contribution to Market Systems Development AREA RESEARCH QUESTION Contribution to market systems development 1. In what ways has KISAN II’s PSE model benefitted farmers and private sector actors? In particular: a. What do the private sector and farmers perceive to be the incentives adopting the PSE model? b. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? KISAN II addresses market systems development under Component 2: Strengthen the competitiveness, resilience, and inclusiveness of selected agricultural market systems. The Activity’s PSE models focused on fostering market linkages through embedded TA provided by JTAs, which were co-funded through the Activity’s grant fund. These models aimed to increase the commercial orientation of farmer beneficiaries through motivating a shift from home consumption￾oriented production to more market orientation through investments in their farming systems and expansion of private sector activity. In some cases, these JTAs fostered upstream linkages, as in the case of the agrovet model that aimed to increase access and sales to goat farmers of veterinary supplies required for improved animal health. Likewise, KISAN II engaged finance institutions and existing agricultural finance schemes to provide financing for inputs for farmer production and private sector. Meanwhile, other PSE models fostered linkages with downstream wholesale markets. These linkages included JTA support to rice millers and traders (though in some cases, rice mills also provided inputs such as seeds and fertilizers, as well as irrigation equipment, required for production of premium varieties). Cooperatives represent a unique case in that cooperative employed JTAs provided TA on production issues while at the same time the cooperatives themselves also facilitated linkages with downstream wholesale markets. 19 | USAID KISAN II EVALUATION REPORT USAID.GOV Figure 7: KISAN II Private Sector Engagement Models Tailoring implementation strategy to specific beneficiary contexts, the IP developed three unique approaches centered on differentiated TA packages to address three farmer typologies: (1) subsistence farmers; (2) semi-commercial farmers; and (3) commercial farmers. For example, in the case of subsistence farmers, the Activity facilitates formation of small groups for cooperatives. For semi￾commercial farmers, KISAN II promotes and facilitates branchless banking services, such as digital payment services and insurance, and cooperative services. Finally, for commercial farmers, the Activity facilitates credit directly through commercial banks leveraging existing finance products.29 Donor staff described these models as useful for three stakeholder groups; farmers, the private sector, and government, which are “all equally winners.”30 In KIIs, the IP described exposure to the PSE models as motivating many farmers to shift towards market-led production as they gained experience with market systems. In addition, training provided through these models also led some farmers to adopt climate smart technology as they became aware of new technology for increasing yields.31 Supporting these statements, documentation provided by IP staff reported that household sales from target value chain commodities increased significantly from the 2018 baseline to year 4 (2021) (see Agriculture Production and Food/Nutrition Security – Conclusions). Meanwhile, documentation reports private sectors business increasing 15-25 percent annually through participation in the PSE models via JTA engagement.32 PERCEPTIONS OF PRIVATE SECTOR ON INCENTIVES FOR ADOPTING PSE MODELS Respondents cited business expansion and formalization, technical capacity and outreach, and expanded contact with producers as the drivers for adopting the PSE models. In KIIs, most grantee private sector respondents cited expansion of contacts with producers to increase their business incentives as their principal incentive for adopting KISAN II PSE models. For example, one respondent stated that through the engagement of JTAs, he “covered more farmers and got more exposure for his business.” Another grantee described utilizing the PSE model to advertise and inform more farmers about his services, “It was mainly about the means to connect with the farmers and present and inform the farmers systematically,” while another said, “We were connected to only 200-250 farmers. Now, we are connected to more than 2,400 farmers.”33 Other grantees described incentives of business expansion and formalization, for example “My business has increased due to interaction and connection with more farmers”. Echoing this sentiment, a trader stated, “The projects come and go; and the benefits of the activities stop after the end of the project no matter how successful the project was. Since the project is implemented through the agrovets, even if the level of success is just average on the short term, it will continue for the long-term.” One grantee related the following story related to business formalization: “I was working abroad, in India, before this project. My father was running this place. After the project, I decided to come over to help my father with management. And probably I will stay back to run this business because the business has increased now, and it’s more organized. Our profit has doubled, and we have expanded our business serving more 29 Social Impact KIIs, implementing partner staff. 30 Social Impact KIIs, donor staff. 31 Social Impact KIIs, implementing partner staff. 32 “Knowledge-based Integrated Sustainable Agriculture in Nepal (KISAN) II Project” PPT by Winrock international, 2021. 33 Social Impact KIIs, grantees. USAID.GOV USAID KISAN II EVALUATION REPORT | 20 farmers and increased input sales.” Similarly, traders, millers and cooperatives expressed that their business, technical capacity, and outreach have been expanded leading to higher profit and improved management. For example, one rice miller stated “We were provided with training on bookkeeping, accounting, and were provided with accounting software, computer, and salary for JTAs. This has helped us in doing business in a better way.” Finally, one rice miller grantee expressed similar sentiments, but also placed emphasis on investment, “Our business has increased due to interaction and connection with more farmers. But it largely depends on investment. According to the miller, investments in installation of modern machines resulted in 30-40 percent business growth.34 PERCEPTIONS OF FARMERS ON INCENTIVES FOR ADOPTING PSE MODELS As discussed in RQ1, KISAN II has been successful in motivating adoption of the improved agricultural practices and technologies it promoted and adopting farmers reported significant expansions of productivity and sales income because of these innovations (see Agriculture Production and Food/Nutrition Security – Findings). These expansions in productivity and sales constituted the most common incentive cited by beneficiary farmers for adopting the PSE model, especially changes in income.35 In addition to potential income gains, in FGDs and KIIs beneficiary farmers frequently cited sales facilitation as another key incentive for adopting PSE models, particularly increased ease of input procurement and marketing. For example, in one FGD with farmer members of a cooperative that sold member output to a rice mill, a respondent stated, "The Rice Mill at least came to our villages and collected rice and we didn't have to worry about the market". Another respondent said, “It is a lot easier that (grantee traders) come to us and collect our produce easily and there is not much stress about having to manage our products and worry about perishing vegetables along with competition in the market. It’s not that everything drastically changed but we have been able to look after our household with the income”, while another declared, “Vegetable farming is profitable. The cooperative is there to provide us loans, and sometimes it gives us seeds and other inputs".36 However, in the case of farmers linked to traders and agrovets, several respondents described marketing support as limited and inconsistent, especially related to forward prices. For example, one respondent stated, “The trader agreed that he would buy all our products during the group formation, but later he bought vegetables from other places where he got a cheaper price [at another nearby wholesale market].” Another said, “We had to face huge losses due to over production and keep our products on the farm until the last moment hoping that the trader would buy all our tomatoes as per the agreement."37 GOVERNMENT BEHAVIOR CHANGES RESULTING FROM KISAN II BUSINESS MODELS In documentary and background research, as well as in KIIs with GON officials, it became apparent that systemic change in the implementation of agricultural development activities related to the PSE business models was stymied by poor coordination between various tiers of the GON. This was especially evident in policy coordination between the federal and other levels i.e., provincial and municipal (see Potential for Scalability and Sustainability – Findings). As a result, behavior change at the provincial and municipal levels was minimal and ad hoc. In KIIs, although respondents recognized that KISAN II provided training on PSE models to GON staff at all three levels, this limited change was clearly attributed to poor coordination. As summed up in an interview with a provincial-level GON staff, “We participated only in one or two joint monitoring visits. Other than that, we are not much aware of what 34 Ibid. 35 Note that virtually none of the beneficiaries interviewed were aware of the term “private sector engagement model” and did not speak in terms of “adopting” these, but instead referred to the impact of activities related to the model they engaged. 36 Social Impact FGDs, beneficiaries. 37 Ibid. 21 | USAID KISAN II EVALUATION REPORT USAID.GOV KISAN-II is doing. No coordination and lack of clarity among the three tiers of the government is the main challenge.”38 However, at the federal level, and in isolated cases at other levels, GON staff recognized advantages to public-private partnership (PPP) models as represented by KISAN II PSE business models, though enthusiasm was inevitably tempered by limited ability to coordinate implementation with other GON levels. For example, one federal level GON official stated the following noting appreciation for PPP models while citing ongoing challenges of coordination and a need for private sector actors to conduct and share their own M&E: “PPPs models are good. The private sector is better at providing seeds, for example. But there is a need for coordination between municipalities as well. The private sector should do monitoring and evaluation on a regular basis because the government can watch.” Likewise, another GON official working at the provincial level who had been exposed to specific project approaches stated, “The rice mill model is working and has been adapted. Specifically, in the rice zone, and government has already initiated some programs for private sector engagement with learnings from KISAN II.” In addition, other officials stated that the GON is working toward a policy to integrate PSE models despite the coordination challenges. For instance, respondents noted how the model Agribusiness Promotion Act39 is being promulgated at the provincial and sub-provincial levels in some cases and is in the process of being finalized in others. Reflecting on how this connects to trends in how GON operates, one GON interviewee stated, “We have entered the decentralized model with three tiers of government... In this way, there have been new efforts in regulations and guidelines to robustly promote the PSE model. Some of the local levels have also created their own model for private sector engagement. So, the KISAN II model and learning are integrated.” However, as expressed by one district-level GON staff interviewed, while the government is gradually adopting the PSE models, progress is impeded by the shortage of extension workers, “Earlier, there used to be around 40 technical workers in the district agriculture office and after the federalism, particularly now, only two of us are in our office. Together with the accountant and drivers, there are seven staff total in our office and providing extension services as per the need is beyond our capacity.”40 FARMER BEHAVIOR CHANGES RESULTING FROM KISAN II BUSINESS MODEL In FGDs and KIIs, interview responses suggested that sourcing of inputs required for sustainable adoption of these upgrades to farming systems were tempered by farmer expectations of free inputs, which stem from previous experiences with projects and government strategies that distributed free inputs. As noted by IP staff, “There is also a kind of reluctance to accept new things from the farmer's side. Farmers expect everything free of cost”.41 As a result, in FGDs, beneficiaries often expressed disappointment with the KISAN II market-led approach that generally required farmers to purchase inputs (though in some limited cases, grantee SMEs provided required varieties of seeds and fertilizers for free, especially for demonstration purposes. In addition, the sporadic availability of hybrid seeds and associated fertilizers represents a disincentive to farmers habitually integrating these inputs into their farming systems. Finally, market competition from Indian products represents a further disincentive to farmer investments due to greater efficiency of Indian production resulting in lower prices for imports, especially in border areas.42 38 Social Impact KIIs, GON staff. 39 The model Agribusiness Promotion Act was issued by the GON in 2017 to promote and support professionalization of agriculture by strengthening the relationship between farmers and agribusiness. 40 Social Impact KIIs, GON staff. 41 Social Impact KIIs, implementing partner staff and FGS beneficiaries. 42 Ibid. USAID.GOV USAID KISAN II EVALUATION REPORT | 22 Notwithstanding these issues, adoption of improved agricultural practices and technologies, as well as expanded consumption of nutritious foods, constitute the principal farmer behavior changes resulting from integration into KISAN II business models (see Agriculture Production and Food/Nutrition Security – Findings). As noted, this is especially the case with vegetable production. One FGD respondent stated, “after the training, farmers got inspired in vegetable farming. The training inspired people who did not plant vegetable before that to plant cauliflower, cucumber. The farmers were connected to [name of agrovet]. Farmers also bought the seeds and other inputs they required from the agrovet.”43 PRIVATE SECTOR BEHAVIOR CHANGES RESULTING FROM KISAN II BUSINESS MODELS In KIIs, numerous grantees expressed positive reactions to their experience with JTA engagement, and many indicated they used this as motivation for further investments into forging upstream linkages to supplier producers. For example, one trader grantee stated, “Farmers got connected with us. I gave seeds/inputs in credit, or I gave them money to buy seeds and inputs and deducted the amount they owed me when they came to sell. Sometimes if they lose their crops and/or struggle to pay for other reasons small credits are forgiven.” In contrast, in numerous cases, these grantees indicated only partial willingness to continue JTA employment following the end of the co-funding grant. For example, one grantee stated, “We got two JTA staffs. After the end of the project, we called a meeting to inform the farmers about the same and committed to continue one JTA service.” 44 Notably however, KIIs and documentary research revealed distinctions in investment willingness between models and value chains. This was most notable regarding cooperative and agrovet employment of JTAs, the majority of which indicated willingness to continue or even expand JTA engagement. As explained by one IP staff: “Cooperatives tend to retain more staff because of their structure. They utilize their staff time for agriculture as well as saving and credit activities if they are Savings and Credit Cooperative Organisations. Agrovets also tend to retain more staff, because agrovets get direct benefit when farmers buy more input from the agrovet, their sales are doubled, tripled. Their sales are directly correlated.” - KII, IP staff. In contrast, in KIIs most rice millers appeared to be least likely to continue investments into JTA employment, citing seasonality of rice as the main reason i.e., because rice purchases only occur during a few specific months each year, they don’t see the rationale for full time JTA employment. Finally, several grantees across all value chains expressed their satisfaction with the model, but indicated that following the end of grant co-financing, they intended to employ family members as JTAs to keep incomes within the household.45 MITIGATING FACTORS TO MARKET SYSTEMS DEVELOPMENT CONTRIBUTION Over the course of research, the ET also noted several pervasive issues that mitigate the impact of KISAN II activities in contributing to market systems development. The main issues include limited commercial orientation and viability of crops; variable access to coaching and TA; constrained access to finance; and risk mitigation measures. Limited commercial orientation / viability – The FTF initiative specifically focuses on addressing food security in target countries. In line with the FTF initiative’s interest to improve resilience of smallholder famers (defined as those holding less than four HA of land), KISAN II targeted inclusion of smallholders, thereby guaranteeing the engagement of vulnerable households. Likewise, several of the 43 Social Impact FGDs, beneficiaries. 44 Social Impact KII, grantee. 45 Social Impact KIIs, grantees. 23 | USAID KISAN II EVALUATION REPORT USAID.GOV value chains targeted by the Activity are dual use i.e., market and home consumption-focused, especially vegetables. These factors also limited inclusion in activities to less market-oriented farmers. As noted by IP staff: “Commercial and semi-commercial producers are more easily reached through the market system since they are already more directly engaged with other market actors. So, it is more effective to connect them with grantees. However, subsistence level farmers have limited engagement with other market actors. So, the approaches need to be customized to this audience.” - KII, IP staff. While these features ensured alignment with the FTF objectives, they also reduced available crop surpluses that could be purchased by co-funded grantees thereby decreasing the marginal profit that could be obtained through extending market linkages and acting as a disincentive for continued JTA employment. In contrast, most market systems development approaches focus on commercial crops, such as coffee, tea, or cacao, for which market linkages are an inherent feature. Variable access to “coaching” and TA – The ratio of JTAs to households was one JTA to ~400, which compares favorably to the ratio of one GON extension staff to 2,000 households. As explained in KIIs, each JTAs provided technical services to about 20 farmer groups, with each group containing about 20 farmers, and it was assumed that a JTA would make at least one visit to each group each month.46 However, in KIIs and FGDs, respondents had mixed reactions to questions about access to coaching and oversight by JTA staff. While in some areas, respondents stated that JTAs were readily available to provide advice and visited frequently, in other areas they reported limited contact. For example, in one FGD a beneficiary farmer stated, "They came to form the group once, and we did not see them again", while another said, "Our JTA only visited once at the beginning of each season to inform us about the processes."47 During validation with the IP, they noted that their records indicate that JTAs would have visited farmers more than once and provided regular monitoring support. Nevertheless, our findings do show a more mixed result. It is unclear whether these variances were due to programmatic challenges or could be attributed to difficulty in JTA's fulfilling their obligations, which would be understandable in the rough terrain of certain areas of Nepal. However, in areas where JTA contact was limited, this represented a challenge to the adoption of improved agricultural practices and technologies by beneficiaries. In addition, while trainings proved to be very effective in encouraging adoption, especially when coupled with training videos, these “timebound” methodologies do not provide farmers with on-demand access to required technical assistance as needs or challenges arise. This access to coaching is often critical for small-scale farmers for whom years may elapse between a training event and the need for specific assistance.48 Constrained access to finance, especially capital expenditure (CapEx) and difficulty in accessing risk mitigation measures – As noted, KISAN II tailored its approaches to finance according to the level of beneficiary market engagement and leveraged existing finance schemes to improve beneficiary access. Also, the IP explained that around 50 percent of Activity partners are cooperatives, some of which have saving and credit core functions and organize services including microfinance, though at a prohibitively high interest because they deal with smaller loans that are not usually collateralized. However, in cases where non-cooperative beneficiaries required access to finance, financial institutions were not consistent in terms of policies and accessing loans entailed longer 46 Social Impact KII, implementing partner staff. 47 Social Impact FGDs, beneficiaries. 48 For example, see Rinck, David, Leah Ghoston, Hariyadi and Cininta Pertiwi “MCC Indonesia Green Prosperity Project Sustainable Cacao Partnership Grants Performance Evaluation - Final Evaluation Report” Social Impact, Inc. for Millennium Challenge Corporation, April 2020. USAID.GOV USAID KISAN II EVALUATION REPORT | 24 processes. In addition, in many cases, forward contracts were simply “verbal agreements” and hence not useful as security for commercial loans. Likewise, although cooperative finance and financial institutions provided some degree of access to finance to meet individual farmer production needs, there was a distinct lack of access to finance for large-scale CapEx items such as cold storage units. In this specific case, lack of cold storage forced harvest season sales when prices are lowest, the alternative being risking spoilage of perishable commodities. Likewise, in KIIs with farmers, IPs and GON staff, respondents stated that crop insurance was nominally available, but difficulty in processing claims made usage undesirable (on the other hand, these respondents reported livestock insurance was widely available).49 RQ2. CONTRIBUTION TO MARKET SYSTEMS DEVELOPMENT – CONCLUSIONS The most significant ways in which KISAN II’s PSE model benefitted farmers and private sector actors was through increasing access to improved agricultural practices and technologies motivating the widespread adoption of these innovations and leading to significant expansions of productivity and sales incomes. The benefits were somewhat hampered by the limited commercial orientation and viability of targeted farmers, variable access to “coaching” and TA, and constrained access to finance, especially CapEx, and difficulty in accessing risk mitigation measures. a. In addition to the benefits described above, the private sector perceives expanded communication with increased numbers of upstream farmer suppliers and greater business formalization as the most significant incentives to adopting KISAN II PSE models as they have perceived higher profits and improved management as a result. Farmers perceive improvements in output, especially vegetables, as the most significant incentives for adopting the PSE models. b. GON staff, especially at the federal level, recognized the effectiveness of the PSE business model, and began integrating similar PPP approaches into their agricultural development strategies. Farmer behavior change is most associated with the adoption of improved agricultural practices and technologies promoted through the model. Private sector actors have only partially adopted the model, and have adapted in several cases, such as through employment of family members to provide potential supplier farmers with TA. RQ3. POTENTIAL FOR SCALABILITY AND SUSTAINABILITY – FINDINGS This section presents the findings of the ET related to RQ 3: Table 9: RQ 3: Potential for Scalability and Sustainability AREA RESEARCH QUESTION Potential for scalability and sustainability 2. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? KISAN II contributes to the GON’s 20-year ADS (2015-2035), especially through influence on the Prime Ministers’ Agricultural Modernization Project (PMAMP), which is designed to enhance productivity and commercialization of major cereals, fish, fruits, and vegetables over the next decade, as well as through other federal and provincial agricultural development activities. As noted, in contrast to KISAN, which focused on direct implementation implementing partner agricultural support staff, KISAN II contributed 49 Social Impact KII, beneficiaries, implementing partner staff, grantees. 25 | USAID KISAN II EVALUATION REPORT USAID.GOV to this strategy through a focus on models for market systems development through support for private-sector actors (see Activity Description). COLLABORATION WITH THE FEDERAL LEVEL OF THE GOVERNMENT OF NEPAL KISAN II leadership has engaged senior GON officials since inception to share its project objectives, implementation modalities, planned activities, and progress. Specifically, the project conducted frequent meetings with the Secretary, Joint Secretaries and other senior officers at Nepal’s Ministry of Agriculture and Livestock Development (MOALD), which is the focal ministry, and with staff at other Nepalese and international agencies. Furthermore, KISAN II staff invited MOALD officials to participate in joint monitoring visits to the field, Kathmandu-based technical events, and monitoring, evaluation and learning briefings to keep them abreast of KISAN II progress and seek advice for adaptations and opportunities to better collaborate with on-going GON programs.50 Perhaps because of these efforts, GON staff at the federal level described coordination between the project and the federal government positively in KIIs. For example, one GON staff at MOALD stated, "KISAN II is one of the entities that has been closely working with the ministry."51 Through these engagements, in addition to several provincial pilot initiatives that integrate PSE models into activities (see Contribution to Market Systems Development – Findings), project documentation and KIIs reflect that KISAN II has exerted a considerable influence on the ADS, noting that the Activity has motivated the GON to allocate federal government grants to 59 municipalities to replicate KISAN II success stories, including the rice mill model. The Activity has also motivated the PMAMP to directly support replication of this model in certain provinces through integration into the Joint Rice Intervention Program (JRIP). This program targets 6,069 farmers (including 4,500 females, and 3,780 from disadvantaged groups) with the objective of introducing cultivation of eight varieties of spring rice on 2,000 HA of land, with support from 14 partners.52 This specific focus on the rice mill model is motivated by the GON objective of increasing domestic production of high-value rice as a means of reducing dependency on imports. According to annual reports, “KISAN II targeted GON engagement resulted in leveraged investments of NPR 169 million (USD 1.45 million) in Year 4, from government organizations for irrigation, agri-machinery, plastic tunnels, custom hiring centers, maize and spring rice production programs, and goat insemination services.”53 As noted in KIIs with donor staff, the GON is motivated to prioritize the commercialization of private sector actors and therefore highly receptive to KISAN II PSE approaches. As one donor staff stated “In Nepal, USAID is leading commercialization of private sectors. But there are many other projects happening in the larger frame of government programming. The government wants to bring the private sectors on board for agriculture services. Government people were heavily engaged in the inception of KISAN II design phase.”54 In KIIs with GON staff, interviewees confirmed that the government is gradually adopting the PSE model but facing challenges in roll-out, largely due to coordination and human resource challenges. As one respondent stated, “They are facing a huge shortage of extension workers as many of the technical experts are in federal offices but not at the local level. There used to be around 40 technical workers in our district agriculture office but after federalism, only two of us are in our office and we must cover two districts.”55 50 “KISAN II Y4 T2 T3 Provisional Plan for Coordination with the GON” Winrock international 2020. 51 Social Impact KII, GON staff. 52 “Knowledge-based Integrated Sustainable Agriculture in Nepal (KISAN) II Project” PPT by Winrock international, 2021. 53 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID. 54 Social Impact KII, donor staff. 55 Social Impact KII, GON staff. USAID.GOV USAID KISAN II EVALUATION REPORT | 26 COORDINATION WITH OPERATIONAL (PROVINCIAL AND MUNICIPAL) LEVELS OF GOVERNMENT OF NEPAL Project documents report that at the district level, KISAN II coordinated with the District Coordination Committee (DCC) and District Chambers of Commerce and Industries (DCCI) to facilitate activities, monitor performance, and share information and models of success. KISAN II field staff coordinate with AKCs, VHLSECs, and the Provincial Coordination Units (PCU) and Project Implementation Units (PIU) of PMAMP, all of which are operational at the district level.56 In addition, in KIIs, implementing staff reported several corollary activities to increase coordination between tiers of the GON. For example, staff stated that “Winrock conducted a study about strengthening agri-business linkages through the three tiers of the government and presented recommendations to the GON.” Project documents also described collaborating with GON programs through joint monitoring in cases where they work with the same farmers to avoid duplication of efforts, provision of training to GON municipality staff on how to implement PSE activities in the field, and preparation of written PSE guidelines at the request of the GON.57 Despite these efforts, both documentary research and KIIs demonstrate that a chronic and pervasive challenge to project implementation is poor coordination between the three tiers of the GON, as well as constraints in resource flows. Figure 8: Government of Nepal - Three Functional Levels As summarized by agriculture analysts, while each tier has required constitutional powers and defined roles, they are disparate, operating independent of each other. “As per the Constitution of Nepal, Nepal has a three-tier Federal system (Federal, Province and Local) and each tier has the constitutional power to enact laws, prepare budgets, and mobilize its own resources. They have delineated roles and responsibilities in each sector. In agriculture, all three tiers have a distinct role. Each of them has power to form policies, standards, and regulations; data management; trade and coordination; implementation of agriculture development activities.”58 “Each tier has also its own supervision system independent from each other and there is no line of command along the hierarchy. Though the functions are defined among the three tiers of the 56 “KISAN II Y4 T2 T3 Provisional Plan for Coordination with the GON” Winrock international 2020. 57 Social Impact KII, implementing partner staff. 58 Bishwakarma, B.K., Uprety, B.R.; Devkota, D. and Devkota T. “Agricultural Functions, Institutions and Policies in the Context of Sectoral Restructuring in Nepal” International Journal of Agricultural Extension and Rural Development Studies. Vol.8, No.2, 2021. 27 | USAID KISAN II EVALUATION REPORT USAID.GOV government, there is weak connection and coordination in all three tiers of the government as they are independent from each other. Priorities differ at different levels. Increased production, productivity and income to the value chain actors depend largely on all three tiers of government working together as a team which is yet to be realized in the context of Nepal.”59 According to donor staff, when KISAN II started, the new state structure was set up but was not fully functional. As described by donor KII respondents, “It was just in the initial phase of executing the federalism process ahead and confusion was occurring among stakeholders. The current challenge is the federalism process is still immature in Nepal and there is still ambiguity at the three tiers of government… The municipal government has fewer capacities, and the agriculture sector is still not a priority sector for municipal governments.” Donor KII respondents went on to explain that, while the GON has made attempts to address these challenges, they persist to some degree, i.e. “In comparison with the previous district level agriculture office, the new Agriculture Knowledge Center (AKC) are getting a sufficient amount of government budget, but to mobilize these budgets, AKCs are under capacitated and have low human resources.”60 In KIIs with provincial and municipal level GON staff, respondents echoed these statements and underlined the negative implications for program roll-out and implementation. For example, one municipal level GON staff stated, “With the change in the government structure, there was a break of chain from the center. There was no one to guide us. There are now meetings with the AKC, but that only started recently.” On resource constraints another commented, “We don’t have vehicles for the technicians. They use their personal vehicle to reach the farmers.” Finally, one municipal level GON staff noted, “At any time, up to 50 percent of staff positions are unfilled. The lack of resources to do our work means that staff often leave to better jobs.”61 RQ3. POTENTIAL FOR SCALABILITY AND SUSTAINABILITY – CONCLUSIONS There is strong potential for integration of the PSE model into GON agriculture programming, with federal government agencies already integrating the approach into several national agricultural development initiatives including the ADS and PMAMP, especially the rice mill model which aligns with GON objectives to reduce dependency on Indian imports. Likewise, provincial government agencies have already launched several similar ad hoc initiatives aimed at strengthening the operations of commercial actors. However, scalability and sustainability are challenged by poor coordination between the three tiers of the GON and resource constraints, especially in human resources. RQ4. ENABLING ENVIRONMENT – FINDINGS This section presents the findings of the ET related to RQ 4: Table 10: RQ 4: Enabling Environment AREA RESEARCH QUESTION Enabling environment 1. To what extent has KISAN II improved the enabling environment for agricultural systems development, specifically for private sector engagement, and institutionalizing the JSR Platform into MOALD? 59 Dahal, H; Karki, M; Jackson; and Pandey D “New State Structure and Agriculture Governance: A Case of Service Delivery to Local Farmers in the Eastern Gangetic Plains of Nepal” Agronomy 2020. 60 Social Impact KII, donor staff. 61 Social Impact KII, GON staff. USAID.GOV USAID KISAN II EVALUATION REPORT | 28 KISAN II implements activities aimed at improving the enabling environment for agricultural systems development under Component 3: Strengthen the enabling environment of selected agricultural market systems. These activities include providing ongoing support to the three tiers of GON through coordination with the MOALD, the Joint National Steering Committee (JNSC), and the Technical Committee (TC), as well as other bodies. KISAN II also provides policy formulation and review support and has carried out studies and engaged with the GON to develop a number of guidelines, regulations and development strategies. The major policy supports include private sector guidelines, food safety regulations, Joint Rice Intervention Program (JRIP) Operating Procedure, Agricultural Sector Policy Think Tank, and Provincial Agriculture development Strategies for Sudurpaschim and Lumbini Provinces. An important area of focus at the federal level was coordinating closely with JNSC to support establishment of the JSR to facilitate KISAN II alignment with GON national strategies and objectives, and coordination between other projects and other development partners. Jointly, at the federal level these activities help evaluate KISAN II approaches and models and facilitate replication in GON initiatives, especially activities under the ADS. POLICY FORMULATION AND REVIEW As reported in project documentation, one key area where KISAN II provides policy formulation and review support is in MOALD efforts to promote and support PSE-related policies and legal frameworks. This engagement also supports ADS-related national and provincial strategies and objectives. For example, at the request of JNSC in 2020, KISAN II prepared PSE guidelines in close consultation and collaboration with MOALD and private sector partners.62 This document provides strategic guidance on providing a broad framework for streamlining private sector engagement in agribusiness development, catalyzing more investment in agricultural enterprises in GON tiers (see Potential for Scalability and Sustainability – Findings). Likewise, the Activity responded to a request from MOALD Department of Food Technology and Quality Control (DFTQC) to draft technical regulations regarding food business registration and licensing, and import-export inspection and certification of food products.63 KISAN II staff also collaborated with MOALD and other stakeholders to develop a goat breeding strategy to promote the goat sector and to standardize goat breeding stock and meat quality. Under this strategy, MOALD formed a National Goat Working Group, with KISAN II as a key stakeholder, to implement the strategy.64 In addition, Activity staff produced several publications and studies that contribute to ADS strategic and policy objectives. These include: ● A Landscape Analysis of Rice Milling Industries incorporating inputs from in-person discussions and virtual validation workshops. ● A food safety booklet focused on farm and harvesting hygiene, water supply quality and irrigation systems, and the proper use of chemicals on vegetable farms and during post-harvest management. ● A rapid assessment on utilization of GON and Nepal Rastra Bank (NRB) relief and post￾COVID-19 recovery measures, specifically focused on farmers and agro-entrepreneurs (including KISAN II grantee partners and beneficiaries, particularly women and DAGs). 62 “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID. 63 “KISAN II Y4 T2 T3 Provisional Plan for Coordination with the GON” Winrock international 2020. 64 Ibid. 29 | USAID KISAN II EVALUATION REPORT USAID.GOV In KIIs with donor and GON staff, respondents familiar with these products described them as useful tools in making the transition from embracing PSE conceptually to putting into practice approaches to engagement and their incorporation into agricultural development strategies. As one GON respondent stated, “Previously, we worked through government agencies and we are not used to working with the private sector… KISAN II has provided practical guidance in this area.”65 In addition a donor staff member said, “We [USAID] are shifting towards working through the private sector across the region, but there is resistance from some, because previously development projects focused on government actors. They don’t have experience with this [private sector engagement]. So, projects like KISAN II are helpful in supporting governments in the region to negotiate this change.”66 Likewise, a representative from a partner project recognized the Activity’s PSE approach as a major shift and noted that most governments across the region will need practical support on how to adopt it. A KII respondent summed up the situation of GON’s lack of experience with PSE models: “Governments in the region are used to government taking the lead on [agricultural] development projects, there's a real lack of experience [in working with private sector] so it's going to take time and support on how to implement these types of projects.”67 INSTITUTIONALIZING THE JOINT SECTOR REVIEW PLATFORM As described in project documentation, through the JSR, KISAN II supports MOALD to jointly review the performance of the agriculture sector, discuss issues constraining achievement of the planned sector targets, agree on actions to resolve issues, and monitor and report progress. The JSR also provides a platform to discuss project and program operations, raise issues and concerns, and seek solutions for enhancing mutual accountability and harmonizing use of resources. Through the JSR, participants have a forum for analyzing the performance of the agricultural sector and for formulation of recommendations for increased effectiveness and efficiency. JSR functions are informed by bi-monthly TC meetings and an annual JSR meeting with MOALD and other ministries, as well as with the Ministry of Land Management, Agriculture and Cooperatives (MOLMAC), municipalities, development partners, and related stakeholders.68 As described by IP staff in KIIs, “There are close links between ADS and JSR. Whatever is reviewed under this mechanism is reviewed in the broader framework of ADS. This is a sector-level mechanism whose role is to coordinate between the government and development partners. It is a more progressive and inclusive kind of mechanism to review the agriculture sector.”69 In KIIs, federal GON staff described the functions of the JSR, but suggested that much work still needed to be done for this forum to be effective, especially in improving coordination between the three tiers of the GON, and with the private sector. For example, one federal level GON KII respondent stated, "We established a review forum called the Joint Sector Review for the Agriculture Development Strategy. Other development partners and stakeholders [in addition to KISAN II] were also engaged in the JSR. In the JSR, we discussed issues related to coordination and policy. Basically, the platform had the main objective of joint implementation, joint review, and joint sharing. The private sector is there but it is not yet part of the technical committee. As the JSR grows and evolves within the ministry, it will be important to have private sector as we progress in the technical committee." Likewise, in a KII with provincial level GON staff, one respondent stated, “JSR has been established at the federal level although not much work has been done through it here so far. It is at least a platform where all the stakeholders including the government can come together and discuss [issues].”70 65 Social Impact KII, GON staff. 66 Social Impact KII, donor staff. 67 Social Impact, implementing partner staff. 68 “KISAN II Y4 T2 T3 Provisional Plan for Coordination with the GON” Winrock international 2020. 69 Social Impact, implementing partner staff. 70 Social Impact KII, GON staff. USAID.GOV USAID KISAN II EVALUATION REPORT | 30 In KIIs with IPs, interviewees opined that realization of the full potential of the JSR would be a long process requiring technical support to stakeholders unaccustomed to JSR and PSE development models. In one KII with IP staff, a respondent stated “There are different agendas in the development partner side as they have their own set of priorities and are not fully aligned with the government. It is a step￾by-step process to align the government and the development partner. Providing technical assistance through the time-bound project for a sustainable mechanism is not the right approach because support for two-three years may not be sufficient. More technical support is required by the government.”71 RQ4. ENABLING ENVIRONMENT – CONCLUSIONS KISAN II has contributed to improvements in the enabling environment for agricultural systems development through providing technical support for policy review and formulation, and practical examples of the application of PSE-related approaches. The Activity also successfully established the JSR Platform in support of the ADS in collaboration with MOALD and development partners. However, there is significant dearth of experience within the GON with private sector-led development models. Full realization of the effectiveness of the JSR will require long-term technical support to the GON and partners. RQ5. GENDER EQUALITY AND SOCIAL INCLUSION – FINDINGS This section presents the findings of the ET related to RQ 5: Table 11: RQ 5: Gender Equality and Social Inclusion AREA RESEARCH QUESTION GESI 5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? GESI is specifically embedded in the purpose of KISAN II through its focus on inclusive income growth. Activities that increase access to market and productive resources and access to finance for marginalized communities are implemented under Component 4: Increase ability of vulnerable communities to act on business opportunities within selected market systems. Likewise, the inclusion of disadvantaged groups and vulnerable people is facilitated by the BLP and ensured through the Activity’s focus on inclusion of smallholder farmers. INCREASING ACCESS TO IMPROVED MANAGEMENT PRACTICES AND TECHNOLOGIES FOR MARGINALIZED COMMUNITIES In KIIs with IP staff, respondents explained that KISAN II did not implement specific targeted interventions to address GESI-related issues, but rather relied on the smallholder four HA maximum landholding criteria to reach women and DAGs. This assumption largely held true -- given the out￾migration in Nepal, women constitute most small-scale farmers amongst the general population. KISAN II did intentionally include women in program design in one area - a focus on specifically constituting women’s groups for the BLP. As a result of these design features, KISAN II reached a preponderance of women farmers through activities to disseminate access to improved practices and technologies. Project documentation reports that, as of May 2021, KISAN II had reached 199,961 farmers, of which 74 percent are women and 59 percent are from vulnerable/disadvantaged groups. In addition, the most recent annual farm survey reported that, in terms of individual farmers accessing these technologies, 71 71 Social Impact KII, implementing partner staff. 31 | USAID KISAN II EVALUATION REPORT USAID.GOV percent are women and 27 percent are youth. In terms of community representation, nine percent of farmers are Dalit, 50 percent are Janajati, 37 percent are Brahmin or Chhetri, three percent are Madhesi, and one percent are Muslim. However, the reasons for the relatively lower proportion of Muslims in the beneficiary population are unclear from the document review and data collection. In addition to the significant percentage of women farmers, in KIIs, respondents reported that the cooperatives are largely female-led.” On the other hand, grantees were overwhelmingly males, and generally came from better-off communities. Data on the gender and vulnerability status of individuals holding leadership positions within KISAN II grantee organizations substantiates this finding. Half of leadership positions are held by males who are not classified as “vulnerable” by Activity markers. Furthermore, less than 20 percent of leadership positions are held by females, though female representation did improve after 2019, as illustrated in Figure 9 below. Figure 9: Gender and Vulnerability Status of Grantee Leadership Source: Wikisan 2.0 database. Vulnerability classifications are pre-defined in the database based on definitions set by the Activity. Some women and DAG respondents noted that the BLP did improve their basic literacy and numeracy, and imparted knowledge on nutrition, farming practices, agri-tools/technologies, and saving and credit. However, the perceived effect of these courses was variable and dependent on many factors including previous educational background, engagement of the course instructor etc. In addition, respondents also noted that COVID also impacted the frequency and intensity of training programs. While the Activity’s effort to develop and implement the BLP is a necessary step for inclusion, findings indicate that more frequent and long-term capacity building is needed to reinforce learning. Despite the fact that the Activity reached a large number of women and DAG farmers, specific gender and community-based barriers limited the full participation of women and DAG farmers in KISAN II activities and mitigated the effects of Activity interventions on outcomes. As stated by IP staff, “Even though the project worked with more than 70 percent women beneficiaries, the project had no targeted interventions, particularly in access to resources, grant subsidies, and household gender dynamics including promoting gender equality and social inclusion, hence, women and DAG couldn’t benefit much from private sector-led support.” In KIIs with donor staff, one respondent explained, “Women and USAID.GOV USAID KISAN II EVALUATION REPORT | 32 DAG are not in a position to provide a lot of revenues to agrovets, traders, millers (resulting low women’s engagement with grantees)."72 Similarly, IP staff noted that the project did not design specific incentives for private sector or financial institutions to work with women, hindering their access to private sector partnership, subsidies, and loans. “The project did not provide any incentives to the grantees to cater to the needs of women and DAG groups. As a result, the private sector did not have the motivation to work with these groups, hence, they were marginalized further. There are provisions of subsidies for marginalized groups, but those subsidies never reach to the poor”73. Commenting on market access challenges, another respondent noted “There is generally a problem of access to the market to women and DAG farmers because of the small volume they produce. It is too small to interest most traders.”74 In KIIs, numerous respondents characterized traditional gender roles and woman control over resources was highly correlated with household proximity to business centers and the relative remoteness of households. Informants described women living near cities as more empowered in terms of expressing their concerns and putting forward their agendas. As noted by one IP respondent, “They take part in community affairs, do some income-generating activities, keep their earnings, and use it as per their wish. On the other hand, in rural areas, traditional gender roles and social norms are heavily embedded in the households. Women were still found to be confined in the houses and doing all the household chores.”75 Additional challenges women face in fully realizing the benefits of KISAN II activities are less flexibility to attend meetings and trainings due to household chores, and limited control over resources. For example, in one FGD, a male respondent in a remote community stated, “This is a women's group. My wife is a member of this group, but I am representing her because she is preparing meals at home." Another female respondent stated, “Husbands are the ones who go out in the training, to the market or other activities. They learn whatever there is to learn, come back home and teach us what they learned. We stay back at home and do the work around here.”76 Likewise, in several FGDs women respondents described being required to hand over all earnings from farm sales to their husbands. As one grantee respondent stated, "The ownership and control of finances are largely with males. However, there have been small gradual changes in terms of gender equality in our society.”77 INCREASING ACCESS TO FINANCE FOR MARGINALIZED COMMUNITIES Because the program did not have specific interventions to address gender barriers aside from the BLP, outcomes related to changes in access to finance reflected outcomes for the overall beneficiary population discussed above, including gender and community specific constraints on activities. According to the most recent farm survey, in Year 4 the value of agriculture-related financing accessed through USG support was almost USD 28 million. Financing was accessed by 33,106 beneficiary farmers, 72 percent of whom were female.78 However, challenges related to control over resources limited financial access for women, even in cases where financial institutions offered concessional lending. For example, in a KII with one financial institution staff member, the respondent noted "We have a provision to provide a loan, sometimes even without collateral, to women, but in practice, we don't issue it without consulting with male members of 72 Social Impact, donor. 73 Social Impact KII, implementing partner staff. 74 Social Impact KII, grantee. 75 Social Impact KII, implementing partner staff. 76 Social Impact FGDs, beneficiaries. 77 Social Impact KII, grantee. 78 “Annual Survey Report – Summary Tables – KISAN II 2021 – Extrapolated ” Full Bright Consultancy (Pvt.) Ltd., 2021. 33 | USAID KISAN II EVALUATION REPORT USAID.GOV the family."79 Women were also found to have faced challenges in accessing loans for commercial farming without land titles. One female respondent shared “I wanted to get a loan from the banks to expand my goat farm. I went to several banks and was told that they required land title as collateral and our land is in my husband‘s name. I shared that with my husband, and he told me that he would not allow me to use his land as collateral [...]. So, I am unable to get loan and expand my business.”80 On the other hand, women and DAGs affiliated with cooperatives appear to have comparatively easier access to loans. As one cooperative member stated in a FGD with female beneficiary farmers, “We take loans but from our own cooperative. Microfinance is here as well but it has created a panic among all of us. It offers a loan but forces us to pay the monthly installment and delay would cause a high penalty.” Both cooperatives and microfinance charge high-interest rates on loans. With the seasonal nature of agricultural income, it is difficult for farmers to pay the loan and its interest in a monthly basis. This is a particular challenge for women and DAG farmers because crops produced are typically consumed with limited surplus for sale, and these households typically do not have other stable income sources to supplement agricultural income. Another female respondent stated, “They [microfinance institutions] are not practical for goat-keeping farmers because we don’t have regular monthly income, and also for vegetables, we only have seasonal income. We can pay to cooperative a little later, but the cooperative’s total capital is very little.”81 RQ5. GENDER EQUALITY AND SOCIAL INCLUSION – CONCLUSIONS A high proportion of KISAN II beneficiary farmers were women and DAGs due to beneficiary selection criteria that limited participation to smallholder farmers (farmers with less than four HA of land,) and the formation of women’s groups for the BLP. Therefore, changes in access to improved practices and technologies, and access to finance generally reflected changes for the overall beneficiary population minus specific barriers that included limited production, traditional gender roles, lack of collateral, and limited control of household resources. However, the concrete efforts to 'increase access to market and productive resources, and improved management practices and access to finance for marginalized communities' was confined to participation in farmers groups and BLP without targeted interventions to address their major barriers. 79 Social Impact KII, grantee. 80 Social Impact Site Visit, beneficiary 81 Social Impact FGD, beneficiaries. USAID.GOV USAID KISAN II EVALUATION REPORT | 34 5. RECOMMENDATIONS AGRICULTURE PRODUCTION AND FOOD/NUTRITION SECURITY “In fact, where there is investment there is profit. Those who invest can make profit.” - Grantee Providing stakeholders with data required for making sound agricultural investments into their agricultural systems is a key element in agricultural market systems approaches. While the KISAN II Activity fulfils its data reporting requirements to donors through data collection on progress toward indicator targets with grantees and sub-samples samples of beneficiary producers, these methodologies do not provide discreet impact information on the beneficiary population. KISAN II and similar projects should consider adopting an individualized recordkeeping system (i.e., crop budgets) for target crops to enhance the analytical capacity of producers. Crop budgets are simple forms that quantitatively estimate anticipated costs and income to record the actual costs of inputs on an on-going basis, and the value of outputs to demonstrate impacts on profitability. Estimates of anticipated costs and incomes based on research can also be utilized by producers to better understand the anticipated impact of various improved agricultural practices and technologies to evaluate likely outcomes of these once adopted. This form also enhances financial literacy and provides a point of reference for technical service providers during farm visits. Figure 10: Sample Crop Budget 35 | USAID KISAN II EVALUATION REPORT USAID.GOV In addition, when aggregated in a database, this data can provide essential investment information to agricultural enterprises on the business case for outreach strategies and technical assistance investments by agricultural enterprises along target value chains, as well as return on investment (ROI) data to GON policy makers to inform its analysis of development strategies. CONTRIBUTION TO MARKET SYSTEMS DEVELOPMENT The KISAN II Activity addresses food security objectives through the inclusion of staple crops (e.g., grain legumes and vegetables) produced by subsistence and semi-subsistence farmers for home consumption and improved nutrition. While these objectives are key to the goals of the USG GFSS, as noted, most market systems development projects tend to focus on a selected cash crop in order to ensure the commercial orientation of producers and market linkages (see Contribution to Market Systems Development – Findings). The KISAN II activity should consider shifting to an enhanced focus on commercial production of these crops through expanding its PSE models to include seed and input dealers that service rice, as well as vegetables and potentially other producers.82 Currently, KISAN II primarily works with inputs suppliers of crops and the goat value chain (agrovets), though cooperatives and some rice millers also provide inputs but to a lesser degree. This enhancement should also focus on addressing noted supply chain weaknesses in the seed supply and input chains for other crops. It should focus on addressing supply challenges in inputs for rice production, which impede the adoption of improved agricultural practices and technologies. In addressing these supply challenges, the activity could also encourage linkages to seed multipliers, wholesale fertilizer83 and related inputs suppliers, and wholesale suppliers of veterinary supplies. Figure 11: Enhanced KISAN II Private Sector Engagement Models This model builds on the relative success of the upstream agrovet model, which supplies inputs and TA to goat farmers. In effect, this enhanced model creates a “hub and spoke” service delivery system like those employed by major commodity companies to ensure raw material supply chain integrity. For example, Mars Inc. deploys a similar model to ensure input and TA delivery to supplier cacao producers in the countries from which it sources.84 The adapted PSE service delivery model would comprise of seed multipliers and wholesale inputs suppliers linked to local input dealers supported by co-funded grants for employment of JTAs to extend input sales and TA delivery. 82 This enhanced focus on commercial production would still enable the Activity to address outcomes related to domestic food consumption and food security, both key areas of interest for the FTF initiative. 83 As fertilizer trade is currently a public sector-controlled function, the ET understands that fostering linkages between farmers and wholesale fertilizers would likely require a different approach than fostering linkages with private sector retailers. 84 “Mars Indonesia – Service Delivery Model, Case Study Report” The Sustainable Trade Initiative (IDH), June 2015. USAID.GOV USAID KISAN II EVALUATION REPORT | 36 Figure 12: “Hub and Spoke” Service Delivery Model A key strength of this model is efficiency in reaching large numbers of farmers for technology transfer and TA, in addition to provision of inputs and supplies. For example, with one seed multiplier or input wholesaler serving 20-30 local input dealers, and each service provider serving 20-30 farmer field schools of 20-30 farmers each, this model can effectively serve between 8,000-27,000 farmers per wholesaler. POTENTIAL FOR SCALABILITY AND SUSTAINABILITY KISAN II leadership has engaged senior officials at the federal, provincial, and municipal levels since inception. However, a chronic and pervasive challenge to project implementation is poor coordination between the three tiers of the GON, as well as constraints in resource flows. By 2019, the GON had established 51 AKCs across Nepal to replace District Agriculture Development Offices (DADOs) previously charged with delivery of agricultural extension. The AKCs role is to manage knowledge required for agriculture modernization in their mandated areas and provide planning and organization to technicians and farmers. These centers are managed by the provincial governments, but also function as a linkage between the federal and municipal governments. Given that the supply chains for the target crops are bound to cross municipal borders, especially under an enhanced model as described above (see Contribution to Market Systems Development), the AKCs are a logical nexus for connecting the federal and municipal levels of government and engaging them in PSE activities. KISAN II should consider closer collaboration with the AKCs, including creating a role for them in management of the grant fund that co-finances employment of the JTAs. KISAN II should also leverage the role of the AKCs in connecting the federal and municipal governments. This could include engagement of the AKCs in the JSR to share lessons learned and experience. This could also include engaging municipal entities in oversight and management of grantee activities in their spheres of activity. Ultimately, the sustainable continuity of PSE approaches will require financing. KISAN II should consider levering its engagement with the MOALD and ADS at the federal level through the JSR to ensure that budget allocation and revenues flow into the grant fund to facilitate continued operation and expansion of the PSE models. ENABLING ENVIRONMENT KISAN II has actively engaged the GON on enabling environment issues related to PSE through support for policy formulation and review, and through production and sharing of a significant number of related guidelines and studies. Creating enabling environment through formulating and adopting policies by GON would have long-term impacts for promoting private sector engagement. Some pertinent policy agendas to be considered: i) support GON strengthening the Agricultural Sector Policy Think Tank (ASPT) at both federal and provincial levels that would bring diverse expertise, experiences and viewpoints into policy options to address emerging policy issues, ii) given the situation of ever-increasing huge volume of rice imports, support GON to formulate a "rice policy to reduce imports", iii) as 37 | USAID KISAN II EVALUATION REPORT USAID.GOV ultimate implementation of ADS is local level, therefore KISAN II is suggested to support municipality for formulating model Agriculture Development Plans that can be replicated in other local levels. The Activity has also successfully supported the launch and establishment of the JSR as a forum for exchange on implementation and lessons learned between the MOALD, the ADS and development partners. However, the regional transition across South Asia to PSE approaches is a long-term shift that entails building expertise and experience in a country long accustomed to state-led agricultural development approaches and unsurprisingly faces some resistance. KISAN II should consider ways in which the JSR can be strengthened, and better institutionalized including expanding the role of provincial and municipal stakeholders and consider ways in which PSE in the JSR can be expanded. This may include strengthening dedicated working groups for discrete value chains, as is already planned in the GON’s annual JSR action plans, to better match the interests of specific private sector participants. In addition, KISAN II should consider ways to expand engagement with DFI, such as the Asian Development Bank and the International Finance Corporation (IFC) of the World Bank, that may have specific interest in PSE approaches and funding. Finally, USAID/Nepal should consider a funding mechanism for long-term technical support to GON in the JSR through on-going TA on PSE approaches and related projects. There are several modalities that could be deployed to implement such a mechanism. For example, the Title VI John Ogonowski and Doug Bereuter Farmer-to- Farmer program is a useful tool to deploy volunteer consultants with specific technical expertise in selected value chains in an advisory role on a short-term basis, especially if implemented in partnership with a US-based organization with access to staff with skills in these areas.85 Since this mechanism is already being used by KISAN II and USAID/Nepal, it could easily be expanded. USAID/Nepal could also consider a partnership for ad hoc exchange between Nepal and countries with similar bodies. GENDER EQUALITY AND SOCIAL INCLUSION KISAN II was relatively successful in reaching a large proportion of female and DAGs farmers due to its beneficiary selection criteria. The Activity also strengthened its focus on female farmers through the farmer groups formed for the BLP. However, KISAN II did not implement specific activities to address inherent barriers to women and DAG full participation in KISAN II activities and factors that mitigate realization of outcomes. The Activity was also less successful in reaching woman and DAG grantees and fostering an environment for their growth. Project design and beneficiary selection should take an inclusive approach ensuring equal participation of all caste and ethnicity in all the project activities by taking extra measures if required. For this reason, future programming should consider more intentional GESI programming by including GESI expertise in the design phase and planning for specific GESI-tailored interventions. KISAN II should consider adopting targeted interventions that address barriers to women and DAG’s full participation. Some illustrative examples of possible activities include: ● A specific focus on connecting women and DAG groups with other stakeholders including local GON agencies to educate them on existing provisions in access to finance. ● Advocacy to GON agencies for women-friendly agricultural tools and technical expertise to design such tools drawing lessons from similar countries. 85 For more on this model, see: “The John Ogonowski and Doug Bereuter Farmer-to-Farmer Program.” U.S. Agency for International Development, June 8, 2021. USAID.GOV USAID KISAN II EVALUATION REPORT | 38 ● Mainstreaming gender and social norms change activities to motivate transformative change, including working with men and boys on awareness-raising activities and creating youth champions among young women and DAG farmers within farmer groups. ● Conduct a study with grantees, finance institutions and vulnerable groups to identify the disincentives and barriers for them to engage with these groups. Findings about the disincentives from the private sector side can inform more strategic approaches for women, minority and vulnerable populations. 39 | USAID KISAN II EVALUATION REPORT USAID.GOV ANNEX A: STATEMENT OF WORK FINAL PERFORMANCE EVALUATION OF USAID/NEPAL Knowledge-based Integrated Sustainable Agriculture in Nepal (KISAN II) STATEMENT OF WORK 1) BACKGROUND The USAID under its FTF initiative has contracted Winrock International to implement the Knowledge￾Based Integrated Sustainable Agriculture in Nepal Activity. Agriculture employs 78 percent of the economically active population in Nepal and contributes 34 percent to the Gross Domestic Product. At the same time, Nepal is food insecure, and the agricultural sector faces persistent challenges, including limited irrigation, increasingly extreme and erratic weather, and insufficient access to quality inputs. Given the obstacles, there is an opportunity to support sustainable agricultural growth for Nepal’s transition into a more food secure country. KISAN II contributes to the GON’s ADS. KISAN II is a five-year, $32.7 million activity which will facilitate systemic changes in the agricultural sector including: (1) greater climate-smart intensification of staple crops and diversification into higher value commodities; (2) strengthening of local market systems to support more competitive and resilient value chains and agricultural related businesses; and (3) improving the enabling environment for agricultural and market systems development. USAID has extended cost ($4.9 M) and time for 24 months of the scope. With this total estimated cost of the award is $37.6M and the award will end in July 2024. KISAN II focuses on market systems and private-sector actors, however, recognizing that social and geographic contexts and barriers can create an uneven playing field for some groups, KISAN II employs a “push-pull” approach to its outreach and engagement with beneficiaries. Push strategies help to integrate women, youth, and marginalized groups into the market system and individuals build the capacity to participate in intensification, diversification, and value addition activities. Pull strategies increase the demand for smallholder production, labor, and related goods and services and improve the affordability and accessibility of skills, resources, inputs, and supporting services needed to participate in competitive markets. KISAN II tailors its approach to empower and graduate farming households and firms into more productive, reliable, and lucrative agricultural enterprises evolving from vulnerable to developing, to commercially minded, and finally to competitive household agricultural enterprises. USAID.GOV USAID KISAN II EVALUATION REPORT | 40 2) DESCRIPTION OF THE ACTIVITY Activity Name Knowledge-based Integrated Sustainable Agriculture in Nepal Implementer Winrock International Contract # AID-367-C-17-00001 Total Estimated Cost $37,600,000 Life of Activity July 12, 2017 to July 11, 2024 Active Geographic Regions 25 districts including 21 districts in the West, Mid-West and Far-West regions and four earthquake-affected districts in the Central region. Development Objective(s) DO2. Inclusive and sustainable economic growth to reduce extreme poverty IR2.1. Agriculture-based income increased IR2.2. Small enterprise opportunities expanded USAID Office Social, Environmental and Economic Development Office, USAID/Nepal 3) PURPOSE OF THE EVALUATION The purpose of this endline performance evaluation is to assess USAID/Nepal’s efforts to strengthen the performance of the selected agriculture market systems against performance indicators and targets. The evaluation will examine how the efforts to strengthen key market actor’s roles in the selected agriculture value chains have improved service delivery to farmers and also examine how the activity has contributed to an enabling policy environment for private sector engagement in agriculture. Similarly, the evaluation will also examine the viability of the interventions in terms of long-term sustainability and scaling up successful business models. The findings from the evaluation will be used to generate recommendations for further improvement and adaptation during the remaining years. The evaluation is also expected to contribute to the mission’s ongoing and future FTF Activities by generating evidence of good practices and learning. Findings from the evaluation will also be shared more broadly with stakeholders in Nepal that may be able to learn from and apply the lessons learned. 4) EVALUATION QUESTIONS The following questions will guide the evaluation process. The endline performance evaluation will assess what is/is not working well in implementation, evaluate progress toward objectives and outcomes, call attention to any unintended outcomes, and provide evidence-based findings and recommendations USAID/Nepal and KISAN II can use to improve activity effectiveness and better achieve the purpose and outcomes outlined above. In answering these questions, the evaluation team must analyze GESI and appropriate disaggregated data, as well as evaluate KISAN II’s contribution to including women and marginalized communities. Evaluation Area Evaluation Question Agriculture production and food/nutrition security 1) Is KISAN II on track to achieve its purpose with respect to: a. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth and marginalized communities? b. the increased incomes used in the consumption of nutritious food by the household members? 41 | USAID KISAN II EVALUATION REPORT USAID.GOV Contribution to market systems development 2) In what ways has KISAN II’s Private Sector Engagement model benefitted farmers and private sector actors? In particular: a. What do the private sector and farmers perceive to be the incentives adopting the PSE model? b. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models?86 Potential for scalability and sustainability 3) To what extent has KII been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? Enabling environment 4) How has KII’s support creating the enabling environment for agricultural market systems contributed, specifically, to: a. Improving private sector engagement in agriculture? b. Institutionalizing the Joint Sector Review Platform into the Ministry of Agriculture and Livestock Development structure? GESI 5) How has KII contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? :87 5) EVALUATION DESIGN AND METHODOLOGY The final performance evaluation for KISAN II will rely on a mixed methods approach, utilizing primary qualitative data, and secondary quantitative data analysis. The evaluation contractor, with feedback from USAID, will further refine a rigorous methodology including a sampling design that will yield meaningful insights into key Activity outcomes as specified by the evaluation questions. The evaluation team, in collaboration with USAID, will finalize the evaluation methodology before fieldwork begins. With the given pandemic situation, this evaluation is largely planned to be conducted on virtual means unless the situation allows for in-person. Fieldwork is expected to cover field testing of data collection instruments and carrying out KIIs and FGDs, and other data collection methods. Other data collection tools may include stakeholder and beneficiary interviews, case studies. A set of instruments must be prepared that asks different groups similar questions, so that the team draws lessons, triangulates data/information, observes trends and gaps, and develops recommendations. USAID expects that, at a minimum, the evaluation team will: ● Review and analyze the existing performance information/data88; ● Familiarize themselves with documentation about KII and USAID’s current assistance in agricultural market systems development in the region. USAID will ensure that this documentation is available to the team from the beginning of this evaluation. ● Meet and/or interview relevant stakeholders, including but not limited to: relevant USAID staff, partners, host government counterparts at appropriate levels, and a representative number of experts working in the sector. ● Carry out rigorous analysis using a variety of different sources to satisfactorily address the evaluation questions. 6) POTENTIAL DATA COLLECTION METHOD ● Desk review of documents and secondary data 86 Examples of KII’s business models include the Rice Miller Model, the agro vet model, and trader and cooperatives models 87 Based on the findings and conclusions of this evaluation question, the ET should provide recommendations on how the activity or interventions can be adjusted to improve inclusion moving forward. 88 This includes performance monitoring data and yearly outcome surveys USAID.GOV USAID KISAN II EVALUATION REPORT | 42 ○ Document review: A thorough review of Activity documents such as the MEL plan and documentation produced for/by KII, and secondary literature in order to identify key trends in agricultural production, food security, and market systems, among others, in Nepal. ○ Review of national, FTF ZOI districts and beneficiary statistics: A thorough review of existing data/information in the ZOI, including baseline, annual reports, monitoring data, WIKISAN 2.0, success stories, training records, and data collection instruments from KII. ● Primary data collection ○ FGDs and KIIs: FGDs and KIIs will be conducted with various stakeholders such as value chain associations, including women’s groups, youth, marginalized communities, and other beneficiaries in order to obtain a deeper insight into the different dimensions of change in capacities, context, attitudes and perceptions of technologies and interventions. ○ Field observations: If the situation allows, the evaluation team will visit and make strategic observations of KII field sites, including meeting with grantees and farmers-women lead households, marginalized communities, cooperative, public and private sector partners (GON offices, companies, agrovets and others) to observe and validate quantitative data collected by IP. The observations can be done simultaneously with the FGDs and KIIs. ● Data Analysis Primary and secondary data must be analyzed and triangulated to robustly answer the above questions. ○ Quantitative secondary data analysis: The Contractor is required to conduct a comparative analysis of existing data from KII field and national statistics on agricultural productivity/yield; analysis of sale of products, consumption of inputs, and private sector extension service support to producers, among others, by geographic, including notation of where KII partners are operating. The evaluation team will map this data to demonstrate KII’s contribution at ZOI, regional, and national levels. ○ Qualitative data analysis: The Contractor must apply a systematic approach to coding, interpreting, and understanding the data. ○ Disaggregation: Data collection and analysis will be disaggregated by sex, age, caste, ethnicity, and geographic regions (hill and terai) of target populations to discern differences and similarities of experiences, perceptions, needs, and barriers, among others. The Contractor will submit the preliminary evaluation design, which will be reviewed by USAID. The field level data collection will begin only after the approval of evaluation design and workplan. The evaluation must operate in line with current USAID and GON policy on the treatment of human subjects when applicable. There are no in-country Institutional Review Board (IRB) requirements for this evaluation, but the Contractor will be required to communicate and collaborate with the Central Bureau of Statistics and with MOALD as the main GON points of contact, and with USAID/Nepal to ensuring respondents are not harmed (i.e., “Do No Harm” principles-especially relevant if there are sensitive questions involved and/or in non-permissive environments), as well as collection and safety of. (See USAID Scientific Research Policy.) 5) EVALUATION TEAM ` The evaluation team should comprise a range of skills directly relevant to the purpose of this evaluation. As emphasis is on ensuring the evaluation team has the skills covered below, the type of positions, position names, and number of positions may be adjusted to ensure a strong evaluation team. The Contractor may propose additional personnel (e.g., translator/interpreter, research assistants/logisticians) as deemed necessary. 43 | USAID KISAN II EVALUATION REPORT USAID.GOV Team Leader/Evaluation Expert (one position): The Team Leader/Evaluation Expert must have extensive experience evaluating or implementing agriculture and food security activities. S/he must have a graduate degree in agricultural economics, economics, agribusiness, or a related field in social sciences, with experience leading and/or evaluating agricultural projects with a market systems approach for USAID or other donor agencies, preferably in Asia. S/he should have experience leading evaluations and assessments for USAID and be familiar with current USAID evaluation policy and guidance and have experience with both quantitative and qualitative research methodologies. The Team Leader/Evaluation Expert should have a background in evaluation methods. This experience should include development of research questions, and critically examining the validity of results frameworks and theories of change within the context that projects are operating within. Experience in evaluation of projects contributing systemic change and sustainability analysis and assessing the development of agricultural market systems. The ideal candidate will have excellent oral and written communication skills in English. Agriculture Sector Specialist (one position): The Sector Specialist is a senior expert on promoting agribusiness and the agriculture sector in Asia and should have extensive experience working with USAID or other donor agencies. The Sector Specialist must have a graduate degree in business, agricultural economics, agronomy, or a related field, from a recognized national or international university, with at least 10 years of experience working in agriculture and agribusiness, including in relation to private sector strengthening in agriculture. Experience leading, or serving as a senior technical expert, in agriculture sector evaluations and assessments for USAID or other donor entities, is preferred. The ideal candidate will have excellent oral and written communication skills in English. Business and Market Systems Expert (one position): The Business and Market Systems Expert must have a master’s level degree in one of the agricultural sciences such as agriculture economics, or an MBA, or another relevant graduate degree. S/he must have a minimum of five years of experience in the development and implementation of agriculture value chain activities. S/he must have a demonstrated experience in business strengthening and value chains. GESI Expert (one position): The Gender and Social Inclusion expert must have a minimum of Master’s Degree in social science or relevant subject area, and at least 5 years’ of relevant experience. The GESI expert must have an extensive knowledge about the GESI issues in Nepal. The GESI expert will work with the evaluation team to develop an evaluation methodology to assess how the project has affected women and other vulnerable groups. Field Researchers: The evaluation team will include Nepali Field Researchers with a bachelor’s degree or equivalent, preferably in agriculture development studies, from a recognized national or international university with at least three years of experience working in the development sector, preferably with a focus on agriculture; OR a Bachelor’s degree with at least eight years of experience. They must have experience working with international organizations and stakeholders; knowledge of Tharu & Abadhi local languages, excellent oral and written communication skills in English; and demonstrable skills utilizing software applications such as Microsoft Excel, Word, PowerPoint, etc. 6) KEY DOCUMENTS FOR REVIEW ● Annual Work Plans ● MEL plans ● Annual and Quarterly Progress Reports ● GFSS Country Plan ● Baseline Report USAID.GOV USAID KISAN II EVALUATION REPORT | 44 ● Monitoring data ● Other documents as recommended by USAID and KISAN II 7) KEY STAKEHOLDERS TO BE CONSULTED ● KII’s beneficiaries ● Private sector partners (including those with grants and MOUs) ● GON officials (Federal MOALD, DOA, Provincial ministries, municipalities) ● Implementing partners, sub-partners, and other USAID-funded Activities in the region ● Other relevant donors ● Relevant NGOs and INGOs ● Banks and other financial institutions ● USAID/Nepal staff ● USAID/WDC staff 8) TIMELINE FOR EVALUATION The total Level of Effort for the evaluation will be approximately three months spread over an six￾month period. The following is a tentative timeline for the evaluation tasks; the detailed timeline will be developed during the team planning meeting and as part of finalizing the evaluation design, in consultation with USAID. 45 | USAID KISAN II EVALUATION REPORT USAID.GOV SN TASKS WEEK OCT ‘21 NOV ‘21 DEC ‘21 JAN ‘22 FEB ‘22 MAR ‘22 1 Sign Consulting Agreements x 2 Desk review x x 3 ET develops and submits the evaluation design report (evaluation instruments, workplan, design) x x 4 USAID reviews and provides comments on the evaluation design report x 5 ET updates evaluation design report based on USAID’s comments and submits final evaluation design report x x 6 USAID in-brief, KISAN II project briefing and team planning x 7 Key Informant Interview with USAID staff, KISAN II staff and key stakeholders in Kathmandu; pilot evaluation instruments x x 8 Roll out of evaluation KIIs and FGDs utilizing evaluation sub-teams led by TL in sample districts x x x x 9 Deliver a presentation of preliminary findings & conclusions to USAID and KISAN II (out brief) x x 10 Translation and transcription of qualitative data collection x x x 11 Data analysis and write draft evaluation report x x x x 12 Analysis and findings / validation workshop x x x x 13 Incorporate feedback from workshop into draft evaluation report; submit report to USAID x x 14 USAID reviews and provides feedback on draft evaluation report (1 round) x USAID.GOV USAID KISAN II EVALUATION REPORT | 46 SN TASKS WEEK OCT ‘21 NOV ‘21 DEC ‘21 JAN ‘22 FEB ‘22 MAR ‘22 15 Finalize report within 10 working days (2 weeks) of receipt of written comments from USAID x x 16 Project closeout and submit final evaluation report to the Development Experience Clearinghouse (DEC) x 47 | USAID KISAN II EVALUATION REPORT USAID.GOV ANNEX B: EVALUATION TEAM Consulting Team ROLE NAME RESPONSIBILITIES QUALIFICATIONS Team Leader David Rinck Oversee the evaluation and coordinate the team, ensure team members understand their roles and responsibilities, and assign individual data collection, analysis, and reporting responsibilities; develop evaluation design to address RQs, including information on data collection and analysis; oversee fieldwork and data collection activities to ensure quality; assume leadership in the preparation, editing and submission of the final report; lead the preparation and presentation of the key evaluation findings and recommendations to USAID; coordinate with SI’s HQ team throughout the evaluation; and brief USAID on the evaluation progress. Mr. Rinck has over 25 years of international assistance experience specializing in agriculture and agronomy, small and micro enterprise development and finance, market and trade development, and project development and evaluation. He further has immense familiarity working in Asia and on USAID projects, having served as a team leader thirteen times. Mr. Rinck holds a bachelor’s degree in agricultural economics from the University of California at Davis and a master’s degree in economics from the University of Chicago. He has also studied economics, in numerous locations around the world including Beijing, Sana’a, Beirut and Damascus. Agriculture Expert Gana Pati Ojha Provide technical guidance and strategic support to the evaluation team leader; assist in the process of drafting and finalizing the evaluation design, data collection instruments, data analysis plan, data collection plan, workplan, data collection tools, briefings, and the evaluation report; assist in crafting briefings and drafting the presentations and the evaluation report; and coordinate with SI’s HQ team throughout the evaluation. Mr. Ojha has 30 years’ experience in development cooperation programs with national and international agencies. He has conducted over 30 evaluations of various types either as a team leader/sole evaluator or a team member. He is a resident of Nepal and is proficient in native Nepal languages and English. He also worked both directly and indirectly with USAID over the course of his professional career six times. He holds a master’s degree in agricultural and extension education from the University of Maryland and a PhD in extension education and development management from the University of the Philippines. USAID.GOV USAID KISAN II EVALUATION REPORT | 48 ROLE NAME RESPONSIBILITIES QUALIFICATIONS Business and Market Systems Expert Bibek Luitel Provide subject matter expertise in developing and implementing agriculture value chain activities; deliver technical inputs in the preparation of reports, briefings, and presentations for USAID/Nepal; support the TL, agriculture sector specialist, and other team members in conducting analysis of agriculture value chains; and provide support in the drafting and editing of the final report, as needed. Mr. Luitel has interdisciplinary social research experience in impact and performance evaluation of development intervention, agriculture sector research, monitoring and evaluation of development intervention, business development, and value chain analysis. He is a resident of Nepal and is proficient in native Nepal languages and English. Additionally, he holds a Master of Business Studies degree in management and business from the Tribhuvan University in Nepal. Gender and Social Inclusion Expert Rita Khatiwada Provide subject matter expertise in GESI topics in the Nepali context; deliver technical inputs in the preparation of reports, briefings, and presentations for USAID/Nepal; support the TL, agriculture sector specialist, and other team members in conducting gender and social inclusion analysis; provide support in the drafting and editing of the final report as needed. Ms. Khatiwada has experience in the field of GESI, research, policy analysis, Monitoring, Evaluation, and Learning, project management, and coordination with various stakeholders. She has carried out several baseline studies, research, and evaluations as a GESI specialist or evaluation expert. She is a resident of Nepal and is proficient in native Nepal languages and English. Rita also worked both directly and indirectly with USAID over the course of her professional career three times. Additionally, Rita holds a master’s degree in peace and conflict studies from the Tokyo University of Foreign Studies. Field Researcher Bandana Sinha Meet with the SI ET at the designated sites at the times communicated by the TL; provide Maitheli-English or Nepali￾English interpretation for SI employees and consultants in KIIs/FGDs with local relevant stakeholders; support SI in conducting data collection during the fieldwork, including notetaking and team debriefs; and if required by SI, contribute to preliminary data analysis and note synthesis. Ms. Sinha has over 12 years of program management and implementation experience, thorough technical experience in facilitation, negotiation, and monitoring /evaluation, and experience translating documents and transcripts into English. She is a resident of Nepal and is proficient in Tharu, Awadhi, and English. Finally, she is the recipient of a master’s degree in conflict, peace, and development studies from the Tribhuvan University in Nepal. 49 | USAID KISAN II EVALUATION REPORT USAID.GOV ROLE NAME RESPONSIBILITIES QUALIFICATIONS Field Researcher Thakur Amgai Meet with the SI ET at the designated sites at the times communicated by the Team Leader; provide Maitheli -English or Nepali-English interpretation for SI employees and consultants in FGDs/KIIs; support SI in conducting data collection during the fieldwork, including notetaking and team debriefs; and if required by SI, contribute to preliminary data analysis and note synthesis. Mr. Amgai has over 20 years of experience in writing, research assistance and research communication, and communication training. He is a research assistant, interpreter and journalist collecting people's stories and interviewing them on the field and on their desks. He trained as a journalist at Media Point Nepal, Mr. Amgai was involved in the design and delivery of compact 'Story Packaging' training to train field staff of development projects to collect stories of successes and challenges from the field and present them as text and/or multi-media content. Amgai has his bylines published at national and international magazines including Newsweek and thethirdpole.com Management Team ROLE NAME RESPONSIBILITIES Project Director Rachel Santos Provide quality assurance for key deliverables, Oversight of progress against work plan, milestones, deliverables, and contract compliance. Provide technical support to the ET on qualitative analysis methods. Project Manager Rashmi Bhat Provide day-to-day oversight of progress against work plan, milestones, and deliverables, ensure contract compliance; oversee travel mobilization and fieldwork logistics before fieldwork and monitored progress during fieldwork; serve as main HQ POC for the ET during fieldwork. Quantitative Expert Benjamin Linkow Oversee and advise on secondary quantitative data analysis. Project Assistant Aaron Ferguson Provide administrative support including consultant contract and invoicing; provided support as requested with other tasks including contract administration, copy editing, and other tasks as needed. USAID.GOV USAID KISAN II EVALUATION REPORT | 50 ANNEX C: DATA COLLECTION TOOLS Informed Consent Thank you for taking the time to meet with us today. My name is [NAME]. I am a researcher from a company called Social Impact, an international consulting company with its headquarters in Virginia, USA that works to improve the effectiveness of development projects around the world through evaluation, capacity building and strategic planning. SI has been asked by USAID to conduct an independent evaluation of a project called Knowledge-Based Integrated Sustainable Agriculture in Nepal II, or KISAN II for short. The objective of this project is to help farming households and agricultural firms to become more productive and profitable by strengthening market linkages for their products and helping them to think more commercially about their activities. This evaluation seeks to assess the effectiveness of the project in achieving this. As independent evaluators, SI does not represent the USAID, the government of Nepal or the implementor of KISAN II (Winrock International). We selected you for an interview because we think your experience with KISAN II activities can help us to understand how effective the project has been and how to improve it. I would like to request that you read (or I will read to you) this Consent Form, to make sure that you understand this evaluation before you decide if you want to participate or not. After I have introduced this evaluation to you and have gone through what is expected of you, and if you agree to participate, I will sign this form indicating your consent. We can give you a copy of this form if you would like. Please ask us to explain anything in this form that you may not understand. Information about the evaluation If you agree to be part of this evaluation, we are going to ask you and other key informants about KISAN II activities that you may have experience with. We would like to have your perceptions of the usefulness of these activities and how they could be improved to achieve more better results. We plan to conduct about 50 interviews similar to this one across the locations where KISAN II worked. The interview is expected to take about 60 minutes. Your rights You have the right to refuse to answer any questions or to stop the interview at any time. You have the right to talk about this evaluation to whomever you choose. Accepting to participate or ending your participation will not affect your working relationship with KISAN II or USAID. Possible risks We do not anticipate any significant risks to you or your organization/facility because of your participation in this interview. However, please note that should you choose to participate in this interview, you will be taking time away from your regular activities, which may affect your routine tasks. Also, some aspects of the interview might affect your feelings regarding your work. Possible benefits The results of this evaluation are expected to inform USAID planning and decision-making and improve strategies for better projects in the future. Your participation in this evaluation will therefore be beneficial to USAID for developing similar programs in the future. By participating in this interview, you will, however, get no immediate and direct personal benefit and there is no guarantee that you will be involved in the activities of future projects. If you decide not to participate in this evaluation 51 | USAID KISAN II EVALUATION REPORT USAID.GOV This interview is voluntary, and you may choose to end your participation at any time. If you decide not to participate, we will accept your decision. Your relationship with KISAN II, USAID or other organizations that provide similar services will not be affected in any way. Confidentiality We would like to record your interview to help us transcribe what you said accurately, but you may choose to participate and not be recorded, with no consequence. If you allow us to record this interview, the recordings will be uploaded to a password protected site and will not be made public. All information you share will be kept confidential and will not be disclosed to anyone in a way that can be linked to you. We will share the opinions you give us in a report to other entities outside of our team, but your answers will be combined with those from other interviews, so nothing you share can be directly linked back to our conversation today. We will not record your name in our data collection tools or notes, or in this consent form. We will also not indicate your name in the any of the reports we prepare, but only your official designation and place of work in an annex to the evaluation report. Please note that (If your official designation is only one of its kind, mentioning it in the annex amounts to identifying you as a participant in this evaluation.) We will not share the information you provide with your peers, supervisors, or friends. Protection from COVID-19 – Only for in-person interviews Given the COVID-19 pandemic, we plan to use safety protocols when we talk such as maintaining a distance of six feet during interviews, wearing face masks, utilizing well-ventilated areas, and using hand sanitizer before, during, and after interviews. If you are worried that these measures are not possible or you cannot adhere to these requests, the study team can contact you remotely for an interview instead. If you or someone in your household or workplace has been feeling sick including having a cough or high temperature in the past two weeks, we would ask you not to participate in this interview face-to-face for your safety and the safety of others. If you have any questions about this evaluation: If you have any questions about this evaluation, you may contact X at Email or phone number. You can also contact the Social Impact Internal Review Board. The contact person is Leah Ghoston; The address is: 2300 Clarendon Blvd, Suite 1000, Arlington, VA 22201; phone number 703-465-1884; email address: irb@socialimpact.com. Do you have any questions before we start? By saying “yes,” and participating in this study, you are indicating that you have heard this consent statement, had an opportunity to ask any questions about your participation, and voluntarily consent to participate. Will you participate in this interview? You may answer yes or no. ◻ Yes, I will participate ◻ No, I will not participate Are you okay with us recording the interview? You may answer yes or no. ◻ Yes, I am okay with recording the interview ◻ No, I am not okay with recording the interview USAID.GOV USAID KISAN II EVALUATION REPORT | 52 KII Guide – USAID Staff Interview Date: Interviewer: Respondent Name: Respondent Organization: Respondent Job Title: BG Informant background (project experiences?) EQ1. Is KISAN II on track to achieve its purpose with respect to - 1.1. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth, and marginalized communities? 1.2. The increased incomes used in the consumption of nutritious food by the household members? RQ1 1. How is KISAN II progress toward improving access and use of improved agriculture technologies and practices measured? How has the project performed against objectives? RQ1 2. What barriers have been encountered to improving access and use to these technologies and practices? How were these addressed? RQ1 3. What evidence exists as to the impact of KISAN II activities on nutrition of beneficiary populations? What approaches have been most successful? EQ2. In what ways has KISAN II’s PSE model benefitted farmers and private sector actors? In particular: 2.1. What do the private sector and farmers perceive to be the incentives adopting the PSE model? 2.2. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? RQ2 1. How have private sector partners (grantees) reacted to the PSE models (for example, the rice miller model, the agrovet model, and the trader and cooperatives models)? What benefits and what challenges have they reported (probe for both positive and negative reactions)? What challenges have impeded the adoption of these models and how were these addressed? RQ2 2. How have market system actors changed their behavior since being exposed to these PSE models (probe for success stories)? EQ3. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? RQ3 1. What changes has the GON made to integrate the PSE models (for example, the rice miller model, the agrovet model, and the trader and cooperatives models) into its own agriculture programming (probe around challenges to integration and examples)? EQ4. To what extent has KISAN II improved the enabling environment for agricultural systems development, specifically for private sector engagement, and institutionalizing the JSR Platform into Ministry of Agriculture and Livestock Development? 53 | USAID KISAN II EVALUATION REPORT USAID.GOV RQ 4 1. What successes has KISAN II had in improving the enabling environment for agricultural systems development (probe around the functions and effectiveness of the JSP platform in enhancing MOALD operations and policies)? EQ5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? RQ5 1. Which approaches (management practices and technologies) has KISAN II applied to increase access to markets and resources by women / DAGs? RQ5 2. Which approaches have worked best to increase access to markets and resources by women / DAGs? What challenges have arisen and how were they addressed? RQ5 3. What are the most significant changes that have resulted from these approaches (probe around potential changes)? RQ5 4. Based on lessons learned during KISAN II, how do you think similar projects in the future could best support women and DAG to increase access to markets and resources? KII GUIDE – Implementing Partner STAFF Interview Date: Interviewer: Respondent Name: Respondent Organization: Respondent Job Title: BG Informant background (project experiences?) RQ 1. Is KISAN II on track to achieve its purpose with respect to - 1.1. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth, and marginalized communities? 1.2. The increased incomes used in the consumption of nutritious food by the household members? RQ1. How is KISAN II progress toward improving access and use of improved agriculture technologies and practices measured? How has the project performed against objectives? RQ2. What barriers have been encountered to improving access and use to these technologies and practices? RQ3. What evidence exists as to the impact of KISAN II activities on nutrition of beneficiary populations? RQ 2. In what ways has KISAN II’s PSE model benefitted farmers and private sector actors? In particular: 2.1 What do the private sector and farmers perceive to be the incentives adopting the PSE model? 2.2 How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? USAID.GOV USAID KISAN II EVALUATION REPORT | 54 RQ2 1. How have private sector partners (grantees) reacted to the PSE models (for example, the rice miller model, the agrovet model, and the trader and cooperatives models)? What benefits and what challenges have they reported (probe for both positive and negative reactions)? What challenges have impeded the adoption of these models and how were these addressed? RQ2 2. How have market system actors changed their behavior since being exposed to these PSE models (probe for success stories)? RQ 3. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? RQ3 1. What changes has the GON made to integrate the PSE models into its own agriculture programming (probe around challenges to integration)? RQ 4. To what extent has KISAN II improved the enabling environment for agricultural systems development, specifically for private sector engagement, and institutionalizing the Joint Sector Review Platform into Ministry of Agriculture and Livestock Development? RQ 4 1. What changes has the GON made to integrate the PSE models (for example, the rice miller model, the agrovet model, and the trader and cooperatives models) into its own agriculture programming (probe around challenges to integration and examples)? RQ 5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? RQ5 1. Which approaches (management practices and technologies) has KISAN II applied to increase access to markets and resources by women / DAGs? RQ5 2. Which approaches have worked best to increase access to markets and resources by women / DAGs? What challenges have arisen? RQ5 3. What are the most significant changes that have resulted from these approaches (probe around potential changes)? RQ5 4. How Based on lessons learned during KISAN II, how do you think similar projects in the future could best support women and DAG to increase access to markets and resources? KII GUIDE – GON Partner STAFF Interview Date: Interviewer: Respondent Name: Respondent Organization: Respondent Job Title: BG Informant background (project experiences?) RQ 1. Is KISAN II on track to achieve its purpose with respect to - 1.1. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth, and marginalized communities? 55 | USAID KISAN II EVALUATION REPORT USAID.GOV 1.2. The increased incomes used in the consumption of nutritious food by the household members? EQ1 1. How well has KISAN II performed in terms of improving access and use of improved agriculture technologies and practices? Which activities have been most effective? EQ1 2. What barriers have been encountered to improving access and use to these technologies and practices? EQ1 3. What evidence exists as to the impact of KISAN II activities on nutrition of beneficiary populations? RQ 2. In what ways has KISAN II’s Private Sector Engagement model benefitted farmers and private sector actors? In particular: 1.3. What do the private sector and farmers perceive to be the incentives adopting the PSE model? 1.4. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? RQ2 1. How have private sector partners (grantees) reacted to the PSE models (for example, the rice miller model, the agrovet model, and the trader and cooperatives models)? What benefits and what challenges have they reported (probe for both positive and negative reactions)? What challenges have impeded the adoption of these models and how were these addressed? RQ2 2. How have market system actors changed their behavior since being exposed to these PSE models (probe for success stories)? RQ 3. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? RQ 3.1. What changes has the GON made to integrate the PSE models into its own agriculture programming (probe around challenges to integration)? RQ 4. To what extent has KISAN II improved the enabling environment for agricultural systems development, specifically for private sector engagement, and institutionalizing the Joint Sector Review Platform into Ministry of Agriculture and Livestock Development? RQ 4 1. What successes has KISAN II had in improving the enabling environment for agricultural systems development (probe around the functions and effectiveness of the JSP platform in enhancing MOALD operations and policies)? RQ 5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? RQ5 1. Which approaches (management practices and technologies) has KISAN II applied to increase access to markets and resources by women / DAGs? RQ5 2. Which approaches have worked best to increase access to markets and resources by women / DAGs? What challenges have arisen? RQ5 3. What are the most significant changes that have resulted from these approaches (probe around potential changes)? USAID.GOV USAID KISAN II EVALUATION REPORT | 56 RQ5 4. How Based on lessons learned during KISAN II, how do you think similar projects in the future could best support women and DAG to increase access to markets and resources? KII GUIDE – Grantee Firm Staff and Intermediary STAFF Interview Date: Interviewer: Respondent Name: Respondent Organization: Respondent Job Title: BG Informant background (project experiences?) RQ 2. In what ways has KISAN II’s Private Sector Engagement model benefitted farmers and private sector actors? In particular: 2.1 What do the private sector and farmers perceive to be the incentives adopting the PSE model? 2.2 How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? RQ2 1. How beneficial were the PSE models (cite relevant model)? What benefits and what challenges have you experienced (probe for both positive and negative impacts)? RQ2 2. Will you continue to use this model (cite relevant model) (probe around how it will be adapted or not)? RQ2 3. How have other market actors (suggest actors, for example finance, input supply, buyers, suppliers) changed their behavior since you adopted this model (cite relevant model)? EQ5: How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? EQ5 1. Which approaches (management practices and technologies) has KISAN II introduced to your firm that have changed access to markets and resources by women / DAGs? EQ5 2. Which approaches have worked best to increase access to markets and resources by women / DAGs? What challenges have arisen and how were they addressed? EQ5 3. What are the most significant changes that have resulted from these approaches (probe around potential changes in terms of access by women / DAGs)? EQ5 4. How do you think similar projects in the future could best support women and DAGs to increase access to markets and resources for businesses such as yours? FGD GUIDE – Beneficiary Groups Interview Date: Interviewer: Respondent Location: Respondent Description (project experiences, gender/DAG?): BG Informant background (if any notable aspects) 57 | USAID KISAN II EVALUATION REPORT USAID.GOV EQ1. Is KISAN II on track to achieve its purpose with respect to - 1.1. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth, and marginalized communities? 1.2. The increased incomes used in the consumption of nutritious food by the household members? EQ1 1. What changes have you made in your farming systems as a result of (cite relevant activity)? EQ1 2. What investments have you made into your farming systems since (site relevant activity)? EQ1 3. What changes have you experienced in your sales of rice, maize, lentil and/or vegetables and goats (probe around increases in income/increases in production)? EQ1 4. How have these changes affected your household consumption/diet (probe around relevant products)? EQ2. In what ways has KISAN II’s Private Sector Engagement (PSE) model benefitted farmers and private sector actors? In particular: 2.1. What do the private sector and farmers perceive to be the incentives adopting the PSE model? 2.2. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? RQ2 1. How have new technologies have you adopted since working with KISAN II (site relevant activity)? RQ2 2. What benefits and what challenges have you experienced (probe for both positive and negative reactions)? RQ2 3. Will you continue to use these technologies in the future (probe around how relevant technologies and methodologies will be adapted or not)? RQ2 4. Has your relationship with other market actors (suggest actors, for example finance, input supply, buyers, suppliers) changed their behavior since working with KISAN II? EQ5. How has KISAN II contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? EQ5 1. How have (cite relevant management practices and technologies) changed access to markets and resources by women / DAGs? EQ5 2. What has most changed access in access markets and resources by women / DAGs? What challenges have arisen (probe around specific activities)? EQ5 3. Have there been any negative changes that have resulted from these approaches (probe around potential changes)? EQ5 4. How do you think similar projects in the future could best support women and DAG to better access to markets and resources? DO GUIDE – Beneficiary Households and Farms Site Visit Date: Interviewer: Site Location: Site Description (crops? area? gender/DAG?): USAID.GOV USAID KISAN II EVALUATION REPORT | 58 Instructions: Ask for consent to visit household and farm site. Let owner(s) know that you will be taking notes and photographs to document your observation. Note whether the following crops are present. If yes, ask if the farming of these has changed over the course of KISAN II activities and how. Has area planted and/or productivity expanded? Has income from this crop expanded? Has consumption expanded? Crops No Yes (Change) Rice Maize Lentil Goat Notes: If present, ask if the use of inputs (fertilizers, seeds, etc.) has changed. If goats are present, has the use of veterinary supplies/inputs has changed. How has the change in inputs affected production? Crops No Yes (Change) Rice Maize Lentil Goat Notes: What post-harvest activities are present? Does the farmer have processing and/or storage sites on the farm or in the household? Has the processing and/or storage changed over the course of KISAN II activities and how? Does this processing and/or storage increase income to the household? Post-Harvest No Yes Processing Storage 59 | USAID KISAN II EVALUATION REPORT USAID.GOV Notes: USAID.GOV USAID KISAN II EVALUATION REPORT | 60 ANNEX D: ILLUSTRATIVE LIST OF RESPONDENTS Donor and Implementing Partner Staff Organization Designation CEAPRED Executive Director DEPROSC Executive Direct & Director Digital Green MEL Director, Senior Grant Manager, Country Engagement Manager OSC Executive Vice President Siddhartha Connecting Inc. President USAID COR, KISAN & KISAN II USAID Program Development and Economics Specialists Winrock International ADS JSR Coordinator Winrock International COP, KISAN II Winrock International DCOP, Agriculture Winrock International DCOPs, KISAN II & Agriculture Winrock International Sectoral Directors and Advisors Winrock International Senior & Associate Director, MEL GON Organization Designation Location(s) Agricultural Directorate Representative Karnali Province AKCs Representatives Baitadi District, Sudur Paschim Province Banke District, Lumbini Province Nuwakot District, Bagmati Province Department of Food Technology and Quality Control, MOALD Senior Officer National District Chambers of Commerce and Industry Representatives Baitadi District, Sudur Paschim Province Kailali District, Sudur Paschim Province MOAFTLM Lumbini Province MOALD Chief - Centre for Crop Development and Agro-Biodiversity Conservation (CCDABC), National MOALD Joint Secretary, Agricultural Development National 61 | USAID KISAN II EVALUATION REPORT USAID.GOV GON Organization Designation Location(s) MOALD Joint Secretary, Planning and Development Cooperation Coordination National MOALD Secretary, Agricultural Development National MOALD Senior Agricultural Economist National MOLMAC Karnali Province Sudur Paschim Province Municipal Governments Mayors, Deputy Mayors, Section Officers Sudur Paschim Province: Patan Municipality, Baitadi District Surnaya Rural Municipality, Baitadi District Kailari Municipality, Kailali District Godavari Municipality, Kailai District Lumbini Province: Nepalgunj Sub-metropolitan City, Banke District Kohalpur Municipality, Banke District Karnali Province Likhu Rural Municipality, Nuwakot PMAMP Project Director National Grantees and Intermediaries Organization Type Location(s) Agrovets Sudur Pashchim Province Dashrathchand Muncipality, Baitadi District Kailari Municipality, Kailali District Joshipur Municipality, Kailali District Lumbini Province Khajura Rural Municipality, Banke District USAID.GOV USAID KISAN II EVALUATION REPORT | 62 Grantees and Intermediaries Organization Type Location(s) Karnali Province Lekbesi Municipality, Surkhet District Bagmati Province Likhu Municipality, Nuwakhot District Cooperatives Sudur Pashchim Province Dashrathchand Muncipality, Baitadi District Kailari Municipality, Kailali District Lumbini Province Khajura Rural Municipality, Banke District Nepalgunj Municipality, Banke District Rapti Sonari Municipality, Banke District Karnali Province Lekbesi Municipality, Surkhet District Bheriganga Municipality, Surkhet District Bagmati Province Likhu Municipality, Nuwakhot District Finance and Insurance Institutions Lumbini Province Nepalgunj Muncipality, Banke District Khajura Municipality, Banke District Goat Farms Lumbini Province Khajura Rural Municipality, Banke District Karnali Province Lekbesi Municipality, Surkhet District Mills Sudur Pashchim Province 63 | USAID KISAN II EVALUATION REPORT USAID.GOV Grantees and Intermediaries Organization Type Location(s) Joshipur Municipality, Kailali District Lumbini Province Kohalpur Municipality – Banke District Rapti Sonari Municipality, Banke District Bagmati Province Likhu Municipality, Nuwakhot District Traders Lumbini Province Kohalpur Municipality, Banke District Rapti Sonari Municipality, Banke District Karnali Province Lekbesi Municipality, Surkhet District Barahatal Municipality, Surkhet District Beneficiary FGD Locations Province District Municipality Bagmati Nuwakhot Likhu Panchakanya Rural Karnali Surkhet Birendranagar Bheriganga Barahatal Lumbini Banke Khajura Rural Rapti Sonari Duduwa Janaki Baijanth Kohalpur USAID.GOV USAID KISAN II EVALUATION REPORT | 64 Beneficiary FGD Locations Province District Municipality Sudur Paschim Baitadi Patan Surnaya Kailali Godawari Kailari Bhajani 65 | USAID KISAN II EVALUATION REPORT USAID.GOV ANNEX E: NOTES ON DATA INTERPRETATION Incomparability of farm survey baseline For farm-level indicators, though fiscal year targets are often used as the primary point of comparison, the baseline survey conducted with farmers in 2018 is also used as a reference point to highlight progress over time. Because the baseline needed to be conducted prior to start of the Activity, the KISAN II team developed a methodology to identify catchment areas and farming households who were most likely to eventually be Activity participants. Given the circumstances, this was a reasonable approach, but this also means that the baseline sample is systematically different than the sample for subsequent farm surveys which only included Activity beneficiaries. It’s entirely possible that farmers surveyed in the baseline survey did not end up participating in KISAN II (or perhaps the most remote or subsistence farmers faced barriers to participation, or conversely perhaps the Activity succeeded in targeting these farmers but less vulnerable farmers in the sample did not participate). In either case, this difference in the type of population sample confounds comparisons between baseline and subsequent surveys unless the baseline sample can be limited to farmers who eventually participated in the Activity. Inflation adjustment of monetary indicators Inflation is an additional factor that clouds interpretation of Activity data. For sales related indicators, Activity documentation includes information on protocols for exchange rate conversion from Nepalese Rupees (NPR) to USD but does not discuss corrections for inflation to enable sales figures to be compared across years in net terms. In a country which typically has a significant inflation rate, these corrections are important for understanding the effective increase in average sales per household over time. In the table below, this issue is demonstrated by the comparative percentage change in average sales by commodity per household sales between 2019 and 2021 adjusted and non-adjusted for inflation. Table 12: Change in Average Sales by Commodity Non-Adjusted and Adjusted for Inflation Average Sales per HH in USD/a Change in Average Sales (%) Commodity FY 2 – 2019 FY 4 - 2021 Inflation adjusted Non-inflation adjusted Rice 445 332 -33.5% -25.4% Maize 89 100 0.1% 12.4% Lentil 64 108 50.4% 68.8% Vegetable 331 859 131.2% 159.5% Source: KISAN II Annual Reports. USAID.GOV USAID KISAN II EVALUATION REPORT | 66 ANNEX F: EVALUATION DESIGN MATRIX EVALUATION QUESTION DATA COLLECTION METHODS AND DATA SOURCES INFORMATION TO BE COLLECTED ANALYSIS METHODS 1. Is KISAN II on track to achieve its purpose with respect to: a. Improving the accessibility and use of improved agriculture technologies and practices to increase crop productivity and income for beneficiaries including vulnerable people, women, youth and marginalized communities? b. The increased incomes used in the consumption of nutritious food by the household members? Document Review ● Baseline and MEAL Plan ● Annual reports KIIs and FGDs ● USAID staff KIIs ● Implementing partner staff KIIs ● Federal, Provincial, District, and Municipal GON staff KIIs ● Beneficiary FGDs Direct Observations ● Beneficiary households and farms Secondary Qualitative Data ● Subjective beneficiary opinions of KISAN II methodologies. ● Changes in farming systems ● Changes in farming technologies ● Changes in investments into farming systems ● Changes in sales vs. home consumption patterns ● Changes in household consumption and diets ● Content analysis ● Triangulation ● Trend analysis ● Expert validation 2. In what ways has KISAN II’s Private Sector Engagement (PSE) model benefitted farmers and private sector actors? In particular: a. What do the private sector and farmers perceive to be the incentives adopting the PSE model? b. How have the behaviors for different market system actors – particularly government, farmers, and the private sector actors – changed as a result of KISAN II’s business models? Document Review ● Annual and Quarterly Reports KIIs and FGDs ● USAID staff KIIs ● Implementing partner staff KIIs ● District, Provincial and Municipal GON staff KIIs ● Grantee firm KIIs Direct Observations ● Beneficiary households and farms ● Subjective beneficiary and grantee opinions of KISAN II methodologies. ● Changes in grantee firm operations and investments. ● Content analysis ● Triangulation ● Trend analysis 67 | USAID KISAN II EVALUATION REPORT USAID.GOV EVALUATION QUESTION DATA COLLECTION METHODS AND DATA SOURCES INFORMATION TO BE COLLECTED ANALYSIS METHODS 3. To what extent has KISAN II been able to integrate its PSE model into the GON’s agriculture programming and are there any challenges faced? Document Review ● Annual and Quarterly Reports KIIs and FGDs ● USAID staff KIIs ● Implementing partner staff KIIs ● Federal, Provincial and District GON staff KIIs ● Subjective GON staff and firm grantee opinions of KISAN II methodologies. ● Changes in extension methodologies. ● Content analysis ● Triangulation ● Trend analysis ● Expert validation 4. How has KISAN II’s support creating the enabling environment for agricultural market systems contributed, specifically, to: a. Improving private sector engagement in agriculture? b. Institutionalizing the Joint Sector Review Platform into the Ministry of Agriculture and Livestock Development structure? Document Review ● Annual and Quarterly Reports ● Special reports and press releases (if any) KIIs and FGDs ● USAID staff KIIs ● Implementing partner staff KIIs ● Federal GON staff KIIs ● Grantee firm KIIs ● Subjective GON staff and grantee firm opinions of KISAN II methodologies. ● Changes in policies and procedures. ● Content analysis ● Triangulation ● Trend analysis ● Expert validation 5. How has KII contributed to increasing access to market and productive resources, such as, improved management practices and technologies, and access to finance for marginalized communities in the ZOIs? Document Review ● Annual and Quarterly Reports ● Baseline and MEAL Plan KIIs and FGDs ● USAID staff KIIs ● Implementing partner staff KIIs ● District, Provincial and Municipal GON staff KIIs ● Beneficiary FGDs Direct Observations ● Beneficiary HHs and farms Secondary Qualitative Data ● Subjective beneficiary opinions of KISAN II methodologies. ● Changes in farming systems ● Changes in farming technologies ● Changes in investments into farming systems ● Changes in sales vs. home consumption patterns ● Changes in HH consumption and diets ● Content analysis ● Triangulation ● Trend analysis USAID.GOV USAID KISAN II EVALUATION REPORT | 68 6. ANNEX G: REFERENCES “Annual Survey Report – Summary Tables – KISAN II 2021 – Extrapolated” Full Bright Consultancy (Pvt.) Ltd., 2021. “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021 Bishwakarma, B.K., Uprety, B.R.; Devkota, D. and Devkota T. “Agricultural Functions, Institutions and Policies in the Context of Sectoral Restructuring in Nepal” International Journal of Agricultural Extension and Rural Development Studies. Vol.8, No.2, 2021. Dahal, H; Karki, M; Jackson; and Pandey D “New State Structure and Agriculture Governance: A Case of Service Delivery to Local Farmers in the Eastern Gangetic Plains of Nepal: Agronomy, 2020. “Economic Survey 2020/21.” Government of Nepal Ministry of Finance., 2022. https://mof.gov.np/public/uploads/document/file/1633341980_Economic%20Survey%20(Engslish)%20 2020-21.pdf. “Feed the Future Knowledge-based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Annual Workplan Year 1” Winrock International for USAID, September 27, 2017. “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 3 Annual Report - July 16, 2019 to July 15, 2020” Winrock for USAID. “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID. “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 5 Annual Work Plan - July 16, 2021 to July 15, 2022” Winrock for USAID. “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Provisional Plan for Coordination with the Government of Nepal - July 16, 2021 to July 15, 2022” Winrock International for USAID. “Feed the Future Nepal Knowledge-Based Integrated Sustainable Agriculture in Nepal (KISAN) II Project - Year 4 Trimester 2 Report - November 16, 2020 to March 15, 2021” Winrock International for USAID and “Annual Survey Report” Full Bright Consultancy (Pvt.) Ltd., 2021. “Gender Equality and Social Inclusion (GESI) Action Plan - July 16, 2021 to July 15, 2022” Winrock International for USAID. Ghimire, R.; Joshi, N. and Ghimire S. “Agricultural Extension Services in Nepal: Past, Present and Future” Innovations in Agricultural Extension, Michigan State University, 2021. Johnson, B.F., P. Kilby “Agriculture and Structural Transformation, Economic Strategies in Late Developing Countries” Revue Tiers Monde, vol. 16, issue 63, 699-699, 1975. Johnston, Bruce F. “Agriculture and Structural Transformation in Developing Countries: A Survey of Research.” Journal of Economic Literature 8, no. 2 (1970): 369–404. http://www.jstor.org/stable/2720471. Johnston, Bruce F., and John W. Mellor. “The Role of Agriculture in Economic Development.” The American Economic Review 51, no. 4 (1961): 566–93. http://www.jstor.org/stable/1812786. “KISAN II Performance Indicator Reference Sheet (PIRS)” Winrock International for USAID, November 2020. 69 | USAID KISAN II EVALUATION REPORT USAID.GOV “KISAN II Y4 T2 T3 Provisional Plan for Coordination with the GON” Winrock international 2020. “Knowledge-based Integrated Sustainable Agriculture in Nepal (KISAN) II Project” PPT by Winrock international, 2021. “Mars Indonesia – Service Delivery Model, Case Study Report” The Sustainable Trade Initiative (IDH), June 2015. “Nepal - Feed the Future Zone of Influence Baseline Report” Westat for USAID, December 2013. “Performance Evaluation of USAID/Nepal’s Knowledge-Based Integrated Sustainable Agriculture and Nutrition (KISAN) Project” CAMRIS International for USAID, February 2018. Reardon, Thomas, and C. Peter Timmer. “Five Inter-Linked Transformations in the Asian Agrifood Economy: Food Security Implications.” Global Food Security 3, no. 2 (2014): 108–17. https://doi.org/10.1016/j.gfs.2014.02.001. Rinck, David, Leah Ghoston, Hariyadi and Cininta Pertiwi “MCC Indonesia Green Prosperity Project Sustainable Cacao Partnership Grants Performance Evaluation - Final Evaluation Report” Social Impact, Inc. for Millennium Challenge Corporation, April 2020. The John Ogonowski and Doug Bereuter Farmer-to-Farmer Program.” U.S. Agency for International Development, June 8, 2021. https://www.usaid.gov/what-we-do/agriculture-and-food￾security/supporting-agricultural-capacity-development/john-ogonowski.