1 Final Performance Evaluation of the Kenya Feed the Future Accelerated Value Chain Development (FTF-AVCD) Project EVALUATION REPORT Submission Date: January 12, 2022 Contract Number: 72061521F00003 Activity Start Date and End Date: August 1, 2021 – January 15, 2022 COR Name: Joseph Oyuga Submitted by: LINC LLC 1100 15th St NW, 4th Floor, Washington, DC 20005 Email: info@linclocal.org Website: www.linclocal.org This document was produced for review by the United States Agency for International Development. It was prepared by LINC under the USAID/KEA-EAA IDIQ, Contract #72061521R000015. 2 Contents Acronyms and Abbreviations…………………………………………………….……………….3 Executive Summary……………………………………………………………………………....6 1. Evaluation Purpose...………………………………………………………………………....24 2. Background on the AVCD Activity………………………………………………………….25 2.1 Overview…………………………………………………………….………………25 2.2 Design and Theory of Change……………………………………………………….25 2.3 Intermediate Results…………………………………………………………………26 2.4 Selected Value Chains……………………………………………………………….27 2.5 Geographic Locations………………………………………………………………..29 2.6 Governance and Management Structure……………………………………………..29 3. Evaluation Questions………………………………………………………………………….31 4. Evaluation Methodology……………………………………………………………………....32 4.1 Structure and Timeline……………………………………………………………….32 4.2 Methodological Approach…………………………………………………………...33 4.3 Data Collection Methods and Sample Sizes…………………………………………36 4.4 Selection and Schedule of Field Site Visits………………………………………….38 4.5 Quantitative and Qualitative Data Analysis………………………………………….39 4.6 Data Integration, Preliminary Findings, and Report Writing………………………..40 4.7 Limitations...................................................................................................................40 5. Evaluation Team Composition..................................................................................................41 6. Ethics and Community Considerations......................................................................................41 7. Evaluation Findings and Conclusions........................................................................................42 7.1 Evaluation Question 1..................................................................................................42 7.2 Evaluation Question 2..................................................................................................69 7.3 Evaluation Question 3..................................................................................................85 7.4 Evaluation Question 4..................................................................................................91 7.5 Evaluation Question 5..................................................................................................99 APPENDICES Appendix 1. Performance Evaluation Scope of Work.................................................................106 Appendix 2. Documents and Literature Reviewed......................................................................112 Appendix 3. AVCD Results Framework…………………………………….............................116 Appendix 4. Map of Kenya showing AVCD project area...........................................................117 Appendix 5. Evaluation County Selection Matrix.......................................................................118 Appendix 6. Expanded Evaluation Matrix...................................................................................120 Appendix 7. Schedule for the Evaluation and Site Visits............................................................121 Appendix 8: Evaluation Work Plan.............................................................................................123 Appendix 9. Team Composition and Roles.................................................................................124 Appendix 10. Key Informant Interview Scripts...........................................................................125 Appendix 11. Checklist for Interviews with AVCD staff............................................................153 Appendix 12. Focus Group Discussion Tool...............................................................................156 3 ACRONYMS AND ABBREVIATIONS Acronym Meaning ADGG African Dairy Genetic Gain AHA Animal Health Assistants AI Artificial insemination ASAL Arid and semi-arid lands ASDS Agricultural Sector Development Strategy AVCD Accelerated Value Chain Development BCC Behavior change communication CA Contribution analysis CAADP Comprehensive Africa Agriculture Development Program CBE Case-based evaluation CBNRM Community Based Natural Resource Management CBOs Community based organizations CDCS Country Development Cooperation Strategy CDDS County Dairy Development Strategy CDR Community disease reporter CDVS County Director of Veterinary Services CEC County Executive Committee CECMs County Executive Committee members CGIAR Consultative Group on International Agricultural Research CHV Community health volunteer CHW Community health worker CIP International Potato Center COP Chief of Party CUG Closed user groups CVCA Climate vulnerability and capacity assessment DDS Dietary diversity score DFA Dairy farm assistants DSM Decentralized seed multiplier DTC Drought tolerant crops DVC Dairy value chain DVS Director of Veterinary Services ECF East Coast fever ERC Ethics Review Committee of Kenyatta University FAO The Food and Agriculture Organization of the United Nations FGDs Focus group discussions FIPS Farm Input Promotions Africa FPO Farmers’ producer organizations FTAI Fixed-term artificial insemination FTF Feed the Future FTFMS FTF Monitoring System GDP Gross domestic product HEO Home Economics Officers HR High rainfall 4 ICRISAT International Crops Research Institute for the Semi-Arid Tropics ICT Information and communication technology ILRI International Livestock Research Institute IMPR Internal Midterm Program Review KABS Kenya Animal Bio-surveillance System KALRO Kenya Agricultural and Livestock Research Organization KAVES Kenya Agricultural Value Chains Enterprises Project KCSAP Kenya Climate Smart Agriculture Program KCDMS Kenya Crops and Dairy Market Systems Activity KEPHIS Kenya Plant Health Inspectorate Services KIIs Key informant interviews KIM Kenya Investment Mechanism KLMC Kenya Livestock Marketing Council KVB Kenya Veterinary Board LMA Local market association LMS Livestock Market Systems LMS AA1 Livestock Market Systems Associate Award 1 LMS AA2 Livestock Market Systems Associate Award 2 LMMF Livestock Market Management Fund LVC Livestock value chain MDD Minimum Dietary Diversity M&E Monitoring and evaluation MEL Monitoring, evaluation, and learning MOU Memorandum of understanding MUAC Middle Upper Arm Circumference NACOSTI National Council for Science, Technology and Innovation (Kenya) NGO Non-governmental organization NHPPlus Nutrition and Health Program Plus in Kenya NPCK National Potato Council of Kenya NTA Non-traditional Area NRM Natural resource management ODK Open Data Kit OFSP Orange-fleshed sweet potato OH Outcome harvesting PAC Program advisory committee PBS Population based survey PMC Program management committee PMO Program management offices PMP Program monitoring plan POs Producer Organizations PREG Partnership for Resilience and Economic Growth PRM Participatory rangeland management PVC Potato Value Chain QCA Qualitative comparative analysis RBM Results-based management RCC Results causal chains 5 REGAL-AG Resilience and Economic Growth in the Arid Lands-Accelerated Growth REGAL-IR Resilience and Economic Growth in the Arid Lands-Improving Resilience RESAKSS Regional Strategic Analysis and Knowledge Support System SA Semi-Arid SBCC Social Behavior and Change Communication SMEs Small and medium enterprises TBE Theory-based evaluation TOC Theory of change USAID United States Agency for International Development USD United States dollars VAD Vitamin A deficiency VC Value chain VSLA Village Savings and Loans Association WAO Ward Agricultural Officers WFP World Food Program ZOI Zones of influence 6 EXECUTIVE SUMMARY This report presents the findings and lessons learned from the final performance evaluation of the Kenya Feed the Future Accelerated Value Chain Development (FTF-AVCD) activity. The evaluation was conducted between August 2021-January 2022 with the primary purpose of assessing AVCD’s performance. The evaluation was led by LINC, together with its partner ICF Macro. Activity Background The FTF-AVCD activity was a five-year $38.6 million World Bank buy-in Public International Organization (PIO) field-support grant to a consortium of Consultative Group on International Agricultural Research (CGIAR) Centers, led by the International Livestock Research Institute (ILRI) with the other members of the consortium including International Crops for Research Institute for Semi-Arid Tropics (ICRISAT) and the International Potato Center (CIP). The program was implemented in two phases: Phase 1 was implemented from October 2015 – September 2018 while Phase 2 was implemented from January 2019 – September 2021. The final year was an extension to complete activities disrupted by the COVID-19 pandemic. The program was implemented in collaboration with national and county governments, the private sector, and civil society, including other USAID projects and partners. The theory of change posited that ‘large scale application of technologies and innovations, in a whole value chain approach, will significantly accelerate the development of agricultural value chains, to rapidly increase food and nutrition security, as well as incomes for all players (producers, processors, marketers) and subsequently contribute significantly to inclusive agricultural and economic development. The key interventions included the identification and scaling of agricultural technologies and innovations to strengthen target value chains (VC) while increasing resilience, agricultural productivity, food security, and economic welfare of farmers, producers, and traders. The four target value chains were livestock, dairy, drought tolerant crops (DTC), and root crops. While the entry point was scaling up the application of agricultural technologies and innovations in the four value chains in high rainfall, semi-arid, and arid areas, the program also sought to address the weakest points of the value chains and benefit all value chain players. The program had the following intermediate results: 1. Improved agricultural enabling environment 2. Expanded markets and trade 3. Improved productivity of selected value chains to ensure that technologies and innovations are widely applied by farmers so as to increase productivity sustainably. 4. Improved nutrition-related behaviors 5. Mainstreaming cross-cutting issues: (i) gender and youth (ii) climate change and environment 7 Evaluation Purpose and Questions As AVCD completed Phase 2, USAID commissioned a final performance evaluation with the primary purpose to assess the activity design and implementation and to inform future activities which harness agricultural technology and innovations within the CGIAR and other research organizations. The performance evaluation will provide learning on innovative approaches for partnering with the CGIAR to implement development interventions and on how partnerships with the national and county government and the private sector can contribute to Kenya’s development. The findings of the evaluation will also inform programming within USAID to improve the impact of future agricultural technology and innovation activities and inform and improve policies. The evaluation explored how the CGIAR centers harnessed their technologies and innovations using a whole value chain approach. The evaluation also determined how the design and implementation could inform future projects to better bring new technologies and innovations to market in the agriculture sector that can address hunger and improve nutrition among vulnerable populations. The evaluation addressed the following five main questions with eight sub-questions. 1. What contribution did AVCD make towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status? 2. How effectively did AVCD address FTF cross-cutting issues, including gender, youth, Natural Resource Management, and climate change among others? 3. To what extent was AVCD’s Theory of Change (TOC) validated by the activity? What were the strengths and weaknesses of the TOC? How can the TOC be improved? 4. To what extent were the CGIAR centers successful in leading the implementation of a development program, given their traditional research focus and mandate? What lessons did the CGIAR centers learn and how did they adapt to scale up technologies? 5. To what extent have AVCD’s efforts in supporting business development services, policies and legislations improved institutional governance (i.e., the configuration and behaviors of participating organizations) leading to sustainable outcomes? Evaluation Methods and Limitations The evaluation applied a theory-based evaluation approach using qualitative analysis complemented by quantitative data. Data were collected through different methods, and systematically triangulated across primary and secondary sources, to validate findings and avoid bias in the evaluative judgement. The evaluation was conducted using the following methods:  Desk review of more than 60 relevant documents, including AVCD program documents, monitoring plans and databases, annual reports, evaluations, post-intervention monitoring reports, beneficiary feedback databases, and government policies and strategies.  67 semi-structured interviews with key informants, including AVCD management and staff, CGIAR regional directors, AVCD partners and beneficiaries, other USAID program 8 staff, government counterparts at the national and county levels, and other development partners.  51 focus group discussions with diverse groups of beneficiaries and stakeholders disaggregated by gender.  Site visits to seven counties including two in the North (Marsabit and Isiolo), three in the West (Bungoma, Homa Bay, and Elgeyo Marakwet), and two in the Southeast (Makueni and Taita Taveta). The sites were purposively identified in consultation with USAID to represent the three main clusters and agro-ecological zones, strategic value chains, and duration of interventions. During the visits, the evaluation team met with stakeholders and beneficiaries and observed physical structures and equipment, and the behaviors and interactions between and among different stakeholders. Data collection was consistent with the approach presented in the evaluation design and Inception Report. Appendix 2 lists the program documents that were reviewed, while Table 2 in section 4.3 includes a matrix of the interviews and focus group discussions that were conducted. AVCD was a complex program which operated across more than 20 diverse counties in multiple value chains over six years. As such, analyzing and synthesizing related performance data is a challenge. Exacerbating this challenge was the low ability of participants to recall information, more pointedly for Phase 1 than Phase 2. Further, some of the intervention areas in Northern Kenya are volatile, as was demonstrated when the evaluation team (ET) faced insecurity challenges during the site visits in Marsabit and Isiolo counties; the ET had to make last minute changes to the sites visits and scheduled interviews. Owing to the unavailability of Population Based Survey (PBS) data, the ET abstracted quantitative data from various project sources based on the evaluation matrix and the AVCD performance measures and indicators so that progress towards targets could be documented and analyzed. Quantitative data were drawn from the FTF Monitoring System (FTFMS), AVCD annual reports and monitoring surveys, interviews with selected key informants, and literature review. For all methods, a gender lens was applied to assess the extent to which differential needs, priorities, voices, and vulnerabilities were taken into account in the design, targeting, implementation, and monitoring of activities. Findings Below, the ET summarizes top level findings, but these are not meant to be exhaustive. Full details are found in section 7 of the report. Evaluation Question 1: What contribution did AVCD make towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status? Subquestions: a. To what extent have the technologies and innovations contributed to improved capacity of households, communities, and market systems in areas affected by recurrent crisis to withstand shocks and stresses? 9 b. How successfully have the nutrition activities been layered and integrated with the core technology components of AVCD to achieve stunting reduction and other nutrition results? c. How have community-level nutrition education and behavior change affected dietary diversity and care practices for women and children? The performance evaluation sought to establish whether the activity enabled improved access to knowledge tools to farmers, increased resilience (buying down risks), and improved nutritional status of households. It similarly examined the achievement of the demand side factors: improving market linkages, realigning regulatory policies, improving access to inputs, strengthening business and financial services, and promoting greater diversification. In this regard, the evaluation assessed the progress on indicators and targets from baseline to the end. In line with the theory of change, AVCD introduced various technologies targeting different value chains in different agro-ecological zones to accelerate the development of agricultural value chains and rapidly increase food and nutrition security. In the DTC value chain, AVCD technologies were designed to enhance access to seed varieties for different agro-ecological zones and digital mobile systems for farmer outreach, while in the potato value chain the program introduced new (early generations of) seed potato varieties and good agricultural practices (GAPs) to improve on-farm yield to at least break-even levels. Other interventions included organizing farmers’ producer organizations (FPOs) in marketing and support to private sector players involved in the potato value chain, to increase incomes and improve household nutrition. In livestock areas, AVCD introduced technologies to enhance rangeland resource management, facilitate policy and legislative reform to improve animal health service delivery, introduced an animal disease surveillance system, worked with a private service provider to enhance access to animal health services, and developed a livestock market information system and trained livestock market associations to enhance pastoralists’ access to markets. In dairy, AVCD provided subsidized fixed-term artificial insemination (FTAI) and East Coast Fever (ECF) vaccination to attract private service providers in the project counties and supported the uptake of Brachiaria (signal grass) fodder to increase dairy productivity and provide alternative sources of revenue. The program also introduced the feed assessment tool (FEAST), developed county dairy strategies and provided training to strengthen FPOs. The results show that AVCD exceeded its target of farmers and others reached with improved technologies or management practices by 9%, reaching 485,258 farmers against a target of 444,975. These farmers dedicated approximately 266,000 hectares to the promoted value chains over the five years (28% higher than the target). The drought tolerant crops and root crops value chains contributed 74% of the beneficiaries. The program interventions catalyzed approximately US $26 million in annual sales from producers and firms receiving assistance; this was equivalent to an estimated average of US $5.3 million per year. Consequently, the program generated approximately US $54.40 of incremental sales per beneficiary during the five-year period; from the unit investment of US $76.94, this is equivalent to a return of US $0.71 per dollar invested. AVCD clearly demonstrated that potential exists to improve productivity in non-traditional (pre￾commercial) growing regions with technology, innovation, and behavioral change. All the 10 technologies introduced for the various value chain were relevant for the arid, semi-arid, and high rainfall areas and addressed real and felt needs and challenges. The technologies were effective and efficient in the various value chain and different county contexts. However, the model of technology dissemination constrained adoption and, hence, achievement of outcomes. Some of the technologies, such as the ECF and FTAI in dairy, were not affordable to farmers and, thus, adoption was limited. In the arid and semi-arid lands (ASAL) areas, beneficiaries include government agencies who benefited from support in drafting legislation; others included livestock market associations and private conservancies. Overall, the program performed very well on yield improvement, albeit with variations across and within value chain components. On average, excluding the livestock value chain, AVCD improved baseline yields of potatoes, DTC, and dairy by 5%, 62%, and 75%, respectively. Within the DTC, performance was mixed; sorghum, finger millet, and pigeon pea achieved significantly higher yield than the baseline, between 97% for pigeon pea and 200% for finger millet; pearl millet and green grams recorded lower yields than baseline. However, except dairy, the attained average yields were below the targets set for each value chain by AVCD; the dairy, DTC and potato achieved 175%, 56% and 64% of their respective yield targets. Ideally, increased yield should signal higher output and revenues, but calculation of income was not feasible because AVCD did not collect cost and income data, despite having identified these indicators for monitoring in its monitoring and evaluation (M&E) framework. From program documents and interviews with beneficiaries, however, a clear picture emerged that significant cyclical input and output price fluctuations meant that increased yield and production did not necessarily lead to higher income. Increased sales were oftentimes accompanied by persistent low prices or sharp drops in prices (and demand) within the same production season or project lifecycle. Consequently, AVCD’s contribution to income is inconclusive and, correspondingly, neither can it be concluded that the program contributed meaningfully to poverty reduction at the household level. Nevertheless, we can conclude that AVCD increased output (availability) and diversified production, which are key ingredients for food security and resilience. Finally, the ET examined how far AVCD spurred inclusive agriculture-led growth in the areas in which it intervened. Whereas AVCD made an effort to address the challenge of marketing, especially related to post-harvest supply gluts and input scarcities, through the formation of FPOs and support to agribusiness SMEs, little success was achieved. This was particularly so in terms of influencing market dynamics. The consensus opinion from discussions with various stakeholders was that FPOs are not the panacea for solving the binding marketing constraints in agricultural value chains, which are diverse and sensitive to local contexts. Intervention activities aimed at building household resilience in the areas covered by the program tended to be light touch, short term (typically less than six months), and involved either one-off or a few interactions. The interventions were implemented either partially or on a much smaller / shallower scale than targeted and, hence, were unable to achieve the intended goal of improving the capacity of households, communities, and market systems to withstand shocks and stresses. Such delivery models were unlikely to lead to institutionalization of the interventions. In all value chains, farmers wanted the interventions continued with a strategy of enhanced outreach. Livestock interventions were not able to reach the pastoralists because of the lack of an 11 appropriate implementation mechanism and sufficient interaction with beneficiaries to create impact. Generally, there was a good uptake of nutritious foods and nutrition indicators improved across the value chains and counties. The nutrition intervention was mainly training, which did not influence women’s access and control. Whereas AVCD reached about half the children under two years (6-23 months) with community level nutrition interventions and reached 85% of individuals targeted with nutrition-related professional training, the median Dietary Diversity Scores (DDS) barely improved from baseline values; differences between the median DDS at baseline and for 2017, 2018 and 2020 were minimal. The median DDS of women of reproductive age (WDDS) increased by 10% from 4.28 to 4.72, which was below the recommended minimum value of 5; similarly, the median DDS for Children 0 – 23 months (CDDS) increased by 13%, from 3.67 to 4.16, which was below the Minimum Acceptable Diet (MAD) of 5 for children. The median household DDS increased 11% from 6.1 at baseline to 6.78, which was higher than the 6 food groups considered in HHDS, both at the baseline and subsequent years. The diversity of diets for women of reproductive age and children were therefore inadequate, and, hence, the nutrition intervention ineffective, especially in reducing stunting. Nutrition education and the sharing of recipes were especially successful. They were well received and spoken of highly. Interviews across all the counties visited, however, confirmed that only limited layering or integration of agricultural production and nutrition interventions occurred. The layering of nutrition improved tremendously when the social behavior change communication (SBCC) aided by the cards became the main intervention. Community level nutrition education and behavior change definitely changed the behavior and care practices for women and children, as demonstrated in several nutrition FGDs. Evaluation Question 2: How effectively did AVCD address FTF cross-cutting issues, including gender, youth, Natural Resource Management, climate change among others? Subquestions: a. Which technology dissemination models were appropriate in the different contexts – High Rainfall, Semi-Arid and Arid areas targeted by AVCD? b. To what extent are the promoted technologies widely available in the target areas (i.e., what is the scale of dissemination), and to what extent has dissemination led to spillover effects that include expanded diverse economic opportunities? Gender disaggregated indicators were measured to establish how men, women, boys, and girls accessed and therefore benefited from interventions. Women and youth inclusion in specified activities was high, with targets being almost achieved or exceeded in 2020 for all indicators identified. Approximately 97%, 128%, and 175% of the gender targets were achieved through the livestock, DTC, and root crops value chains, respectively. Targets for youth were similarly exceeded, with 143% and 167% of the targeted youth in root crops and DTC, respectively, participating in activities to increase access to productive economic resources. Generally, more 12 men (54%) were reached than women (46%) and more girls (52%) than boys (48%). Nevertheless, the data provided were patchy and not supported with baseline data, and it was unclear how the targets were arrived at. AVCD targeted its varying agro-ecological zones with different types of technologies and approaches, including: (i) developing value chains in non-traditional and pre-commercial areas, (ii) developing supply systems for seed, animal health products, and services, (iii) leveraging existing infrastructure of both government and other development programs, (iv) individual and collective delivery, (v) a “One-size fits all” approach to value chain development, not aligned with the different ecological zones and local contexts, and (vi) the use of private service providers. AVCD was fairly successful in pushing different types of technologies and innovation across value chains and counties. Post-harvest handling was the most adopted technology, at 45%, followed by crop genetics (improved seeds) and pest and disease management; about 24% of the technologies and practices adopted were on livestock. Furthermore, most adopters of technologies for crops were women (58%); about 19 were youth. At the same time, the ET found that most of the technologies and innovations introduced via AVCD are not widely available in the program areas. In Elgeyo Marakwet, Homa Bay, and Bungoma counties, the ET found that technologies disseminated during Phase 1 were either available to a very limited extent or non￾existent after beneficiary farmers abandoned them after the close of the program. Farmers provided different reasons for disadoption, including unavailability of inputs (particularly seed), high vulnerability to climatic fluctuations, unsuitability to existing production system, incompatibility with farmer preferences, inadequate agronomic and management knowledge, and unaffordability. Some of the technologies, such as three groundnut varieties introduced by ICRISAT, the ECF vaccine and FTAI semen / hormones promoted by ILRI, and groundnut shellers, were not readily available in the local markets. The approach to technology dissemination was largely supply-driven, including direct provision of free or subsidized seed, drugs, vaccines, and services. In most cases and VCs, the program assumed direct procurement roles on behalf of beneficiaries. This was informed by the realization that supply systems for these inputs and services were underdeveloped for the value chains and within the program counties. Complementary interventions sought to build local seed systems through the establishment of community seed banks and support to seed producers. The use of community-based service providers, such as Community-Based Facilitators (CBFs) by ICRISAT, Dairy Farmer Assistants (DFAs) by ILRI, and Community health Volunteers (CHVs) for nutrition proved more effective, and should be expanded in similar programs. The AVCD program components developed strong working relationships with county governments as implementing partners and also tapped into existing programs, both USAID and non-USAID, for faster and easier entry. The overdependence on government structures, which were not adequately resourced and had other competing activities, posed implementation challenges; it worked better in areas where the value chains were less established and had fewer or no programs, but less so in areas with more established value chains and competing programs. 13 Standard technology dissemination and value chain development strategies were applied irrespective of the local context and needs of the beneficiaries. The program ended up promoting value chains and technologies even in areas where there were relatively low interest or priorities; this led to poor uptake and/or high disadoption rates. The ET did not find any evidence that AVCD conducted any studies or consulted with stakeholders and beneficiaries in selecting and prioritizing technologies and practices for different contexts. Moreover, key informants at the counties, including county government officials, reported that effective collaboration with AVCD only materialized after the interventions had already commenced. In high rainfall (HR1) areas, the use of private service providers appeared a more acceptable model of service provision and extension. In the dairy value chain, for instance, the use of private artificial inseminators and animal health assistants worked well in most areas; specifically, the uptake of conventional AI persisted even in counties where county government subsidy had stopped. In the livestock value chain, the Community-based Disease Surveillance (e-surveillance) system showed potential to quickly detect disease outbreaks to enable timely intervention, but was discontinued due to myriad challenges, principally the concern of the national government (the Directorate of Veterinary Services - DVS) that it was competing with the existing Kenya Animal Bio-surveillance System (KABS). Dairy technologies and practices disseminated, such as FTAI and the ECF vaccine, were not only unaffordable to a majority of farmers but also required large investment in the delivery and market infrastructure that were nonexistent in the target counties. For instance, the need for an expensive cold chain, undividable large minimum vial packages, a relatively short lifespan after unpacking, and lack of critical mass of animals to vaccinate made the administration of the ECF vaccine unsustainable, in addition, the few farmers who adopted Bracharia fodder on small parcels of land later opened the farms into open grazing instead of maintaining and harvesting the fodder. However, the introduction of some of the technologies, such as FTAI and Brachiaria, spawned new opportunities that exist to date. For example, public and private service providers trained on the technologies were still providing conventional animal breeding and health services at the time the ET visited selected counties. Similarly, some non-dairy farmers, especially the youth, adopted Brachiaria production, harvesting, packing, and trading as a commercial venture. The program introduced new (early generations of) seed potato varieties and good agricultural practices (GAPs) to improve on-farm yield to at least break-even levels. Others included organizing FPOs in marketing and support to private sector players involved in the potato value chain, resulting in increased incomes and improved household nutrition. Potato farmers were found knowledgeable on the potato varieties and selected the varieties to grow according to their preference. In Taita Taveta County, CIP helped a private firm successfully produce 7,000 cuttings that were distributed to farmers. The firm has since been registered by the Kenya Horticulture Directorate as certified potato seed producer, but limited marketing and low awareness of the technology and preference for old seed potato among farmers constrained its market. 14 Findings from field visits suggest that most potato farmers in Elgeyo Marakwet, Taita Taveta, and Bungoma had moved from the new varieties introduced by AVCD back to their local varieties or abandoned production altogether due to seed supply constraints. In Elgeyo Marakwet, for example, potato farmers returned to existing varieties, such as Shangi, Romano and Nyayo, after exhausting the certified foundation Unica seed provided by AVCD. Whilst the Diffused Light Store (DLS) technology for potato storage was found in all the growing counties, the stores were mostly non-operational at the time of the evaluation because they either were incomplete or lacked sufficient potato volumes to store. In the DTC value chain, ICRISAT introduced KARI Mtama 1, Seredo and EU Red 1 sorgum varieties and high-yielding and fast maturing finger millet varieties in Elgeyo Marakwet but only a few of the farmers interviewed by the ET were still growing them, largely because of unfamiliarity and poor markets. Similarly, groundnut farmers in Soy South Ward (EMC) never adopted new groundnut varieties (Mwangaza and Ndovu) introduced by ICRISAT/Egerton beyond the demo farms, and still grow their local varieties – Cheplambus and Zambia. Groundnut farmers cited poor germination rates, high vulnerability to water stress and excessive rainfall, early sprouting if not harvested promptly, cross-pollination and varietal contamination, and lack of seed as the leading reasons for not adopting the new varieties. Some evidence of spill-over effects of the technologies and practices disseminated were found. Most notable are the nutrition dialogue cards that were adopted/adapted for upscaling by some county governments and development partners, such as UNICEF in Taita Taveta. Evaluation Question 3: To what extent was AVCD’s Theory of Change validated by the activity? What were the strengths and weaknesses of the TOC? How can the TOC be improved? The performance evaluation analyzed the theory of change (TOC), the results causal chains (RCC), and project pathways to assess the validity, relevance, and plausibility of the causal assumptions made. It examined whether core assumptions at program commencement continued to be valid through implementation phases, and any readjustments made. It also sought to understand the effectiveness of the Push-Pull strategies for each value chain component and different county contexts. Related to this was an assessment of whether the expected effects were achieved uniformly across various categories of farming households, and the reasons underlying any variations. The conclusion from the analysis is that the AVCD program TOC was not fully realized. The intervention was big on the push forces, targeting mostly producers rather than all the value chain actors; there was limited application of a whole value chain approach, as anticipated in the TOC and delivery pathways. While there was an appreciable attempt at large scale application of technologies and innovations, the approach to technology dissemination was largely supply￾driven; the production enhancing technologies were not accompanied by deliberate attempts to address service provision and marketing, among others. The program was less effective in developing sufficient “pull” mechanisms and delivery infrastructure early on, including developing strategies for creating a conducive business environment and stimulating the market in tandem with the push interventions. Limited success in catalyzing the pull forces resulted in a 15 program that resembled more the traditional TOC for most agricultural development and nutrition programs than an accelerated whole value chain development program. Despite the fact that AVCD made technologies and innovations available to farmers, the mode of delivery made effective uptake difficult. First, supply markets for agricultural inputs and services are weakly developed in most program areas, which are predominantly non-traditional production and marketing areas. AVCD did not sufficiently appreciate the fact that private sector players, particularly agribusiness SMEs, are risk averse and will only invest where they are certain of sufficient and reliable quantity and quality, that supply chains are effective or at least can be made to work more effectively, and that effective demand exists for the products they are selling to farmers. The assumption that the private sector would quickly follow AVCD into the value chains where it had extended technologies to farmers and created an enabling policy environment was not validated. Some of these issues resulted from incomplete implementation of the delivery pathways. Apart from providing inputs and working on the policies to create an enabling environment, the program did not pay sufficient attention to value chain and market research, including stakeholder mapping and analysis to understand the functioning and potential of existing supply chains. Regardless of the above findings, the ET is cognizant of the fact that the program implementation period in most areas was too short and intermittent, and challenges too numerous, to make for credible evaluation and firm conclusions about the TOC. Historically, technology and innovation uptake and behavioral changes take time and the trajectory of change is more often than not non-linear. While AVCD was implemented for a cumulative six years, disruptions in the aftermath of 2017 general elections during Phase 1, shifting of zones of influence after Phase 1, and cessation of most Phase 2 activities due to COVID-19 meant that AVCD was effectively implemented for no more than two years in most counties and areas. In new Phase 2 counties and value chains, such as in Taita Taveta / Bungoma and DTC / potato, AVCD had been on the ground for less than a year before COVID-19 pandemic halted activities in early 2020. Moreover, intervention activities lasted no more than one year in any single intervention area; for instance, actual interaction with beneficiaries in the DTC and LVC was either for a season (seed distribution and agronomic training) or six months at most. As a result, this evaluation concludes that the AVCD TOC was unproven, but not completely invalidated. Evaluation Question 4: To what extent were the CGIAR centers successful in leading the implementation of a development program, given their traditional research focus and mandate? What lessons did the CGIAR centers learn and how did they adapt to scale up technologies? Subquestions: a. Since CGIAR centers’ core mandate is research and the development of agricultural solutions and the AVCD activity involved scaling the adoption of the technologies, what lessons were learned on the best way of bringing innovations to market? 16 b. Apart from the CGIAR centers, to what extent did the activity involve other private sector actors (private sector engagement), including lessons learned on how to effectively engage the private sector in project activities? The AVCD program was implemented within the domain of international and national research systems so as to anchor research into development. As leading providers of research and knowledge, and having developed a range of technologies and innovations with great potential to contribute to Kenya’s inclusive agricultural sector growth and improved nutrition, the program sought to harness the technologies, innovations, and capacity within the CGIAR system to contribute to Kenya’s development objectives. In this regard, the research institutions involved in its implementation, while having the technical knowledge of agricultural research, may not have had the hands-on experience in implementing a development program. Therefore, the program was reviewed and assessed for implementation to understand adherence to development objectives and intermediate results at field level. The ET finds that the evidence of implementation effectiveness is mixed. In certain areas, the AVCD implementation teams were effective, and in others, they were less effective. A level of interdependence and complementarities needed to be built across various project and implementation components to facilitate technology transfer, adoption, and the development of supporting systems and structures. To its credit, the program leveraged its core institutional strengths and existing institutional infrastructure of the respective CGIAR centers and integrated its activities with county governments and other development agencies in most areas. There was a good level of coordination with county governments, the national government, and other development partners in facilitating the formulation of a number of policy and regulatory frameworks across all four value chains. While most of these had stalled at the legislative or implementation stage, a few had influenced behaviors of selected counties; examples include the potato strategies in Elgeyo Marakwet, Bungoma, and Taita Taveta, and the Kenya Agri-nutrition Strategy. Adaptive change and program flexibility is key to value chain development. The centers proved highly adaptive to changing circumstances and landscapes. In Phase 1, the program learned from the weak intervention at the farmer level and shifted delivery models and implementation strategy in the later part of the intervention. It further narrowed the scope in Phase 2, but with no strategy of reaching outgroups (neighboring farmers not engaged in the project). Also, at the onset of COVID-19, the program quickly adapted and changed strategy by using various digital platforms to reach and support beneficiaries. The centers, however, were less effective in leveraging the internal institutional capital available through the AVCD program, preferring instead to continue with a “disciplinary silo” approach, thereby missing opportunities for building complementarities and drawing from lessons learned by one center, even in locations with multiple value chain components. This also applies to cross-cutting issues like nutrition, climate smart agriculture, and gender and social inclusion. Despite being one of the central pillars of the program, private sector engagement was suboptimal. Except for seed and animal health provision, private sector engagement was limited and peripheral, particularly so with regards to the development of market systems for inputs and output. In some cases, it is unclear to what extent due diligence was performed on prospective 17 private sector partners before engaging or supporting them, and the ET did not find such evidence. As regards the monitoring and evaluation systems, the indicators in the TOC were tracked and aligned with the USAID indicator handbook, which demonstrated a clear linkage between the support provided by AVCD and USAID FTF initiative objectives. The good practices and lessons learned from Phase 1 helped shape the design and implementation of the Phase 2 approach and migration to the MEASURE database platform. In future programming, the ET recommends that more attention be paid to ensuring that databases are accessible and easy to use, have clearly defined variables, have sufficient internal documentation, and are consistent, reliable, and complete. Evaluation Question 5: To what extent have AVCD’s efforts in supporting business development services, policies, and legislations improved institutional governance (i.e. the configuration and behaviors of participating organizations) leading to sustainable outcomes? The evaluation examined the extent to which the program addressed the constraints across the four value chains through policy and institutional support interventions. Issues investigated included how value chain coordination mechanisms (i.e. linkages with county governments), information and power asymmetries, and regulatory and normative institutional arrangements were addressed. The evaluation assessed the level of ownership by county governments of the institutional mechanisms that have been created, in terms of budgetary allocations, staffing, political commitment, and their durability and resilience. The latter examined the extent to which the regulatory and governance mechanisms are inclusive, participatory, and enjoy legitimacy with farmers, and hence, have the potential to transcend the program lifespan. Some of the key findings are summarized here. AVCD promoted a number of policies and legislative frameworks with the aim of creating an enabling environment where increased production by farmers would combine with a concomitant improvement in the policy environment, thus creating vibrant markets and trade in the agricultural inputs and produce supported. AVCD could be said to have been fairly successful in advocating for the development of policy and legislation; however, a majority of these existed only in draft form at the time of the evaluation, because the policy and regulation development processes stalled at the legislative stage - at either the County Assemblies or national Parliament. Counties cited human and financial resource limitations, competing policy priorities by different county government officials, and inadequate political will in the County Assemblies to conclude the processes. AVCD performance in business development services (BDS) was less successful. There were attempts to support the establishment and development of FPOs across the AVCD components. However, as entities expected to promote the production and marketing of agricultural products, most FPOs remain institutionally weak, and in most cases, poorly led and managed. Nonetheless, AVCD’s engagement in policy and institutional development produced notable outputs and outcomes, including: (i) several counties designating AVCD value chains as flagship 18 projects, (ii) counties allocating more resources, including budgets, extension staff, and seconded staff to organizations supported by AVCD, (iii) several counties taking or scaling up AVCD activities through other programs, (iv) the National Agri-nutrition Strategy being adopted for agri-nutrition programming by government and nongovernmental organizations, and (v) Agricultural Development Steering Committees and Coordination Units established in most counties to improve coordination of interventions among development partners. Key Lessons Derivable from the Evaluation Findings The evaluation’s findings provide strong evidence of the performance of the AVCD program. While the program performed admirably well in many areas, certain design, implementation, and management challenges impeded effectiveness in some areas. Below, the ET provides top-level lessons learned for future consideration. Lessons from interventions aimed at contribution towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status 1. Learn from and integrate the vast indigenous knowledge. While new approaches and technologies on managing grazing lands are welcome, there was a lot to learn from the vast knowledge and experience of traditional management practices. Particularly, pay attention to local complexities in designing interventions, for instance related to breaches of pasture grazing norms, charcoal burning, and illegal enclosures. Moreover, there is a need to anchor the guidelines in law for better enforcement. 2. Interventions aimed at building household resilience need to be deliberate, intensive, and collaborative over longer periods of time. Light-touch activities carried out over short durations of time, typically less than six months, and one-off or a few interactions do not lend well to the institutionalization of the intervention approaches. 3. Value chain interventions need to integrate more intentional and robust climate change response, mitigation, and adaptation. Given the diversity in climatic effects, mere introduction of drought tolerant crops is insufficient; rather, applying a broad array of climate-smart agriculture (CSA) technologies and innovation, buttressed with robust institutional arrangements, would help address some of these constraints. Climate vulnerability and capacity assessments (CVCAs) at intervention design stages could help program implementers understand local climatic conditions and constraints, and how they are likely to impact interventions and outcomes. 4. Farmer producer organizations are not the panacea for solving the most pressing and diverse marketing problems in agricultural value chains. While the FPOs achieved success in mobilizing farmers into production and, to a limited extent, coordinating seed procurement, they had not engaged in tangible marketing activities nor developed any marketing strategies or plans at the time of the evaluation. In all the interviews conducted with FPOs and other stakeholders, marketing was cited as the weakest link. County government officials and other stakeholders were in agreement that there is an urgent need to re-think and test different interventions beyond the FPOs to enhance market systems and markets development. Some examples of complementary approaches include geographic organization and clustering of production, hub-and-spoke clustering of 19 farmers around established anchor producers or marketers, first-mile agribusiness enterprises, formation of marketing common interest groups (CIGs), and establishment of innovation and marketing platforms, among others. 5. Program interventions should not be used for technology research and development outside the legal and regulatory frameworks. Several of the promoted technologies were under development or prototypes for further development and proofing. Examples include the groundnut sheller and groundnut seed varieties that were under trial and development during Phase 1 of AVCD and only released at the end of or after the phase. Promising technologies disseminated prematurely can discourage uptake and continued use. 6. For sustainable seed supply systems, it is prudent to engage both the formal and informal systems, including community seed producers. However, seed systems that are too dependent on rain-fed production in the same areas as the intervention are not sustainable in all the USAID ZOIs. Potato, DTC and orange fleshed sweet potato (OFSP) farmers lost foundation seed and seedlings after prolonged drought and excessive rainfall led to widespread crop damage and failures in the implementation areas. Farmers suggested that seed production should be located in areas with more favorable climatic conditions and outside the production areas, and technologies, such as irrigation, be provided to mitigate against climatic fluctuations. In developing the supply systems, it is important to be cognizant of the fact that the legal and regulatory framework is not yet sufficiently formed to support nonconventional seed systems. Such unconventional systems include the production and sale of clean (as opposed to certified) seed, seed sales by community seed banks, and the use of apical cuttings as planting materials for potato. 7. Participation in and benefits from technology and innovation interventions across gender and social groups are dependent on many other factors. Value chain development programs should therefore make a more deliberate effort to integrate gender and social inclusion from design to implementation. For example, it is imperative that project implementers understand that success of common technology interventions on seed, breeding services, or improved management practices are subject to access to land, animals, inputs, income, machinery/equipment, and dedicated support services, among other; these are more often than not owned mainly by men over 35 years old. Gender inclusion is also dependent on the perceived value of the enterprise; men tend to be more interested in commercially valuable enterprises. Although more women than men adopted technologies, only about 50% of married women had access to the income derived from production; women expressed a reluctance to engage in any agricultural production whose proceeds they had limited control over. Therefore, it is imperative to mainstream gender and other cross cutting themes at the beginning of the program, including developing systematic indicator tracking systems. Lessons from nutrition interventions 8. Cultivate and utilize existing government infrastructure early on. AVCD’s strong relationships with county governments helped the nutrition interventions a great deal, especially during the COVID-19 lockdown when government departments of health remained active as they worked on pandemic prevention interventions. Consequently, while other projects stalled, the nutrition intervention continued because the community 20 health volunteers continued their nutrition work as they worked on sensitizing communities on COVID protocols. 9. The three pathways to better nutrition need to be complementary but do not always work in tandem. The production pathway would have been even more impactful had energy and time saving processing technologies been part of the technology package promoted by the program. Although women were trained on production technologies e.g. multi￾cropping, processing e.g. shelling, drying and storing maize in hermetic bags and raised vermin roof granaries, as well as use of energy and hence time saving stoves and water harvesting technologies, in addition to food groups and how to prepare them, they were not facilitated by the program to access these technologies, and did not, therefore, benefit from the knowledge. Where mechanical grain shellers (time saving technologies) were available, quantities of grain produced were inadequate to sustain their use. An action plan on how to empower women from the various value chain contexts beyond training, and looking into ways of increasing access, decision making, time burden, and workload could have enhanced the effectiveness of this empowerment pathway. Access to suitable credit terms, which can be enhanced by collective action, and affirmative action, such as the provision of energy stoves and water storage tanks, could have reduced the time burden and workload for women. 10. Consider affordability. AVCD’s focus on single value chain per beneficiary potentially undermined potential nutrition benefits of the interventions; most households were too poor to afford the recommended foods. Additional support was therefore needed to develop supplemental or complementary value chains, e.g. the establishment of kitchen gardens and creation of agribusiness opportunities. More specifically, in Elgeyo Marakwet County most nutrient-dense food produced (green grams and ground nuts) was sold because growing communities barely ate it owing to its unfamiliarity. Communities growing them did not know how to prepare it. Green grams have poor storability, because of its high susceptibility to weevils, but it also has higher cash generating potential. Farmers sold most green grams because of their poor storability. Field days were held to teach farmers how to prepare the various foods and motivate communities to utilize the nutrient dense DTC grains and legumes. Hermetic bags could be a solution to the weevil problem if they had been promoted. Mothers indicated that most of the required foods were not produced at home or in the farms owing to the semi-aridity of the EMC agro￾ecological zone and had to be bought. Most households were too poor to afford the recommended foods. Other challenges cited were unsuccessful attempts to raise chicken for eggs and meat as well as some crops because of pests and diseases. The mothers recommended that they be supported to establish kitchen gardens and to create agri￾business opportunities for mothers. 11. Take a broader view of nutrition, beyond farming mothers, caregivers, and children. There was limited nutrition outreach to other members of the community, specifically men, who determine resources available for procurement of diverse foods. Educating mothers or caregivers on nutrition while excluding those most responsible for food choices and acquisition was an oversight. One positive example was a gender sensitive intervention that was reported to have been conducted to address a negative gender nutrition intersectionality problem. In Isiolo, naming new born babies is a male gender role and a baby cannot be breast fed before the naming. If the father is away from home at birth, some babies are denied breast milk after birth and made to wait till their fathers’ 21 return to name them. To protect newborn babies from this community from hunger, the program initiated a father-to-father sensitization program, whereby fathers that are involved in harmful practices learn from those that are gender and nutrition conscious. 12. The program and government nutritionists struggled with disciplinary silo syndrome, where coordination of nutrition activities was considered a preserve of trained nutritionists. The tension created between nutritionists and agriculture officers curtailed collaboration and delayed the delivery of nutrition interventions. There is, however, an openness and interest within county governments to integrate the two departments in future nutrition programming, including engaging Home Economists and the agribusiness departments; future programming, therefore, should take this into consideration. 13. Identify nutrition training needs and develop training materials at the beginning of the interventions so that they can be implemented with other activities. According to the nutrition strategy, the impact of increased productivity on diets should be stronger where nutrition education, social and behavior change communications, and women's empowerment interventions are meaningfully incorporated into agricultural interventions. Lessons from intervention on cross cutting issues 14. Cross-cutting issues are equally important and require intentional and well-planned integration/mainstreaming and layering by qualified professionals across implementing centers representing each issue and controlling a realistic percentage of the budget. Enhanced cross disciplinary coordination of plans, intervention, and evaluation could have improved cross-cutting outcomes. 15. Beneficiary targeting is key to successful implementation and needs to be carefully thought through, with evidence, before project launch. According to AVCD annual reports and KIIs with staff, the project initially targeted individuals with interventions, thereby reaching only a few people; the targeting method was revised mid-course to prioritize group-based interventions and greater utilization of county extension machinery. The revision resulted in intervention targets being achieved or exceeded over the rest of the life of the project. 16. Engage specialist resource persons at the beginning of the program to develop common cross-cutting strategies. For instance, a gender and social inclusion strategy would have better guided intervention design and targeting. 17. The potentially high (sunk) cost associated with technology, innovation, or enterprise uptake and / or failure demands enhanced interaction with beneficiaries and highly adaptable evidence-based delivery models. While the program may easily have absorbed any losses incurred from the technologies disseminated, it would be too expensive for farmers and private enterprises were they to invest at market cost. More so, during implementation, program implementers should conduct regular (more than once a year) monitoring of interventions to establish progress in knowledge and practice, share experiences among value chain actors, and apply key lessons to programming. 18. Effective integration of cross-cutting issues requires specific knowledge, skills, and expertise. Studies of value chain, market and social dynamics, and testing existing models before designing interventions and launching activities should inform program and intervention design. The ET did not find evidence that AVCD conducted value chain and market studies or adapted evidence from existing studies in designing interventions 22 and guiding implementation. Various key informants interviewed, both government and non-government partners, and beneficiaries, were categorical that designing and implementing the interventions should have started with a value chain and markets analysis, including analysis of gender and stakeholder differences, to better understand the complex issues relevant to robust intervention design and implementation. Lessons from the theory of change evaluation 19. Theories of change involving the dissemination of technology, innovation, and behavioral change take time to prove and validate. It requires even a longer period of time when it entails a “whole value chain” approach. The term “accelerated” value chain development primarily based on technology and innovation adoption therefore was a misnomer. Moreover, the huge challenges across whole value chains require much more investment in the enabling environment and holistic understanding of context, which are beyond just one program. Perhaps a more parsimonious TOC would have been proven and validated faster. That is, a TOC that tackled challenges specific to certain nodes of the value chain. 20. Comprehensive value chain, market, stakeholder, and organization assessments are key ingredients to effective realization of the TOC. Most of the challenges experienced, especially with the pull forces, would have been anticipated from such analyses. 21. Interventions should first be tested on a smaller scale while being closely monitored. Red flags raised during monitoring must be taken seriously and issues addressed. Key lessons for the CGIAR centers 22. Partnership with county governments and private sector players to directly reach target beneficiaries can enhance acceptance and adoption of technology and innovation. AVCD should be credited for the strong relationships it developed with county governments, and the centers achieved most success in interventions where they actively engaged county governments and other development partners. These partnerships would have been even more effective with greater, repeated, and longer interaction between implementing partners and the beneficiaries. For private sector players, being pre-commercial areas, the duration of engagement needs to be longer and the partnership cultivated early; importantly, the delivery approach should be more commercial oriented. 23. Maintain flexibility. The Centers proved highly adaptive to changing circumstances and landscapes, including shifting some delivery models and implementation strategies in Phase 2 and utilizing digital platforms as a result of the COVID-19 pandemic. This nimble and adaptable approach yielded results and should be encouraged. 24. Linking of the research-policy-practice nexus needed to be stronger for effective intervention design and implementation. In certain instances, the project worked with county governments in the belief that the latter would reach farmers with the interventions; in other instances, the project worked with county governments and farmers, but did not involve farmers sufficiently in the policy making processes. 25. Establish dedicated regional offices to support implementation. While assigning county government focal points and a field support staff helped with coordination and rapid deployment, it was inadequate to effectively support the complex projects. This is particularly true for focal points, who have competing demands and interests from other 23 programs. In addition, project implementers needed an interface between farmers and the county governments to strengthen engagements at that level. 26. Engage competent last-mile farmer-facing implementers with proven specialization and experience working with various value chain communities for direct interface with beneficiaries. As research and technology institutions, the CGIAR centers have limited development programming experience and could have considered delegating some of the direct implementation. The program attempted this through FPOs, but these were only emergent and unlikely to have been effective partners besides serving different purposes and interests. 27. Properly structure implementation partnership and engagement frameworks. While county government collaboration was admirably successful for some VCs and counties, the engagements were not structured. In addition, no capacity assessments were conducted to understand the capabilities of partners before engagement. Nevertheless, it is important to maintain brand identity even in partnerships. In interviews and discussions at the county level, respondents struggled to identify AVCD or the partner centers with most interventions because several other projects, including past and present USAID programs, had supported similar interventions. 28. Work with farmers and other key stakeholders to identify and understand what is best for them. This includes co-generation of intervention and implementation designs. Contexts in which farmers decide whether to adopt a particular technology or not are complex, and are not static. There are a number of considerations to be made by farmers, and a number of factors that affect adoption decisions. Rapid changes, i.e. occasioned by seasonality, local political and economic dynamics, imply that intervention strategies working in one context might not necessarily work in another. 29. The effect of the intervention was felt better where AVCD had repeated engagements with target beneficiaries over an extended period of time. Intervention effects on new technologies have better results where there is a deliberate effort to understand the contexts in which interventions occur, coupled with meaningful engagement over slightly longer periods of time. This led to a narrower approach in Phase 2 where the CGIAR centers worked with only a limited number of wards or groups of farmers. 30. Interventions focusing on policy change are political processes that require time and a deep understanding of the politics around these policies. AVCD supported the development of several bills and strategies, but most of them are yet to be enacted. Similarly, a number of policies and strategies supported at county and national level are yet to be fully operationalized. Two lessons are important in this regard: ● Political Economy Analysis (PEA) and organizational capacity assessments would be necessary to understand what is feasible, in what contexts, and the vested interests that could facilitate or block policy change. ● Sufficient time, resources (financial and technical), and effort should be devoted to promoting policy change. The assumption that policy is technical, and only technocrats need be engaged should be discarded for a more political approach to influencing policy change. 24 Key lessons for policy, organizational and institutional development 31. Pay sufficient attention to the legislative organs at the county and national levels, and engage them early, because they are the critical players in the policy process. 32. Involve communities and key stakeholders in the development of policies early and meaningfully. Inclusive agricultural growth includes inclusive development of the rules of the game. The preferred top-down approach to policy making led to policies having “form” and not “function”, and thus risk rejection by the target beneficiaries. 33. While agriculture is a devolved function and value chain development interventions were implemented at the county level, agriculture policy, institutions, regulation and trade are national government functions. The weak engagement with national government, informed by the notion that the program needed only engage with targeted counties undermined the development of critical policies and influencing national regulatory institutions, such as KEPHIS. National government remains a stumbling block to policy making and enforcement, oftentimes countermanding the county governments and rejecting policies enacted by CAs. 34. Institutional support should be connected to interventions. For instance, while AVCD provided commendable support to the Council of Governors Agriculture Sector Secretariat (CoGASS), more could have been achieved had the support been linked to other AVCD interventions across the value chains. There is no indication that the support was connected, for instance, to the support to the development of policies and strategies at county levels. 35. FPOs need greater support over much longer periods to reach minimum operational competence. Whereas efforts to link FPOs to county government structures are noble, county governments are themselves institutionally weak; furthermore, there are risks of politicizing the management and operations of the FPOs as county political elite seek to exert control over voting blocs. Support to FPOs, therefore, should be decoupled from other engagements with county governments. 1. EVALUATION PURPOSE The USAID/Kenya and East Africa Mission commissioned a final performance evaluation in mid-2021 to assess the effectiveness of the Kenya Feed the Future-Accelerated Value Chain Development (FTF-AVCD) program’s design and implementation. AVCD was, at the time, nearing the end of its five-year period of performance and USAID sought to use the evaluation to capture lessons to inform the design of future activities which harness agricultural technology and innovations within Consultative Group on International Agricultural Research (CGIAR) and other research organizations. The evaluation was also designed to provide learning on innovative approaches of partnering with the CGIAR to implement development interventions, how partnerships with the national and county government and the private sector can contribute to Kenya’s development, and to inform programming within USAID. After a competitive procurement, LINC was selected to lead the evaluation, together with its partner ICF Macro. 25 2. BACKGROUND ON THE AVCD ACTIVITY 2.1 Overview The Feed the Future-Accelerated Value Chain Development (FTF-AVCD) activity was a five￾year, $38.6 million World Bank buy-in field-support grant to a consortium of Consultative Group on International Agricultural Research (CGIAR) Centers led by the International Livestock Research Institute (ILRI). Other consortium members included the International Crops for Research Institute for Semi-Arid Tropics (ICRISAT) and the International Potato Center (CIP). The program was implemented in two phases: Phase 1 was implemented from October 2015 – September 2018 while Phase 2 was implemented from January 2019 – September 2021. The final year was an extension to complete activities disrupted by the COVID-19 pandemic. The key interventions of FTF-AVCD included the identification and scaling of agricultural technologies and innovations to strengthen target value chains while increasing resilience, agricultural productivity, food security, and the economic welfare of farmers, producers, and traders. 2.2 Design and Theory of Change One of the objectives of the USAID/Kenya Country Development Cooperation Strategy (CDCS) 2014-2020 was inclusive, market-driven, and environmentally sustainable growth. Since Kenya is highly dependent on agriculture, increasing inclusive growth in agriculture is essential for increasing household food security, improving nutrition status, and resilience. The USAID/Kenya FTF strategy seeks to contribute to sustainably reducing poverty and hunger in Kenya through support to selected value chains and regions, especially those with higher poverty rates and greater vulnerability to climate change. To achieve this goal, FTF identified inclusive agricultural sector growth and improved nutrition status of women and children as its two strategic objectives. The AVCD theory of change (TOC) posited that “large-scale application of technologies and innovations, in a whole value chain approach, will significantly accelerate the development of agricultural value chains, to rapidly increase food and nutrition security, as well as incomes for all players (producers, processors, marketers) and subsequently contribute significantly to inclusive agricultural and economic development” (see Section 7.3 below for a detailed examination of the TOC). The TOC was formulated as a departure from traditional agricultural development strategies that focused on research, extension, supportive infrastructure, and marketing as separate programs and entities (‘silos’). It sought to integrate research, extension, inputs, and marketing institutions to work better for agricultural development through a more holistic and inclusive approach that takes into account the whole value chain – from technology development to consumption - with greater emphasis on markets and benefits from increased income. AVCD used a two-pronged approach to achieve the goal and development objectives of economic growth, inclusive agricultural growth, and improved nutrition status for women and children. The first front focused on “push” strategies, such as improving access to knowledge 26 tools (technologies and innovations), buying down risk (increasing resilience), improving nutritional status, and enhancing natural resource management. The second front sought to exert a “pull” effect, including improved market linkages, re-aligning business regulatory policies, improving inputs access, strengthening business and financial services, and promoting greater diversification. The push-pull forces were expected to work in tandem to make the value chains competitive and productive by creating value chain players that are more market oriented. The expected result was to catalyze a competitive, productive, market-oriented value chain that would spur farming investments into viable businesses. 2.3 Intermediate Results The program had the following intermediate results: 1. Improved agricultural enabling environment: providing empirical evidence to support policy decisions, supporting producer organizations with management skills and information, and supporting all value chain partners with knowledge-based needs identified through value chain analysis. The program targeted to produce seven policy briefs and support strengthening 27 producer organizations and 18 private sector business enterprises in seed systems within the program period. 2. Expanded markets and trade: The program sought to increase the value and volume of market access of the products in each value chain by at least 15% and the gross margins by 15-30% from baseline by the end of the program period. The specific targets at each value chain level included: a) Build capacity for all farmers and at least a producer organization per target county to utilize marketing information for the specific products. b) Develop ICT-based marketing information system packages and develop capacity for local organizations to maintain and manage the information system for the livestock and dairy value chains. c) Assist farmers and producer organizations to link with regional or international output markets. d) Institute marketing-based interventions to increase livestock sales by 12% and have 30 local market associations (LMAs), traders, or private businesses participating in the livestock value chain. e) Assist farmers to increase total value of sales at farm level by more than 25% from baseline. f) Increase the gross margins from sale of potatoes and sweet potatoes by 30% from baseline. g) Assist farmers to link with processors and traders of the value chain products. 3. Improved productivity of selected value chains: The program sought to ensure that technologies and innovations were widely applied by farmers so as to increase productivity in a sustainable manner. The specific productivity targets for each value chain included: 27 a) The area under improved range management practices and improved livestock technologies will be increased by 15% and 20% respectively in the targeted Feed the Future zones of influence. b) The area under improved crop management practices will increase by 15% in the targeted Feed the Future zones of influence. c) A total of 250,000 farmers and pastoralists will be applying improved technologies and innovations as a result of the program’s interventions. d) Gross margins (per production unit) in the production of the targeted value chains will increase by over 12%. e) The number of households using improved quality seed of the selected drought tolerant crops will increase by 30% and the area under improved varieties will increase by 15%, while the productivity of these crops will increase by 20% from baseline during the program period in the targeted Feed the Future zones of influence. 4. Improved nutrition and related behaviors: improve access and diversity of food among the targeted beneficiaries and improve the quality of food at household and market level by improving storage, processing and cooking techniques in 150,000 households. 5. Cross-cutting issues a) Gender: work with community organizations and households to promote women’s engagement in value chain activities and their control over incomes derived from their participation. Each of the value chains was to conduct comprehensive gender analysis and include issues identified as pertinent in the program activities. b) Climate change and environment: introduce climate-smart technologies and farming practices, in particular, drought and disease tolerant crop varieties in semi-arid areas, livestock technologies, and innovations that enhance the resilience of people and environment, and incorporate environment management practices to ensure sustainability and best management of natural resources. c) Youth: target the youth, especially in enterprise development and the service delivery part of the value chain. 2.4 Selected Value Chains The program was comprised of the livestock value chain, the dairy value chain, the root crops value chain (potato and sweet potato), and drought tolerant crops value chain.  Livestock value chain: According to the national livestock policy, 60% of Kenya’s livestock are found in arid and semi-arid areas, which are also home to approximately 30% of the human population (GOK, 2019; FAO 2021). The land-use system in these areas is predominantly pastoralism and extensive grazing. The region has the highest levels of poverty in the country, affecting more than 70% of the population, exacerbated by poor accessibility and frequent droughts (World Bank, 2018). The impact of frequent droughts is severe hunger, malnutrition, and extensive loss of livelihood, with diminished resilience. Interventions therefore focused on improving the incomes and nutritional status by increasing income sources through increasing access to improved grazing areas, 28 enhancing access to markets, developing a mobile-based marketing information application, and improving disease control and livestock management practices. Others included support for relevant policy and legislative frameworks to create an enabling environment facilitating livestock marketing.  Dairy value chain: Demand for milk and other dairy products is rising in Kenya and East Africa, driven by increasing population, incomes, per capita consumption, and changing dietary patterns. The demand currently outstrips available supply. Dairy farming is predominantly smallholding and the value chain involves many resource-poor marketers and processors. Farmers with 1-3 cows dominate the dairy industry and produce about 80% of the milk in the country, but at very low productivity levels and low hygiene standards. The dairy value chain, therefore, plays a critical role in food security and income generation through direct and indirect consumption. In addition, milk and milk products contribute significantly to nutritional food security by supplying protein, vitamins, and minerals to the diet. Accelerated development of the dairy industry is essential for food security, economic welfare, and improvement of the nutrition status of the population in the country. AVCD’s dairy value chain interventions focused on increasing milk productivity, establishing dairy business hubs to enhance milk marketing, and establishing nine innovation platforms to facilitate the value chain. During phase 2, the dairy value chain components focused on building the capacity of county governments in dairy development, improving advisory services for dairy, and building the capacity of producer organizations to spearhead market-led dairy development.  Roots crops value chain: The program worked on Irish and Sweet potatoes. Sweet potato roots and leaves provide high pro-vitamin A carotenoid at high levels of bio-accessibility within local diets. This includes beta-carotene, which reduces vitamin A deficiency (VAD) in women and young children. Opportunities within the seed potato value chain offer diverse entrepreneurial opportunities for smallholder farmers to large-scale professional farms and businesses, contributing to business development and economic growth. Targeted technologies included the scaling up of seed systems with the potential to double smallholder potato yields from quality seed, and significantly increasing the nutritional quality and economic value of sweet potatoes through improved orange￾fleshed sweet potato (OFSP) varieties.  Drought tolerant crops value chain: Food scarcity and food quality are compounded by a lack of diversity of crops that can be grown in climatically unfavorable environments in both the dryland and HR1 areas. Drought tolerant cereals, such as sorghum, finger and pearl millet, and legumes and pulses (groundnuts, green grams, and pigeon peas) presented a possible solution to food security and improved nutrition in these areas. These crops provided additional benefits for sustainable farming by fixing nitrogen in the soil, which can lead to more sustainable cropping systems for smallholder farmers that cannot afford to use inorganic fertilizers. Additionally, there is a growing market for these cereals and legumes, and their commercialization would help enhance food safety. 29 2.5 Geographic Locations During Phase 1, activities were spread across 103 sub-counties within 21 counties in all Feed the Future Zones of Influence in Kenya. The geographic scope was narrowed to 15 counties in Phase 2 due to: (i) USAID’s greater emphasis on mainstreaming its new policy on the journey to self￾reliance (J2SR), (ii) the need to demonstrate how local entities benefited and that their capacities were built, and (iii) the results of an earlier evaluation that found that programs were thinly spread and, therefore, FTF needed to reduce its footprint and increase depth. Consequently, USAID reduced the Zone of Influence from 25 to 17 in Phase 2, including the 15 that were subsequently selected for Phase 2. Table 1 below shows the counties where each value chain component was implemented, across the Phases. Table 1: Target Value Chains and Counties Value Chains Target Counties Phase I Phase II Dairy Busia, Homa Bay, Kisumu, Kitui, Makueni, Migori, Siaya, Vihiga Makueni, Kisumu, Siaya, Vihiga, Busia, Homa Bay, Migori Livestock Garissa, Isiolo, Marsabit Turkana, Wajir Turkana, Marsabit, Wajir, Isiolo, Garissa Drought tolerant crops Busia, Elgeyo-Marakwet, Kitui, Makueni, Tharaka Nithi, Siaya Kitui, Makueni, Taita Taveta Root Crops Potato Elgeyo Marakwet, Meru, Nandi, Uasin Gishu Bungoma, Taita Taveta OFSP Homa Bay, Migori, Busia, Bungoma 2.6 Governance and Management Structure AVCD was led by ILRI, together with two CGIAR consortium partners, ICRISAT and CIP. The program was implemented in collaboration with national and county governments, the private sector, and civil society. As the lead implementing institution, ILRI served as the fiduciary and had overall responsibility for delivering the program targets and was directly accountable to USAID as the main grantee institution. ILRI hosted and provided all support services to the program management office (PMO) to ensure that all management, monitoring, and evaluation activities were accomplished on time. It allocated office space for the PMO, mobilized support staff, and recruited the program management staff. The PMO, led by the Chief of Party (COP), was tasked with routine management, oversight, and leadership of the program, and was accountable to ILRI management. The COP handled all program management functions while ILRI provided financial management, auxiliary administrative, and other support services. The PMO coordinated the sub grant process and logistics and in-kind support to partners to avoid duplication of effort. However, each center was allowed to subcontract other partners when applicable and be responsible for managing such contracts. There was a program management committee (PMC) comprising the COP, monitoring 30 and evaluation (M&E) officer, the nutrition expert, and the managers of each of the value chains. The PMC was responsible for: 1) Reviewing and standardizing work plans and budgets from the individual value chain components 2) Reviewing and standardizing quarterly progress reports from the individual value chains 3) Planning joint field activities such as field days and baseline surveys 4) Planning joint communication activities 5) Planning cross-cutting issues The PMC met regularly, at least once a month, to conduct management business. The COP was assisted and supported by the M&E officer and a communications officer. The PMC also included a food and nutrition specialist to oversee the implementation of nutrition activities in the value chain components. The COP was responsible for the day-to-day management of the program and the external linkages with USAID and all partners during the program period. The program advisory committee (PAC) provided oversight and quality control to the PMO, including: 1) Reviewing and advising on the performance of the PMO 2) Reviewing progress reports against the set targets and advising on how to enhance effectiveness 3) Advising on follow up linkages with national and country governments and other USAID supported projects 4) Advising on accountability, sustainability, and visibility of the program 5) Assisting the PMO to establish and strengthen linkages with other projects of the participating centers 6) Advising on private sector participation and public-private sector linkages 7) Undertaking any other relevant functions requested by ILRI and USAID Each participating center nominated a member to the PAC, who was at the level of management and capable of making management decisions on behalf of the center. The PAC was comprised of representatives from the following organizations: 1) ILRI (chair) 2) CIP 3) ICRISAT 4) USAID/Kenya Mission 5) A representative of the national government 6) A representative of county governments 7) A representative from the private sector 8) Program manager (secretary) The PAC met regularly as necessary, but not less than once each quarter. 31 Each of the value chain components was managed by a value chain manager recruited by the participating center and seconded to and reporting to the PMO. Each of the components had administrative staff supported by an M&E officer, but financial management services, communication, public relations, and other cross cutting services were provided centrally by the PMO. The value chain manager was expected to perform the following tasks: 1) Participate in the program management committee 2) Be the main link on technical and implementation matters between the PMO and the relevant center 3) Execute program activities and assess progress to ensure that targets are met 4) Ensure effective communication and the necessary synergies with the PMO 5) Ensure that the program is on track and the funds are made available on time 6) Compile and submit all program activities and financial reports as required by the PMO 7) Convene program meetings with implementing partners to review progress towards delivery 8) Provide oversight over the program activities to ensure delivery as required results. Each of the value chain components was to ensure that there was adequate staff to implement program activities at the field level. Value chains with adequate staff seconded existing field staff to the project while those in remote areas, e.g. the livestock value chain, recruited field coordinators in each county, who were either staff of ILRI or recruited by partners. 3. EVALUATION QUESTIONS The performance evaluation was based around the following evaluation questions and sub￾questions provided by USAID: 1. What contribution did AVCD make towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status? ● To what extent have the technologies and innovations contributed to improved capacity of households, communities, and market systems in areas affected by recurrent crisis to withstand shocks and stresses? ● How successfully have the nutrition activities been layered and integrated with the core technology components of AVCD to achieve stunting reduction and other nutrition results? ● How have community-level nutrition education and behavior change affected dietary diversity and care practices for women and children? 2. How effectively did AVCD address FTF cross-cutting issues, including gender, youth, Natural Resource Management, climate change among others? ● Which technology dissemination models were appropriate in the different contexts – High Rainfall, Semi-Arid and Arid areas targeted by AVCD? 32 ● To what extent are the promoted technologies widely available in the target areas (i.e., what is the scale of dissemination), and to what extent has dissemination led to spillover effects that include expanded diverse economic opportunities? ● Cross-cutting issues 3. To what extent was AVCD’s Theory of Change (TOC) validated by the activity? What were the strengths and weaknesses of the TOC? How can the TOC be improved? 4. To what extent were the CGIAR centers successful in leading the implementation of a development program, given their traditional research focus and mandate? What lessons did the CGIAR centers learn and how did they adapt to scale up technologies? ● Since CGIAR centers’ core mandate is research and the development of agricultural solutions and the AVCD activity involved scaling the adoption of the technologies, what lessons were learned on the best way of bringing innovations to market? ● Apart from the CGIAR centers, to what extent did the activity involve other private sector actors (private sector engagement), including lessons learned on how to effectively engage the private sector in project activities? 5. To what extent have AVCD’s efforts in supporting business development services, policies, and legislations improved institutional governance (i.e. the configuration and behaviors of participating organizations) leading to sustainable outcomes? 4. EVALUATION METHODOLOGY The methodology described was intentionally developed to address the evaluation questions listed above, as well as the contextual and geographic factors described in section 2. 4.1 Structure and Timeline The evaluation proceeded according to three distinct phases, elaborated below. Phase 1, Preparation and Inception: This first phase ensured the evaluation plan was responsive to the objectives, coherent, and well-coordinated. The ET held key meetings with USAID, AVCD, and other stakeholders; began the literature review; produced an Inception Report; and conducted a Partner Stakeholder Workshop to introduce the evaluation questions and methodology, finalize tools and the timeline, and secure buy-in and cooperation for the evaluation. This phase concluded after incorporating into the Inception Report the feedback from the Partner Stakeholder Workshop and lasted from August 1, 2021 (contract award) until August 30, 2021 (Final Inception Report submitted). Phase 2, Data Collection: The evaluation used a mixed-methods approach that included secondary data (literature) review, the analysis of quantitative data (obtained from project documents), and qualitative data obtained from project documents, key informant interviews, 33 focus group discussions, and site visits. This phase lasted from approximately September 1 – October 4, 2021. Phase 3, Data Analysis, Report Preparation, and Dissemination: The evaluation team analyzed the collected data, presented initial findings to USAID, incorporated feedback, and prepared the draft final evaluation report and accompanying materials. This phase lasted from October 15, 2021 – December 10, 2021. 4.2 Methodological Approach The core purpose of the evaluation is to help USAID draw inferences regarding whether the AVCD intervention is demonstrably ‘proven’ and whether to replicate it elsewhere or scale it up (either to larger numbers of the same target organizations or to new countries). Methodologically, the first set of questions pertains to internal validity (cause-effect relationships between activities and results) concerns, while the second set speaks to external validity (or generalization and extrapolation). If appropriately integrated, answers to both sets of questions can serve the broader purposes of enhancing learning (so that subsequent decisions regarding an intervention design and implementation are made more prudently) and accountability (so that outcomes can be explained on a firm foundation to intervention recipients, managers, funders, and taxpayers). It is important to make a distinction between performance evaluation (the approach adopted for this assignment) and impact evaluation. According to the guidance provided by the USAID Program Cycle Operational Policy (ADS Chapter 201), performance evaluations encompass a broad range of evaluation methods.1 They often incorporate before–after comparisons but generally lack a rigorously defined counterfactual (i.e. what would have occurred without the intervention, often measured through a comparable non-treatment group). Performance evaluations may address descriptive, normative, and/or cause-and-effect questions. Among the types of evaluation given as examples of performance evaluation include developmental, formative, outcome, and process or implementation evaluation. Outcome evaluation, the type of performance evaluation adopted for this assignment, may be used to assess whether intended outcomes of the program or organizational approach were achieved, but cannot be used to establish causal attribution. To establish a counterfactual and direct attribution, impact evaluations must be carried out. The AVCD program interventions were complex in nature. Evaluating outcomes of complex interventions is often difficult, because the complexity of the interventions is compounded by corresponding the implementation contexts. In essence, finding a clear causal link that can be wholly attributed to one mechanism or activity might not be possible, because there are usually multiple causes for any one observed outcome.2 Single causal mechanisms or processes can similarly contribute to several outcomes. Even when carefully designed, fully supported, and faithfully implemented, complex 1 ADS Chapter 201 Program Cycle Operational Policy, available at https://www.usaid.gov/sites/default/files/documents/201.pdf 2 Theory-Based Approaches to Evaluation: Concepts and Practices (2012) available at http://www.tbs-sct.gc.ca 34 interventions are characterized by the highly variable outcomes they generate over time, space, and groups.3 Complex interventions, therefore, do not lend themselves easily to evaluation approaches that seek to measure the causal effects of an intervention’s activities on outcomes, and to establish a counterfactual for observed change. A more useful approach is to understand the specific steps in a causal chain through which various outputs and outcomes are produced and to test for and verify the presence of the specific causal mechanisms responsible for change, including the role of context. Most evaluations of complex programs tend to prefer theory-based approaches involving either the use of mixed methods (quantitative and qualitative) or qualitative evaluation methods. The incentive for using theory-based evaluation methods derives from the premise that programs are embodiments of theories. First, programs comprise an expectation that the introduction of a program or policy intervention will help ameliorate a recurring problem. Secondly, program designs involve an assumption or set of assumptions about how and why program activities and resources will bring about change for the better.4 In view of the above, this performance evaluation combined a mixed-methods with a theory￾based approach: ● Mixed-method approach to ensure data collection and analysis were informed by a feedback loop combining a deductive approach (drawing from predefined analytical categories) with an inductive approach allowing space for unforeseen issues or lines of inquiry overlooked at the outset; this can also lead to capturing unintended outcomes, negative or positive, of AVCD interventions. ● Theory-based approach to assess AVCD’s contribution to strategic outcomes through reconstructing the Theory of Change (TOC) implicit in the AVCD logical framework and narrative, showing the expected causal relationships between activities, outputs, and strategic outcomes as well as risks and assumptions. The reconstructed TOC was then used to assess the effectiveness of AVCD activities by (i) verifying the TOC internal logic, (ii) measuring the extent to which activities and outputs have been delivered, (iii) assessing whether assumptions have proven valid and risks managed and mitigated, and (iv) looking at trends in outcome indicators. In line with this approach, data were collected through different methods – and systematically triangulated – across primary and secondary sources to validate findings and avoid bias in the evaluative judgement. The full evaluation matrix is found in Appendix 7. Data collection methods applied to this performance evaluation included: Quantitative Approach 3 According to Woolcock (2019), this is a product of how the intervention’s structural characteristics and implementation modality interact with contexts in inherently idiosyncratic ways. 4 Brad Astbury and Frans L. Leeuw, Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation, American Journal of Evaluation 2010 31: 363 DOI: 10.1177/1098214010371972 available at http://aje.sagepub.com/content/31/3/363 35 The ET proposed an approach that would compare targets and actual achievements before and after the AVCD intervention in the value chains and counties, i.e., at the end of the project (Year 5) as well as comparing Year 1 and Year 5 estimates. In this case, the ET had intended to compare beneficiaries and non-beneficiaries, where the beneficiaries are the counties, value chains, households, and enterprises reached with different interventions. For purposes of measuring impact, three outcome scenarios were to be analyzed. First, for each performance indicator, a delineation of the differences within-group (beneficiary and non-beneficiary) and between-group (beneficiary versus non-beneficiary) before and after the intervention. These were to be used to calculate the mean differences within and between groups to assess whether performance improved significantly. In addition to the two main groups, since interventions were not applied uniformly (i.e. unequal treatment intensity) – i.e., not all counties received similar interventions and not all beneficiaries received all the intervention treatment, the beneficiary group was to be partitioned further by the intensity of intervention to examine the mean differences in outcomes between subgroups to draw insights on the effect of specific combinations of interventions. That is, understand whether specific interventions were impactful on their own or as a package. The effectiveness of the proposed evaluation approach depended on information on the selection criteria for beneficiaries and the availability of credible population-based survey (PBS) data – baseline, midterm and end line, covering key impact and outcome indicators, resilience indicators, and custom indicators developed by the program. However, these were unavailable.5 In the absence of PBS data, the ET used data from the Feed the Future Monitoring System (FTFMS), AVCD annual monitoring surveys, and other sources, including interviews with selected key informants and additional literature review to generate data. The following analyses, among others, were performed on the data: ● Descriptive analysis of data from FTFMS to compare targets and actual achievements at the end of the project (Year 5). The analysis compared Year 1 and Year 5 estimates. ● Content analysis of AVCD reports that describe achievements and factors related to performance, considering programmatic shifts. Qualitative Approach The quantitative data was complemented by qualitative information derived from interviews, observations, and textual analysis. Qualitative methods are less concerned with demonstrating whether emergent findings are representative of the larger population from which they are drawn, since such claims are rarely made or expected. Qualitative methods are especially useful when the interventions to be evaluated increase in complexity (i.e., require many discretionary and face-to-face transactions, and are contentious), when the context itself is highly variable (and perhaps volatile), when the quality and availability of existing data is poor, and when insights are sought on specific types of impacts on specific groups. Qualitative methods are also useful for understanding the nature of the causal links/relationships between project activities and 5 Per USAID, an end term evaluation of AVCD’s PBS was carried out and a draft report produced, but the data and the report could not be released at the time of this evaluation. In addition, there was no standard baseline survey conducted for the program. Instead, each value chain component conducted its own baseline survey using own methods and tools. 36 outcomes of interest, which are not easily amenable to quantitative measurement. The qualitative data is not intended to be representative but to illustrate, triangulate, and explain the quantitative results, and gather detailed information relating to the project design and implementation, as well as the sustainability of outcomes and lessons learned. For all methods, a gender lens was applied to assess the extent to which differential needs, priorities, voices, and vulnerabilities of women, men, boys, and girls were taken into account in the design, targeting, implementation, and monitoring of project activities. 4.3 Data Collection Methods and Sample Sizes The performance evaluation was conducted through the following methods:  Desk review of AVCD program documents and reports; AVCD program design, delivery models and pathways, monitoring plans and data, and outcome monitoring reports, risk register, annual reports, USAID reports, evaluations, post-intervention monitoring reports, beneficiary feedback databases, and other relevant documents; government policies and strategies and reports; strategies and reports from USAID. Appendix 2 contains a detailed bibliography.  Semi-structured key informant interviews (KIIs), including with AVCD management and staff, including field staff and focal points; AVCD partners and beneficiaries; USAID programs; government counterparts at the national and county levels; other development partners; managers and technical staff from cooperating partners;  Focus group discussions (FGDs) with beneficiaries. The ET visited seven out of fifteen AVCD Phase 2 counties, and a sample of activities covering all value chains in different sites. A detailed sampling strategy was developed at the inception stage in consultation with USAID.  Direct observation: the ET visited a sample of activities covering all value chains in different sites to observe physical structures and equipment, and the behaviors and interactions between and among different stakeholders. However, the short time in the field did not allow for in-depth behavioral observations and, therefore, only opportunistic cursory behavioral observations were made.  Other data collection approaches were used based on data needs identified during the inception phase. The KIIs and FGDs were conducted by the evaluation key personnel, including the Evaluation Team Leader, subject matter experts, and qualified research assistants. The range of stakeholders interviewed was intended to promote the participation of different groups, including representatives of direct and indirect beneficiary groups and types, as well as visits to project￾supported entities. These groups included direct and indirect female and male beneficiaries to avoid potential biases within the existing time and logistical constraints. Since there are too many potential informants to follow up in depth, AVCD facilitated the selection of the most information-rich persons to answer key questions. County headquarters relating to the selected site were visited for interviews with relevant government officials, private sector stakeholders, and community representatives. 37 The ET conducted all interviews and FGDs in the languages the participants were most comfortable speaking. We anticipated that most interviews would be conducted in English or Kiswahili, but recognized that some respondents, such as farmers, may prefer to speak languages or dialects spoken in the selected communities. The ET obtained informed consent from all respondents. The ET planned to digitally record all FGDs, with permission from the respondents. However, given the necessity to observe COVID-related precautions – such as holding FGDs outdoors and with physical distancing – the audio quality was often subpar. In those circumstances, the FGD moderator took notes or back-up note takers were used. The Team Leader and other evaluators periodically reviewed a sample of the notes and audio recordings for quality and thoroughness of interviewing and facilitation techniques and followed-up with moderators about any improvements needed. The team met weekly to review progress and discuss key findings and impressions, describe any difficulties or challenges encountered in the field, and find solutions. For the KIIs, the interviewer took own notes. The number of interviews and FGDs is presented in Table 2. During the evaluation design phase, the ET set a target to complete 94 KIIs and 34 FGDs, and subsequently completed 67 KIIs and 51 FGDs. Some of the planned key informant interviews, especially those with county governments and farmer’s producer organizations, were turned into FGDs because of the desire of the officials to be interviewed as a group for complementarity purposes. In addition, the number of FGDs with community groups increased because some counties had more than one value chain. Overall, the ET sufficiently covered the targeted groups and obtained fairly representative and robust data to inform the evaluation. Table 2. Summary of Qualitative Data Collection Respondent type Target Achieved Number of KII Number of FGD KI I FG D AVCD Program / Nairobi 6 2 8 2 USAID staff 4 4 CGIAR Centers 4 5 Development Partners and collaborators 8 10 National Government Officials 10 10 AVCD field staff representatives 14 10 County Government Officials 14 2 12 5 Private Sector representatives 20 8 7 Community Groups: Farmers / Mothers 14 30 37 38 Total 94 34 67 51 Abstraction of Quantitative Data: Based on the evaluation matrix and the AVCD performance measures and indicators, the ET abstracted quantitative data from various project sources so that progress towards targets could be documented and incorporated into the evaluation analysis and findings. Data Collection Instruments: The ET developed comprehensive semi-structured interview guides for all KII that were specific to the type of respondent and which addressed the evaluation and learning questions as outlined in the final evaluation protocol, as well as FGD guides for community groups. These tools are presented in Appendices 11-13. 4.4 Selection and Schedule of Field Site Visits The evaluation scope of work required that the ET visit seven counties including, two in the North, three in the West (Western-1, Nyanza-1 and Rift Valley-1) and two in the Southeast. The sites were identified with inputs from USAID, and came from each of the three clusters below: ● North Cluster: two (2) from Turkana, Marsabit, Isiolo, Garissa, and Wajir. ● West Cluster: three (3) - One from Kakamega, Busia, Vihiga, and Bungoma; - One from Siaya, Homa Bay, Migori, and Kisumu; - One from Nandi, Trans Nzoia, Uasin Gishu, and Elgeyo Marakwet. ● Southeast Cluster: Two (2) from Kitui, Makueni, Taita Taveta, Tharaka Nithi, and Meru. The ET used a combination of criteria to assess counties for site visits, including: ● Number of value chains implemented, with first priority given to those with multiple value chains; ● Duration of intervention – whether phase 1, 2 or both; ● Strategic value chains implemented in specific counties – e.g. LVC; and, ● Whether the county is a USAID ‘Prosper County’6 Using these criteria, seven AVCD counties were purposively selected to enable close to representative coverage of agro-ecological zones and the various value chains through which technologies and innovations were disseminated.  Marsabit and Isiolo – Livestock  Taita Tavata – Potato and DTC  Makueni – DTC and Dairy  Elgeyo Marakwet – Potato and DTC  Bungoma – Potato and OFSP  Homa Bay – OFSP and Dairy 6 See details in http://www.usaid.gov/sites/default/files/documents/1860/Prosper-Counties-Fact-Sheet.pdf 39 The evaluation team split into two groups and visited different counties simultaneously; the whole team visited the first county – Taita Taveta, as a trial run before splitting up to the West and Eastern clusters. Each group used approximately four days per county, including two days of travel to and from each county, conducting KIIs and FGDs, and completing site visits to speak with beneficiaries and when feasible, observe the relevant technology or innovation. For respondents missed during the county visits, interviews were conducted virtually. During site visits, the ET took notes and, when possible, pictures on how technology and innovation are used at different points of the value chains, how farmers are engaged, and the extent to which youth and women are participating. When appropriate, the team also elected to review documentation related to technology and innovation. The Task Order Manager and Evaluation Team Leader provided weekly updates to USAID during the field work, highlighting any issues or problems as well as enriching experiences. Updates were provided via virtual meetings and via email. 4.5 Qualitative and Quantitative Data Analysis Triangulation of findings from different sources forms part of the analysis to substantiate the findings and to develop conclusions. Triangulation involves comparing different sources (e.g., primary quantitative PBS data, primary qualitative data, secondary documentation) collected by different team members and through different methods (e.g., KIIs, FGDs). For primary qualitative data collected through a single method focused on a single activity (e.g., FGDs relating to group activities and achievements), triangulation involves comparing locations and sites and beneficiary status (i.e., women, men). In this way, gender-sensitive triangulation was possible. Thematic analysis was conducted for the KII and FGD respondents, organized by research question. Details of the analytical approaches are contained in Appendix 7. Quantitative Data Analysis Plan: AVCD collected data to monitor project outcomes. This data measure achievement against the project’s targets. The data were carefully organized, reviewed, and analyzed to further highlight the project’s achievements and help answer the evaluation questions. Selected quantitative analyses were provided using AVCD-reported monitoring data through the MEASURE platform. The quantitative data analyses involved the identification and analysis of trends, themes, and patterns (as well as outliers), disaggregated by livelihood zone, gender, and wealth, where relevant. As part of the triangulation process, themes or patterns were examined to determine whether they were coming from multiple stakeholder levels and categories; observations or comments that only came from a single source or a single category of stakeholder were given less weight. Efforts were made to explain gender differences in terms of the contributing underlying factors, including power dynamics. At the end of the field data collection period, the ET participated in an internal debrief and analysis meeting to discuss and develop the emerging findings, lessons, conclusions, and recommendations. Initial findings and conclusions were shared with USAID during a debriefing presentation at the end of the fieldwork for discussion to elicit feedback, verification, and correction of facts. 40 4.6 Data Integration, Preliminary Findings, and Report Writing Results obtained from the quantitative and qualitative data, including data collected from the desk review, were integrated to tell a comprehensive story about the effectiveness of interventions and the approach of promoting and bringing to market new innovations and technologies that are inclusive, create an enabling environment, and have the potential to improve the nutrition, food security, and resiliency of rural populations. This report addresses each research question individually, and presents findings cutting across all respondents and data sources. Key lessons were gathered as well as recommendations for further program initiatives. The ET presented its preliminary findings and conclusions to USAID/Kenya, AVCD staff, and key stakeholders in October 2021 for validation and feedback. The feedback from the presentation and other interactions were then used to complete this draft evaluation report. 4.7 Limitations The ET relied on program documents for both Phase 1 and Phase 2, a review of surveys reports and data, quarterly and annual project monitoring reports, and respondent’s accounts to inform its findings. The team attempted to get diverse perspectives from AVCD staff, program partners, program beneficiaries, and other private sector partners to ascertain how well the program performed and its impact on institutional governance for improved inclusive agriculture growth that impacts the nutrition and resiliency of vulnerable households. Whereas the evaluation exercise was successful, the ET faced some challenges and limitations as follows. Short timeframes to interact with documents to prepare the inception report and equally short periods allocated for field visits were challenging. Further, in some cases a suboptimal mobilization of identified groups for FGDs and KIIs exacerbated this challenge. The challenge of participant recall was also pronounced for some beneficiaries, more pointedly for Phase 1 than Phase 2. Associated with the COVID 19 pandemic and its various dynamics were some rushed and patchy intervention implementations. Some of the intervention areas in Northern Kenya are volatile, as was demonstrated when the ET team members faced insecurity challenges in Marsabit and Isiolo counties that required the team to make last minute changes in the sites to visit for FGDs, which resulted in a limited participation of women in that subset of planned FGDs. The disruption also resulted in changes in scheduled interviews with key informants. The ET was also unable to access some documents and data or received them late. For example, the anticipated Population Based Survey (PBS) data that the team planned to use in the quantitative analysis was unavailable. The team chose to use the data available in the FTFMS topline indicators instead, which were not adequate to address most of the evaluation questions. Obtaining data to fill the data gaps proved a challenge because available datasets were sometimes fragmented or raw and had to be obtained from different databases, often in different CGIAR centers. Collecting, cleaning, organizing, and analyzing raw data, and collating all data required to respond to the evaluation questions, took considerable time and effort. As a result, the preliminary analyses relied more on qualitative data collected from field visits. Over time, however, adequate secondary data were obtained from datasets and reports, and used in quantitative analysis. 41 5. EVALUATION TEAM COMPOSITION The evaluation team is comprised entirely of highly experienced Kenyan key personnel. It is led by Team Leader, Dr. Hezekiah Agwara, a development economist with 23 years of experience in economic development research, policy analysis, and program support across Africa, particularly Kenya. He was joined by three Senior Evaluators including: Agriculture/Livestock Value Chain (LVC) Specialist, Joyce Mutua; Governance Specialist, Opiyo Makoude; and Nutrition and Gender Specialist, Dr. Elizabeth Waithanji. Together, they bring several decades of experience in high-profile evaluation designs and implementation, complementary subject matter expertise in agriculture and livestock value chains analysis, appreciation of technology and innovations in agricultural development, and understanding of market systems. The evaluation team has substantial experience working in Kenya in cooperation with national and county governments, public and private sectors, and is skilled in participatory methods to facilitate stakeholders’ engagement and learning. This team is supported by non-key staff including a Data Analyst/Statistician, a Qualitative Specialist, a Logistician, and LINC and ICF technical and management staff. The roles and responsibilities of each member are outlined in Appendix 10. 6. ETHICS AND COMMUNITY CONSIDERATIONS Ethical practices were observed during the fieldwork, and the overall quality of the evaluation process and outputs were ensured through proper planning and preparation, the application of relevant guidelines (where available), and LINC/ICF’s internal quality review board mechanism. The evaluation was consistent with and met the standards of USAID’s evaluation policy. Prior to conducting the field work, the ET obtained approval from the Kenyatta University Ethics Review Committee (ERC). A discussion of risks and possible benefits of this evaluation was provided to the respondents prior to any KIIs or FGDs. Consent forms describing the evaluation, evaluation procedures, and risks were given to the participants and informed consent required prior to starting interviews. Consent forms were approved by the ERC and the participants were asked to read and review the document. Upon reviewing the document, the evaluator explained the evaluation to the participants and answered any questions that arose. The participants had the opportunity to discuss the evaluation among themselves, family members, and/or associates, and consider it prior to agreeing to participate. Group leaders were required to review the consent form on behalf of their respective groups after consultation with members. The participants were allowed to withdraw consent at any time throughout the course of the process. Confidentiality is a high priority for the evaluation team, partners, and collaborators. The evaluation protocol, documentation, data, and all other information generated were held in strict confidence. No information concerning the evaluation, or the data, will be released to any unauthorized third party without prior written approval. The findings of the evaluation will be disseminated to relevant parties involved upon completion of the exercise. 7. EVALUATION FINDINGS AND CONCLUSIONS This section presents the findings according to the evaluation questions. The performance evaluation answers the questions based on the objectives described in program documents, assessing implementation from project initiation to the present, and makes conclusions based on 42 findings, identifies opportunities and challenges, and formulates recommendations for improvements in future programming. It further outlines the issues identified from document reviews, quantitative data analysis, and qualitative data collection and analysis. For easy flow, the section is organized by evaluation question. 7.1 EVALUATION QUESTION 1 What contribution did AVCD make towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status? The performance evaluation examined the nature and extent of the contribution that AVCD made towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status. This entailed an examination of both the aggregate contribution and specific contributions of each of the value chains. In particular, it sought to establish whether the activity enabled improved access to knowledge tools to farmers, increased resilience (buying down risks), and improved nutritional status of households. It similarly examined the achievement of the demand side factors: improving market linkages, realigning regulatory policies, improving access to inputs, strengthening business and financial services, and promoting greater diversification. In this regard, the evaluation assessed the progress on indicators and targets from baseline to end of the project, taking into consideration findings of the internal midterm program review (IMPR) 2017 and Annual Report 2020 on lagging targets. Table 3 summarizes the results on selected topline indicators. Table 3: Selected Topline indicators Cumulative Target (2016- 2020) Cumulative Achieved (2016- 2020) Percent change Total 2016/17 2017/18 2018/19 2019/20 4.5.2 (2). Number of hectares of land under improved technologies or management practices with USG assistance DVC 25,915 48,078 185.5 225.6 218.9 163.7 28.3 LVC 1,571,023 1,599,500 95.5 89.1 101.9 100.8 DTC 154,147 187,730 121.8 79.9 114.7 148.1 142.7 RC 12,669 19,360 152.8 32.7 118.2 114.0 315.0 Total 1,778,754 1,766,199 99.3 91.0 104.7 104.9 110.4 Total (excl. LVC) 207,731 265,690 127.9 108.5 128.0 135.8 110.4 4.5.2 (5). Number of farmers and others who have applied improved technologies or management practices with USG assistance DVC 58,507 63,996 109.4 64.7 128.3 133.4 48.7 LVC 79,950 63,835 79.8 131.4 87.3 75.4 53.4 DTC 171,180 219,834 128.4 107.0 127.2 123.5 167.5 RC 136,738 137,593 100.6 67.0 106.0 140.2 105.3 Total 444,975 485,258 109.1 86.0 113.3 117.3 112.7 HL.9-2 & 3.1.9 (15). Number of children under two (0-23 months) or under 5 reached with community-level nutrition interventions DVC 9,577 12,738 133.0 LVC 26,300 28,421 108.1 DTC 8,268 26,110 315.8 RC 40,725 38,999 95.8 Total 86,070 107,411 124.8 EG.3.2-26. Value of annual sales of producers and firms receiving USG assistance 43 DVC 2,006,402 7,978,767 397.7 LVC* 4,500,000 8,693,681 193.2 DTC 1,200,000 4,775,000 397.9 RC-Potato 4,000,000 4,929,210 123.2 Total 11,706,402 26,376,658 225.3 Notes: * Includes sales of live animals, animal carcasses, and milk The results show that AVCD reached 485,258 farmers and others with improved technologies or management practices, who committed about 266,000 hectares to the target value chains over the five years.7 Both numbers exceed program targets. The majority of the beneficiaries (74%) came from the DTC and root crops value chains. The program interventions catalyzed approximately USD 26 million in annual sales from producers and firms receiving assistance, equivalent to about $5.3 million per year, with all value chains significantly exceeding their sales targets by an average of 125%. Consequently, the program generated approximately $54.40 of incremental sales for each beneficiary farmer and others during the five-year period; this was significantly lower than the unit investment of $76.94. While it is possible the program generated other non￾monetized benefits, these were not reported and the ET did not attempt to estimate them. AVCD collected only limited yield and cost data for all of the value chains, despite productivity and profitability being key indicators of performance in program documents (AVCD PPR 2017 and 2020). The ET used data extracted from reports and datasets provided by VC components to generate estimates of changes in yield and gross margins. Overall, the program performed very well on yield improvement, albeit with variations across and within VC components. Table 4 illustrates the program’s effect on yield, suggesting that, on average, AVCD improved baseline yields of potatoes, DTC, and dairy by 5%, 62%, and 75%, respectively. Within the DTC, performance was mixed; sorghum, finger millet, and pigeon pea achieved significantly higher yield than the baseline, between 97% for pigeon pea and 200% for finger millet; pearl millet and green grams recorded lower yields than baseline. Nevertheless, except for the DVC and groundnuts (which did not have yield targets), the realized yields were all below the program targets set for each value chain. The dairy, DTC and potato achieved 175%, 56% and 64% of their respective yield targets; within DTC, sorghum (42%) and pearl millet (38%) performed poorly against targets, while pigeon pea did better (74%). Table 4: Average yield by value chain Value chain Average yield Change, % Annual change, % Baseline Target Attained Relative to target CAGR 2017 2018 2020 Milk ( liters / cow) 2.90 5.1 175 15 17 44 7 Crops (ton/ha): DTC (median): 0.28 0.80 0.45 56 13 42 -18 56 Sorghum 0.24 1.20 0.50 42 20 117 -52 100 7 The ET excluded the LVC from calculating the area because of highly inconsistent numbers reported and data quality issues. Despite numerous attempts to reconcile the data, the ET was unable to validate the numbers or make them useful for analysis. In fact, various AVCD reports, including the IMPR 2017, had questioned the credibility of the LVC numbers and several revisions followed. 44 Finger millet 0.10 0.80 0.30 38 32 380 -58 50 Pearl millet 0.63 0.80 0.39 49 -11 -79 200 28 Pigeon pea 0.30 0.80 0.59 74 18 -33 195 36 Groundnut 0.25 0.30 5 96 -63 67 Green gram 0.72 1.00 0.50 50 -9 -58 77 -6 Potatoes 10.90 18.00 11.47 64 1.3 0 10 -4.2 Source: AVCD Phase 1 & 2; MPRO Database The LVC did not collect a lot of yield data; only Phase 2 data was available – 2019 and 2020. According to AVCD, the LVC surpassed its yield targets (live animals off-take in kg p.a.) for cattle, camels, goats, and sheep by 165%, 265%, 174%, and 152%, respectively. This it attributed mainly to availability of pastures and water due to good rains in 2019 and 2020. In addition, AVCD claimed that increases in body weight of animals was a result of improved resource management using the PRM toolkit, which led to increased availability of pasture in some areas. The ET was unable to verify this claim. Since most of the value chains were introduced in new non-traditional areas, baseline and end line yield data could have been affected by smaller numbers of producers and sampled beneficiaries. For example, the ET suspects that the 75% improvement in milk yield, from 2.9 liters to 5.1 liters per cow daily, masks data quality issues (specifically the baseline data) because the baseline yield was significantly lower than past estimates from the same counties. It is possible that either the baseline milk yield was understated or the timing (seasonal) of the baseline and annual surveys differed significantly. For instance, a Baseline Survey for USAID￾KAVES (USAID-KAVES, Baseline Survey 2013) found the average daily milk yield per cow across 22 FTF counties was 5.5 liters in 2013.8 Moreover, Table 5 shows that AVCD’s own annual surveys for the dairy value chain between 2017 and 2020 reported higher average daily milk yield of 4.9 to 5.3 liters across all the breeds. It is therefore improbable that milk yields nearly doubled in only one year of AVCD implementation. Table 5: Average milk production per cow per day Type of cow 2017 2018 2020 Pure breeds: N 208 108 179 Mean 11.2 11.0 9.9 Median 10.0 10.0 10.0 Crosses: N 350 453 601 Mean 5.7 6.2 5.7 Median 5.0 5.0 5.0 Local: N 395 386 318 8 Makueni, Kisumu, Kisii, Kitui, Machakos, Siaya, and Vihiga counties recorded 4.8, 3.5, 5.3, 5.3, 4.8, 6.4, and 3.8 liters, respectively. Furthermore, USAID-KAVES analysis of Tegemeo (2016) and Njarui et al. (2016) found the national median milk yield in 2014/15 was between 6.1 and 7.0 liters per cow per day (8.1 liters among smallholder farmers). 45 Mean 1.8 1.7 1.8 Median 1.5 1.5 1.5 ALL: N 953 947 1,098 Mean 5.3 4.9 5.2 Median 4.6 4.1 4.8 Source: AVCD DVC Annual Surveys The ET finds that AVCD clearly demonstrated that potential exists to improve yield in non￾traditional (pre-commercial) growing regions with technology, innovation, and behavioral change. In another five years, the ET estimates that the program would have achieved its targets for DTC (except green grams) and improved potato yield to 12.4 t/ha, which would be higher than the national average (9.2 t/ha). These would amount to relatively good yield for the non￾traditional growing areas. Ideally, increased yield should signal higher output and revenues, but calculation of income requires information on cost of production and operations. AVCD did not collect cost and income information, despite having identified these indicators for monitoring in the original M&E framework. From program documents and interviews with beneficiaries, however, the picture that emerges is that increased yield and production did not necessarily mean higher income due to higher cost of production and price fluctuations. As an induced technology / innovation program, the improved technology promoted by AVCD came at significantly higher cost than existing technology – both the cost of acquisition and funding the more intensive management practices. These higher costs potentially impacted profitability despite increased yield. For example, the cost of certified potato seed was about 2.2 times the locally acquired seed (Kshs. 3600 per 50-kg, c.f. Kshs. 2500 per 75-kg); this is equivalent to 120% increase. With additional cost of agrochemicals (six spraying cycles per season), labor for intensive management, and packaging, the 5% increase in yield means negative returns. Indeed, potato farmers in Elgeyo Marakwet and Bungoma reported negligible or high negative returns from production. For example, officials and members of the Elgeyo Marakwet Potato Marketing Cooperative Society (EMC-PMCS) Ltd estimated that member growers were making negative returns of Kshs. 35,000-65000 per acre because of extremely high cost of production (Kshs. 65,000-97,000 per acre) and low prices (Kshs. 300-600 per 75-kg bag of ware potatoes). In Bungoma, farmers in Ndalu Ward in Tongaren estimated earning gross returns ranging from Kshs. 5,000 to Kshs. 19,000 per acre. In addition to the effect of cost of production, increased sales were accompanied by either sticky prices or sharp drops in price (and demand) within the project lifecycle. In Marsabit County, for example, pastoralists in Merrile reported improvements in livestock sales in absolute numbers over certain periods, but maintained that prices had remained more or less stagnant over the last three years. The pastoralists also reported a severe dip in sales over the COVID-19 period, and in times of drought. The ET could not verify this information, because, while AVCD collected data on livestock prices and volumes of sales from the livestock markets, the data or data analysis output was not provided to the ET. A clear pointer to this can be found in the reported value of livestock sales in 2020, which showed a sharp drop of 68% from the previous years’ annual average. 46 Likewise, in Makueni, the DVC and DTC farmers interviewed by the ET reported cyclical fluctuations in prices of produce. For example, DVC farmers interviewed at the Kaiti Dairy Cooperative reported heavy losses sustained from deaths of cows due to foot and mouth disease and the negative effects of COVID-19 on access to dairy inputs. This led to higher production costs, sharply lower milk output (by about one-third pre-COVID, on average), and general lower demand for milk. In the DTC value chain, farmers cited climatic and cyclical price fluctuations and lack of (or delayed) seed as major impediments to increased production and income. In Elgeyo Marakwet and Bungoma Counties, potato farmers reported persistently low prices (Kshs. 300-1000 for an extended of 75-100 kg) that, together with high cost of production, made their enterprises financially unviable. Generally, the accounts from beneficiaries across the value chains differed from those provided by the AVCD project staff and documents. Several reasons might account for these variations. First, farmers could have been reporting their current condition and ignoring previous episodes of increased sales, which, at times, fails to account for intervention effect. Secondly, it could be that AVCD staff were referring in general to trends over the five years – which in their view had seen improvements. More realistically, drawing from interviews with a diversity of stakeholders, the ET concluded that AVCD staff’s perspectives were generally at odds with the actual experiences of farmers, who reported not experiencing notable improvements in income from the program activities. Because of lack of data on household incomes and ET’s inability to extract recall income information from interview respondents, the determination of AVCD’s contribution to income is inconclusive. Correspondingly, it cannot be concluded that the program could have contributed significantly to poverty reduction at the household level. We can conclude, however, that AVCD interventions increased output (availability) and diversified production, which are key proxies for food security and resilience. One indicator of this success is the program increasing nutrition indicators by about 10%. Finally, the ET examined how far AVCD spurred inclusive agriculture-led growth in the areas in which it intervened. To some extent, it can be said that women and youth participated in the various value chains. In the LVC, women and youth were included in collecting levies in livestock markets, which was previously not the case. However, this level of involvement occurred on the fringes of key decision-making processes, which is still male dominated, and thus could be viewed as tokenistic, rather than meaningful. 7.1a. To what extent have the technologies and innovations contributed to improved capacity of households, communities, and market systems in areas affected by recurrent crisis to withstand shocks and stresses? The capacity to withstand shocks mainly occurred through the livestock and DTC value chain interventions. These are discussed in turn in the sub-sections that follow. 47 7.1. a.1. Livestock Value Chain In the livestock value chain, AVCD developed a number of interventions in the five target counties of Northern Kenya to address various constraints facing pastoralist communities, including policy challenges that limit market access and livestock mobility. These included inadequate access to financial services and markets, limited availability of animal health services, low levels of literacy and technical capacity, and a lack of integrated policies on management of natural resources. The program hypothesized that by addressing these constraints, it would accelerate the development of livestock marketing to improve the incomes and nutritional status of 60,000 households by increasing fodder production and access to improved grazing areas, creating additional local market associations and private sector enterprises, and applying improved livestock management practices. AVCD interventions included enhancing rangeland resource management, undertaking policy reform to improve animal health service delivery, implementing an effective and sustainable animal disease surveillance system, enhanced access to animal health care using the private sector, and promoting market access by improving the market systems. First, the program trained Livestock Market Associations (LMAs), for example, in Merille in Marsabit and Ngare Mara in Isiolo, to improve access to livestock markets and, by extension, improve livestock sales and household incomes. Second, interviews with LVC component staff indicate that the program supported business-to-business (B2B) interactions among buyers of livestock from the major Kenyan towns and its beneficiaries. Third, AVCD helped develop a Livestock Market Information System (LMIS) to help farmers’ access better markets and prices; however, the ET visited Merille market in Marsabit County and established that the system was not operational. Fourth, AVCD introduced Participatory Range Management (PRM) practices to organize grazing lands and ensure availability of pastures during drought. Finally, LVC staff reported supporting an improved goat breeding program in Isiolo, where farmers selected the best breeds and ILRI supported with castration of those that were less resilient to diseases and changes in climate. However, the breed improvement intervention was stopped in Phase 1 due to lack of funds. Other interventions, such as support for animal vaccinations and improvements in supply of animal drugs, were aimed at improving animal health and, concomitantly, reducing loss of livestock. The program worked with the private sector, mainly Sidai Africa, in extending vaccinations and access to clinical services and veterinary drugs to farmers. To this end, AVCD also piloted a county-level electronic animal disease-reporting system (e-surveillance) and facilitated the training of Community Disease Reporters (CDRs) on syndromic identification and reporting of disease and linked them to county livestock extension officers. The system was layered on a closed user group (CUG) developed in collaboration with Safaricom to enable interactions among the CDRs and the county departments of veterinary services. The motivation behind the interventions in the LVC was to “buy down” risk and increase resilience among pastoralists. In spite of the above, interventions aimed at building household resilience in the areas covered by the livestock value chain tended to be light touch. LVC activities tended to be carried out over short durations of time, typically less than six months, and engagements between the project staff and the pastoralists tended to be either one-off or a couple 48 of interactions that did not lead to any institutionalization of the intervention approaches. On the PRM, the pasturelands rehabilitated were smaller in size, compared to AVCD reports and total area available and the size of herds, as demonstrated in Kargi and Ngare Mara Wards in Marsabit and Isiolo counties, respectively, and detailed below. The various initiatives are discussed in more detail in the subsections below: i) Sustainable utilization of rangelands. Sustainable utilization of the rangelands and rangeland resources are key to the development of the arid and semi-arid land (ASAL) areas as they play a vital role in fostering the economic, social, and environmental wellbeing of pastoralists. AVCD sought to ensure that communities in these counties could utilize rangelands resources in a more adaptable and sustainable manner that guaranteed continued supply of pasture across climatic cycles. ILRI developed a PRM toolkit to support community leadership and inclusiveness in the use of rangeland resources. The kit considered the interests, positions, and needs of all rangeland users and offered opportunities for negotiations among the different stakeholders to agree on the future of pastoral land use. It also provided a suitable and legitimizing process of communal land and resource tenure that fit with both the priorities of pastoralists as well as government bodies. In collaboration with the LMS AA2 program under Mercy Corps and respective county governments, AVCD introduced PRM in four counties to train communities on how to organize the use of grazing lands. The intervention involved training of trainers (TOTs) drawn from the county government on rangeland management practices, who would then go to train the community. To facilitate this, the Ward grazing lands management committees were established and trained on PRM; these included Kapurin Turkana, Golbo in Marabit, Ngare Mara and Kiina in Isiolo, and Bute in Wajir. In its 2018 annual report, AVCD reported to have rehabilitated a total of 1,5002,022 ha of land, with the capacity to feed about 579,200 heads of cattle owned by 8,473 households in Turkana, Marsabit, Isiolo, and Wajir Counties. The ET established from key informant interviews that land rehabilitation was undertaken in Turkana’s Kapur Ward and Wajir’s Bute Ward, where climatic conditions are much more favorable to pastureland establishment. In the rest of the Wards, NRM issues were addressed during training; LMS AA2 officers confirmed having received the training and found the tool useful in harmonizing the approach. However, the ET established that grazing land rehabilitation was undertaken in a much smaller scale than reported. For instance, the Kargi Ward committee members interviewed by the ET could not confirm whether any rehabilitation was undertaken, contrary to AVCD reports of 752,800 ha. rehabilitated; in fact, the committee was unaware of such an intervention, apart from ILRI’s work with university students in the area to collect vegetation data. Other key informants confirmed to the ET that actual rehabilitation of grazing land in Kargi Ward was not undertaken. To further put the numbers reported into perspective, the Northern Rangeland Trust (NRT) confirmed having worked with ILRI in Phase 1 to rehabilitate 350 acres in five community conservancies in Marsabit and Isiolo counties; the Nasuulu Community Wildlife Conservancy Committee confirmed this assertion. Taken together, the ET estimates that only about 1,700 acres were rehabilitated in the two counties. 49 From the findings on the two counties, the ET could not validate the 1.5 M ha of rehabilitated grazing land reported by AVCD. Similarly, due to lack of data to support the reports on rehabilitated land, the ET could not validate the reported over-achievement on key performance indicators, including EG.3-10, -11, -12 - Yield of targeted agricultural commodities among program participants with USG assistance (live animals off take in kilograms) and EG.3.2-26 Value of annual sales of farms and firms receiving USG assistance. The high achievement on these indicators was more due to availability of pastures and water due to good rains (AVCD Annual Report, 2020) than the implementation of the PRM Toolkit. Despite the shortfalls, community members interviewed noted that where PRM had been implemented, they were able to coordinate their grazing, and that pastures took longer to get depleted. A more indirect result was that the PRMs had brought several communities to co￾manage particular rangelands. For instance, in the Ngare Mara area of Isiolo County, the ET met with the management committee of a conservancy where four communities – Borana, Turkana, Samburu, and Somali – were co-managing the conservancy. This model of shared management had lessened inter-communal conflicts in an area characterized by violent skirmishes over access and utilization of grazing land and watering points. Elders reported that it was now possible to call other elders from another community and report incidents of lost or stolen livestock, which are often traced and returned to the owners. The wards grazing lands management committees in both Marsabit and Isiolo counties expressed interest in training on better strategies on managing grazing lands to improve existing traditional practices and enhance resilience. In addition, there is need for recognition of the grazing lands management systems by the county governments, including integration in the county spatial plans, to strengthen adherence to the grazing lands rules. ii) Development of rangeland management laws and regulations, and livestock and veterinary laws. In the last two years of implementation, the project supported the Wajir and Marsabit counties to develop rangeland management bills, which are at different stages in the legislative process. The implementation of the two bills was to provide a platform integrating the PRM principles into law to facilitate the: ● Promotion of sustainable management of rangelands through community and customary mechanisms for protection and management of pastures and other natural resources; ● Strengthening of community and customary management of grazing areas; ● Promotion of inter-community access to rangelands resources; and ● Establishment and protection of seasonal rangeland reserves among others. AVCD also assisted the national Directorate of Veterinary Services to update existing legislation in animal health service delivery to facilitate smooth running of the livestock sub-sector. It supported the DVS to review existing laws and policies to allow the regulations to reflect on current practices and enhance the participation of private sector in preventive measures. As a result, ten veterinary laws were consolidated into three different bills, namely, the Animal Health Bill, Veterinary Public Health Bill, and Livestock Bill. These were still under review at the national legislature at the time of AVCD closure. 50 iii) Establishment of animal disease e-surveillance system. The AVCD program developed an animal disease e-surveillance system that could quickly detect disease outbreaks to enable timely intervention. It supported the county Departments of Veterinary Services in Marsabit, Isiolo, and Turkana to set up the system, which was meant to allow timely flow of disease outbreak information to the sub-county veterinarians and, thereafter, to the CDVS. Interviews with ILRI staff reported that ICT training for county officers was undertaken and a mobile phone app rolled out in the three counties involving CDRs and veterinary officers at the sub-county and county levels. AVCD also trained the CDRs on syndromic identification and reporting of disease and linked them to county livestock extension officers. Veterinarians collected data output on the online system, which was then analyzed to capture the animal diseases being reported in various parts of the county. The findings from these systems enable the CDVS to advise animal health services providers and agro-vets on the type of drugs to stock at specific times of the year and to provide information on emerging disease outbreaks. According to AVCD staff, this significantly reduced physical movement of county veterinarians because the CDRs complemented the county technical officers. To enhance communication between CDRs and the Veterinary officers, the project collaborated with Safaricom PLC to introduce a CUG platform in which members within the group can communicate at a cost to the counties. Three key informants, including veterinarians and livestock experts at ILRI, reported that the system was stopped by the national government following conflicts over custody of the information and concerns that it ran parallel to or was supplanting the Kenya Animal Bio￾surveillance System (KABS).9 This position was confirmed by the DVS and CDVS offices, who reiterated that the system was a duplication of effort since the national office had already rolled out a similar system, including in the five LVC counties.10 In addition, the key informants reported that the DVS office was concerned about the security of “rumor” reports (animal disease information collected or shared by nonprofessionals). From interviews with key informants, it was unclear why AVCD sought to bypass an established national surveillance system for a closed county-based system. The government recommended that the AVCD system interfaces with the KABS and any future interventions should support the implementation and maintenance of the KABS. Such support could include capacity building of government system implementers, training and equipping of disease reporters, including on the use of the disease reporting system, and capacity building of county government officers on data analysis, interpretation, and utilization. 9 Kenya developed and rolled out an open-source mobile phone-based domestic and wild animal disease reporting system in 2017 and collected data over two years to investigate its robustness and ability to track disease trends. The Kenya Animal Bio-surveillance System (KABS) application was built on the Java® platform, freely downloadable for android compatible mobile phones (available on Google Play Store), and supported by web-based account management, form editing, and data monitoring and reporting (Njenga et al., 2021). The application was integrated into the surveillance systems of Kenya’s domestic and wild animal sectors by adopting their existing data collection tools, and targeting disease syndromes prioritized by national, regional, and international animal and human health agencies. 10 Responding the to the Draft Evaluation Report, AVCD reported that the system was operational in Turkana County and provided log in details to turkana.ilridsd.ilri.org (username: Turkana and password: Turkana-DSF2018). 51 Even without the stoppage by the national government, the system had faced numerous challenges. According to the CDRs, while they had provided information to the online platform, the information was not synthesized and communicated back to the community. Interviews with Isiolo CDVS staff informed the ET that the county lacked the capacity to analyze the data collected into information that could be relayed to the community. Furthermore, it was established that the county governments had not been paying for the CUG platform at the time the AVCD program closed, a matter that threatened the sustainability of the project. While there was an attempt to have the counties meet the cost of the CUG, as a strategy to J2SR, that may not be sustainable long term due to changing political interests; a funding mechanism involving the beneficiaries should be explored. iv) Improving access to animal health services by pastoralists. To enhance access to animal health services, the program partnered with LMS AA2 and Sidai Africa to facilitate the provision of mobile clinical services and veterinary drugs at watering points and during vaccination camps. An interview with Sidai in Isiolo confirmed that they were contracted by AVCD to provide veterinary services to livestock farmers for a period of six months. The partners also collaborated in public vaccination campaigns, including recent mass vaccinations against Rift Valley Fever (RVF) in Isiolo. According to LMS AA2, AVCD facilitated rapid access to the RVF vaccine through ILRI, a process which would otherwise have taken longer because the administration of vaccines is highly regulated by the government. While the initiative was good, there was a need to fully conceptualize matters of viability for the private enterprises to ensure sustainability of the activity beyond the project period and ensure compliance with regulatory requirements. According to an animal health expert at ILRI, the mobile clinical service was found to be profitable and sustainable if it included the provision of vaccination programs, which is periodic. The government was, however, reluctant to engage the private sector in the provision of veterinary services. Upon a review of existing regulations on animal vaccinations, AVCD established that no law in Kenya barred private service providers from providing vaccination to livestock and thereafter pursued the matter with the government offices to ensure inclusion of private service in vaccination programs. Senior staff at the DVS and the Kenya Veterinary Board confirmed that both private sector and government can participate in the provision of vaccinations, with the private sector providing the services to farmers who can afford the service and government offering the services to other farmers who are unable to afford paying for the service as a public good. There is, however, a need to ensure strict adherence of vaccine handling practices by all parties involved and, according to LMS AA1), that the public option does not undermine private provision. AVCD therefore supported both the public and private sector involvement in vaccination provision. v) Enhancing Livestock Market Information Systems. Livestock markets play an integral part in the lives of communities living in remote pastoral areas; pastoralists sell their livestock, access market information, and purchase food and farm input supplies, among others. Further, the livestock value chain has many actors who solely derive their livelihood from livestock related enterprises such as the butchers, livestock traders and transporters, and boda boda riders, among others. It is also a critical revenue stream for all the counties in northern Kenya. Therefore, any shock to the livestock market has a negative effect on the livelihoods of all the actors along the value chain. To enhance access to information, the AVCD project initiated the development of 52 an online LMIS to advise pastoralists on competitive prices for livestock in various markets. However, the ET visited Merille market in Marsabit County and established that the system was not operational. While traders reported providing information on prices, the same was not processed and relayed back to them and pastoralists to inform market decisions. The following quote captures the prevailing situation: “We provide information on livestock prices but we don’t get feedback on market prices at other markets. We would like to get livestock and milk market information even from Nairobi so that traders don’t lie to us” (Key Informant, Merille market, Marsabit County). Livestock traders instead relied on the National Livestock Marketing Information System whose data was online. While traders were able to access the online information, most of the pastoralists were not able to access information on prices and thus remained vulnerable to price information asymmetry. According to the respondents representing the pastoralists, there is need for the livestock market information to be communicated through the mass media, such as local radio stations, to better advise on applicable livestock prices. Overall, while AVCD attempted to implement different technologies to increase incomes in pastoral communities, these efforts were either partial or applied on a much smaller scale than targeted and reported. Consequently, the intended goal of improving capacity of households, communities, and market systems in areas affected by recurrent crisis to withstand shocks and stresses was not achieved. The following lessons are derivable from the PRM interventions: ● Learn from and integrate the vast traditional knowledge. While new approaches and technologies on managing grazing lands are welcome, there was a lot to learn from the vast knowledge and experience of traditional management practices. ● Involve the government in managing grazing lands. The rules and norms governing grazing lands are not anchored in local law, even though the importance of rangelands is recognized by county governments and their utilization anchored in the Community Lands Act 2016. Enforcement is left to customary norms and obligations, which most pastoralists ignore; invasion by external pastoralists is common. While numerous development partners are making progress in bringing these communities together, there is a need to anchor the guidelines in law for better enforcement. ● There is need for a well-structured partnership and collaboration with county governments and other partners in identifying roles and targets to be achieved to realize meaningful implementation of the PRM. ● Pay attention to local complexities in designing interventions. Breaches of pasture grazing reported to the authorities are not always dealt with. In the Golbo area of Marsabit, rampant charcoal burning degrades pastures and erodes resilience of the pastoralists. However, Kenya Forestry Service officers are reportedly unable to control charcoal burners, which negates the efforts to conserve pastures. In Isiolo, illegal enclosures by influential pastoralists in community owned pastures interfere with rangeland management, and hence, the resilience of pastures and households. 53 7.1.a.2 Drought Tolerant Crops Value Chain The drought tolerant crops (DTC) value chain interventions were in the ASALs, where food and nutrition insecurity are compounded by lack of diversity in the crops suitable for harsh environments. The ASALs are characterized by high temperatures, low and erratic rainfall, frequent droughts, poor soils, and inadequate infrastructure. These adverse conditions are exacerbated by inadequate extension support, input supply systems, and storage and marketing that are not supportive of commercialization. The overall goal of the DTC component was to reduce poverty and increase food and nutrition security, and specifically: ● Establish a reliable system for the supply of improved seed of selected DTC to farmers; ● Improve extension, advisory, and marketing information services with a digital mobile technology platform; ● Increase incomes though reliable marketing systems; and, ● Improve consumption and feeding practices among women, infants, and children. These were coupled with the promotion of technologies to increase yield. Establishing seed systems. A seed system is one of the key requirements for the development of any value chain. While Kenya has well-organized seed systems for major cereal crops, such as maize, wheat, barley, and others, there is no organized seed system for the DTCs. ICRISAT therefore collaborated with Egerton University and county governments to develop strategies to improve access to seed for selected crops in different target counties. To identify appropriate seed system interventions, the program carried out a baseline survey and a seed supply and demand situation study. From the findings, the project introduced farmers to improved varieties by distributing free seed; about 150,000 farmers received seed. The project promoted sorghum, green gram, and pigeon pea in all the counties. In addition, in Makueni, farmers were given finger millet and cowpea; in Kitui, pearl millet and cowpea; in Elgeyo Marakwet, groundnuts and pearl and finger millets; and in Taita Taveta, groundnuts. The interventions were implemented using the County Ward Agricultural Officers (WAOs), who identified the farmers to receive seed and provided extension education and training to the recipients. For sustainable seed supply systems, the project engaged both the formal and informal seed systems. Formal seed supply included ICRISAT, Egerton University, and private companies, such as Dryland Seeds, Faida Seeds, and Agrosoy Seeds Ltd. For informal seed systems, the project supported community seed producers, who could recycle seed up to three times. It promoted the bulking and distribution of seed by establishing 36 seed banks, which produced about 58 tons of seed during Phase 1. However, seed production declined sharply to 14 tons during Phase 2 due to inadequate rains and reduced seed demand as the project scaled down to only 14 Wards, among other reasons. The program supported field development trials of new seed varieties, leading to the release of three pigeon pea, two finger millet, and two groundnut varieties. As a result of the intervention, farmers were able to access quality seed of the right varieties, affordably. Based on FGDs held with farmers and interviews with key informants, farmers confirmed receiving seeds from the program and also purchasing limited quantities from community seed 54 producers. However, poor rains led to crop failure for three consecutive seasons in some areas at the time of the evaluation. Having not harvested much, most seed producers and seed banks were not operational. In some Phase 2 areas, farmers were still waiting for their first seed supplies at the time of the evaluation. Improve extension and marketing information services with a digital mobile technology platform. The project interventions revolved around setting up farmer training systems and identifying a digital agricultural training platform. In collaboration with the county extension, the project selected farmers who were trained on good agricultural practices (GAPs) after receiving seed. The project also set up 48 demo farms as learning farms for farmers in given clusters; in Taita Taveta, it partnered with the United Nations Food and Agriculture Organization (FAO) to access its beneficiary database and use its Field Schools as learning centers. In addition, the project set up a computer-based application to provide mobile-based weather and climate advisories. Key informants from the three counties visited by the ET confirmed having benefited from interventions that helped enhance packaging of extension information, including the development of GAPs manuals, post-harvest manuals, seed production manuals, cookbooks, and brochures for various seed varieties. Beneficiary farmers confirmed receiving digital advisory messages on weather, good agronomic practices, post-harvest management, and nutrition. Increase incomes through reliable marketing systems. The project planned to work with value chain actors and other development partners to improve market access and margins for farmers, including training on post-harvest handling and quality. Focus was also on technologies that reduce the use of human labor while improving quality during post-harvest processing, e.g., shelling and threshing; women and youth were particularly encouraged to take up the technologies, including as commercial service providers. Indeed, women farmers in Taita Taveta confirmed to have taken a loan to purchase a maize sheller. However, some of the promoted machinery and equipment were prototypes licensed to local fabricators to develop; an example is the groundnut sheller developed by Egerton University. While the sheller significantly reduced groundnut shelling labor and time, farmers expressed concerns over its quality, largely related to challenges with the local fabrication model adopted by the promoters. Farmers and key informants interviewed for this evaluation in Elgeyo Marakwet and Taita Taveta counties reported problems with poor quality grain from the sheller due to excessive splitting. Egerton University confirmed encountering issues with the shellers, which they recalled for further development and withdrew licenses from some local fabricators. There was effort to address the challenge of marketing, especially related to post-harvest gluts and scarcities through formation of apex FPOs. Agribusiness SMEs involved in the marketing of grains were also supported to develop business plans. These cooperatives were modeled on the traditional approach of farmers or groups coming together to form critical masses of producers and/or buyers. It was clear from KIIs with FPO officials and other stakeholders, including FGDs with farmers, that the cooperative model had not yielded the desired effect, particularly in influencing market dynamics. The consensus opinion from discussions with various stakeholders was that apex FPOs are not the panacea for solving the most pressing marketing problems in agricultural value chains, which are diverse and sensitive to county contexts. While the FPOs achieved success in mobilizing farmers into production and, to a limited extent, coordinating seed procurement, they had not engaged in tangible marketing activities nor developed any 55 marketing strategies or plans at the time of the evaluation. In all the interviews conducted with FPOs and other stakeholders, marketing was cited as the weakest link. County government officials and other stakeholders were in agreement that there is an urgent need to re-think and test different interventions, beyond the FPOs, to enhance benefits for producers and other value chain actors in commodity markets systems and markets development. Some examples of complementary approaches include geographic organization and clustering of production, hub￾and-spoke clustering of farmers around established anchor producers or marketers, first-mile agribusiness enterprises, formation of marketing common interest groups (CIGs), and establishment of innovation and marketing platforms, among others. Whereas farmers had adopted some GAPs as well as learned ways of preparing the crops, which were new to most of them, access to extension services remained a major constraint. Farmers interviewed in Makueni, Taita Taveta, and Elgeyo Markwet counties narrated their struggles understanding the varieties and getting assistance with agronomic, climatic, and crop protection issues. A majority of farmers mentioned seeds and occasional SMS weather information alerts as the only interventions received. However, except for groundnuts, many farmers could not recall or discuss the seed varieties promoted under AVCD. The ASALs would have benefited from higher yielding DTC varieties of green gram, millets, sorghum, pigeon pea, and groundnuts, but consecutive seasons of poor rains during AVCD implementation curtailed major gains. The new DTCs were meant to help farmers avoid total crop failure, and storage of produce in ways that preserve the quality of harvests, e.g., hermetic storage bags, aimed at enhancing resilience to shocks and stresses at the community level. However, these assumptions did not hold up to the vagaries of climate particularly well, although AVCD sent SMS weather alerts. FGDs and interviews with DTC farmers in Taita Taveta and Elgeyo Marakwet, and OFSP farmers in Bungoma and Homa Bay, confirmed that consecutive seasons of poor rains and drought had wiped out nearly all the new planting materials provided for multiplication. For potatoes, the damage from excessive rainfall, hailstones, and frost bites were common constraints identified by farmers across the three potato growing counties the ET visited, but no solution reported. Information on innovative approaches to deal with the effects of climate, especially for relatively new value chains and technologies, were particularly in high demand but not paid much attention by the program. More intentional and robust interventions on climate change response, mitigation and adaptation, e.g. applying climate-smart agriculture (CSA) technologies, would have helped address some of these constraints. Such technologies and innovations include early planting, drought-escaping or water-tolerant varieties, micro-irrigation, soil and water conservation, on-farm NRM practices, zero/reduced tillage, and intercropping and fallowing, among others. Future interventions would need to be coupled with CSA approaches to value chain development. Findings from key informant interviews suggest that the institutional capacity of the county agriculture offices was generally high and the officers demonstrated genuine interest in supporting the interventions. From the executive to WAOs, there were high levels of awareness about the technologies and innovations, and challenges encountered and feasible ways of addressing them. Whereas this high capacity could not be attributed to AVCD, the program 56 support must have helped enhance it. For improved impact, the county officers emphasized the need for co-creation of future interventions, working together in identification of suitable varieties and partners, and layering and integrating of interventions being implemented by multiple partners. There is a gender dimension to the benefits of the resilience enhancing capacity of the various VCs. For strategies covering the entire community, both women and men are likely to benefit equally if access to the products of those interventions are affordable, which can be enhanced by providing favorable credit terms. Benefits from seeds are subject to access to land to plant them and land is owned mainly by men older than 35 years. Value addition technologies relevant to household consumption are accessible to everybody, but women are the main household diet planners and most likely to use them. Several lessons learned from the DTC value chain interventions include:  For improved impact, there is need for co-creation of future interventions, working together with intended beneficiaries in the identification of suitable varieties and partners and layering and integrating of interventions being implemented by multiple partners.  Value chain interventions needed to integrate more intentional and robust climate change response, mitigation and adaptation. Given the diversity in climatic effects, mere introduction of drought tolerant crops is insufficient; rather, applying a broad array of climate-smart agriculture (CSA) technologies and innovation would help address some of these constraints.  Farmer producer organizations (or cooperatives) are not the panacea for solving the most pressing and diverse marketing problems in agricultural value chains. County government officials and other stakeholders were in agreement that there is an urgent need to re-think and test different interventions, beyond the FPOs, to enhance benefits for producers and other value chain actors in commodity markets.  Program interventions should not be used for technology research and development outside the legal and regulatory frameworks. Several of the promoted technologies were under development or prototypes for further development and proofing. Examples include the groundnut sheller and groundnut seed varieties developed by Egerton University, on which farmers expressed challenges with quality, availability, and performance. Promising technologies disseminated prematurely can discourage uptake and continued use.  For sustainable seed supply systems, it is prudent to engage both the formal and informal systems, including community seed producers. However, seed systems that are dependent on rain-fed production systems are not sustainable in the selected USAID ZOI. Moreover, it is important to be cognizant of the fact that the legal and regulatory framework is not sufficiently formed to support the development of nonconventional seed systems.  Participation in and benefits from technology and innovation interventions across gender and social groups are dependent on many other factors. Programs should therefore make a more deliberate effort to integrate gender and social inclusion into the program and implementation design. For example, it is imperative that project implementers understand that success of common technology interventions on seed, breeding services 57 or improved management practices are subject to access to land, animals, income or machinery/equipment, which are owned mainly by men older than 35 years. 7.1b. How successfully have the nutrition activities been layered and integrated with the core technology components of AVCD to achieve stunting reduction and other nutrition results? AVCD’s nutrition strategy states that AVCD interventions could improve nutrition via three pathways, namely: consumption of nutrient-dense foods produced; purchase of nutrient-dense foods using income from sale of surplus food produced; and, women empowerment, to enhance ability to access and make decisions on how to utilize the food produced and income obtained from sales. The strategy targets women to increase both physical access through own production and economic access through sale of surplus produce and using some of the money raised from sales to purchase other diverse nutrient-dense foods not grown by the household.11 The approach to nutrition intervention was the same across all the value chains. The main form of intervention was training, with delivery models similar across the VCs. The program utilized Sub-County Nutrition Officers from Health department and Home Economics Officers (HEOs) from Agriculture/Livestock departments, and senior Ward Officers. These were trained as trainers and then dispatched to train field workers, e.g., community health assistants, volunteers, workers, and extension officers. The field workers then trained groups of farmers / producers, livestock owners, and mothers and caregivers of young children. The three types of audience were as follows: ● Primary: Mothers and caregivers constituted target audience and reached through community and social mobilization for nutrition training ● Secondary: Spouses of mothers, in-laws and other family members, community religious leaders, county and sub-county technical staff for behavior change communication (BCC) ● Tertiary: County decision makers – Governors, CEC Members, and MCAs for advocacy The subtopics in the training on improved nutrition behavior included the following: ● Knowledge of the need to breastfeed (exclusively for the first six months) ● Appropriate weaning practices ● Diversified diets ● Food safety ● Appropriate food preparation ● Sanitation Nutrition KII respondents indicated that despite the identification of the training topics, training materials were not ready at the beginning of implementation.12 This information was corroborated by county government KIIs, Community Health Volunteers, and FGD participants. Therefore, nutrition interventions were delayed as well as delivered differently in the various 11 AVCD nutrition strategy, 2016. 12 The delay was attributed to the intentional wait by nutritionists in the program to complete the development of content and a tool (the nutrition SBCC cards/booklet) for use by government and other nutrition stakeholders in all counties nation-wide. 58 value chains in the beginning (Table 6). Nutrition training occurred at learning farms, producer organizations, and pastoral / farmer / women groups. The LVC team indicated that after training, the community trainees developed action plans for the strategic rollout of the lessons. They also intended to provide nutrition content for the “Shamba Chef” educational program.13 In addition, the OFSP nutrition teams collaborated with teams in four counties on the use of the “Malezi Bora” and similar nutrition education platforms to enhance nutrition information dissemination.14 The ET did not find information on content of the nutrition interventions for the dairy and potato value chains. Table 6: Content of nutrition training LVC DTC OFSP ● Integration of agriculture and nutrition ● Promotion of improved nutrition through agriculture ● Strategies to improve dietary diversity and consumption ● Milk hygiene ● Household food security ● Consumption of nutrient dense foods ● Dietary diversity and food utilization through training on cooking methods ● Enhancement of OFSP utilization ● Training on BCC using cards and posters ● Evaluating acceptability of OFSP through interviews with consumers Owing to the late (3 – 4 months) start in delivery of content at the beginning of implementation by the nutrition team, layering and integration of the nutrition component to the core technology components was less than ideal. This was reflected in the nutrition indicators presented in the AVCD 4th Quarterly Report, July to September 2016 (Table 7). Table 7: Top line Nutrition indicators for the period July – September 2016 Nutrition indicator Target Actual 3.1.9.(1): Number of people trained in child health and nutrition through USG supported nutrition programs 12,874 2,286 (18%) 3.1.9.(5): Number of children under 5 reached by USG supported nutrition programs 24,704 13,006 (53%) 13 Shamba Chef is a reality TV show that travels across Kenya exploring what people eat, where they get it from, and how they cook it. Aired in English and Kiswahili on Thursdays and Sundays by Citizen TV, it brings a mix of cleaner cooking solutions, nutritious recipes, and expert advice on the best foods to grow and eat. (https://shambachef.com/) 14 Malezi-Bora is a mobile application, available in local languages, with audio content regarding maternal and newborn health. It works on ultra-low-cost handsets and leverages village social networks and utilizes a pay-it￾forward business model, designed to incentivize mothers to share its content. (https://healthmarketinnovations.org/program/malezi-bora) 59 In the subsequent quarter (October to December 2016), there was an increase in effort, but the teams were still using the methods adopted in the previous quarter. While encouraging results were reported, actual performance was not comparable to the previous quarter, because the top line indicators had changed. Table 8: Top line Nutrition indicators achievements for period October - December 2016 Nutrition indicator Target Actual HL.9-4 Number of individuals receiving nutrition-related professional training through USG-supported programs 798 532 (67%) HL.9-2 Number of children under two years (0-23 months) reached with community level nutrition interventions through USG-supported programs 2,700 5,515 (204%) By the time of the IMPR, 2017, performance on nutrition indicators was still lower than other top line indicators. On average, the HL.9-4 indicator achieved only 17% of the target (LVC, 5%; DVC, 24%; DTC, 26%; and, RC, 16%). Better progress was recorded for HL.9-2, which achieved 60% of the target, on average (LVC, 30%; DVC, 14%; DTC, 511%; and, RC, 44%). The low achievement in the number of people professionally trained in nutrition was mainly attributed to the training approach, whereby training was individual-based and training materials varied with the different value chains according to their priorities. The program later opted to focus on training Community Health Extension Workers (CHEWs), Community Health Volunteers (CHVs), and other Public Health, Ministry of Agriculture and Livestock staff as TOTs. These TOTs were expected to train community members over two days per community targeting men and women household members with children between 0-23 months. AVCD contributed to the development of a revised National Agri-nutrition resource manual, which was being developed by the government with other stakeholders. The manual was intended to help standardize messages across the value chains and be used as a blueprint for developing community agri-nutrition training material. While prior to adoption of the manual, approaches, pathways, and messages differed across the VCs, there was general consensus that, once approved, all the VCs would pay special attention to enhancing the mainstreaming of nutrition in their activities using a common TOT approach and exploiting opportunities for leveraging and learning across the VCs. The agri-nutrition manual was completed and launched in 2018 and the AVCD team was confident that availability of the cards to all value chain components could enable adequate integration and layering of nutrition interventions with the core technology components. As a result, performance of indicator HL.9-2 rose from 16% in 2017 to 48% in 2018, whereas indicator HL.9-4 rose from 60% to 85% in the same period. Table 9: Top line Nutrition indicators achievements for the Year 2018 Nutrition indicator Annual- Year 3 Life of the program 60 Target Achieved % Target Achieved % HL.9-4 8,614 4,165 48 16,252 11,983 74 HL.9-2 21,756 18,430 85 40,148 67,168 167 During Phase 2, HL.9-4 was no longer captured because the nutrition intervention approach changed from training individuals to training TOTs, who then trained community members. The nutrition intervention going forward cascaded knowledge transfer down to three levels of county governance. At the county headquarters, senior staff from the departments of agriculture (home economics), health (nutrition), and education (education officers) were targeted as partners to oversee the implementation of the nutrition components. At the sub-county level, heads of home economics and nutrition conducted TOTs for CHEWs, early childhood development (ECD) coordinators, and CHVs, who then trained the community (including lead farmers at demo farms and mothers and caregivers at health centers and homes). The community resource persons – lead farmers, CHVs, dairy farmer assistants (DFAs), women groups, and CDRs - were trained as trainers in nutrition and the need for social behavior change communication (SBCC) around nutrition practices using the SBCC dialogue cards. They were issued these cards and engaged in the training of project beneficiaries. Later, and in some places, CHVs and WAOs worked together to improve integration and layering of nutrition by conducting nutrition education in the same forum and trainings for good agricultural practices (GAPs). Supporting and sustaining the involvement of CHVs, who worked as volunteers remained a challenge. The high achievement of HL 9-2 in 2020 was attributed to a change of strategy whereby AVCD in collaboration with county governments facilitated CHVs and CHEWs with extra allowances for protective materials against COVID-19. This enabled them to continue with community nutrition training. The achievement of targets for indicator HL.9-2 for the four value chains during the project period 2017 – 2020 show that the DTC performed extremely well in 2017 and remained consistent in subsequent years, while the potato VC performed consistently well during all four years. Table 10: Top line nutrition indicator (HL.9-2) for 2019 and 2020 Year Annual- Year 4 Life of the program Target Achieved % Target Achieved % 2019 14,900 10,509 71% 23,000 10,509 46% 2020 14,818 16,745 113% 23,000 27,254 118% In the LVC areas, pastoralists receiving production interventions did not receive nutrition interventions, while women in settlements received nutrition but not production interventions. In fact, production and nutrition interventions were implemented in different wards. Some CDRs were CHVs and were expected to train in nutrition, but they could not cover all required households. Previously, nutrition services were linked to health and provided by UNICEF; agri￾nutrition was therefore a new concept. The trainers also demystified food taboos, e.g., the 61 Tupendane Group from Isiolo explained that pregnant women did not eat eggs or liver because the community believed that these two foods made the baby in the stomach very big and would cause problems at child birth. They also believed that only cowards went to give birth in hospitals and the worst kind of cowards got assistance at childbirth e.g. caesarean section. In another community, the child has to wait for the father to come and name it before it breastfeeds. This may take days if the father is away. Such babies are fed water with sugar and goat milk. As a result, father to father support groups were formed to sensitize men about nutrition and such harmful cultural actions. Qualitative data to fill the gaps arising from quantitative data was collected from the seven counties visited and are summarized as follows. Taita Taveta: AVCD intervened in two value chains in the county, DTC and potato, in Phase 2; the DVC piloted in Phase 1 was not successful and, therefore, withdrawn. Food security in Taita Taveta suffers from a lack of required foods and low levels of income in most dry parts of the county, which is juxtaposed against production abundance in a small corner of the highlands. From FGDs and KIIs conducted in the county, there was evidence of attempts to integrate and layer nutrition interventions with the DTC and potato technology interventions. The two value chains, however, implemented nutrition interventions separately. Mothers at selected intervention sites confirmed having received seed and training on nutrition, feeding of children, and sanitation, but limited training on agronomic practices. This is because nutrition training was conducted at the sub-county level, whereas GAPs were trained at the ward level. Similarly, nutrition and production components were disconnected and different groups received different interventions. Interviews with WAOs indicated that they were not trained on nutrition and could, therefore, not integrate nutrition in the GAPs training. Since there were no nutrition interventions in many places the WAOs worked, they could have helped integrate nutrition messaging with the technology interventions if they had been trained as trainers in nutrition. Makueni: AVCD intervened in two value chains in Makueni, dairy and DTC. Food security in Makueni suffers from a lack of required foods and low levels of income. Among the value chain commodities available in the county are maize, beans, pigeon peas, cow peas, sorghum, and finger and pearl millet. Foods given to children include omena (silver cyprinids), beans, green grams, cassava, spinach, milk, paw paws, oranges, and watermelon; eggs are available but rarely given to children due to cultural taboos. From FGDs and KIIs conducted in the county, there was evidence of attempts to integrate and layer nutrition interventions with the DTC technology interventions, but none in the DVC interventions. Mothers at selected DTC intervention sites confirmed having received seed and training on nutrition, feeding of children, and sanitation, but no training on agronomic practices. This is because in Makueni (and Kitui), nutrition training was conducted at the sub-county level, whereas GAPs were trained at the ward level. Similarly, nutrition and DVC production components were disconnected and different groups received different interventions. Wards where nutrition training occurred recently – Nunguni and Emali - were not covered by DVC. Mothers indicated that most of the required foods were not produced at home or in the farms owing to the semi-aridity of the agro-ecological zone and had to be bought. Most households were too poor to afford the recommended foods. Other challenges cited were unsuccessful attempts to raise chicken for eggs and meat as well as some crops because of 62 pests and diseases. The mothers recommended that they be supported to establish kitchen gardens and to create agri-business opportunities for mothers. Marsabit and Isiolo: In Isiolo, LVC activities covered different areas from those covered by nutrition interventions. Dialogue cards were shared with CHVs, expectant women, women with children below five years, and youths, among others. There, was however limited reach out to other members of the community especially men who determine resources available for procurement of diverse foods. The women were also trained and benefited from demos on kitchen gardens. However, there was limited adoption of the kitchen gardens at home. The following were recommended areas for nutrition needs from a group in Isiolo: ● Refresher course for CHVs ● Training on food preservation to serve during the dry seasons ● Involvement of the whole family in training ● Support for other income generating activities/agribusiness for youth and women ● Motivation / allowances for CHVs ● Training for youths ● Continued use of the SBCC tool kit, which was helpful, as the pictures helped communicate to the less educated trainees ● Women should be used to train other women, as well as identify their training needs. Elgeyo Marakwet: A formative study on nutrition interventions provided lessons on the most appropriate approaches to enhance consumption of nutrient dense foods. The challenge in Elgeyo Marakwet is that so much nutrient-dense food was produced but most of it was sold, with growing communities barely eating them due to unfamiliarity, culture, poor storability, and also their higher cash generating potential. Field days were held to teach farmers how to prepare the various foods and motivate communities to utilize the nutrient dense DTC grains and legumes. There was also the “Smart Food Campaign” to influence communities to change their feeding behavior and eat smart foods.15 One reason farmers sold most green grams was their poor storability – they are highly vulnerable to weevil attacks. Hermetic bags could be a solution to the weevil problem had they been promoted. Another reason is that green grams earn good money, and people are not used to consuming it. Bungoma: Agriculture interventions started in April 2019 while the nutrition component started later in June. AVCD used CHVs and dialogue cards and first targeted the learning farms (LFs). The LFs, however, did not have enough mothers with children 6-23 months or under 60 months. The CHVs therefore made household visits to reach the target children population. Initially, they targeted individuals at home, leaving WAOs to train farmers on GAPs at the LFs. The approach later changed and became two-pronged. The CHVs joined the WAOs and taught farmers at the LFs, but also visited households and trained people in their homes on their own. It was only later 15 To improve the nutritional status of women in the reproductive age and children below five years, an awareness drive under the Smart Food campaign was launched in selected counties in Kenya. The aim was to promote increased consumption of nutrient dense, drought tolerant crops (sorghum, millets, pigeon pea, groundnut, cowpea and green gram) and appropriate dietary practices in the project areas using social behavior change communication approaches. (https://www.icrisat.org/smart-food-nutrition-activities-kick-off-in-kenya/) 63 in the intervention that WAOs were trained on how to use the dialogue cards and shared their training platforms with the CHVs, but did not train farmers on nutrition. Homa Bay: The ET interviewed OFSP and dairy value chain stakeholders and found that nutrition and production components were disconnected, and different groups received different interventions. This is because nutrition training was conducted at the sub-county health centers, and messaging implemented at the MoH Village Units, whereas GAPs training was done at the ward level. Nutrition was first implemented through the department of health and then later followed by production training by the WAOs; this is because a previous CIP project, Mama Sasha, was implemented through the Ministry of Health.16 Moreover, OFSP and DVC activities were implemented separately. For example, in Rangwe Sub-County, where nutrition interventions occurred under OFSP, no DVC intervention was reported by the respondents, most of whom were unaware DVC was being implemented in the area. Further, there is no evidence that the DVC implemented any direct nutrition interventions in the County. 7.1c. How have community-level nutrition education and behavior change affected dietary diversity and care practices for women and children? First, an increase in food production and income from sale of products does not necessarily translate in an increase in the diversity of food consumed. Second, training in nutrition increases the knowledge of food types, their importance, and the necessity to consume them, but without access and control of these foods and income from their sale, women will not benefit. A significant change in a dietary diversity score is likely to happen if the women, in addition to knowing the food types and their importance, are empowered enough to have access to and control over the food and money generated from the sale of surpluses. In all the rural communities where the various beneficiaries are located, strong gender stereotypes exist. For example, in the cropping areas visited, women were allocated plots large enough for subsistence production to feed their families; it is unlikely that the dietary diversity score (DDS) could have increased significantly. Whereas AVCD reached about half the children under two years (6-23 months) with community level nutrition interventions and reached 85% of individuals targeted with nutrition-related professional training, the median DDS barely improved from baseline values. At baseline in 2016, the median DDS of women of reproductive age (WDDS) across all value chains was 4.3 (out of the 10 recommended food types); which was slightly higher than the minimum recommended DDS of 5.0. The baseline WDDS was much lower among DTC and LVC beneficiaries (3.2 and 3.5, respectively, but above average among the DVC and RC beneficiaries. In the DTC, the overall WDDS from three counties did not meet the Minimum Dietary Diversity (MDD) (5.0), which suggests poor diet quality; however, slightly more than half (51.5%) of the sampled WRA consumed five and above food groups (DTC Annual Survey 2021). Similarly, the median DDS for Children 0-23 months (CDDS) increased by 13%, from 3.67 to 4.16, which was below the Minimum Acceptable Diet (MAD) of 5. Finally, the median household DDS (HDDS) increased 11% from 6.1 at baseline to 6.78, 16 The Sweet Potato Action for Security and Health in Africa (SASHA) project has the idea of encouraging mothers to get the health care they needed and increase consumption of the orange-fleshed sweet potato, a highly nutritional potato that promised to reduce under-nutrition in sub-Saharan Africa. (https://healthmarketinnovations.org/program/mama-sasha-project) 64 which was higher than the minimum of 6, both at the baseline and subsequent years. The diversity of diets for women of reproductive age and children were therefore inadequate, and, hence, the nutrition intervention ineffective, especially in reducing stunting. Table 11: Dietary Diversity Scores Baseline Achieved % change Women Dietary Diversity (WDDS): DVC 4.83 4.72 -2% LVC 3.50 4.73 35% DTC 3.20 4.53 42% Potatoes 5.00 5.67 13% OFSP 4.28 4.62 8% Median 4.28 4.72 10% Household Dietary Diversity (HDDS): DTC 6.30 7.13 13% Potatoes 6.10 7.10 16% OFSP 6.45 Median 6.10 6.78 11% Child Dietary Diversity (CDDS): DTC 3.10 4.17 34% Potatoes 4.10 4.15 1% OFSP 3.23 3.42 6% Median 3.67 4.16 13% Source: AVCD Phase 1 and 2 MPRO Database The DTC component compared the reports on food consumed (not DDS) by households in general, women of reproductive age, and children 6-36 months old. With the exception of cereals and other vegetables, there were significant declines in consumption of most foods between 2017 and 2018. The PVC evaluation report (2020) indicated that training did not result in an increase in dietary diversity from baseline values in spite of holding an average of four training sessions in Bungoma and five in Taita Taveta. It was similar to the dietary diversity obtained in 2018 before the trainings (Potato Value Chain report, 2018). According to the nutrition strategy, the impact of increased productivity on diets should be stronger where nutrition education, social and behavior change communication (SBCC), and women's empowerment interventions are incorporated into agricultural interventions. Women’s empowerment is important in general because it determines a woman’s access to decision making over resources such as income, time expenditure that determines caring capacity, and workload that determines a woman’s ability to participate in competing activities. An action plan on how to empower women from the various value chain contexts could have enhanced the effectiveness of this empowerment pathway. Training about access, decision making, time burden and workload is not adequate. The program could have identified linkages with actors / partners who could have enabled women to access finance (through for example affordable credit); time saving and workload reducing technologies such as energy saving stoves, water storage tanks and shelling and threshing equipment by introducing them systematically. 65 The nutrition strategy, therefore, was incomplete without an action plan for empowerment of women in ways to enhance their benefits from the production enhancing technologies. Consequently, the program effect on care practices of women and children is unclear, because the nutrition engagement was too short and limited in scope and depth to be impactful. While some changes in hygiene practices were stated, most beneficiaries interviewed were unable to name any particular nutrition benefits nor classify the different types of foods for children, mothers, and/or adults. Moreover, the intervention missed out on young single mothers living with parents or guardians. Community level nutrition education and behavior change may have affected dietary diversity in some communities in the HR1 areas, which have diverse food types. Behavior and care practices for women and children definitely changed as demonstrated in the nutrition case study presented below. Story of change: Case study from Nutrition FGD interview with mothers from Sirikwa Village, Bungoma County Women that benefited from the nutrition intervention when pregnant or lactating were targeted for this activity. The interview was one of the many FGDs lined up, but these women stood out. They were excited about the intervention because, as they said, the lessons they learned changed their lives completely. The case study helped the evaluators establish whether the intervention was appropriately targeted, if they achieved the desired effect on the poorest individuals, if the lessons were relevant, acceptable and sustainable. The ten women interviewed were beneficiaries of the nutrition intervention more than other interventions. Out of the ten, two planted Irish potatoes, an AVCD promoted crop. The rest planted other (non AVCD promoted crops). Most discussants stated that they had been trained using BCC dialogue cards. They were reminded of the basic lesson.17 The response to the question, “What have you been taught?”, was followed by the question, “What did you do before you learnt this?” Their responses are summarized below. Lesson taught Practice prior to lesson  Pregnant mothers should be clean, live in clean homes, eat well and rest a lot. We were shown the different types of food and how to grow them – we were taught how to make kitchen gardens and prepare nutritious foods. We had to tend them. We were also taught how to make dish drying racks and dry our clean dishes. We became so busy tending the garden and cleaning our homes that we no longer had time to visit neighbors. We thought pregnancy was a disease, we felt lethargic and sick. We would visit neighbors and beg for food. We’d neglect our families and our children could beg from other neighbors too. By the time we delivered the babies we were very weak and had no milk in our breasts.  It is important for every household to have a pit latrine and a hand washing facility like a tippy tap. We did not have latrines – we used to share latrines, up to five households per latrine. We also went to the bush to relieve ourselves.  Mothers should have children in the hospital to avoid birth related complications as well as have the mother and child checked for wellness and children immunization program initiated.  Now we go to hospital and are taken care of and advised what to eat and feed the baby when of When the time came, we walked out of the house with a razor blade and string and gave birth at the grass outside the home, by the garden. “Sometimes, someone could pass by… even your father in-law, but what can you do in that situation.” We’d cut and tie umbilicus. Then we’d go back to the house and sleep 17 For children, 8 food types, including breast milk, are recommended and at least 5 food types should be eaten daily – e.g. eggs, fish, fruits, milk, vegetable. For mothers, ten food types are recommended and at least 5 of these food types should be eaten daily – e.g. fruits, milk, green grams, groundnuts. 66 weaning age. Mothers and children are surviving more than before the intervention.  Even when we give birth at home, we take the child for BCG immunization and return home and bleed. Sometimes to death. Two out of every ten women died at child birth or from birth related complications.  Lactating mothers should have a lot of food and fruit so that they can manufacture milk to feed the child exclusively for six months. Now we are all feeding children on breast milk only for the first six months.  We have been taught to befriend our babies and play with them and to clean the teats before and after a child suckles. Also to feed the child frequently.  We ate poorly during and after pregnancy. Before a month after birthing was over, the child was already very hungry so we gave them porridge in the first month.  We did not intentionally make time to play with the baby or clean the teats before and after every feed.  After six months we were taught how to wean children using weaning foods like potatoes and bananas in milk, sukuma, cabbage, managu, nduma, avocado, bananas, beef, fish and eggs. We weaned our children too early, but denied them eggs stating that eggs made babies dumb. Summary conclusions on nutrition and SBCC interventions Generally, there was a good uptake of nutritious foods across the value chains and counties studied – nutrition education and sharing of recipes was fairly successful. The production pathway of the intervention was the most successful, while the income pathway the least successful. In-between, women empowerment was modestly successful. The production pathway would have been even more impactful had energy and time saving processing technologies, such as pulverizers and shellers, been used. Whereas these were available, the production quantities were inadequate to sustain their use. Inconsistency in reporting of nutrition data across value chains and over multiple reporting periods made a more accurate analysis impossible. For example, the LVC disaggregated some data on training and on children reached by sex. DTC disaggregated their data on technology adoption by sex and that on food consumption by household, WRA, and Children 6 – 12 months. There was rich disaggregated data at different VCs and during different periods. The only consistent nutrition data at the aggregate level was that obtained for the number of children under two years (0-23 months) reached with community level nutrition interventions through USG-supported programs (HL.9-2). Across all the counties visited, and VCs, a common pattern is that there was limited layering or integration of agricultural production and nutrition interventions. These were implemented largely as separate activities, with nutrition interventions lagging production interventions by 3-4 months in most areas and VCs. Implementation of the nutrition intervention started about 3-4 months after the commencement of the technology and agronomy interventions. This is because the nutrition team had to conduct assessment, communication, analysis, design and development, action, and evaluation (ACADAE) before implementation. The ACADAE approach could have caused the delay in harmonizing the nutrition intervention. It would have been necessary to adapt this highly academic approach to a more simplified version that is still effective for development programming, for example, enabling enhancement of women dietary diversity (WDD) and the minimum adequate diet (MAD) for children. Interviews with county government staff found that the WAOs were not trained on nutrition and could, therefore, not integrate nutrition in the GAPs training they provided to farmers. This is 67 despite the fact that there were many more WAOs than CHVs in the counties, which could have eased the burden of integrating and layering nutrition with the technology trainings had they been trained. Since WAOs are mostly holders of Diploma in Agriculture and some are degree holders, they could have supported delivery of nutrition intervention. However, the nutritionists were adamant that using non nutritionists could have disadvantaged the intervention because of the risk of inadequate grasp of the subject or “task shifting”, i.e., placing lower priority on nutrition relative to their core agriculture extension mandate. In Elgeyo Marakwet, asked why they did not train WAOs to disseminate nutrition messaging, the county nutrition team was adamant that using CHVs is the best approach because of the threat of “task shifting”. They instead demanded that the county recruit more nutrition professionals. The problem is that, in many places, there were no nutrition interventions and, therefore, the WAOs would have helped, and expressed interest in being trained on nutrition. Even where there were possibilities for recruiting more nutritionists, pending this, it would have been prudent to engage the WAOs, and have them work under close supervision of the Sub-County Nutrition Coordinators (SCNC) and the Home Economics Officers (HEOs). For example, training the WAOs on nutrition messaging may have helped address the challenge of farmers eating seeds, which occurred in EMC. The Bungoma CECM Agriculture, and his staff, however, expressed an openness and interest in integrating the two departments in future nutrition programming, particularly to engage the newly-established HOMEC department and the agribusiness department. Overall, there was awareness and recognition of the dialogue cards provided by AVCD as a useful training tool, but it was felt that its application could have been deeper. In all counties visited, FGD participants and some key informants considered nutrition and SBCC training offered by AVCD as inadequate – both in scope and depth of interaction with beneficiaries. Some communities stated that they were trained only once, which is contrary to USAID requirement of at least two trainings before reporting a training activity. According to nutrition KIIs, it is possible that training was done more than once, but subsequent trainings had high attrition - many dropouts, especially for home-based targets; group targets had a lower dropout rate. The project used existing government infrastructure, which helped a great deal, especially during the COVID-19 lockdown. While other projects stalled, the nutrition intervention continued because the CHVs continued to work on nutrition as they worked on sensitizing communities on COVID protocols, once they were issued with personal protective equipment (PPE). There was a challenge of coordinating nutrition activities between the CGIAR centers, probably a carryover from their research tradition that focused on the respective disciplines. The program nutritionists struggled with this disciplinary silo syndrome, which appeared to be stronger than what they had observed in predominantly development projects. Another challenge was that the nutrition component was lumped together with other cross cutting issues, such as gender, and undertaken by non-nutritionists who were untrained, poorly understood, or had little interest in, nutrition. In addition, nutrition key informants stated that the program should have identified nutrition training needs and made nutrition training materials ready at the beginning of the project / program to be implemented with other project activities. 68 The layering of nutrition improved tremendously when the social behavior change communication (SBCC) aided by the cards became the main intervention. Community level nutrition education and behavior change definitely changed the behavior and care practices for women and children, as demonstrated in several nutrition FGDs. However, the nutrition intervention focused primarily on training, and therefore did not meaningfully influence women’s access and control. Although many women and children were reached, the women’s dietary diversity scores (WDDS) and Children’s Dietary Diversity scores (CDDS) improved only marginally, by 10 and 13%, respectively. Lessons from nutrition and SBCC interventions  Cultivate and utilize existing government infrastructure early on. AVCD’s strong relationships with county governments helped the nutrition interventions a great deal, especially during the COVID-19 lockdown when government departments of health remained active as they worked on pandemic prevention interventions. Consequently, while other projects stalled, the nutrition intervention continued because the community health volunteers continued their nutrition work as they worked on sensitizing communities on COVID protocols.  The three pathways to better nutrition need to be complementary but do not always work in tandem. The production pathway would have been even more impactful had energy and time saving processing technologies been part of the technology package promoted by the program. Although women were trained on production technologies e.g. multi￾cropping, processing e.g. shelling, drying and storing maize in hermetic bags and raised vermin roof granaries, as well as use of energy and hence time saving stoves and water harvesting technologies, in addition to food groups and how to prepare them, they were not facilitated by the program to access these technologies, and did not, therefore, benefit from the knowledge. Where mechanical grain shellers (time saving technologies) were available, quantities of grain produced were inadequate to sustain their use.  An action plan on how to empower women from the various value chain contexts beyond training, and looking into ways of increasing access, decision making, time burden, and workload could have enhanced the effectiveness of the empowerment pathway. Access to suitable credit terms, which can be enhanced by collective action, and affirmative action, such as the provision of energy stoves and water storage tanks, could have reduced the time burden and workload for women.  Consider affordability. AVCD’s focus on single value chain per beneficiary potentially undermined potential nutrition benefits of the interventions; most households were too poor to afford the recommended foods. Additional support was therefore needed to develop supplemental or complementary value chains, e.g. the establishment of kitchen gardens and creation of agribusiness opportunities. More specifically, in Elgeyo Marakwet County (EMC) most nutrient-dense food produced (green grams and ground nuts) was sold because growing communities barely ate it owing to its unfamiliarity. Communities growing them did not know to prepare it. Green grams have poor storability, because of its high susceptibility to weevils, but it also has higher cash generating potential. Farmers sold most green grams because of their poor storability. Field days were held to teach farmers how to prepare the various foods and motivate communities to utilize the nutrient dense DTC grains and legumes. Hermetic bags could 69 be a solution to the weevil problem if they had been promoted. Mothers indicated that most of the required foods were not produced at home or in the farms owing to the semi￾aridity of the EMC agro-ecological zone and had to be bought. Most households were too poor to afford the recommended foods. Other challenges cited were unsuccessful attempts to raise chicken for eggs and meat as well as some crops because of pests and diseases. The mothers recommended that they be supported to establish kitchen gardens and to create agri-business opportunities for mothers.  Take a broader view of nutrition, beyond mothers, caregivers, and children. There was limited nutrition outreach to other members of the community, specifically men, who determine resources available for procurement of diverse foods. Educating mothers or caregivers on nutrition while excluding those most responsible for food choices and acquisition was an oversight. One, a gender sensitive intervention was reported to have been conducted to address a negative gender / nutrition intersectionality problem. In Isiolo, naming newborn babies is a male gender role and a baby cannot be breast fed before the naming. Some newborn babies were denied breast milk after birth because their father was away, and made to wait till their fathers’ return to name them. To protect newborn babies from this community from hunger, the program initiated a father-to￾father sensitization program, whereby fathers that are involved in harmful practices learn from those that are gender and nutrition conscious.  The program and government nutritionists struggled with disciplinary silo syndrome, where coordination of nutrition activities was considered a preserve of trained nutritionists. The tension created between nutritionists and agriculture officers curtailed collaboration and delayed the delivery of nutrition interventions. There, however, is an openness and interest within county governments to integrate the two departments in future nutrition programming, including engaging Home Economists and the agribusiness departments; future programming, therefore, should take this into consideration.  Identify nutrition training needs and develop training materials at the beginning of the interventions so that they can be implemented with other activities. According to the nutrition strategy, the impact of increased productivity on diets should be stronger where nutrition education, social and behavior change communications, and women's empowerment interventions are meaningfully incorporated into agricultural interventions. 7.2 EVALUATION QUESTION 2: How effectively did AVCD address FTF cross-cutting issues, including gender, youth, Natural Resource Management, climate change among others? The data presented here is the aggregate data relevant to gender inclusion and analysis constituting the topline indicators, annual survey indicators, and value chain specific indicators. Annual data, from multiple sources, were collated and used to measure change as a result of the interventions. Where available, annual targets for 2017, 2018, 2019, and 2020 were aggregated and termed cumulative targets. Data were not consistently available for all years. For gender indicators, data were only available for 2020. Table 12 represents the gender topline indicators for only 2020 and cumulative indicators for children from 2016 - 2018. Table 12: Topline indicators achievements for Gender and Nutrition for 2016 – 2020 70 Indicator ID Indicator Target Achieved % achieved GNDR – 2 (LVC) % female participants of USG nutrition sensitive agriculture activities consuming a diet of minimum diversity 30 29 97 GNDR – 2 (DTC) Percentage of female participants in USG￾assisted programs designed to increase access to productive economic resources 50 64 128 GNDR – 2 (PVC) ” 40 70 175 Youth – 3 (LVC) Percentage of participants in USG-assisted programs designed to increase access to productive economic resources who are youth (15-29 years) 30 43 143 Youth – 3 (DTC) ” 15 25 167 Women and youth inclusion in specified activity was high with targets being almost achieved or exceeded in 2020 for all indicators identified. 97% of the women targeted through the LVC were realized as participants of nutrition sensitive agriculture activities and as consumers of a diet of minimum diversity; and 128% of the targeted women through the DTC, and 175% targeted through the PVC designed to increase access to productive economic resources. Targets for youth were similarly exceeded, with 143% and 167% of the targeted youth in PVC and DTC, respectively participating in activities to increase access to productive economic resources. In the annual survey, gender disaggregated indicators were measured to establish how men and women and boys and girls accessed and therefore benefited from interventions. Generally, more men (54%) were reached than women (46%) and more girls (52%) than boys (48%). These differences were slight (Table 13). From the limited data, women and youth were reached and access to interventions by gender appears similar. Nevertheless, the data provided were patchy and not supported with baseline data. Further, it was unclear how the targets were arrived at and what difference achieving them would have made. Table 13: Gender ratio of adult and children reached by interventions Indicator ID Indicator Men (%) Women (%) 3.1.9(1) Number of people trained in child health and nutrition through USG-supported programs (2016 and 2017) 2,730 (53) 2,451 (47) 3.1.9(15) Number of children under five reached by USG-supported nutrition programs (S) (Disaggregation of parents (2016) 1,363 (55) 1,095 (45) 22165 (51) Number of children under two years (0-23 months) reached with community-level nutrition interventions through USG￾supported programs (RAA) Boys￾21,030 (49) Girls – 22,165 (51) 4.5.2(7) Number of individuals who have received USG supported short-term agricultural sector productivity or food security training (RiA) (WOG) (2016,2017,2018, 2019 and 2020) 82,173 (44) 105,354 (56) No identifier Number of people using climate smart information of implementing risk reducing actions to improve resilience to 60,634 (51) 58,360 (49) 71 climate change as supported by USG assistance (2017 and 2018) Qualitative analysis In the DTC, women are at the center of most farming activities. Women reported playing a lead role in decisions around production and sale of crops. While ownership of land is still vested in men, the increased role women played in the DTC can be built further upon to lead them to work towards land ownership or cooperatives. Consequently, production of the DTC in larger farms often attracts the participation of men, who might want to control income from sales. The youth, on the other hand, are involved mostly in processing – i.e. threshing of grains, packaging, and transport to markets. Below the ET highlights notable gender and social inclusion findings from certain counties: Taita Taveta: The Taita Taveta Potato Producers Community-based Organization (CBO) is a mixed gender group of 200 registered members consisting of 70% women (youth and older), 10% male youth, and 20% older men representing ware and seed potato producer groups. The male dominated seed producer groups are few (we counted four) with remaining female dominated groups being ware potato producers. Representation of women in the DTC VC in Taita Taveta is the opposite of the potato VC. Asked why this is the case, male respondents from this mixed gender group supported the statement that “men are patient learners and women find planting very time consuming and do not want to be tied to that work” (Male DTC FGD respondent). Responding to the statement, one young female FGD respondent stated “shamba ya masharti sitaki”, which is loosely translated to mean that she does not want a garden with conditions. Other women supported this statement, confirming that only about 50% of married women have access to agricultural income and, therefore, making women reluctant to participate in DTC production. The situation represented by the two contrasting value chains in Taita Taveta, a highly commercialized (DTC) and a non-commercialized (PVC) value chain, explains the gender distribution of opportunities and benefits. Before commercialization of the DTC crops being controlled by men, these crops were termed “orphan crops” and were owned and managed by women. Commercialization is sometimes associated with the adoption of new technologies, which require resources and may reduce the role of women who often cannot afford new technologies (Fischer et al., 2012) and or introduction of new, often formal markets, inaccessible to women (Njuki et al., 2011). Commodities generating lower average revenues are more likely to be controlled by women, whereas men control high revenue generators, often sold in formal markets because income and resources in most households are not pooled and allocated (ibid.). Crops like green grams and groundnuts fetch very good prices and men are reluctant to share proceeds from their sale. FGDs with farmers in Taita Taveta suggested that only about 50% of the women in the community had access to the income obtained from sale of valuable DTCs. An important lesson from these findings is that benefits of technological change are not equitably distributed across gender and social classes; for instance, successful technological change can have the unintended consequence of making women worse off when its application requires resources that are out of their control or costs beyond their financial means. 72 Elgeyo Marakwet: The CECM Agriculture and Irrigation reported that the county government is very intentional in the inclusion of women, youth, and disabled persons to improve gender equality through the county equitable development act. Unlike Taita Taveta, where ware potatoes are produced by women and seed potatoes by men, the situation is reversed in EMC. Seeds are produced mainly by women’s groups, which they bulk and sell as a business. Because land holding per capita is larger in Elgeyo Marakwet than other counties, women are able to secure at least an acre for own production. Women raise funds and support each other through the merry￾go-round or rotating savings and credit associations (ROSCAs) to afford seeds.18 Youth groups are also involved in business. Groundnut production is big in Elgeyo Marakwet but shelling by hand is extremely difficult. Mechanical shellers were tried, but they could not be adjusted by grain size; they ended up crushing or breaking large-sized nuts. On whether there are gender considerations in DTC, key informants interviewed at the county offices reported that about 60% of AVCD beneficiaries were women; youth participation is limited because of a general disinterest in crop production. This is partly due to limited access to land and weak control over proceeds of farming. The DTC component made deliberate efforts to target the youth with training, starting with a target of 5% and then moving to 10%; ultimately, male youth constituted 15-17% of the DTC beneficiaries. At the various FGDs, participating young mothers, especially single mothers, stated that they were largely forgotten by the project despite their rising population. With the exception of the component that targeted young adults on nutrition, there was no intervention targeting young mothers. Consequently, there is a need to develop interventions for young single and landless mothers. In both value chains, it is not clear how far AVCD drew in particularly households in the lower economic quintiles to participate in value chain activities. Inclusivity is thus difficult to identify, and to attribute. Several lessons emerge from the AVCD intervention on cross cutting issues, among others: ● Cross-cutting issues are equally important and require intentional and well-planned integration/mainstreaming and layering by qualified professionals across implementing centers representing each issue and controlling a realistic percentage of the budget. Enhanced cross disciplinary coordination of plans, intervention, and evaluation could have improved cross-cutting outcomes. ● Beneficiary targeting is key to successful implementation and needs to be carefully thought through, with evidence, before project launch. According to AVCD annual reports and KIIs with staff, the projects initially targeted individuals with interventions, thereby reaching only a few people; the targeting method was revised mid-course to prioritize group-based interventions and greater utilization of county extension machinery. The revision resulted in intervention targets being achieved or exceeded over the rest of the life of the project. 18 A “merry-go-round” is a small social organization where members contribute a small sum of money on a regular basis, often every week. Each time money is collected, the full sum is paid out to one of the members. By participating, members are essentially putting money away until it comes back to them as a larger sum. 73 ● Engage specialist resource persons at the beginning of the program to develop common cross-cutting strategies. For instance, a gender and social inclusion strategy would have better guided intervention design and targeting. ● The potentially high (sunk) cost associated with technology, innovation, or enterprise uptake and / or failure demands enhanced interaction with beneficiaries and highly adaptable evidence-based delivery models. While the program may easily have absorbed any losses incurred from the technologies disseminated, it would be too expensive for farmers and private enterprises were they to invest at market cost. More so, during implementation, program implementers should conduct regular (more than once a year) monitoring of interventions to establish progress in knowledge and practice, share experiences among value chain actors, and apply key lessons to programming. ● Studies of value chain, market and social dynamics, and testing existing models before designing interventions and launching activities should inform program and intervention design. The ET did not find evidence that AVCD conducted value chain and market studies or adapted evidence from existing studies in designing interventions and guiding implementation. Various key informants interviewed, both government and non￾government partners, and beneficiaries, were categorical that designing and implementing the interventions should have started with a value chain and markets analysis, including analysis of gender and stakeholder differences, to better understand the complex issues relevant to robust intervention design and implementation. 7.2a. Which technology dissemination models were appropriate in the different contexts – High Rainfall, Semi-Arid and Arid areas targeted by AVCD? While most of the counties targeted by the AVCD program are categorized as either Semi-Arid or Arid, some of the counties receive relatively high rainfall; these include Bungoma, Nandi, Vihiga, Migori, and parts of Taita Taveta, Elgeyo Marakwet, Homa Bay, Siaya, Busia. AVCD targeted these agro-ecological zones with different types of technologies using various technology dissemination models, as described below. Developing value chains in non-traditional and pre-commercial areas. The program focused on introducing or upscaling value chains in non-traditional areas, both across and within counties. For DVC, the establishment of Dairy Business Hubs/Innovations platform, community￾based animal health extension (enhanced agency network), Fixed Time Artificial Insemination (FTAI), vaccination against ECF, fodder production, and the FEAST and Teckfit platforms were aimed at building a critical mass of dairy animals that would generate interest and crowd-in private service providers. In the PVC, the promotion of potatoes in non-traditional areas of Bungoma and subsistence systems of Taita Taveta succeeded in creating awareness and generated interest within the communities and county government that unlocked additional production and investment. In the DTC, AVCD successfully introduced groundnuts growing in Taita Taveta for the first time. The approach to technology dissemination was largely supply-driven, including direct provision of free or subsidized seed, drugs, vaccines, and services. In most cases and VCs, the program assumed the role of direct procurement on behalf of beneficiaries. This was informed by the realization that supply systems for these inputs and services were underdeveloped for the value 74 chains and within the program counties. While this approach facilitated access to unavailable seed, animal health products, and services, it proved less effective and inefficient. Across all counties and VCs, beneficiaries interviewed reported delayed delivery or non-delivery of seed, semen, vaccine, and other inputs. In some cases, farmers waited for seed from AVCD even when they could plant their own seed or purchase from local agrovets. As such, there is limited evidence of ownership of the technologies disseminated, with most farmers reporting inability or unwillingness to spend their own resources in procuring them without project support. In the DTC value chain, for instance, farmers in Makueni reported that AVCD delayed in making its seeds available, but interviews with AVCD staff reported having provided seed only once; this means farmers expectations were at odds with the program strategy. Developing supply systems for seed, animal health products and services: ● In the DTC and potato value chains, seed diversification was well disseminated, particularly following the establishment of seed bulking centers and farmers engaged in quality seed production and distribution of seeds suitable for the different agro-ecological zones. Some of the crop value chains which benefited from the seed diversification technology include groundnuts, green grams, sorghum, pigeon peas and potato. ● Interventions to build local seed systems through the establishment of community seed banks and support to seed producers were largely ineffective. Most seed banks are not operational and seed producers lack capacity to source or multiply sufficient quantities for their catchment areas. In some cases, like potatoes in Bungoma, the demand for bulked or procured seed was relatively low because of its high cost, i.e., Kshs. 3600 per 50-kg bag (c.f. Kshs. 2500 for extended 75-kg bag from local markets or farms). ● Community-based service delivery. The use of community-based service providers (CBSPs), such as Community-Based Facilitators (CBFs) by ICRISAT, Dairy Farmer Assistants (DFAs) by ILRI, and CHVs in nutrition education proved more effective, and should be expanded in similar programs. The success of these CBSPs is their proximity to, knowledge of, and familiarity with the local communities, including tastes and preferences and social norms and codes of behavior. They performed better where the program sufficiently trained them and facilitated (transport and lunch) their outreach to farmers and households. In other cases, such as the KASBONDO Cooperative, the DFA was not paid but allowed to charge members standard fees for services provided. The program used existing infrastructure of both government and other development programs, which helped a great deal in disseminating technologies and management practices: ● AVCD developed strong working relationships with county governments as implementing partners, with a model of delivery highly dependent on county governments structures, specifically county extension officers. In addition, the program tapped into existing programs, both USAID and non-USAID, for faster and easier entry. However, except in the LVC and DTC, where there was joint planning under the Partnership for Resilience and Economic Growth (PREG) and Southeastern Kenya (SEK) networks, engagement with county governments and other partners were not structured to clearly specify roles and responsibilities and thereby extract firm commitments from the partners. In most cases, the program wrongly assumed that county government or partners would support its interventions merely from verbal commitments. An example is Homa Bay County, where the 75 county government failed to facilitate animal health assistants (AHAs) who were to take over the provision of conventional AI, ECF vaccinations, and pest (tick) control; it also did not monitor progress of the interventions, such as disease prevalence and mortality rates after AVCD intervention. ● The dependence on government structures, which were not adequately resourced and have other competing activities and programs, posed implementation challenges. It worked better in areas where the promoted value chains were less established and had fewer or no programs, but less so in areas with more established value chains. In livestock, for example, AVCD collaborated with county governments and the private sector to provide vaccinations and animal drugs to farmers in remote parts of the five counties, at livestock markets, or at livestock watering points. While AVCD provided the vaccines, the county governments provided the supply infrastructure, which the private provider (Sidai Africa) used to sell animal drugs to pastoralists. It is reported that 90 percent of the farmers who brought their animals to be vaccinated bought at least one animal drug, but the initiative did not last more than three months because pastoralists were reluctant to buy genuine drugs at market prices, largely due to the ubiquity of cheap counterfeit drugs in local markets. ● A less successful technology dissemination model used by AVCD involved mobile ambulatory services on designated routes in which the county government and AVCD would cover specific routes in the counties, and the livestock keepers would bring their livestock to be vaccinated at designated points along the route. This initiative was implemented by Sidai Africa but collapsed due to high cost of provision and reluctance by pastoralists to purchase the more expensive quality drugs because of ubiquitous availability of cheap counterfeit drugs. ● The dairy intervention did not involve county livestock officers as much as the other value chains. Although the livestock department personnel were aware of the DVC work, they did not work closely with the DVC team. Where they worked with county governments, there was a tendency to pick individual officers and, thereby, sidestepping established county or departmental governance structures. A conversation on approaches to the different value chain interventions could have prevented some mistakes, like excluding the county veterinary department personnel from the interventions. Individual-based versus collective dissemination of technologies and management practices. In Phase 2, the program adapted its initial individual-based outreach model toward using FPOs as the medium for farmer mobilization, input procurement, and marketing. Effort was redirected to facilitating the formation or capacity building of FPOs, including cooperatives, to turn them into last-mile intermediaries with farmers. Where available, AVCD worked with FPOs already under development by other programs, e.g. TANAFACO in Taita Taveta (KCEP), EMC-PMCS in EMC (ASDSP), and KASBONDO FCS (PICO/KAVES). In counties where no functional FPO existed, the program facilitated their formation and training of leaders, e.g., Nandi Potato Grower’s Farmer Cooperative and KWESTO in Bungoma (with GIZ and county government). The FPOs facilitated under FPC were modestly successful while those in the other value chain components achieved only limited success. “One-size fits all” approach to value chain development. AVCD applied standard value chain development strategies across the three main commodity categories – crops, dairy and livestock, counties, and agro-ecological zones. Within commodity categories, similar strategies and 76 approaches to value chain development were applied. The overarching approach was to push standard pre-determined technology packages irrespective of the felt needs of target beneficiaries and local context. For example, in all three counties where DTC was implemented and visited by the ET, while green grams and groundnuts appeared in higher demand than finger millet, pearl millet, pigeon peas, and sorghum, AVCD tended to include all the crops in its basket. In Elgeyo Marakwet, for example, farmers in Soy South Ward did not sufficiently take up sorghum, finger millet, green grams, and pigeon pea promoted by ICRISAT and Egerton University because of their preference for groundnuts – the traditional crop in the area, and unfamiliarity with the rest. In other areas, such as cases where multiple value chains were implemented, AVCD did not integrate its interventions; it instead implemented the different value chains in different locations. An example of this is Rangwe Sub-County in Homa Bay, where beneficiaries of the OFSP intervention were not aware of dairy interventions in other parts of the area; this is despite the fact that dairy and OFSP enterprises should ideally be complimentary – OFSP vines/peels as animal fodder and dairy for animal manure. Use of private service providers. Overall, in high rainfall areas, including the DVC and PVC, linkage to private service providers appeared to be a more acceptable model of service provision and extension by the beneficiary farmers. In the DVC, for instance, use of the private sector in promoting AI worked well in most areas. Although the FTAI technology was less successful due to the high cost of provision and low conception rates, the conventional AI system continued to be adopted by farmers even in counties where county government subsidy on the same had been stopped. Dairy farmers were more willing to pay private AI service providers. Similarly, farmers were more receptive to the use of private input providers in the propagation of Brachiaria seeds and splits, with farmers willing to pay for the inputs including land preparation services. For extension, the use of dairy farm assistants (DFAs) was well accepted by beneficiaries because of its perceived value beyond the project lifecycle. In the semi-arid areas, the farmer-to-farmer extension models were well received in the dairy, potato, and DTC VCs, resulting in significant technology dissemination. Farmers provide extension either as a community service or at a much lower rate that is affordable to smallholder farmers in the semi-arid areas. In addition, use of demonstration plots for fodder establishment, potato production, and DTC enhanced adoption. In the arid areas that comprised mostly the Northern Kenya counties, interventions focused on community involvement in rehabilitation of grazing lands, private sector in establishment of conservancies, and re-seeding of pasture lands. Market linkage through the introduction of inputs suppliers, service providers, and produce off takers, and developing market information systems. For all areas and all value chains, linkage to markets was less successful. The livestock value chain attempted to establish livestock market information system (LMIS), but the system was not installed and, hence, no results realized. For potatoes, market linkage for ware potato and apical cutting seedlings remained a challenge; however, enhanced adoption of potato storage technologies can help farmers sell when prices improve. Similarly, there is a market for apical cuttings but its uptake requires more awareness, farmer education, and regulatory support. For DTC, markets for groundnuts in Taita Taveta seemed more promising, with companies such as Upfield and INSTA expressing interest in procuring produce from some of the new varieties. In contrast, the sorghum market in Taita Taveta collapsed following the withdrawal of East African Maltings Ltd 77 (EAML) due low volumes, long distances, and COVID -19 pandemic, which affected alcohol consumption. While green grams and pigeon peas had access to markets, prices still remained a challenge with farmers citing oppression by middlemen. 7.2b. To what extent are the promoted technologies widely available in the target areas (i.e., what is the scale of dissemination), and to what extent has dissemination led to spillover effects that include expanded diverse economic opportunities? AVCD was fairly successful in pushing different types of technologies and innovation across value chains and counties. Analysis of the AVCD Performance Plan & Report (PPR) 2020 indicates that various technologies and innovations were disseminated during the five years of the program. Table 14 and Figure 1 illustrate adoption of various technology by male and female households. Figure 1 shows that post-harvest handling was the most adopted technology, at 45%, followed by crop genetics (improved seeds) and pest and disease management at 38% each; about 24% adopted livestock management practices. In contrast, cultural practices (10%), marketing and distribution (5%), irrigation (2%), and value addition were the least adopted innovations. Disaggregating beneficiaries by gender shows that adopters of technologies for crops were women (58%), and about 19% of adopters was the youth (5 -29 years old). Table 14: Year 2020 application of technologies and innovations by program beneficiaries Indicator/ Disaggregation Baseline Value Target (FY20) Actual (FY20) % Achieved % of total Number of individuals in the agriculture system who have applied improved management practices or technologies with USG assistance [IM-level] 29,034 54,422 65,806 121 100 Value chain actor type: Small holder producers 29,034 54,422 65,806 121 100 Sex: Male 9,848 25,280 27,827 110 42 Sex: Female 19,186 29,142 37,979 130 58 Age: 15-29 3,239 17,520 12,362 71 19 Age: 30+ 25,795 36,902 53,444 145 81 Management practice or technology type: Crop genetics 10,553 17,000 25,111 148 38 Management practice or technology type: Cultural practices 4,876 5,345 6,361 119 10 Management practice or technology type: Livestock management 11,000 12,354 15,879 129 24 Management practice or technology type: Pest and disease management 16,123 18,701 25,041 134 38 Management practice or technology type: Irrigation 98 981 1,448 148 2 Management practice or technology type: Marketing and distribution 1,390 2,870 3,050 106 5 78 Management practice or technology type: Post-harvest handling and storage 9,800 18,750 29,400 157 45 Management practice or technology type: Value-added processing 34 245 407 166 1 Source: Performance Plan & Report (AVCD PPR, 2020) Figure 1 While the program was fairly successful in pushing different types of technologies and innovations, most of the introduced technologies and innovation are unavailable in the target areas. The ET visits to Elgeyo Marakwet, Homa Bay, and Bungoma counties found that most technologies disseminated during Phase 1 are largely non-existent after farmers abandoned them. For instance, only a handful of farmers in FGDs conducted by the ET were still growing OFSP in Bungoma and Homa Bay, and groundnut, sorghum, millet and pigeon pea varieties introduced in Elgeyo Marakwet. Farmers provided different reasons for disadoption, including unavailability of seed, high vulnerability to climatic fluctuations, and unaffordability. Some of the technologies, such as the groundnut varieties introduced by ICRISAT and the ECF vaccine promoted by ILRI, were not yet in the local markets; the groundnut varieties were introduced as part of R&D for release. Specific examples of technologies and innovations include: 38 10 24 38 2 5 45 1 0 10 20 30 40 50 Adoption as a % of total beneficiaries 1 Year 2020 Application of managment practices and Technologies by beneficiaries Management practice or technology type: Crop genetis Management practice or technology type: Cultural practices Management practice or technology type: Livestock management Management practice or technology type: pest and disease management Management practice or technology type: irrigation Management practice or technology type: Marketing and distribution Management practice or technology type: post harvest handling and storage Management practice or technology type: value added processing 79 DTC value chain ● In Elgeyo Marakwet, except for groundnuts, farmers in Soy South Ward did not sufficiently take up the sorghum, finger millet, green grams, and pigeon pea varieties disseminated because of their preference for groundnuts – the traditional crop in the area, and unfamiliarity with and poor markets for the rest. For example, while ICRISAT introduced KARI Mtama 1, Seredo and EU Red 1 sorghum varieties and high-yielding and fast maturing finger millet varieties, only a few of the farmers interviewed by the ET were still growing them. Similarly, groundnut farmers in Soy South Ward (EMC) never adopted new groundnut varieties (Mwangaza and Ndovu) beyond the demo farms, and still grow their local varieties – Cheplambus and Zambia. Groundnut farmers cited poor germination rates, high vulnerability to climate (intolerance to water stress and excessive rainfall), easy sprouting if harvest is delayed, cross-pollination and varietal contamination because farmers plant different varieties in blocks on the same parcels, and lack of seed as the leading reasons for not adopting the new varieties. DTC farmers in Taita Taveta were still waiting for seed supplies from ICRISAT/Egerton at the time the ET visited the county, and those that received seed for multiplication had lost most of them as a result of three consecutive poor seasons. ● Several promoted seed varieties and technologies were not ready for market. For example, the three groundnut varieties promoted in EMC were under development and only formally released after the end of Phase 1; they were unavailable locally, including in agrovet shops. Because seed was not readily available in the markets, the program promoted local seed producers and seed banks to fill the gap while courting private seed companies to set up operations in the areas. However, they operated for only about a year and are currently largely non-existent. Moreover, the private seed companies selected to commercialize the seeds, e.g., Agrosoy Ltd, Dryland Seeds, and Faida Seeds, proved incapable of setting up operations in the program areas because they had no local footprints nor experience with the specific seeds. An additional bottleneck was that all the seed varieties promoted by AVCD are open-pollinated and, therefore, unattractive to private seed companies. ● Digital innovations were vital for monitoring, training, and extension advisories that were disrupted by the COVID-19 pandemic in 2020. The program quickly adapted and changed strategy by using various digital platforms to reach and support beneficiaries. Interactions with farmers, SMEs, and FPOs were mainly through the SourceTrace DATAGREEN platform, WhatsApp, and Zoom. These platforms were used for dissemination of information related to proven technologies using bulk SMS messaging features. The digital interactions and online platforms presented an opportunity to continue engaging the beneficiaries but encountered numerous challenges, including lack of smartphones and shortages of internet bundles and airtime. By the end of the 2019/2020 reporting period, the program reported sending out 444,332 extension advisory text messages to 11,228 beneficiaries who had mobile phones with the capacity to receive the texts. The ICT platform was also useful for the program in coordinating activities during the period of restricted movement associated with the pandemic, where DFAs and AI service providers were submitting breeding and other associated field data. In addition, the dairy component used the platform to manage the subsidy program for FTAI and ECF vaccination interventions. 80 Livestock value chain ● Technology dissemination in the livestock value chain achieved limited success because the delivery model was insufficiently implemented. Technologies such as the online disease e￾surveillance system, livestock markets, and PRM would have been widely available in target areas, but numerous challenges constrained their dissemination. Implementation of the livestock markets systems was not completed, limiting availability. On PRM, while the county TOTs were trained, the actual land rehabilitation message was not taken to the community-owned lands. ● The innovative Community-based Disease Surveillance (e-surveillance) system showed potential to quickly detect disease outbreak to enable timely intervention. The platform improved early warning and response to livestock disease outbreaks, hence, enhancing resilience in Northern Kenya. Based on its achievements and its promise, the IMPR 2017 recommended more investment to upscale and out scale it and crowd-in the private sector in veterinary input supply toward creating a robust animal healthcare delivery system. To support data collection and reporting of syndromic animal diseases by CDRs for the e￾surveillance system, AVCD introduced the use of a Closed User Group (CUG). The technology is said to have enabled real time reporting of disease syndromes by the CDRs, including triggering county veterinary officers to respond where syndromes reported pointed to a disease outbreak. However, the effects were limited because, first, there was reluctance by the county governments to adopt and pay for the system beyond AVCD, and, second, the national government opposed it for attempting to bypass the national system - the Kenya Animal Bio-surveillance System (KABS). The intervention stopped after training and initiating the online disease reporting in four counties. Secondly, data and information collected was not communicated back to inform decisions of livestock keepers. The data was mainly analyzed and managed by AVCD and county governments, with little involvement of communities or their representatives in establishing appropriate media for information dissemination. Likewise, even though AVCD developed a system that collected regular market information on livestock sales, the data was never shared with the farmers. Most farmers lack sufficient formal education, and radio was reported to be a much friendlier medium, but this option was never explored or used. Information seemed to have been extractive, i.e., collected more to suit the needs of project implementers than the beneficiaries. Importantly, the innovation was found at odds with the national animal disease surveillance and reporting system, forcing the scaling back of its implementation. ● At the end of AVCD, the system was still being used, but with mixed levels of commitment in terms of budgetary allocations. The system faced the following challenges: (i) Lack of a coordination mechanism for data collected in various counties. (ii) Competition / duplication to the KABS, whereby counties under AVCD were not reporting to the KABS. (iii)Data security and risk of publishing unverified animal disease data by CGIAR centers. (iv)Inadequate capacity within counties to analyze reported data. (v) Safaricom not considering the CUG as a community service but a commercial activity, which raised the cost of running the CUGs In hindsight, AVCD should have supported the KABS and, thereafter, focused its resources on continuous training of counties on disease reporting and data analysis. Moreover, there was a 81 need for more consultation with the DVS office to avoid any misunderstandings and provide an opportunity for counties to play a bigger role in disease surveillance, specifically cultivating a regional approach in disease surveillance for the frontier counties. Finally, the program should have coordinated more with other development partners in that space for greater synergy and leverage. Dairy value chain ● The DVC component catalyzed the build-up of a critical mass of improved cows through subsidized FTAI and ECF vaccination that began to attract private service providers in the project counties. This was expected to reduce the cost of services and, in turn, the cost of dairy production, thereby triggering further interest among farmers and the growth of the dairy value chain. The ET scrutinized program reports and data and additional information from qualitative interviews to gain better perspectives on these technologies. It found that the two technologies were less successful, largely because of complex infrastructure and logistics required to deliver them, cost, and unavailability in the local market. The ECF vaccination, for example, was less successful due to delays in starting the intervention, low uptake, lack of regular spraying to control ticks for the vaccinated animals, and limited follow-up by CDVS due to limited funding. Importantly, the current ECF vaccine delivery system is unsuitable for most target areas – the ECF vaccine has to be stored in liquid nitrogen at -186oC, packed in doses of 40 vials, and stay viable in ice for only 2 hours after reconstitution. Further, at about KSh.1000 per service, it is relatively expensive for most dairy farmers and county governments. ● For FTAI, the realized conception rates were low, e.g., 42% and 30% for Makueni and Homa Bay counties, respectively, against a minimum requirement of over 60 per cent. At the time of the field visits, limited FTAI provision was confined to cooperative societies. The low success rate of FTAI was attributed to the following: (i) Too many cows synchronized at the same time but scattered over wide areas such that AI was administered too late for some of them (in the process missing the heat window); and, (ii) Repeat heat cycle for the unsuccessful inseminations was never reported by some farmers and, where reported, inseminators were not always available. ● Public and private animal health service providers were trained in the technologies and are still providing conventional AI services. These AI services continue because they are already commercially available and, there are new crossbred cattle that have been vaccinated against ECF. If successful, all crossbred calves will require ECF vaccination and regular spraying, and hence expanding the market. ● The ET also sought to further understand how the improved feeds and feeding, and specifically Brachiaria (signal grass) fodder, contributed to increased dairy productivity and alternative sources of revenue for commercial fodder producers. The project reported to have trained 6,000 farmers on the value of fodder establishment and conservation and supplied Brachiaria seeds amounting to 565 kilograms for fodder establishment while also encouraging farmers to invest in the establishment of other fodder varieties; the amount of seeds distributed was expected to have established more than 1,000 acres. Visits to and interviews with dairy stakeholders, including FPOs, in Makueni and Homa Bay counties to establish the extent to which the push and pull interventions had been implemented found the 82 outcomes mixed. Brachiaria establishment had produced significant results, including an estimated 400 acres established in Makueni County with several farmers engaged in commercial fodder production. In Homabay County, members of the Kasbondo Dairy FCS increased the area under improved fodder from 106 acres to 716 acres (80% Brachiaria), although adoption was predominantly among larger scale farmers. In Homa Bay, the CDVS was of the opinion that farmers never understood Brachiaria before adoption and thus allocated only small parcels of land and did not avail labor for harvesting. Further, there was no training provided on fodder harvesting and preservation. Importantly, the fodder processing and packaging method promoted by AVCD proved incompatible with dairy farming systems; for instance, the Homa Bay CDVS informed the ET of their discovery that cows love lush Brachiaria, not the baled hay. Save for a few medium - to large-scale commercial farmers and the cooperatives, most farmers stopped tending the Brachiaria fields and left them for open grazing, and thereby killing the plants. In some areas, however, Bracharia production was commercialized by non-dairy farmers, and thus leading to indirect technology benefits. In hindsight, the program should have focused on other fodders in addition to Brachiaria. ● In collaboration with the KCDMS program, it adapted an ICT platform from the African Dairy Genetic Gain (ADGG) program aimed at improving efficiency in breeding service delivery. The platform was instrumental in strengthening the capacity of county governments to oversee implementation of breeding activities, which were also embedded in the dairy development strategies. Many counties implementing breeding programs were using the platform to monitor delivery of AI services and expressed great satisfaction with the platform, while providing feedback for improvement. For Makueni, it was reported that the platform reduced the tedious paperwork associated with administration of subsidy and management of claims filed by service providers to reduce pilferage. The ET sought to further understand the workings of the system, its contribution to dairy value chain development, and its cost and sustainability, including how far efforts to engage county governments have gone and how effective they have been. It was found that the ICT platform had not been installed at the time of the evaluation. ● The program also introduced the feed assessment tool (FEAST) though it is yet to be adopted, with cost of feeds being reported as a major challenge. Root Crops value chains The OFSP variety was launched in Homa Bay, Migori, Busia, and Bungoma counties in Phase 1 and quickly reached its targets by the end of the phase and, therefore, discontinued. The ET sought to understand the drivers of what appeared to have been a stunning success with interviews in Homa Bay and Bungoma counties. It found that while the recipients embraced the variety and generally recognized its nutritional benefits, the intervention delivery model focused on one-off supplies of small quantities of vines at health centers for multiplication. Minimal support was provided to farmers who planted on agronomic practices, particularly to address challenges of inadequate supplies of vines, climatic variations, pests and diseases, and marketing. In Homa Bay and Bungoma, the ET found that nearly all farmers abandoned OFSP immediately, once the AVCD program came to an end. The leading reasons for disadoption was the collapse of vine supply systems due to drought conditions that wiped out the vines supplied or multiplied and underdeveloped markets. The OFSP farmers interviewed indicated that they are short of 83 planting materials. However, some agronomic practices, such as ridging, were still being practiced, but on traditional or local sweet potato varieties. In Tongaren, Bungoma, some former OFSP farmers were adopting Irish potato using the GAPs knowledge acquired from OFSP training. ● Irish potatoes for seed and ware are produced in Taita Taveta, Elgeyo Marakwet, Nandi, and Bungoma. In all counties, new (early generations of) seed potato varieties and good agricultural practices – spacing, hilling, use of fertilizers, dehulming, and application of pesticides and anti-frosting agents - were introduced. Interventions were informed by the finding that on-farm yield was too low for farmers to break even; this was attributed to farmers not having adequate knowledge of good agricultural practices (GAPs). For example, the baseline survey indicated an average of 8 t/ha and 6.3 t/ha in Bungoma and Taita Taveta counties, respectively, much lower than the 10 t/ha required to break even. The value chain interventions included increased access to quality seed and good agronomic practices and knowledge, improving the enabling environment, county government investments and business support leading to increased productivity. Others included organizing FPOs in marketing and support to private sector players involved in the potato value chain resulting in increased incomes and improved household nutrition. However, the ET found that potato FPOs supported in Taita Taveta, Bungoma, and Elgeyo Marakwet counties had not acquired functional capacity for seed production. ● In Taita Taveta and Bungoma counties, male and female potato farmers were knowledgeable on the potato varieties and selected the varieties to grow according to their preference. For example, farmers from the high rainfall areas chose the short dormancy period Shangi variety so that they could replant them sooner during the multiple cropping seasons. Farmers that planted Unica stored them in a warm place to shorten the three-month dormancy period. ● In Taita Taveta, CIP introduced apical cuttings technology for potato seed production in 2020 and supported a private firm (Taita Papa) as the producer. The firm faced challenges of poor production resulting in a loss of 1,050 mother cuttings at the start of the project; this was attributed to poor installation of the screen house and temperature controls, poor quality tissue culture splits received from the supplier, and water logging at establishment. However, following technical support from CIP, the firm successfully matured 300 mother splits that produced 7,000 cuttings for planting. CIP bought more than 75% of the cuttings and distributed them to farmers, while the rest were bought by individual farmers. Taita Papa has since been registered as certified potato seed producer by the Horticultural Crops Directorate (HCD), but low awareness on the technology among farmers has constrained its market; farmers still prefer the old technology of seed potato. ● Potato farmers in Elgeyo Marakwet, Taita Taveta, and Bungoma moved from the new varieties introduced by AVCD to their local varieties or abandoned production altogether due to seed supply constraints. In all three counties, potato farmers could not get supplies of the preferred varieties that had shown the most promise in their areas, such as Unica and Shangi. In EMC and Bungoma, farmers resorted to selecting own seed or buying seed from local markets after supplies of free seed ceased and/or seed multiplication or procurement failed. In Elgeyo Marakwet, for example, potato farmers returned to varieties, such as Shangi, Romano and Nyayo, after exhausting the certified foundation Unica seed provided by AVCD. Similarly, in Bungoma’s Elgon area, farmers reverted to their local variety, Alga, after trying the foundation certified seed AVCD supplied to demo farms. 84 ● For the potato seed multipliers, there is good business as the demand for seed potatoes is still very high. The findings from KIIs and FGDs indicated that more farmers prefer growing the Shangi variety, which is high-yielding and preferred across market segments. This has led to abandonment of other varieties and, thereby, shortages of supply. However, the main challenge with Shangi is its poor storability – short dormancy period, and high losses during processing – deep eyes. For ware potato production, farmers reported unfavorable markets and prices, with an extended bag of potatoes going for Kshs. 300-1000 in Bungoma, Elgeyo Marekwet, and Taita Taveta. ● The farmers across the board were discouraged by the poor pricing and desperate for appropriate storage facilities where potatoes will remain dormant and can be sold after the glut season. The Diffused Light Store (DLS) for potato storage were found in all the potato counties, the latest being Bungoma. These stores were mostly non-operational at the time of the evaluation because they either were incomplete or lacked sufficient potato volumes to store. As such, economic and financial viability of the DLS remains questionable. Storage of ware potato did not seem to provide an effective solution to potato marketing issues. Because of the prevailing market dynamics where traders prefer buying potatoes on-farm from individual farmers using extended bags or tins, potatoes graded and stored in standard packages had no markets. An example of this is the potato farmers in Ndalu Ward, Tongaren Sub-County, Bungoma, who the ET found stuck with several bags of good quality potato stored at their local cooperative awaiting a buyer. Most of the technology access issues are attributable to a lack of functional seed/livestock market systems and supply chains to support the disseminated technologies and innovations. The ET’s assessment from the literature review, KIIs and FGDs is that AVCD was generally weak on this aspect of value chain development. The farmers formed producer groups or cooperatives to help address challenges of marketing and low prices. However, discussions with the PO members and leadership revealed that they do not have a clear pathway to address the issue of poor marketing and prices. Markets and pricing are very complex matters, but these POs seem to have been developed as a panacea for all challenges, including seed production and produce aggregation, but with no marketing strategy or plan. Other factors have to do with the appropriateness of some of the technologies disseminated, largely related to unavailability of delivery infrastructure and critical masses to sustain them, and costs beyond the financial capacity of beneficiary farmers. The ET assessed the effects of COVID-19 on livestock sales due to market closures by analyzing the data for 2019 and 2020. The data show that the total annual value of incremental livestock sales dropped sharply (68%) from an average of USD 194,072 in 2017/2018 ($225,000 in 2018) to USD 62,199 in 2020. Other than the effect of COVID-19 restrictions, other extraneous factors limited the effect of the DVC interventions on milk production and marketing. For instance, dairy farmers reported cyclical fluctuations in prices of produce. In Homa Bay, mismanagement and fraud at Kasbondo Dairy FCS Ltd caused a 70% drop in milk deliveries (from 300 liters to 180 liters daily) and the deactivation of the cooling plant as the number of active farmers dropped to 40 (out of 1,200 members). Another significant factor was the location of the cooling plant in an area largely inaccessible during the rainy season, when milk production is highest. Farmers interviewed at Kaiti Dairy Cooperative narrated the losses they incurred due to foot and mouth disease; one farmer reportedly lost seven cows, another four cows, and another four calves. COVID-19 also 85 exerted pressures on farmers, making it hard for them to purchase inputs for dairy production. The consequence has been significant increases in the cost of milk production at the same time milk production dropped sharply, by one-third from pre-COVID levels, on average. The lower volumes produced faced the additional challenge of low demand for milk in the area, due to financial pressures arising from COVID-19. Some evidence of spill-over effects of the technologies and practices disseminated were found. Most notable are the nutrition dialogue cards that have been adopted/adapted for upscaling by some county governments and development partners, such as UNICEF in Taita Taveta. Other examples include the following:  In Taita Taveta, the FAO has adapted the CBFs and agro-weather and extension messaging approaches used by ICRISAT (SourceTrace platform) for its own programing in Taita Taveta. The agro-weather and extension alerts disseminated through ICRISAT in Taita Taveta have reportedly reached populations beyond AVCD beneficiaries (per FAO County Rep.). FAO is upscaling groundnut development and has attracted a large offtaker (INSTA Ltd – a member of the Chandaria Industries) that requires a minimum of 600 acres under groundnuts.  In Elgeyo Marakwet, the county government retained many interventions introduced by AVCD through other programs. For example, AVCD induced the county to allocate a budget for seed multiplication - Sh30 million in FY2021/22 under the Kenya Devolution Support Project (KDSP),and took over some of the AVCD activities through ASDSP, KSCAP, and the new Emergency Locust Response Project (ELRP) that seeks to enhance livelihoods through restoration and nutrition. For instance, ASDSP took over the groundnut and potato value chains – specifically to promote the apical cuttings technology from CIP.  In Bungoma, the Kenya Cereal Enhancement Program (KCEP) started supporting potato production as a COVID-19 mitigation measure after AVCD exited.  In Bungoma and Homa Bay, sweet potato farmers that were engaged in OFSP interventions have adapted some of the agronomic and management practices learnt through the program, such as ridging and side-planting.  The LMS AA2 program adopted the PRM approach for its intervention on improving grazing lands management. 7.3 EVALUATION QUESTION 3 To what extent was AVCD’s Theory of Change (TOC) validated by the activity? What were the strengths and weaknesses of the TOC? How can the TOC be improved? The performance evaluation analyzed the theory of change, the results causal chains (RCC), and project pathways to assess the validity, relevance, and plausibility of the causal assumptions made. It examined whether core assumptions at program commencement continued to be valid through implementation phases, and any readjustments made. It sought to understand the effectiveness of the Push-Pull strategies for each value chain component and different county contexts. Related to this was an assessment of whether the expected effects were achieved uniformly across various categories of farming households, and the reasons underlying any variations. Likewise, the evaluation tested the context/contingent factors that had an effect on the 86 TOC, the implications of the TOC for scaling up technology, and its potential impact for poverty reduction and enhancing household food and nutrition security. A value chain is constituted by a number of steps, as illustrated in the figure below. In the case of AVCD, these were articulated in the intervention pathways for the different value chains with a focus on increasing production, consumption, and marketing through dissemination of technology/innovation, training and advisory support, and linkages to markets. Figure 2: An illustration of a generic value chain Guided by Figure 2, comprehensive analysis of each value chain was necessary to understand each node and identify activities, actors (by gender and age), and resources at every level and node. In addition, the VC analysis should identify anticipated opportunities and benefits from program activities and clearly demonstrate how they should be distributed equitably among the actors and beneficiaries. Actions, actors, and resources to ensure equitable distribution of opportunities and benefits should be demonstrated in a representative TOC developed and explained – including all assumptions made during its development. The AVCD program TOC (Figure 3) envisaged that pushing technologies, innovations, inputs, knowledge, services, and nutrition and behavioral messaging to current value chain players would increase production and productivity and, thereby, trigger a response from the pull side of the value chain. The pull-side included enhancing access to markets, business development services, and financial services, and developing good policies. These, combined, would attract market-oriented VC players to develop a competitive and productive VC; the latter would ultimately lead to inclusive agricultural and economic growth and improved nutrition. 87 Figure 3: The AVCD theory of change The program identified intermediate results, including improved agricultural enabling environment, expanded markets and trade, improved productivity of selected value chains, improved nutrition-related behaviors, and several cross-cutting issues. The ET finds that the AVCD program TOC was not fully realized. The intervention was big on the push forces targeting mostly producers rather than all the value chain actors. While the TOC was a whole chain approach covering both production and marketing interventions, there was limited application of a whole value chain approach, as anticipated in the TOC and delivery pathways. The promotion of the production enhancing technologies was not accompanied by deliberate attempts to address service provision and marketing, for example. The program should have been more deliberate on developing sufficient “pull” mechanisms and delivery infrastructure early on, including developing strategies for creating a conducive business environment and stimulating the market in tandem with the push interventions. The internal midterm program review (IMPR) 2017 flagged the insufficient attention to the pull forces, noting that more effort had been devoted to the achievement of the push, as opposed to pull, interventions. The ET assessed the extent to which the program in general, and the value chain components in particular, realigned intervention strategies in line with the IMPR recommendations to enable the achievement of the pull components of the theory of change. It is unclear whether the recommended corrective measures were taken or not. Reviews of Phase 2 documents and data suggest even lower effort toward unlocking pull forces, including what appears to be a near-complete cessation of private sector engagement after 2018. Lack of the pull force component resulted in a theory of change scenario illustrated in Figure 4. In this regard, AVCD was actually an accelerated agricultural production and consumption 88 model, which is the traditional TOC for most agricultural development and nutrition programs; that is, to improve nutrition, increase food availability, and incomes. Figure 4: An illustration of the resultant change after AVCD intervention The following narratives use specific examples to illustrate the position of the ET on the representation of the TOC. There was an appreciable attempt at large scale application of technologies and innovations. The approach to technology dissemination was largely supply-driven. In the dairy value chain, for example, AVCD promoted Fixed Time Artificial Insemination (FTIA) to stimulate demand for and interest in AI services. In the livestock value chain, AVCD layered animal health campaigns by teaming up with the private sector to sell animal vaccines and drugs to farmers. the program also promoted the mobile-based disease e-surveillance system in the livestock value chain and the ICT platform from African Dairy Genetic Gain (ADGG) to improve efficiency in breeding service delivery. In the potato and DTC value chains, AVCD promoted new crops and the supply and multiplication of new seed varieties. These and other technologies were promoted in the belief that they would stimulate subsequent demand for the technologies. DTC farmers in Makueni reported that AVCD gave them seeds but oftentimes delayed in making the seeds available. The supply of seeds was not, in certain cases, coupled with training on establishment and agronomic practices. Interviews with AVCD staff indicated that seeds were only given once, which differed from some of the accounts given by farmers. Similar experiences were witnessed in the livestock and dairy value chains, where vaccination and animal health campaigns did not trigger the desired effects of sustained technology uptake. In the livestock value chain, counterfeit drugs are a major problem, pushed by vested interests in the supply chain. Even though farmers find these cheaper, they are ineffective and dangerous. It appears from the ET’s data collection and analysis that AVCD had not sufficiently appreciated the negative effects that vested interests in the counterfeit animal drugs supply chain would have on the project’s disease prevention efforts. 89 In conclusion, despite the fact that AVCD made technologies and innovations available to farmers, the mode of delivery made effective uptake difficult. First, supply markets for agricultural inputs and services are weakly developed in most areas targeted by AVCD, largely because, except for the LVC, the program was implemented in predominantly non-traditional production and marketing areas (NTAs). AVCD did not sufficiently appreciate the fact that private sector players, particularly agribusiness SMEs, are risk averse and will only invest where they are certain of sufficient and reliable quantity and quality, that supply chains are effective or at least can be made to work more effectively, and that effective demand exists for the products they are selling to farmers. The assumption that the private sector would quickly follow AVCD into the value chains where it had extended technologies to farmers and created an enabling policy environment was not validated. There is need for sustained engagement with actors to create lasting market opportunities that attract private service providers. Apart from providing inputs and working on the policies to create an enabling environment, careful thought needed to have been given to value chain and market research, including stakeholder mapping and analysis (characterization of actors and their interests) to understand the functioning and potential of existing supply chains. In the LVC component, a systematic VCA could have revealed the negative effects that vested interests in the counterfeit animal drugs supply chain would have on the project’s animal health efforts and developed interventions to address the problem (e.g. an anti-counterfeit drugs campaign through different media). Analysis of AVCD reports and documents did not yield any records of value chain or market analysis conducted to guide the specific interventions. Weaknesses in delivery (e.g. sustained engagement with farmers was lacking in certain value chains) compromised the validity of the TOC. Informed by the TOC, program implementers disseminated technologies and innovations with little attention to local complexities and context. Consequently, inadequate attention was paid to understanding the capabilities of local systems, which were found to have weak internal management systems, inadequate market information, and weakly linked to support systems. Working with groups could have been more advantageous to the program if sufficient time was taken to identify groups of actors of desired gender and age for each of the value chain nodes. Where suitable groups did not exist, the program could have patiently facilitated their formation. Comprehensive value chain analysis, including stakeholder mapping and profiling, and organizational capacity assessments would have guided such a process. The introduction of orphan crops in the market could have been embraced by the more health￾conscious urban population, and there are well-known processors that could have partnered with the DTC VC, in the presence of suitable value proposition and incentives. For potatoes, the Bungoma County potato strategy should be tested and adapted in potato producing counties since the issues it addresses are similar. A thorough PVC analysis should be done in partnership with active market players that have been running successful potato aggregation businesses. Likewise, the DVC too should have partnered with successful dairy processors to commercialize dairy enterprises in the pre-commercial areas. Systematic value chain analyses with specialists and potential partners would have identified these actors and informed interventions. 90 In the PVC, the program did not adequately appreciate challenges in supply and demand for potatoes, including shortages of suitable potato varieties and fluctuations in supply due to the predominantly rain fed production system. The latter leads to periods of gluts and scarcity and, therefore, price surges and ebbs that generally discourage production. Due to lack of production and marketing standards as well as low levels of adoption of good agronomic practices by farmers, the quality of produce is generally poor, which leads to low quality products, high production, marketing and processing waste, or total rejection of potatoes by buyers. Regardless of the above findings, the ET is cognizant of the fact that the program implementation period in most areas was too short and challenges too numerous to make any firm conclusions. While AVCD was implemented for a cumulative six years, disruptions in the aftermath of 2017 general elections during Phase 1, shifting of zones of influence after Phase 1, and cessation of most Phase 2 activities due to COVID-19 meant that AVCD was effectively implemented for no more than two years in most counties and areas. In new Phase 2 counties and value chains, such as in Taita Taveta / Bungoma and DTC / potato, AVCD had been on the ground for less than a year before COVID-19 pandemic halted activities in early 2020. Moreover, intervention activities lasted no more than one year in any single interventions area; for instance, actual interaction with beneficiaries in the DTC and LVC was either for a season (seed distribution and agronomic training) or six months at most. Historically, technology and innovation uptake and behavioral changes take time and the trajectory of change is more often than not non-linear. For example, technology adoption is known to assume an S-curvature depicting rates of adoption over time (introduction, traction, maturity, and saturation) – at different speeds, with long periods required for some technologies to gain traction and reach maturity and saturation.19 Any hypotheses or claims about change processes around technology, innovation, and behavior change ought to incorporate time (by when it is reasonable to expect that a net impact will be attained) and the high likelihood that the trajectory of that change will be non-linear (e.g., a J-curve, S-curve or step function). Paying inadequate attention to changing circumstances and the possibility of non-linear impact trajectories can lead to claims about impact that turn out to be premature, thereby forming an inaccurate basis for future projections.20 As a result, this evaluation concludes that the AVCD TOC is unproven, but not completely invalidated. Several lessons are derivable from the foregoing analysis: 19 Gleaned from Michael DeGusta, “Are Smart Phones Spreading Faster than Any Technology in Human History?”, MIT Technology Review, May 9, 2012. Traction is defined as the time it takes the technology to move from consumer availability to 10% penetration (percent of total households or consumers). In the USA, for example, it took the telephone, electricity, television and mobile phone about 25, 30, and 12 years, respectively, to gain traction. Maturity refers to the time it takes for the technology to move from 10% to 40% penetration. Again, it took the telephone, electricity and the computer 39, 15 and 14 years, respectively, in the US to reach maturity. Other technologies, such as radio, smart phone, tablet, computer and internet, took less than 10 years to gain traction, and equally shorter time to reach maturity (about 3 years for smart phone and television). Once matured, most technologies typically take 10-15 years to reach saturation (moving from 40% to 75% penetration). 20 According to Ravallion (2009), the assessment of short-term impacts generates a “myopia bias” that can lead not only to erroneous conclusions but also to decisions to scale-up programs without knowing the underlying factors of impact that can lead to negative spillovers. 91 ● Theories of change involving the dissemination of technology, innovation, and behavioral change take time to prove and validate. It requires even a longer period of time when it entails a “whole value chain” approach. The term “accelerated” value chain development primarily based on technology and innovation adoption therefore was a misnomer. Moreover, the huge challenges across whole value chains require much more investment in the enabling environment and holistic understanding of context, which are beyond just one program. Perhaps a more parsimonious TOC would have been proven and validated faster. That is, a TOC that tackled challenges specific to certain nodes of the value chain. ● Comprehensive value chain, market, stakeholder, and organization assessments are key ingredients to effective realization of the TOC. Most of the challenges experienced, especially with the pull forces, would have been anticipated from such analyses. ● Interventions should first be tested on a smaller scale while being closely monitored. Red flags raised during monitoring must be taken seriously and issues addressed. 7.4 EVALUATION QUESTION 4 To what extent were the CGIAR centers successful in leading the implementation of a development program, given their traditional research focus and mandate? What lessons did the CGIAR centers learn and how did they adapt to scale up technologies? The AVCD program was implemented within the domain of international and national research systems so as to anchor research into development. As leading providers of research and knowledge, and having developed a range of technologies and innovations with great potential to contribute to Kenya’s inclusive agricultural sector growth and improved nutrition, the AVCD program sought to harness the technologies, innovations, and capacity within the CGIAR system to contribute to Kenya’s development objectives. In this regard, the research institutions involved in its implementation, while having the technical knowledge of agricultural research, may not have the hands-on experience in implementing a development program. Therefore, the program was reviewed and assessed for implementation to understand adherence to development objectives and intermediate results at field level. The ET endeavored to evaluate their performance to generate lessons, identify strength and weaknesses, and adaptation capacity. Based on document reviews and data analysis, the evidence of implementation effectiveness is mixed. In certain areas, the AVCD implementation teams were effective, and in others, they were less effective. A level of interdependence and complementarities needed to be built across various project and implementation components to facilitate technology transfer, adoption, and the development of supporting systems and structures. To its credit, the program integrated its activities with county governments and other development agencies in most areas it was implemented. A good example of this was in the livestock value chain, where AVCD used the institutional infrastructure offered by the PREG and partnerships with county governments to roll out training on participatory rangeland management (PRM), animal disease reporting, and animal health campaigns. Similarly, in the DTC and potato value chains, AVCD worked in partnership with county governments in improving access to quality seeds. There was a good level of coordination with county governments, the national government, and other development partners in facilitating the formulation of a number of policies, laws, and regulatory frameworks across all four value chains. 92 For the most part, each CGIAR center leveraged its core institutional strengths and existing institutional infrastructure. However, the Centers were less effective in leveraging the internal institutional capital available through the AVCD program, preferring instead to continue with a “disciplinary silo” approach. Senior managers at the centers interviewed for this evaluation made subtle references to competition for resources, turf, and profile building among the centers as having been evident; one respondent reported that there is reconfiguration and restructuring underway to address the problem. Because of the competition and turf wars, opportunities for building complementarities and drawing from lessons learned by one Center seem to have been missed, even in locations where multiple VC components were implemented. This also applies to the treatment of cross-cutting issues like nutrition, climate smart agriculture, and gender and social inclusion, which either sat uncomfortably or were ignored in some program components and interventions. The presence of a nutrition strategy kept the nutrition component visible, but a failure to integrate nutrition in the VCs from the onset caused a delay in the implementation of the nutrition intervention. Nutrition interventions were initiated later than core VC interventions, yet it is possible that an early introduction of nutrition education would have helped with value chain and technology adoption. In some counties, nutrition interventions were conducted at different locations from value chain intervention, despite two of the three pathways to better nutrition having been increased production and incomes resulting from the VC intervention. There also appeared to be inadequate preparedness, or a lack of flexibility, of the nutrition team that required the conduct of assessments before implementing social behavior change interventions. Because some of the interventions were over a short period, nutrition interactions were few, about once in the project lifespan, and rushed. As such, most beneficiaries had forgotten the lessons by the time of the evaluation, including in new Phase 2 counties or locations. There was also a tendency for nutritionists insisting on having exclusive control over the delivery of nutrition messages, despite their professional ranks being very thin - available only up to the sub￾county level. Discussions with nutrition and other implementing stakeholders agreed that the ideal situation would be having nutrition specialists at the ward levels. Pending their recruitment, the consensus opinion among key informants across the counties visited was that existing professionals, including WAOs and Home Economists, would work more closely with CHVs. Some nutritionists, however, pointed to the risk associated with task-shifting, whereby one does more of what they are familiar with than the things they are tasked to do. Most county government stakeholders emphasized the need to change the implementation TORs toward KPIs that are more specific to the project rather than the training or expertise of personnel. This practice is the norm in most development agencies but was lacking in the research institutions and among the county staff. Absence of program gender and a climate-smart strategies resulted in inadequate attention, except for occasional mentions in AVCD reports. Furthermore, the quantitative data was not consistently disaggregated by gender beyond the household type, except in some value chains, such as LVC, where some data were gender disaggregated. The very low percentages of female￾headed households (19.3, 18.0, 25.2 and 14.2 in 2016, 2017, 2018 and 2020, respectively) made it impossible to compare data by household type. Therefore, it is unclear if the program beneficiaries were representative of local populations. 93 7.4a. Since CGIAR centers’ core mandate is research and the development of agricultural solutions and the AVCD activity involved scaling the adoption of the technologies, what lessons were learned on the best way of bringing innovations to market? ● Adaptive change and program flexibility is key to value chain development. The Centers proved highly adaptive to changing circumstances and landscapes. In Phase 1, the program learned from the weak intervention at the farmer level and shifted delivery models and implementation strategy in the later part of the intervention. It further narrowed the scope in Phase 2 (e.g. working with only a few farmers’ groups), but with no strategy of reaching outgroups. At the onset of COVID-19, the program quickly adapted and changed strategy by using various digital platforms to reach and support beneficiaries mainly through SourceTrace DATAGREEN platform, WhatsApp, and Zoom. ● Partnership with county governments and private sector players to directly reach target beneficiaries can enhance acceptance and adoption of technology and innovation. Despite early struggles with implementation, including program management structure and ineffective delivery models and approaches, the CGIAR centers achieved most success in interventions where they actively engaged county governments and other development partners. However, these partnerships would have been more effective with greater, repeated, and longer interaction between implementing partners and the beneficiaries. For private sector players, the engagement needs to be over a longer period of time and the approach to technology dissemination more commercial oriented to be effective. ● Linking of the research-policy-practice nexus needed to be stronger for effective intervention design and implementation. In certain instances, the project worked with county governments in the belief that the latter would reach farmers with the interventions; in other instances, the project worked with county governments and farmers, but did not involve farmers sufficiently in the policy making processes. ● Establish dedicated regional offices to support implementation. While assigning county government focal points and a field support staff helped with coordination and rapid deployment, it was inadequate to effectively support the complex projects. This is particularly true for focal points, who have competing demands and interests from other programs. In addition, project implementers needed an interface between farmers and the county governments to strengthen engagements at that level. ● Engage competent last-mile farmer-facing implementers with proven specialization and experience working with various value chain communities for direct interface with beneficiaries. As research and technology institutions, the CGIAR centers have limited development programming experience and could have considered delegating some of the direct implementation. The program attempted this through FPOs, but these were only emergent and unlikely to have been effective partners besides serving different purposes and interests. ● It is important to maintain clear identity and brand in partnerships and collaboration. In FGDs and KIIs at the county level, the respondents struggled to identify AVCD or the partner centers with most interventions because several other projects, including past and present USAID programs, had supported similar interventions. These compromised accountability and transparency between partners, and interventions and outcome reporting. The collaboration between KCDMS and ILRI is a notable case in point. While 94 the two programs developed a joint work plan in counties they were present, ILRI played the dual (and potentially conflicting) role of the implementer of the DVC component and a consortium member of KCDMS, where it was being paid fees for its work. This presented a challenge in separating AVCD from KCDMS interventions, and thus made an assessment of the contribution of each difficult. It is highly probable that both programs reported the same outputs and outcomes from the joint interventions. These various projects were not collaborating and had at times undermined the program by giving farmers free or wrong inputs. 7.4b. Apart from the CGIAR centers, to what extent did the activity involve other private sector actors (private sector engagement), including lessons learned on how to effectively engage the private sector in project activities? Despite being one of the central pillars of the program, private sector engagement was suboptimal throughout implementation. Except for seed and animal health provision, private sector participation was limited and peripheral; this is particularly so with regards to the development of market systems for inputs and output. In the LVC, the program fostered business-to-business interactions among established buyers of livestock/livestock products and the pastoralists. However, the evidence that this approach worked, and was sustained, is lacking, with interviews with pastoralists not finding any existing examples of such engagement. The support to Livestock Market Associations similarly lacks convincing evidence of AVCD’s role in market stimulation. These markets were established with the support of other partners and AVCD only offered short term capacity building in the form of training. Its contribution in this regard is unclear, and farmers did not report a marked improvement in private sector participation as a result of AVCD’s interventions. Findings from the FGDs and KIIs indicate that efforts at supporting the establishment of apex FPOs did not have the desired effect of sufficiently stimulating private sector participation. Farmers still complained of weak supply/input markets and ineffective links to product markets. Where there was private sector engagement, the companies or organizations took long to set up operations (most were not present in various locations of interest at the time of the evaluation) and the period of engagement was too short for meaningful impact. For example, one of the companies engaged to support private provision of seed for DTC, Agrosoy Ltd, had no presence in Taita Taveta, Elgeyo Marakwet, and Makueni counties. In the LVC, Sidai Africa Ltd faulted the duration of engagement, six months, as too short to make any impact. In several cases, it is unclear to what extent due diligence and organizational capacity assessments were performed on prospective private sector partners before engaging or supporting them, and the ET did not find such evidence. Examples include engaging Green Forest as the main off taker of groundnuts in Elgeyo Marakwet, Organi for OFSP in Homa Bay and Bungoma, Agrosoy Ltd, Leldet and Dryland Seed for DTC seed, and Bubayi to produce potato seed in Bungoma County, among others. In Taita Taveta, while Taita Papa was supported to propagate potato apical cuttings, there is no evidence that an assessment was conducted on the commercial viability of the investment, a business plan developed, nor marketing strategy to guide further investments. In Homa Bay and Makueni counties, AVCD built milk cooling plants for dairy farmer cooperatives without conducting feasibility studies and assessing their capacity 95 to operate and maintain the plants. The lack of due diligence and subsequent failure of the companies to effectively support farmers led to ineffective engagement and/or nonperformance, and potentially undermined technology adoption and continued use. Some of the successful private sector engagement include Safaricom PLC supporting the mobile￾based Closed User Groups (CUG) platform for the animal disease e-surveillance system. In addition, the PVC successfully linked farmers to private seed suppliers through the National Potato Council of Kenya’s Viazi Soko platform; one result of this was KWESTO FCS in Bungoma procuring 100 bags of certified seed through the Nakuru-based Agribom Ltd. Lessons ● The effect of the intervention was felt better where AVCD had repeated engagements with target beneficiaries over an extended period of time, than where engagement was thin and not sustained. Intervention effects on new technologies have better results where there is a deliberate effort to understand the contexts in which interventions occur, coupled with meaningful engagement over slightly longer periods of time. This led to a narrower approach in Phase 2 where the CGIAR centers worked with only a limited number of wards or groups of farmers. However, even with this limitation, there was no framework of how it would be scaled up to reach other farmers. There were also numerous studies conducted on the impact of interventions being implemented by the project, but the scale of use of the study findings to inform actual program implementation remains limited. ● Contexts in which farmers decide whether to adopt a particular technology or not are complex, and are not static. There are a number of considerations to be made by farmers, and a number of factors that affect adoption decisions. Rapid changes, i.e. occasioned by seasonality, local political and economic dynamics, imply that intervention strategies working in one context might not necessarily work in another. Consequently, it is incumbent upon program implementers to work with farmers and other key stakeholders to identify and understand what is best for them. This includes co-generation of intervention and implementation designs. ● Interventions focusing on policy change are political processes that require time and a deep understanding of the politics around these policies. AVCD supported the development of several bills and strategies, but most of them are yet to be enacted. Similarly, a number of policies and strategies supported at county and national level are yet to be fully operationalized. Two lessons are important in this regard: o Political Economy Analysis (PEA) and organizational capacity assessments (OCA) would be necessary to understand what is feasible, in what contexts, and the vested interests that could facilitate or block policy change. o Sufficient time, resources (financial and technical), and effort should be devoted to promoting policy change. The assumption that policy is technical, and therefore only technocrats need be engaged, should be discarded for a more political approach to influencing policy change. AVCD reported during the evaluation that the activity largely ignored the national government, despite agriculture policy and strategy being a national government function, and legislatures (parliament and county assemblies), despite most policies and strategies requiring legislation to be enacted and enforced. 96 ● The activity could have been more successful had there been deliberate and structured engagements with key stakeholders from the design and scheduling stages. Programs should properly structure partnerships and engagements through formal and clear engagement frameworks, including: o Among the four value chain component partners o Between the four VC partners and private sector actors – specifically the business community and local NGOs with hands on development experience especially in agricultural value chain development to a point that the projects have become fully fledged agriculture value chain businesses. o Between AVCD and county governments, especially of co-designing and implementing context specific interventions. o Between AVCD and national government on cross-cutting policy and regulatory issues. While county government collaboration was admirably successful for some VCs and counties, the engagements were not structured. Across all counties visited, there is no evidence of documented structured engagement; instead, the centers took county governments at their word and assumed adequate capacity to drive implementation. The same was true for collaboration with other USAID programs – outside the PREG and SEK frameworks. Importantly, the ET found no evidence that capacity assessments were conducted to understand the capabilities of partners before engagement. Most of the partnerships, therefore, including with private sector players, were not informed by technical and administrative capabilities of the partners. 7.4c. Program Management, Monitoring, Evaluation, and Learning An additional element of the performance evaluation entailed an assessment of the program management structure with regards to coordination issues identified in AVCD’s internal midterm program review (IMPR) report of June 2017. Because the program implementers may not have had adequate experience with development programing, the program provided excellent opportunities for learning. Monitoring and evaluation was therefore a key component of the program. While the ET scrutinized the data and reports in-depth to extract key information on the various outputs and outcomes, the sheer volume of M&E data and reports and slow data release made comprehensive analysis challenging. The following findings are derivable from the analysis. The program established a monitoring and evaluation system that not only tracked the performance of the program but also provided evidence on key outcomes. It was guided by: 1) The program results framework 2) Individual value chain frameworks 3) Relevant FTF indicators with further details to measure progress against results 4) Use of ICT to ensure effective data capture, storage, and management 5) Research results on the theory of change and processes for scaling up of technologies and innovation 97 The program results framework provided details on indicators, means of verification, and the assumptions to achieve the results. Monitoring and evaluation focused on compliance monitoring for output delivery and progress toward outcomes. The program M&E working group comprising the PMO and ILRI M&E officers led the activity in the following functions: 1) Support review of logical framework and other M&E tools for the consortium 2) Develop and update the activity and M&E plan 3) Develop a database for the assembly of tracking data for the entire consortium 4) Support implementation teams in collecting data for tracking and reporting 5) Ensure that program deliverables are tracked 6) Analyze data from implementation to inform overall progress and learning 7) Provide data to online USAID databases as appropriate 8) Provide the necessary documentation and materials and support to technical teams 9) Follow-up on implementation level issues that pertain to M&E The PMO also developed a work plan and program performance monitoring plan (PMP). The consortium M&E Working Group coordinated the generation of baseline data in collaboration with other USAID-funded programs. Implementation of the program was based on the Results Based Management practice. The program was expected to deliver targets that were arrived at through rigorous value chain analyses. The targets for each value chain component contributed to the intermediate results of the entire program. The PMO made it clear to each project team and explicit to all partners that program components or interventions not achieving desired outputs were to either be restructured or terminated upon review by and recommendation of the PAC. Unused funds from terminated projects were reallocated in consultation with USAID and ILRI. AVCD’s M&E was implemented from inception of the program following the program TOC. The outcomes and output indicators in the TOC were tracked and aligned with the USAID indicator handbook. This demonstrated a clear linkage between the support provided by AVCD and USAID FTF initiative objectives. AVCD employed monitoring systems in Phase I with data collected using Open Data Kit (ODK) and stored in a central repository at the ILRI portal. The database captured baseline, annual survey data, and continuous monitoring datasets. In Phase 2, based on the lessons from Phase 1, an independent database was designed and developed on the MEASURE platform with a dedicated team at ICRISAT to manage all beneficiary data, annual survey, and continuous monitoring data for both Phases 1 and 2, as well as design unique beneficiary identification codes. The database was designed to collect real time data using a mobile app and reflected on the database dashboards. The data for baselines, annual surveys, and continuous monitoring surveys were disaggregated by sex and geography. The idea behind migrating to the MEASURE database was to reinforce innovative tools such as mobile apps to manage data collection, analysis, and visualization at the value chain program outcome and output levels. The good practices and lessons learned from Phase 1 helped shape the design and implementation of the MEASURE database platform. However, like any database, it came with a few challenges that can be addressed in the next programming of similar USAID programs of this nature. i. Audit trail 98 This is a complete record of changes to the master copy data, datasets, or survey tools for collecting data in monitoring and evaluation program. The AVCD project value chains changed the questionnaires as the project progressed; some within the same phase and some from Phase 1 to Phase 2. However, the changes were not documented in an audit trail. It is good practice to keep records of any decisions made about the data and the analysis, rather like a log book to accompany the data. A well-maintained audit trail, log book, or notebook greatly eases the subsequent tasks of writing reports on the data and of answering data queries. A fundamental requirement of any scientific or monitoring and evaluation cross-sectional datasets is for the work to be repeatable and easy to work with by persons outside the project team or new staff. It is also good practice to keep the data with the codebook or do-file of any analysis done on the dataset for ease in understanding the analysis involved in cleaning and analyzing the data. Therefore, project managers and research teams should establish a documented routine for regularly making safe copies of the data with any changes updated in the audit trail document. ii. Backup The project had clear guidelines on how to manage and archive data. In Phase 1, they used ODK for data collection and stored the data on an online database using ODK central data repository. The baseline and annual survey data were backed up on the ILRI portal. The portal was designed well and had all the data sets archived well and capabilities of exporting the data in many data analysis formats (e.g. CSV, STATA, SPSS etc.). In Phase 2, the project decided to change the data collection and backup system to the MEASURE database. The data collection was well designed with farmers given unique barcodes for scanning with real time updates of the database and dashboards. The major challenge with the MEASURE database is the documentation (audit trail) and lack of options for export formats for the database. The MEASURE database only has options for Excel CSV export. This makes it difficult for anyone who wants to analyze the data in any statistical software because the CSV does not come with the codebook for the data. The CSV format will require recoding the data before analysis. Another challenge is that the options for exporting the data as one record or many datasets are limited. Monitoring and Evaluation experts spend a lot of time preparing data for analysis. This includes converting data to suitable formats, merging data sitting in different files, and summarizing data or datasets for a project. The time spent in this pre-processing step can be greatly reduced if the database has many options for the export format. iii. Data Archiving The data and programs from a development program must be archived in a central database or repository in such a way that they are safe and can be accessed by a subsequent user. The way data, datasets, and any relevant information (e.g. audit trails, questionnaires and codebooks) are transferred to the archive should be planned from the outset of the program. Careful planning is very helpful throughout the project because it helps to promote a consistent directory structure and naming convention for computer files, and also encourages the recording of all steps in the project. The archive is more than a permanent storage place for the files used for the analysis. It must give access to all the information (reports, cleaned raw data, pictures, recordings from FGDs, Questionnaires etc.) from the experiment or program. During the operational phase of the AVCD program, the data/information about the implementation was partly in the computer, partly on paper and other media (such as photographs), and partly in the minds of the project team. The archived data only contained computerized data and excluded all the relevant, non- 99 ephemeral information that is in the minds of the research team. Data that could not be archived electronically should have still been recorded in the archive. Most of the rich data was lost due to the absence of a proper archiving scheme. In conclusion, data should be/have: ● Accessible: hence easy to access (in a centralized database) by many users who have commonly available software (STATA, SPSS, SAS or Excel format) ● Easy to use: so that the field data collection forms, and what will be entered into the computer are similar and backed with documentation of any changes in the audit trail. ● Clearly defined variables: the units of measure and codes used (labels for names of variables) should be as clear as possible backed by code books and questionnaires. ● Consistency: of names, codes, units of measurement, and abbreviations (AVCD data variable names and dataset names should be consistent from baseline to end term for ease in analysis of panel data). ● Reliable: archive should be as free from errors as possible (data should be cleaned before archiving-AVCD data was not cleaned when archived). ● Internal documentation: documentation should be complete with regard to: procedures for data collection, sampling methodology and sampling units used; the structure of the archive (how different files are related); a list of all computer files in the archive; a full list of variables and notes on how to treat missing values; summary statistics for crosschecking the information in the archive; and any warnings and comments that need to be observed for data usage. ● Complete: if possible, data should be stored in copies of: the data capture field forms; the data management log-book; a description of derived/calculated variables and how they were derived. Describe the format of the files, e.g. Excel 2007, SPSS ver. 15, etc. Number of records in each file and number of variables in each file. Questionnaires used to collect the data and any reports derived from the database. 7.5 EVALUATION QUESTION 5 To what extent have AVCD’s efforts in supporting business development services, policies, and legislations improved institutional governance (i.e. the configuration and behaviors of participating organizations) leading to sustainable outcomes? The evaluation examined the extent to which the program addressed the constraints across the four value chains through policy and institutional support interventions. Issues investigated included how value chain coordination mechanisms (i.e. linkages with county governments), information and power asymmetries, and regulatory and normative institutional arrangements were addressed. The evaluation assessed the level of ownership by county governments of the institutional mechanisms that have been created, in terms of budgetary allocations, staffing, political commitment, and their durability and resilience. The latter examined the extent to which the regulatory and governance mechanisms are inclusive, participatory, and enjoy legitimacy with farmers, and hence, have the potential to transcend the program lifespan. The ET examined interventions for their relevance, appropriateness, and effectiveness at three levels: 100 ● Interventions initiated at the system level, i.e. those transcending the governance of specific value chains (e.g. with the Council of Governors and relevant Ministries). ● Farmer’s producer organizations (FPOs) level: their composition and functionality, in particular, the extent to which farmers perceive the FPOs to be an embodiment of their interests and aspirations, including effectiveness in leveraging farmers’ bargaining power with other market and value chain players. The management of FPOs were assessed to understand whether they have the requisite organizational structures for effective management. ● Informal institutional arrangements that govern access to value chain resources (e.g. information, technology, inputs, markets, etc.). Value chains exist as nested structures with a wider production system, and both formal and informal institutional arrangements affect relationships, norms, and practices. The evaluation assessed how both formal and informal application of regulatory norms, for instance, by county governments, affect access to value chain resources and productivity. AVCD promoted a number of policies and legislative frameworks with the aim of creating an enabling environment where increased production by farmers would combine with a concomitant improvement in the policy environment, thus creating vibrant markets and trade in the agricultural inputs and produce supported. Examples of these include: ● In the LVC, advocating for the compression of ten veterinary laws developed under the LVC into three bills: Animal Health Bill, Veterinary Public Health Bill, and Livestock Bill.21 AVCD also supported the Directorate of Veterinary Services to develop a national Livestock Masterplan. At the county level, the project supported the development of county livestock policies, livestock market bills, rangeland management bills and policies, and various regulations that would give effect to these policies or legislative frameworks. In Isiolo, the Sale-Yard Bill and the Rangeland Management Policy have been signed into law, but the Rangeland Management Bill has not been passed by the County Assembly. In Marsabit, the County Livestock Policy was to be launched in October 2021. In Marsabit, Wajir, Turkana, and Garissa counties, the status of policy and legal frameworks remain unclear; for example, the Marsabit County Rangeland Management Bill 2019 and the Wajir County Rangeland Management Bill that stalled at the technical review and executive review stages due to COVID-19. Also unclear is the status of the Marsabit Livestock Trade and Marketing Bill and the regulations to operationalize the Isiolo Livestock Market Management Fund (LMMF) Act 2019, which were awaiting approvals of the respective CAs. ● In the PVC, AVCD successfully advocated for the recognition of (the Irish) potato apical cutting technology and its incorporation into the seed certification system by KEPHIS. The program also contributed to the review and launch of the Bungoma County Potato Strategy, in collaboration with GIZ and the county government. ● In the DTC, AVCD reported reluctance by policy makers in allowing seed developed by farmers to be recognized as quality declared seeds. This is the practice in other countries (i.e. 21 In addition, seven regulations had been technically reviewed and improved to operationalize the Livestock Bill – i.e., the Animal Breeding Regulations, Animal Feeds Regulations, The Establishment of the Livestock Training Institutions Regulations, The Pig Industry Regulations, The Poultry Industry Regulations, and The Livestock and Livestock Products Marketing Regulations. 101 Tanzania) where farmers develop seed under the supervision of research organizations, and even though the seeds are not certified, they are recognized as quality declared seeds. ● In the DVC, AVCD worked with county governments toward the development of County Dairy Development Strategies. The Busia, Migori, Makueni, Homa Bay, Kisumu, Siaya, and Vihiga strategies that stalled due to COVID-19 restrictions remain in limbo. None of the seven county dairy development strategies has been finalized; in Homa Bay, the strategy had not been published at the time the ET visited the county. While AVCD could be said to have been fairly successful in advocating for the development of policy and legislation, most exist only in draft form. Counties cited human and financial resource limitations and inadequate political will in the County Assemblies to conclude the processes. Most policy and regulation development processes have stalled at the legislative stage, at either the County Assemblies or Parliament. For example, the amalgamated livestock laws have stalled at Parliament. In Isiolo, the Sale-Yard Bill and the Rangeland Management Policy have been signed into law, but the Rangeland Management Bill has not been passed by the County Assembly. In Marsabit, the County Livestock Policy was to be launched in October 2021. In Marsabit, Wajir, Turkana, and Garissa counties, the status of policy and legal frameworks remain unclear. For example, it is unclear what became of the Marsabit County Rangeland Management Bill 2019 and the Wajir County Rangeland Management Bill that stalled at the technical review and executive review stages due to COVID-19. Moreover, none of the seven County Dairy Development Strategies (CDDS) has been finalized, and the Taita Taveta Potato Strategy is stuck at the County Assembly because it was not sufficiently engaged in its development. What is evident is that counties have been slow in developing or embracing the policies and regulations, which respondents explained as reflective of weak institutional capacities and conflicting policy priorities by different county government officials. In addition, interested communities, including political representatives, were not sufficiently involved in the development of policy and legislative frameworks. This inadequate community involvement meant that either the policies and laws were not fully reflective of the priorities and interests of key value chain actors or, once operational, more effort would be required to popularize them and obtain community buy-in. The ET found the following: ● Government officials in all the counties visited felt the impact would have been greater had they been engaged at the planning and development stages for an opportunity to co￾create and tailor the interventions to their unique contexts instead of instructions on already finished projects. ● Communities were not meaningfully involved in the development of policies, except in seed certification where there was minimal community involvement. Public participation is a core tenet of policy development. There seemed to have been preference for a top￾down approach, which might lead to policies having “form” and not “function.” ● Importantly, while agriculture is a devolved function, agriculture policy and regulation is a national government function. AVCD had only weak engagement with national government, informed by the notion that the program need not engage with national government, because it largely targeted counties. 102 While AVCD could be said to have been fairly successful in advocating for the development of policy and legislative frameworks at national and county levels, the same cannot be said of its performance in business development services (BDS). There were attempts to support the establishment and development of farmer producer organizations (FPOs) across the AVCD components. However, as entities expected to promote the production and marketing of agricultural products, most FPOs remain institutionally weak, and in most cases, poorly led and managed. Farmers still complained of weak input and service markets and ineffective links to product markets. Most of the FPOs facilitated were inherited from other programs and are weak and barely operating; they will need greater support over much longer periods to reach minimum operational competence. Efforts to link FPOs to county government structures are noble. Nonetheless, county governments are themselves institutionally weak, and there are risks of politicizing the management and operations of farmer producer organizations and cooperatives as county political elite seek to exert control over voting blocs in the counties. Interviews with county government officials, AVCD staff, and farmers confirmed the institutional weaknesses in the leadership and the FPOs. There also seems to be resistance to infusing the FPOs with more progressive leadership by influential members. During Phase 2, the program engaged in discussions with the Council of Governors Agriculture Sector Secretariat (CoGASS). The CoGASS is one of the 18 committees established under the Council of Governors secretariat, a governmental body created under Section 19 of the Intergovernmental Relations Act (no. 2 of 2012). Discussions between the CoGASS and AVCD started in 2019, culminating in the development of a document which gave effect to the partnership in September 2020, with AVCD supporting the Secretariat for one year (September 2020 – September 2021). Under this partnership, AVCD recruited a technical advisor to support the CoGASS. The partnership was premised on the assumption that AVCD support to the CoGASS would in turn strengthen the capacity of the Council of Governors who would further support the counties to improve food systems in the counties. This support was seen as critical to enabling the Council of Governors to be able to effectively steer national legislative processes on agriculture, strengthening the Council to reach out to the private sector and to obtain direct support from donors, instead channeling everything through the national government. From interviews with the CoGASS, the following were cited as the main achievements from the partnership: 1. It contributed to the goal of strengthening the CoG to offer support to county governments to deliver on their mandate of strengthening county food systems through legislation, budgets, training, and the development of appropriate structures. It was reported that CoGASS regularly supports the counties in the development of model policies, legislation, and regulations. Interviews with county government officials did not mention this, because it was said that CoG dealings with counties is at the level of county executives. 2. Support for the CoG to look into national policies, regulations, and standards within the lens of devolution. Respondents at CoGASS mentioned that the Secretariat has enabled the CoG to be a co-creator of policies, laws, standards, and regulations – a great improvement over the previous periods when the national government took advantage of capacity gaps at the CoG to formulate policies and laws with very little consultation or 103 involvement of the counties. Reportedly, an arrangement was reached with the Senate, the National Assembly, and Ministry of Agriculture that no laws or policies pertaining to devolved functions would be taken to Parliament if the CoG had not fully participated in the process of developing it. An example was given of the tea sector, and the livestock masterplan, which were characterized by initial contentions between the two levels of government, until the national government yielded to pressure from the counties. 3. Guidelines were developed for the CEC Caucus, which brings CECs for agriculture across all the 47 counties. The guidelines provide structure to the discussions and engagements within the Caucus. 4. Development of instruments for cascading national policies and laws to the counties. These will help in guiding counties on how to domesticate national laws and policies. 5. Transfer of skills through the technical expert and short term technical assistance to the Secretariat. 6. AVCD supported the CoG in its strategic plan development process. Commendable support seems to have been offered by AVCD to the Secretariat. However, more could have been achieved if the support was linked to other AVCD interventions across the value chains. There is scarcely any indication that this support was connected, for instance, to the support to the development of policies and strategies at county levels. It is possible that CoGASS participated in the processes of formulating laws and policies at the national level, but it is unclear that this was done as part of AVCD support. Even the engagement by the CoG in the development of the national Livestock Masterplan seem to have started much earlier, before AVCD formalized its partnership with the CoGASS. Engagement in policy development produced notable outputs and outcomes, including: 1. Several counties designated value chains promoted by AVCD as flagship projects, e.g. potato in Taita Taveta, and developed VC-specific strategies for the first time, e.g. potato strategies in EMC, Bungoma, and Taita Taveta, and dairy development strategies in seven pre-commercial counties. 2. Counties allocated more resources, including budgets, extension staff, and seconded staff to organizations supported by AVCD. Examples include: o Bungoma County allocating land and budget (KSh.5.8 million) to KWESTO FCS for potato seed production and storage facilities. o Taita Taveta County supported the potato and NAFAKA cooperatives with office space and resources through KSCAP. o Elgeyo Marakwet allocating the potato cooperative a budget (KSh.30 million proposed for FY2021/2022 for seed production under the KDSP. 3. Counties taking or scaling up AVCD activities through other programs, such as KSCAP and KCEP. Examples include: o Elgeyo Marakwet facilitating the adaptation of key AVCD interventions and activities by ASDSP and KSCAP. o KCEP promoting potato production in non-traditional zones of Bungoma County as a COVID-19 response through diversification from maize farming. 4. The National Agri-nutrition Strategy being adopted for agri-nutrition programming by government and nongovernmental organizations. 104 5. Agricultural Development Steering Committees and Coordination Units established in most counties. Several lessons are however derivable from interventions aimed at policy, organizational and institutional development:  Pay sufficient attention to the legislative organs at the county and national levels, and engage them early, because they are the critical players in the policy process.  Involve communities and key stakeholders in the development of policies early and meaningfully. Inclusive agricultural growth includes inclusive development of the rules of the game. The preferred top-down approach to policy making led to policies having “form” and not “function”, and thus risk rejection by the target beneficiaries.  While agriculture is a devolved function and value chain development interventions implemented at the county level, agriculture policy, institutions, regulation and trade are national government functions. The weak engagement with national government, informed by the notion that the program needed only engage with targeted counties, undermined the development of critical policies and influencing national regulatory institutions, such as KEPHIS.  Institutional support should be connected to interventions. For instance, while AVCD provided commendable support to the Council of Governors Agriculture Sector Secretariat (CoGASS), more could have achieved had the support been linked to other AVCD interventions across the value chains. There is no indication that the support was connected, for instance, to the support to the development of policies and strategies at county levels.  FPOs need greater support over much longer periods to reach minimum operational competence. Whereas efforts to link FPOs to county government structures are noble, county governments are themselves institutionally weak; furthermore, there are risks of politicizing the management and operations of the FPOs as county political elite seek to exert control over voting blocs. Support to FPOs, therefore, should be decoupled from other engagements with county governments. 105 APPENDICES Appendix 1. Performance Evaluation Scope of Work Appendix 2. Documents and Literature Reviewed Appendix 3. AVCD Results Framework Appendix 4. Map of Kenya showing AVCD project area Appendix 5. Evaluation County Selection Matrix Appendix 6. Expanded Evaluation Matrix Appendix 7. Schedule for the Evaluation and Site Visits Appendix 8: Evaluation Work Plan Appendix 9. Team Composition and Roles Appendix 10. Key Informant Interview Scripts Appendix 11. Checklist for Interviews with AVCD staff Appendix 12. Focus Group Discussion Tool 106 Appendix 1. Performance Evaluation Scope of Work SECTION C – STATEMENT OF WORK C.1. DESCRIPTION OF THE ACTIVITY The Feed the Future Accelerated Value Chain Development (FTF-AVCD) activity is a five year $38.6 million World Bank buy-in Public International Organization (PIO) field-support Grant to a consortium of Consultative Group on International Agricultural Research (CGIAR) Centers led by the International Livestock Research Institute (ILRI). The other members of the consortium include International Crops for Research Institute for Semi-Arid Tropics (ICRISAT) and the International Potato Center (CIP). The FTF-AVCD Program harnesses the technologies and innovations within the CGIAR to contribute to the development objectives of the USAID Country Development Cooperation Strategy (CDCS) for Kenya in line with the strategic goal and objectives of Kenya’s FTF Strategy. The Goal of FTF is “to sustainably reduce poverty and hunger in the Feed the Future Zones of Influence in Kenya” and the development objective is “Increased inclusive agricultural growth and improved nutrition status of participants in the targeted value chains in the Feed the Future zones of influence in Kenya.” The FTF-AVCD targets value chains that are of national priority for food and nutrition security and cover a range of ecosystems, including those that have been affected by severe drought over the years and resource-poor farming communities. These value chains include livestock, dairy, drought tolerant crops and potatoes (both sweet potatoes and Irish potatoes). The FTF-AVCD theory of change is based on whole value chain analysis to produce good agricultural policies, improve productivity, expand markets and trade to ensure adequate, diverse and quality food and nutrition. The activity uses the same results framework as the FTF strategy for Kenya. The program has therefore identified the following intermediate results: 1. Improved agricultural enabling environment; 2. Expanded markets and trade; 3. Improved productivity of selected value chains and; 4. Improved nutrition related behaviors. The activity is implemented through the national and county governments, private sector and civil society to reach at least 460,000 small holder farmers, pastoralists, private sector and civil society organizations in 20 counties namely Turkana, Marsabit, Isiolo, Garissa, Wajir, Kitui, Makueni, Taita Taveta, Busia, Vihiga, Siaya, Homa Bay, Migori, Tharaka Nithi, Meru, Nandi, Trans Nzoia, Uasin Gishu, Elgeyo Marakwet, and Bungoma. The number of counties were reduced to twelve in the last two years of the project period after rationalization. C.2. BACKGROUND The USAID Kenya CDCS 2014-2019 had inclusive market driven, environmentally sustainable economic growth as one of the key development objectives for Kenya’s economic transformation. The FTF program was one of the major programs being implemented under this development objective to achieve food security, resilience among people and environment, 107 ecosystem management, adaptation to climate change while enhancing private sector participation. The FTF-AVCD activity is designed to apply the whole value chain approach to identify interventions from both the ‘push’ (supply) and the ‘pull’ (demand) side to accelerate the value chain development. The interventions from the push side of the chain include improving access to knowledge tools (technologies and innovations); buying down risk (increasing resilience); improving nutritional status; and enhancing natural resource management. The interventions from the pull side of the chain include improved market linkages by establishing producer organizations as marketing institutions and linking them with consumers; re-aligning business regulatory policies; business and financial services for producer organizations; and conducive government policy environment. The theory of change posits that ‘large scale application of technologies and innovations, in a whole value chain approach, will significantly accelerate the development of agricultural value chains, to rapidly increase food and nutrition security, as well as incomes for all players (producers, processors, marketers) and subsequently contribute significantly to inclusive agricultural and economic development. The application of this theory of change slightly differs in each value chain as shown below. Livestock Value Chain: The theory of change in the livestock component include work on both push forces, by increasing rangelands productivity and improving animal health to increase the supply of livestock in the markets, and pull forces, by enhancing market performance through supporting improved livestock sales, information sharing and adopting enabling policies and legislative environment. Policy and legislative reforms are expected to improve pastoralist participation in competitive and inclusive markets and enhance informed sales decisions at the household level, thereby boosting both the demand and supply sides. Improved rangeland productivity will increase the supply of quality livestock to the markets, resulting in the dual effect of increased milk availability for households, and improved income from livestock sales. Additionally, improved animal health, through enhanced private sector participation and a revitalized surveillance system with a real time response framework, will result in healthier animals that mature faster and fetch better prices. Resource savings are expected to occur because of minimal livestock losses from disease outbreaks and the limited use of public resources to combat disease outbreaks. Dairy Value Chain Component: The Theory of change for the dairy value chain is informed by the low critical mass of livestock holding and also low milk yield per animal at household level which cannot support dairy farming as a business enterprise. The Theory of change therefore hypothesizes that the “push” force would target improved delivery of breeding (Fixed Time AI based on estrous synchronization), animal health (ECF vaccination to secure breeding gains) and feed technologies (High yielding & protein dense forages such as brachiaria grass). To link the “push” and “pull” sides would require investments in the capacity strengthening of county governments to be able to plan and facilitate dairy sector investments. The dairy component therefore works with county governments and other stakeholders to develop implementable county dairy development strategies. In addition, the components aim to invest in enhancing the capacity of local Farmers’ Producer Organizations (FPO) to be the drivers of dairy sector growth 108 and markets. The FPOs are to provide the platform for coordinated delivery of technologies and innovations in breeding, animal health, feeding and advisory services, as well as bring into bear the pull force of marketing and value addition. Potato Value Chain Component: In both sweet potatoes and Irish potatoes value chain analysis, lack of a seed system and weak farmer producer organizations were identified as the main constraints to accelerating the value chain development. To transform potato farming into a private sector led and commercial-oriented value chain, the potato component supports the development of a sustainable seed system and farmer producer organizations to drive the potato value chain under county government investments ensuring an enabling and supportive business environment. Through the county agriculture office, the components also support wide application of good agricultural practices (GAP) among farmers to increase productivity and production, combined with establishing farmer producer organizations as key players in sourcing for inputs and to market the production. Drought Tolerant Crops Value Chain Component: Drought tolerant crops value chain component deploys whole value chain approach with technologies and innovations to upgrade the value chains from the point of accessing quality seed to the point of availing more food for consumption and surplus grain to market. The theory of change states that if the activity enhances its collaboration with national and county government partners, the private seed companies, and farmer producer organizations (FPOs) to develop a sustainable seed supply system, and if the capacity of FPOs is developed to become business hubs for collective action, then productivity improvements will occur leading to improved households diets and nutrition, and increased benefits through economies of scale for more competitive prices of inputs and products. According to its design, the AVCD activity was expected to closely collaborate with other existing FTF activities and thus contribute to the overall FTF goal of poverty reduction and reduced stunting. Among the FTF activities AVCD collaborated with include the Resilience and Economic Growth in Arid Lands (REGAL-AG and IR) and the Kenya Agricultural Value Chain Enterprises (KAVES) and their partners to enhance resilience, agricultural productivity, food security and overall economic welfare of farmers, producers and marketers throughout the value chain. While the entry point was to scale up application of agricultural technologies and innovations, the program was to address the weakest points of the value chain while ensuring that all value chain actors benefit. The activity also worked closely with Tegemeo Agricultural Policy Reform and Analysis Project phase II (TAPRA II), Africa Lead, and the Regional Strategic Analysis and Knowledge Support System (RESAKSS) on baseline surveys, institutional capacity analysis and policy development, respectively. In addition, the program was expected to work closely with Government, public research, and extension institutions as well as the private sector to tap from and leverage the enormous human capacity in the country. AVCD also collaborated with the Nutrition and Health Program (NHP) Plus activity to improve the nutrition and health status of Kenyans, including addressing the critical issues of nutrition that have not received adequate attention in the past such as improving access and demand for quality nutrition at community level; strengthening nutrition commodity management; and improving food and nutrition security as it relates to malnutrition and stunting. 109 Purpose of the Evaluation This performance evaluation is commissioned by USAID/KEA at the end of the FTF-AVCD activity which is in its fifth year. The primary purpose of the evaluation is to assess the activity design and implementation to inform the design of future activities which harness agricultural technology and innovations within the CGIAR and other research organizations. The performance evaluation will also provide learning on the innovative approach of partnering with the CGIAR to implement development interventions and on how partnerships with the national and county government and the private sector can contribute to Kenya’s development. The evaluation aims to capture lessons learned to (1) Improve impact of future agricultural technology and innovation activities; (2) Inform and improve policies; (3) Strengthen the design of future activities, by harnessing the innovations and technologies developed by the CGIAR and other research organizations, identifying good practices/approaches for engaging CGIAR in implementation processes and pinpoint opportunities for development partnerships between CGIAR and research organizations and GOK national and county governments and the private sector. The primary audience for the evaluation report will be USAID/KEA, CGIAR centers, and the Government of Kenya at both National and County levels. It will contribute to the pool of knowledge on sustainable agricultural technology dissemination which can inform other partners. Also, the evaluation findings will contribute to global learning on how to effectively work with the CGIAR to achieve development objectives. Evaluation Questions 1. What contribution did AVCD make towards achieving Kenya’s FTF goals of poverty and hunger reduction and FTF objectives of inclusive agricultural led growth and improved nutritional status? a. To what extent have the technologies and innovations contributed to improved capacity of households, communities and market systems in areas affected by recurrent crisis to withstand shocks and stresses? b. How successfully have the nutrition activities been layered and integrated with the core technology components of AVCD to achieve stunting reduction and other nutrition results? c. How have community-level nutrition education and behavior change affected dietary diversity and care practices for women and children? 2. How effectively did AVCD address FTF cross-cutting issues, including gender, youth, Natural Resource Management, climate change among others22? a. Which technology dissemination models were appropriate in the different contexts – High Rainfall, Semi-Arid and Arid areas targeted by AVCD? b. To what extent are the promoted technologies widely available in the target areas (i.e., what is the scale of dissemination), and to what extent has dissemination led to spillover effects that include expanded diverse economic opportunities? 22 This will be further refined by the evaluators 110 3. To what extent was the AVCD’s Theory of Change (TOC) validated by the activity? What were the strengths and weaknesses of the TOC? How can the TOC be improved? 4. To what extent were the CGIAR centers successful in leading the implementation of a development program, given their traditional research focus and mandate? a. Since CGIAR centers core mandate is research and the development of agricultural solutions and the AVCD activity involved scaling the adoption of the technologies, what lessons were learnt on the best way of bringing innovations to market? b. Apart from the CGIAR centers, to what extent did the activity involve other private sectors actors (private sector engagement), including lessons learned on how to effectively engage the private sector in project activities? 5. To what extent has the activity’s business development services, policies and legislations improved institutional governance and sustainable outcomes? Evaluation Methodology While the evaluation team is expected to use a mixed method approach due to its ability to handle complexities related to program implementation, the team is free to propose an alternative plausible methodology. The proposed methodology, which will be reviewed by USAID and AVCD technical teams, may include the following data collection techniques: Desk review: The evaluation team will undertake a detailed desk review to inform the evaluation of the project. This review will cover all the relevant project documents including all AVCD project documents - original and updated activity design reports, concept notes, annual work plans, annual reports, quarterly reports, and mid-term assessment report which have been prepared between the year 2015-2020, which will be provided by AVCD. In addition, the technical team will avail to the evaluation team the performance data reported in the FTFMS and other USAID strategy and project documents. The review will also cover relevant government of Kenya national and county governments, relevant donor funded programs and published agriculture literature. Key Informant Interviews: The Evaluation Team will conduct interviews with USAID/KEA team, AVCD team, relevant Government of Kenya (national and county) staff, AVCD partner staff, private sector organizations, participating community members, and others as appropriate. A list of proposed interviewees will be submitted with the inception report. The final list of interviewees shall be agreed upon in consultation with USAID and AVCD team. Focus Group Discussions: The Evaluation Team may also consider carrying out Focus Group Discussions (FGDs) to obtain the required data from project participants where applicable. The groups shall be constituted in consultation with the AVCD team. Site Visits: To further compliment information generated through key informant interviews and desk review, field visits shall be carried out. The Evaluation Team is expected to come up with a suitable sampling methodology to identify the locations or organizations to be visited within 22 selected counties within the project area. In selecting the various locations for site visits, the Evaluation Team should ensure all the project components and/or sectors are catered for (various 111 value chains), the varying contexts (HR1, SA2 and NAL) are appropriately sampled, and that the logistics and cost involved with the different sites are put into consideration. The evaluation team should consider visiting at least two counties from each zone. Please see details specified in the Place of Performance subsection of Section C. Prior to the start of data collection, the evaluation team will develop and present, for USAID/KEA and AVCD review and approval, a data collection and analysis plan that details how data will be collected, transcribed and analyzed, including what procedures will be used to analyze qualitative and quantitative data from interviews. Operating Constraints The evaluation is expected to take place between Mid-June - August 2021. The evaluation team is expected to be aware of the security situation in the northern regions and areas bordering Ethiopia, Uganda and Somalia where insecurity may adversely impact travel and other operations. USAID and AVCD will be available to provide specific travel updates before field data collection begins. The team should also factor in the COVID-19 situation which may adversely impact the evaluation in various ways. Participation USAID/KEA, AVCD, other FTF partners collaborating with AVCD and the Resilience and Food Security team in Washington will participate in the evaluation including during the SOW development, the evaluation methodology and deliverables. The USAID technical team will coordinate inputs from all partners and be available to provide any clarification. AVCD program management team will also be available to provide the needed documentation and facilitate linkage to key informants, communities and households who are participating in the activity. USAID staff may accompany the evaluation team to the field and be involved in key informant interviews. Standards The technical response including methods and final evaluation must be consistent with and meet the standards of USAID’s evaluation policy, which is available here: https://www.usaid.gov/evaluation/policy. Place of performance The evaluation team is expected to cover seven (7) counties including, two (2) in the North, three (3) in the West (Western-1, Nyanza-1 and Rift Valley-1) and two (2) in the Southeast for approximately 1.5-2 weeks of field work. The counties are drawn from AVCD target counties including Turkana, Marsabit, Isiolo, Garissa, and Wajir in the North, Kitui, Makueni, Taita Taveta, Tharaka Nithi, and Meru in the Southeast, Kakamega,Busia, Vihiga, and Bungoma in Western, Siaya, Homa Bay, Migori, and Kisumu in Nyanza, Nandi, Trans Nzoia, Uasin Gishu, and Elgeyo Marakwet in Rift Valley. 112 Appendix 2. Documents and Literature Reviewed ADS, Chapter 201: Program Cycle Operational Policy. Available at https://www.usaid.gov/sites/default/files/documents/201.pdf AVCD. 2016. Accelerated Value Chain Development (AVCD) Program. AVCD Nutrition Strategy 2015-2019. AVCD. 2016. Accelerated Value Chain Development Project Baseline Survey. Dairy, Staple Crops, Livestock and Roots and Tubers Value Chains AVCD. 2017. Accelerated Value Chain Development (AVCD) Program. Final Report on Internal Mid-Term Program Review (IMPR). June 2017. AVCD. 2017. Accelerated Value Chain Development Program(AVCD) 2015/16 Annual Progress Report February 2017 AVCD. 2017. Kenya Accelerated Value Chains Development Program. Financial year 2016/2017 Quarter 1 Progress Report. January 2017 AVCD. 2018. Accelerated Value Chains Development (AVCD) Program Annual Report (October 2017 to September 2018). October 2018 AVCD. 2018. Feed the Future - Accelerated Value Chain Development Program for Kenya. Potato Value Chain – Annual Survey Report. November 2018 AVCD. 2019. Accelerated Value Chains Development (AVCD) Monitoring, Evaluation and Learning (MEL) Plan AVCD. 2019. Accelerated Value Chains Development (AVCD) Program Annual Report (January to December 2019). Annual Report 2019 AVCD. 2019. Fact Sheet: Accelerated Value Chain Development Program (AVCD) AVCD. 2020. Accelerated Value Chains Development (AVCD) Program Financial year 2020 Report (January to September 2020) AVCD. 2020. Feed the Future - Accelerated Value Chain Development Program for Kenya Project Document Phase II (January 2019 – December 2020). Theme: Accelerating Value Chains Development (AVCD) for Smallholder Farmers and Pastoralists with Technologies and Innovations in Kenya Consortium of CGIAR Centers (ILRI, ICRISAT and CIP). AVCD. 2020. USAID Feed the Future Kenya Accelerated Value Chain Development (AVCD) – Potato Value Chain Endline Survey Report November 2020. AVCD. 2020. USAID Feed the Future Kenya Accelerated Value Chain Development (AVCD) – Potato Value Chain. Endline Survey Report, November 2020. AVCD. nd. Dairy Production Pathway AVCD. nd. Improved and Enabling Environment Interventions. AVCD. nd. Phase 2 AVCD_LVC Investments/ Intervention by County/ Sub-counties/ Wards/ Community Units AVCD. nd. Staple Crops Grain Production Pathway 113 AVCD. nd. Staple Crops Seed Production Pathway AVCD. nd. Strategic Animal Health Services and Interventions AVCD. nd. Strategic Feed and Forage Services and Interventions AVCD. nutrition strategy, 2016. Bamberger, Michael, Vijayendra Rao and Michael Woolcock (2010) ‘Using Mixed Methods in Monitoring and Evaluation: Experiences from International Development’, in Abbas Tashakkori and Charles Teddlie (eds.) Handbook of Mixed Methods in Social and Behavioral Research (2nd revised edition) Thousand Oaks, CA: Sage Publications, pp. 613-641 Brad Astbury and Frans L. Leeuw, Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation, American Journal of Evaluation 2010 31: 363 DOI: 10.1177/1098214010371972 available at http://aje.sagepub.com/content/31/3/363 Chesoli Rose and Rao Ganga 2021. Annual Survey Report of the USAID Feed the Future Kenya. Accelerated Value Chain Development (AVCD) Program, Drought Tolerant Crops Component. Phase 2. Clark, Vicki Plano and Manijeh Baidee (2010) ‘Research Questions in Mixed Methods Research’, in Abbas Tashakkori and Charles Teddlie (eds.) Handbook of Mixed Methods in Social and Behavioral Research (2nd revised edition) Thousand Oaks, CA: Sage Publications, pp. 275-304 County Government of Garissa, USAID and AVCD nd. County Government of Garissa. Guidelines for Livestock Market Operations during COVID-19. County Government of Isiolo, USAID and AVCD nd. County Government of Isiolo, USAID and AVCD. Guidelines for livestock market operations during COVID-19 County Government of Marsabit, USAID and AVCD nd. County Government of Marsabit. Guidelines for Livestock Market Operations during COVID-19. County Government of Turkana, USAID and AVCD nd. County Government of Turkana. Guidelines for Livestock Market Operations during COVID-19. County Government of Wajir, USAID and AVCD nd. County Government of Wajir. Guidelines for Livestock Market Operations during COVID-19. DeGusta, Michael. “Are Smart Phones Spreading Faster than Any Technology in Human History?” MIT Technology Review, May 2012. https://www.technologyreview.com/2012/05/09/186160/are-smart-phones-spreading-faster-than￾any-technology-in-human-history/. FAO. 2021 Kenya’s ASALs, the Emerging Frontier for Opportunities and Development. Blog post. FAO Kenya. https://www.fao.org/archive/from-the-field/detail/en/c/202711/ Fischer, E., Qaim, M. 2012. Gender, agricultural commercialization, and collective action in Kenya. Food Sec. 4, 441–453. https://doi.org/10.1007/s12571-012-0199-7 FTF. 2018. Global Food Security Strategy (GFSS) Kenya Country Plan April 2018 GOK / USAID. nd. Community Dialogue Cards on Agri-nutrition. For use by Community Own Resource Persons. 114 Government of Kenya. 2019. The National Livestock Policy Higgins, P., and H. Alderman. 1997. Labor and women’s nutrition: The impact of work effort and fertility on nutritional status in Ghana. Journal of Human Resources 32 (3): 577–595. International Livestock Research Institute (ILRI). 2015a. Feed the Future Kenya Accelerated Value Chain Development program. Accelerating value chains development for smallholder farmers and pastoralists with technologies and innovations. Program document. Nairobi, Kenya: ILRI. International Livestock Research Institute (ILRI). 2015b. Feed the Future Kenya Accelerated Value Chain Development program. Accelerating value chains development for smallholder farmers and pastoralists with technologies and innovations. ILRI Project Report. Nairobi, Kenya: ILRI Kiome R., 2016. Kenya Accelerated Value Chains Development Program. Financial year 2015/16 Quarter 3 Progress Report. August 2016 Kiome, R.M., Wamwere-Njoroge, G., Parker, M., Audi, P., Rao, J., Muoki, P. and Bamanya, B. 2018. The Accelerated Value Chain Development Program national conference report. Nairobi, Kenya: International Livestock Research Institute (ILRI). KIT. 2017. Increasing Youth Engagement in the AVCD Programme. Draft Report, 4 August 2017. Royal Tropical Institute (KIT), The Netherlands. Mayne, J. (2008). Contribution Analysis: An approach to exploring cause and effect. Brief 16, Institutional Learning and Change (ILAC) Initiative Mutie Ianetta, Miano Grace, Bamanya Barbara, Wamwere – Njoroge. G. 2017. USAID Feed the Future Kenya Accelerated Value Chain Development (AVCD) - Livestock component DRAFT Annual Report, October 2017 Njarui D M G, Gichangi E M, Gatheru M, Nyambati E M, Ondiko C N, Njunie M N, Ndungu￾Magiroi K W, Kiiya W W, Kute C A O and Ayako W. 2016a. A comparative analysis of livestock farming in smallholder mixed crop-livestock systems in Kenya: 1. Livestock inventory and management. Livestock Research for Rural Development. Volume 28, Article #66. Retrieved August 22, 2017, from http://www.lrrd.org/lrrd28/4/njar28066.html Njenga et al., 2021 Njuki, J., Kaaria, S., Chamunorwa, A and Chiuri W. 2011. Linking Smallholder Farmers to Markets, Gender and Intra-Household Dynamics: Does the Choice of Commodity Matter? European Journal of Development Research 23(3):426-443 https://www.researchgate.net/publication/227469109_ Njuki, J.M., Wyatt, A., Baltenweck, I., Yount, K., Null, C., Ramakrishnan, U., Girard, A.W. and Sreenath, S., 2016. An exploratory study of dairying intensification, women’s decision making, and time use and implications for child nutrition in Kenya. The European Journal of Development Research, 28(4), pp.722-740. Quisumbing, A. R. (2003). Food aid and child nutrition in rural Ethiopia. World Development, 31(7), 1309-1324. 115 Ragin, C. (1984). ‘What is Qualitative Comparative Analysis (QCA)?’ Available at: http://eprints.ncrm.ac.uk/250/1/What_is_QCA.pdf Ravallion, Martin. (2009.) ‘Evaluation in the Practice of Development’ World Bank Research Observer 24(1): 29-53 Swindale, A. and Bilinsky P. 2006. Household Dietary Diversity Score (HDDS) for Measurement of Household Food Access: Indicator Guide VERSION 2. Food and Nutrition Technical Assistance (FANTA) III. USAID. Swindale, A. and Bilinsky P. 2010. Months of Adequate Household Food Provisioning (MAHFP) for Measurement of Household Food Access: Indicator Guide VERSION 4. Food and Nutrition Technical Assistance (FANTA) III. USAID. Tegemeo Institute. 2016. Report of a Study on Assessing the Cost of Production Structures in Dairy Systems in Kenya. Nairobi, Kenya: Egerton University, pp. 43. Tegemeo Institute. Poverty Assessment in Rural Kenya: Explaining Trends in FTF ZOI. Technical Report by Miltone Ayieko, Tim Njagi, Milu Muyanga, John Olwande, Joseph Opiyo, and Kevin Onyango. April, 2017 Theory-Based Approaches to Evaluation: Concepts and Practices (2012) available at http://www.tbs-sct.gc.ca USAID-KAVES. Baseline Survey 2013 USAID Prosper Counties – See details in http://www.usaid.gov/sites/default/files/documents/1860/Prosper-Counties-Fact-Sheet.pdf Wamwere-Njoroge, G., Makau, J.M., Ng’ang’a, E.W., Kutu, A. A., Osman, A. J. and Wanyoike, J.N. 2021. Feed the Future Accelerated Value Chain Development (AVCD) Program Livestock value chain annual survey report, October 2020, ILRI. Wamwere-Njoroge, G., Makau, J.M., Rutsimba, E. Ng’ang’a, E.W., Kutu, A.A., Osman, A, Wanyoike, J.N., Hambe, H.A, Omosa, E.B., Kiara, H. K, and Bett, B. 2015. Feed the Future Accelerated Value Chain Development. (AVCD) Program Effect of COVID-19 on livestock value chain actors and food systems in northern Kenya International Livestock Research Institute (ILRI) March 2021. Wilson-Grau, R & Britt, H. 2012. Outcome Harvesting. Ford Foundation. Woolcock, Michael. Reasons for Using Mixed Methods in the Evaluation of Complex Projects, CID Faculty Working Paper No. 348, March 2019 World Bank. 2018. Boosting Prosperity, Improving Equity in North and North Eastern Kenya Available at https://www.worldbank.org/en/news/feature/2018/05/08/boosting-prosperity￾improving-equity-in-north-and-north-eastern-kenya 116 Appendix 3. AVCD Results Framework (as presented in program proposal) 117 Appendix 4. Map of Kenya showing AVCD project area 118 Appendix 5. Evaluation County Selection Matrix County Value chain Duration Prosper County? Select? Remarks P1 P2 Isiolo LVC Y Y Y Y More diverse interventions which are at advanced levels of adoption including; Rangeland management bill, Community Based Natural Resource Management (CBNRM), Land rehabilitated, Livestock Market Management Fund (LMMF)2019, Livestock Sale Yard bill enacted (2016), E- disease surveillance system, mobile information marketing systems Marsabit LVC Y Y N Y Received similar interventions as Isiolo yet slower in adoption and attainment of targets. Selected for contiguity with Isiolo and comparison to identify factors impending adoption Turkana LVC Y Y N N Desk reviews show limited adoption of interventions and, hence, may not present significant additional information compared to Marsabit. It will be assessed through desk review and AVCD staff interviews. Garissa LVC Y N N N Will be assessed through desk review and AVCD staff interviews. Wajir LVC Y Y N N Will be assessed through desk review and AVCD staff interviews. Kakamega DVC Y N Y N Only one VC, implemented in phase one, hence may not present much learning opportunities. Busia DVC, PVC Y N N N Implemented two VCs, but slow in adoption, hence may not present much learning opportunities. Will be assed under Desk review and AVCD staff interviews. Vihiga DVC Y Y N N Will be assessed through desk review and AVCD staff interviews. Bungoma OFSP, PVC Y Y N Y Reported to have recorded relatively higher yields than Taita Taveta, so a good case for comparative analysis. In addition, as a scale-up county for potato, it presents rich comparative case with counties in phase 1 Siaya DVC, DTC Y Y N N Will be assessed through desk review and AVCD staff interviews. 119 Homa Bay DVC, OFSP Y Y N Y Received diverse interventions in two VCs in both phases, so provides a good case for assessment of progress. Migori DVC, OFSP Y Y N N Only beneficiary dairy cooperative in Rongo is closer to the Homa Bay cluster (border). If time and resources permit, hold an FGD with the Rongo Coop. Otherwise, it will be assessed through desk review and AVCD staff interviews. Kisumu DVC Y Y Y N Will be assessed through desk review and AVCD staff interviews. Nandi PVC Y N N N Only a single value chain implemented so a less rich comparator, unlike Elgeyo Marakwet with two VCs Uasin Gishu PVC Y N N N Will be assessed through desk review and AVCD staff interviews. Elgeyo Marakwet DTC, PVC Y N N Y Phase 1 only and multiple value chains for richer comparative analysis. It presents an opportunity understand how the progress made in phase 1 has been sustained and provides comparison with other regions Kitui DTC Y Y N N Will be assessed through desk review and AVCD staff interviews. Makueni DTC, DVC Y Y Y Y Provides an opportunity to learn the performance of two VCs over the entire duration of the program Taita Taveta DTC, PVC Y Y N Y Provides an opportunity to learn the performance of two VCs over the entire duration of the program. In addition, as a scale-up county, it presents rich comparative case with counties in phase 1 Tharaka Nithi DTC Y N N N Will be assessed through desk review and AVCD staff interviews. Meru PVC Y N N N Will be assessed through desk review and AVCD staff interviews. Source: ET’s analytical review of documents 120 Appendix 6. Expanded Evaluation Matrix Please see the attached file. 121 Appendix 7. Schedule for the Evaluation and Site Visits Evaluation Logistics and Timelines Output Comments Date (approximate) Inception meeting Clear and mutual understanding of the technical requirements of the award August 10, 2021 Inception report Provides a projected timeline and describes the methodology in detail and includes the final data collection tools, interview protocols, limitations, and ways to mitigate them, and an operational workplan August 20,2021 Stakeholder Workshop To review, refine and accept the work plan and tools August 30, 2021 Key Informant Interviews With USAID, IP staff, CGIAR Center leadership, Government officials, private sector representatives September 7-14, 2021 Field Visits To conduct KIIs, FGDs and site visits September 15 – October 1, 2021 Weekly Updates Field work updates – virtual, including any issues or problems encountered September 17, 23, 30 October 7 Presentation Preliminary findings and conclusions October 8, 2021 Draft report Draft Evaluation Report December 10, 2021 Debrief Presentation Major findings to the technical team and other key stakeholders. Discuss the methodology, challenges, key findings, conclusions, and recommendations. January 5, 2022 Final Report Submitted to USAID along with all raw data. The report will be based on the provisions of the USAID evaluation policy and include comments and suggestions from USAID/KEA, AVCD program team and implementing partners January 12, 2022 Executive Summary Briefing document: 2-3 pager January 12, 2022 122 Field Logistics and Timelines County Comments Date Nairobi Evaluation Team September 7-14, 2021 Taita Taveta Evaluation Team September 15-17, 2021 Nairobi Debrief Evaluation Team September 18-20, 2021 Marsabit Team 1 September 21-23, 2021 Elgeyo Marakwet Team 2 Isiolo Team 1 September 24-27, 2021 Bungoma Team 2 Makueni Team 1 September 29-October 1, 2021 Homa Bay Team 2 Debrief & Analysis Evaluation Team October 4-9, 2021 123 Appendix 8. Evaluation Work Plan Kenya AVCD Performance Evaluation Work Plan Task Project Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Inception Deliverable 1: Initial Meeting between USAID and the Contractor evaluation team to clarify roles and responsibilities, logistical issues, and timelines (Aug 10) * Deliverable 2: Inception Report, which describes the methodology in detail and includes the final data collection tools and an operational work plan (Aug 20) * Deliverable 3: USAID/Partner/Stakeholder Workshop to review, refine and accept the work plan and tools (Aug 30) * Implementation Deliverable 4: Key Informant Interviews and FGDs with USAID, IP staff, and other informants Deliverable 5: Fieldwork Deliverable 6: Fieldwork updates in person or via email/virtually as agreed at the initial meeting, including any issues or problems encountered (Sep 17, 23, 30, Oct 7) * * * * Analysis and Reporting Deliverable 7: Presentation of preliminary findings and conclusions of the evaluation to USAID/Kenya and key stakeholders (Oct 8) * Deliverable 8: Draft Evaluation Report (Dec 10) * Deliverable 9: Debrief/Presentation of Report. The evaluation team will present the major findings to the technical team and other key stakeholders.(TBD) * Deliverable 10: Final Evaluation Report submitted to USAID along with all raw data. (TBD) * Deliverable 11: A 2–3-page Briefing Document and an Executive Summary (TBD) * * dates are approximate and subject to adjustment with USAID. 124 Appendix 9. Team Composition and Roles The table below lists the primary roles and sector specialization of each evaluation team member: Team Member Role Task Order Manager Serve as the main point of contact for award management, communications with and reporting to USAID/KEA; manage subcontractors; ensure the implementation is done within the approved scope, timeline, and budget. Team Leader Leading the team; serve as the main point of contact for stakeholders, compile reports; evaluate the crop and livestock value chain components. Agriculture/Livestock Value Chain (LVC) Specialist Lead data analysis; compile reports; evaluate the crop and livestock￾based value chain components. Nutrition and Gender Specialist Evaluate the nutrition and gender subcomponents of each of the value chain components. Governance Specialist Evaluate the governance subcomponents of each of the value chain components. Qualitative Analyst Support qualitative data collection. Record and transcribe KIIs and FGDs with key stakeholders, code transcripts and support data analysis and reporting. Data Analyst/Statistician Support quantitative data abstraction and analysis. Nutrition Expert Provide nutrition back-up support to the evaluation team including data collection and analysis. Logistician Support outreach to stakeholders in Nairobi and counties to coordinate scheduling of KIIs and FGDs. Manage travel logistics including air and ground transportation, lodging, and security arrangements. 125 Appendix 10. KEY INFORMANT INTERVIEW SCRIPTS Accelerated Value Chain Development Performance Evaluation 1. Authority This evaluation is conducted under the Statistics Act 2006. The information provided in this interview will be treated in strict confidence and used for research and informational purposes only. Please be assured that none of your responses to the questions will be shared publicly and that any information you provide will be used exclusively for the purpose of the evaluation. 2. Objective The USAID/Kenya and East Africa Mission has recently commissioned a final performance evaluation of the Kenya Feed the Future Accelerated Value Chain Development (FTF-AVCD) Program, a five-year activity implemented by a consortium of Consultative Group on International Agricultural Research (CGIAR) Centers led by the International Livestock Research Institute (ILRI). The other members of the consortium include the International Crops for Research Institute for Semi-Arid Tropics (ICRISAT) and the International Potato Center (CIP). The purpose of the evaluation is to assess the design and implementation of the AVCD program to inform the design of future activities which harness agricultural technologies and innovations within the CGIAR and other research organizations. It will also provide lessons on the innovative approach of partnering with the CGIAR centers to implement development interventions and how partnership with the national and county governments, and the private sector can contribute to Kenya’s development. LINC LLC, in partnership with ICF Macro, is conducting the evaluation. I would like to introduce to you the team which will be conducting the evaluation and request your assistance in responding freely to their inquiries on the subject. To ensure the success of the evaluation, they will need to conduct brief interviews or focus groups with the staff and/or leadership of multiple institutions and amongst community groups that have interacted with AVCD. We respectfully request your assistance in facilitating the successful scheduling of meetings and interviews. 3. Instructions The following notes are useful in completing the interview. Read carefully and understand before completing. More specific notes are provided as filling instructions to the relevant sections for your reference. i). Who to complete this form Selected value chain and market actors at the national and county levels that are engaged in the AVCD Activity. ii). Period covered The information required is for the years 2015 to 2021, where applicable. iii). Completion and submission of the Form The respondent is encouraged to generate and retain a copy of the completed interview. All sections should be completed. Any parts deemed inapplicable should be indicated as "not applicable". 4. Secondary Sources Some of the information required for this enquiry can be obtained from published material – reports or data. Kindly attach or link copies of any relevant supporting documents. 5. Other Comments In case you have any comments or information relating to the data provided, please insert them in the comment box found on page 5. For any further inquiries regarding this evaluation, please contact Patrick Lohmeyer (plohmeyer@linclocal.org). We value your cooperation in this activity and thank you for your support. Sincerely, Hezekiah Agwara Team Leader, AVCD Performance Evaluation 126 QUESTIONS FOR DAIRY BREEDING AND ANIMAL HEALTH SERVICES (BAHS) KEY INFORMANTS Name…………………………………………………... Organization…………………………… Position…………….…………………………………. Contacts…………………………………………………… County / Value Chain…………………………………………………………………………………… 1. How was breeding and vaccination services (BAHS) being delivered to farmers in this area before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 2. How did the team enhance dairy farmers’ access to BAHS under the project? (Probe for what contributed to success under the project.) a. At initiation of the project ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 3. Was there a difference in access to BAHS under the project for older men and women and male and female youth? …………. (yes or no) a. If yes, did the difference in access change over the life of the project? (yes or no). If yes, how? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………. b. Why did it change or not change? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 4. What shortcomings/challenges/weakness did you observe in the BAHS component of the project? (Probe for the causes and what the project should have done or continue/stop doing.) a. At initiation of the project ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… 127 5. What enabling factors did you observe in the project? (Probe for the causes and what the project should have done or continue/stop doing.) c. At initiation of the project ………………………………………………………………………………………………………………… d. At project implementation phase ………………………………………………………………………………………………………………… 6. How did you solve the challenges/shortcoming named in 3? a. At initiation of the project ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… 7. How best should similar project (s) address the named challenges in future? (Probe for what the project should start/keep/stop doing and adjustments or improvements to be made.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 8. In what ways can agriculture benefit from lessons learned on the BAHS component of this project? ……………………………………………………………………………………………………………………… ……………………………………………………………………………..………………………………………… 9. Please give any other views concerning the BAHS for this area? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………… 128 QUESTIONS FOR AGRICULTURE EXTENSION SERVICES KEY INFORMANTS Name………………………………………………….. Organization…………………………… Position…………….…………………………………. Contacts…………………………………………………… County / Value Chain…………………………………………………………………………………… 1. How was agro-extension and advisory services (AEAS) being provided to farmers in this area before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………………………… …………………………………………………………………………………………………………… 2. How did your organization influence beneficiary farmers’ access to AEAS under the project? probe for what contributed to success under the project? a. At initiation of the project ………………………………………………………………………………………………………………… …… b. At project implementation ………………………………………………………………………………………………………………… …… 3. Were there similar projects or interventions during the period of implementation? If yes, name them………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………… 4. List the technologies, Innovations or management practices that you implemented because of the project. ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 5. Did you observe any change in yield/productivity? If yes, what was the change ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 6. What are you doing with the additional production? Home consumption, gift or sell? …………………………………….. 7. What is the change in income as a result of the extra yield marketed? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 129 8. Would you still continue to use the technology, innovation or management practice without AVCD support? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 9. Was there a difference in access to AEAS under the project for older men and women and male and female youth? Yes or No (Probe and capture responses related to each population group) a. If yes, did the difference in access change over the life of the project? Yes or No. If yes, how? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… b. Why did it change or not change? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 10. What shortcoming/challenges/weakness, if any, did you observe in the AEAS component of the project? (probe for the causes and what the project should continue/stop doing) a. At initiation of the project ………………………………………………………………………………………………………………… b. At project implementation ………………………………………………………………………………………………………………… 11. How did you solve the challenges/shortcoming named in 3? a. At initiation of the project ………………………………………………………………………………………………………………… b. At project implementation ………………………………………………………………………………………………………………… 12. How best should similar project (s) address the named challenges in future? Probe for what the project should start/keep/stop doing and adjustments and improvements to be made ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 13. In what ways can agriculture benefit from lessons learned on the AEAS component of this project? ……………………………………………………………………………………………………………………… …………………………………………………………………………………………………..…………………… 14. Please give any other views concerning the AEAS for this area? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 130 QUESTIONS FOR KEY INFORMANTS OF FARMER PRODUCER ORGANIZATIONS / LIVESTOCK MARKETING ASSOCIATIONS Name: ………………………………………………….. Organization: …………………………… Position: …………….…………………………………. Contacts: …………………………………………………… County / Value Chain: …………………………………………………………………………………… 1. Did you work with beneficiary farmers under the AVCD project? 1= Yes 2=No 2. How were farmers organized before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 3. If YES in 1, what was the intervention to your organization by the AVCD Project? ……………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………. 4. How did you and your team enhance farmer organization under the project? Probe for what contributed to success: a. At initiation of the project ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… 5. Were the initiatives implemented on organizations under the project new to you and your team or had been tried before? (Probe for what constituted innovation.) Yes / No a. If Yes, what of the elements did you find new? ………………………………………………………………………………………………………………… b. If No, what had you and your organization tried before the project? ………………………………………………………………………………………………………………… 6. How did the interventions affect your business? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 7. Was there a difference in the way older men and women and male and female youth organized? Yes or No c. If yes, did the difference in organization change over the life of the project? Yes or No. If yes, how? 131 ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… d. Why did it change or not change? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 8. What shortcomings/challenges/weakness did you observe in the organization approaches of the project? (probe for the causes and what the project should continue/stop doing) a. At project initiation phase ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… 9. How did you solve the challenges/shortcomings named in 5? a. Solutions at initiation phase ………….………………………………………………………………………………………………… b. Solutions at implementation phase …………………………………………………………………………………………………………… 10. How best should similar project (s) address the named challenges in future? (Probe for what the project should start/keep/stop doing and adjustments and improvements to be made.) ……………………………………………………………………………………………………………………… …………………………………………………………………………………………………………… 11. In what ways can agriculture benefit from the gains and lessons learned on the organization component of this project? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………… 12. Please give any other views concerning farmer organization and marketing for this area …………………………………………………………………………….………………………………………… ……………………………………………………………………………………………………………………… …………………………………………………………………………………………………… 132 QUESTIONS FOR KEY INFORMANTS OF COMMUNITY HEALTH WORKERS AND VOLUNTEERS (CHWVS) AND COUNTY NUTRITION TEAMS Name………………………………………………….. Organization…………………………… Position: …………….…………………………………. Contacts: …………………………………………………… County / Value Chain…………………………………………………………………………………… 1. Did you work with beneficiary households under the AVCD project? 1= Yes 2=No 2. How was agri-nutrition education and information provided before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 3. If yes in 1, how did you and your team enhance agri-nutrition education and information under the project? Probe for what contributed to success: a. At initiation of the project ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………. b. At project implementation phase ………………………………………………………………………………………………………………… …………………………………………………………………………………………………………… 4. Were the initiatives implemented on agri-nutrition education and information under the project new to you and your team or had been tried before? (Probe for what constituted innovation.) Yes / No a. If Yes, what of the elements did you find new? ………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………. b. If No, what had you and your team tried before the project? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………… 5. Was there a difference in the way agri-nutrition education and information was provided for older men and women and male and female youth? Yes or No a. If yes, did the difference in provision change with inception of the project? Yes or No. If Yes, what changed? ……………………………………………………………………………………………………………..…… ……………………………………………………………………………………………………………. 133 b. Why did it change or not change? …………………………………………………………………………………………………………………. 6. Was agri-nutrition education and information service targeted? a. Who was targeted? …………………………………………. b. Why? ……………………………………………………………. c. Was targeting beneficial? Yes or No. If Yes, what were the three key benefits? 1. 2 3. 7. What shortcomings/challenges/weakness did you observe in the agri-nutrition component of the project? (Probe for the causes and what the project should continue/stop doing) a. At project initiation phase ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… 8. How did you solve the challenges/shortcomings named in 5? a. Solutions at initiation phase ………….………………………………………………………………………………………………… b. Solutions at implementation phase ………………………………………………………………………………………………………………… 9. How best should similar project (s) address the named challenges in future? (Probe for what the project should start/keep/stop doing and adjustments and improvements to be made.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 10. In what ways can the community benefit from the gains and lessons learned from the agri￾nutrition component of the AVCD project? (Probe for what categories of (people/households) in the community benefited from the agri-nutrition intervention) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 11. Please give any other views concerning agri-nutrition situation and interventions for this area …………………………………………………………………………….………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 134 QUESTIONS FOR KEY INFORMANTS OF COMMUNITY BASED NATURAL RESOURCE MANAGEMENT (CBNRM) COMMITTEES Name………………………………………………….. Organization…………………………… Position…………….…………………………………. Contacts…………………………………………………… County / Value Chain…………………………………………………………………………………… 1. Did you work with beneficiary livestock farmers under the AVCD project? 1= Yes 2=No 2. How were communities organized to manage natural resources before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 3. If yes in 1, what was the intervention to your CBNRM by the AVCD Project? ……………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………………. 4. How did you and your team enhance NRM under the project? (Probe for what contributed to success:) a. At initiation of the project ………………………………………………………………………………………………………………… ………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… ……………………………………………………………………………….……………………...………… 5. Were the NRM initiatives implemented under the project new to you and your team or had been tried before? (Probe for what constituted innovation.) Yes / No a. If Yes, what of the elements did you find new? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………… b. If No, what had you and your organization tried before the project? ………………………………………………………………………………………………………………… ……………………………………………………………………………….……………………...………… 6. Was there a difference in the way older men and women and male and female youth organized to manage natural resources? …………. (yes or no) a. If yes, did the difference in organization change with inception of the project? …………. (yes or no). If yes, what changed? 135 …………………………………………………………………………………………………………………. b. Why did it change or not change? …………………………………………………………………………………………………………………. 7. What shortcomings/challenges/weakness did you observe in the CBNRM component of the project? (probe for the causes and what the project should continue/stop doing) a. At project initiation phase ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… ……………………….……………………………………………… 8. How did you solve the challenges/shortcomings named in 4? a. Solutions at initiation phase ………….……………………………………………………………………………………………………… ………………………………………………………………….………….…………………….……………… b. Solutions at implementation phase ………………………………………………………………………………………………………………… ……………………………………………………………………………….………………………………… 9. How best should similar project (s) address the named challenges in future? (Probe for what the project should start/keep/stop doing and adjustments and improvements to be made.) ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ………………………………………………………………………………………………… 10. In what ways can agriculture benefit from the gains and lessons learned on the NRM component of this project? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………………………………………………………...………………………………………………… 11. Please give any other views concerning NRM for this area …………………………………………………………………………….………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ………………………………………………………………………............ 136 QUESTIONS FOR AGGREGATION PARTNER KEY INFORMANTS Name………………………………………………….. Organization…………………………… Position…………….………………………………….Contacts…………………………………………………… County / Value Chain…………………………………………………………………………………… 1. How was agricultural marketing and information being done and communicated before introduction of the AVCD project? Probe for ineffectiveness ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 2. Did you work with beneficiary farmers under the AVCD program? 1= Yes 2=No 3. If yes in 2, what was the nature of intervention by the project or the beneficiary farmers? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 4. How did you and your team enhance farmers’ access to markets under the project? Probe for what contributed to success: a. At the initiation phase ………………………………………………………………………………………………………………… ……………………………………………………………………………………… b. At project implementation phase ………………………………………………………………………………………………………………… ……………………………………………………………………………….……………………... 5. Was there a difference in access to markets for older men and women and male and female youth? Yes or No a. If Yes, did the difference in access to markets by gender and age change with inception of the project? Yes or No. If yes, what changed? ………………………………………………………………………………………………………………… …………………………………………………………………………………………………………. b. Why did it change or not change? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………. 6. What shortcoming/challenges/weakness did you observe in the marketing component of the project? (probe for the causes and what the project should continue/stop doing) a. At project initiation phase ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 137 b. At project implementation ………………………………………………………………………………………………………………… …………………………………………………………………….…………………………………………… 7. How did you solve the challenges/shortcomings named in 4? a. Solutions at initial phase ………….……………………………………………………………………………………………………… ………………………………………………………………….………….…………………….……………… b. Solutions at implementation phase ………………………………………………………………………………………………………………… ……………………………………………………………………………….………………………………… 8. How best should similar project (s) address the named challenges in future? Probe for what the project should start/keep/stop doing and adjustments and improvements to be made. ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 9. In what ways can agriculture benefit from lessons learned on the marketing component of the project? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………………………………………………………...………………………………………………… 10. Please give any other views concerning agricultural aggregation and marketing for this area …………………………………………………………………………….………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 138 QUESTIONS FOR AGRICULTURAL FINANCE INSTITUTIONS KEY INFORMANTS Name………………………………………………….. Organization…………………………… Position…………….………………………………….Contacts…………………………………………………… County / Value Chain…………………………………………………………………………………… 1. How was agricultural finance information being communicated to farmers before introduction of the AVCD project? Probe for ineffectiveness ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 2. How did your team enhance farmers’ access to financial information and products under the project? Probe for what contributed to success: ……………………………………………………………………………………………………………………… 3. Was there a difference in access to financial information and products for older men and women and male and female youth? …………. (yes or no) a. If yes, did the difference in access to financial information and products by gender and age change with inception of the project? Yes or No. If Yes, what changed? ………………………………………………………………………………………………………………… b. Why did it change or not change? …………………………………………………………………………………………………………………. 4. What shortcoming/challenges/weakness did you observe in the financing component of the project? (probe for the causes and what the project should continue/stop doing) ……………………………………………………………………………………………………………………… 5. How did you solve the challenges/shortcomings named in 3? ……………………………………………………………………………………………………………………… 6. How best should similar project (s) address the named challenges in future? Probe for what the project should start/keep/stop doing and adjustments and improvements to be made ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 7. In what ways can agriculture benefit from lessons learned on the financing component of this project? ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… 8. Please give any other views concerning agricultural financing for this area? ……………..……………………………………………………………………………………………………… Inception Report, Kenya FTF-AVCD Final Performance Evaluation Page 139 of 164 QUESTIONS FOR TRADE AND INDUSTRY ASSOCIATIONS KEY INFORMANTS Interviewer: ____________________ Date: _______________ Respondent name and position: _______________________________________ Other respondents’ names and positions [If any]: _______________________________ I. GENERAL INFORMATION ABOUT THE ASSOCIATION 1. Legal name of the association and when it was registered ____________________________; ____________________ 2. Which regions of the country/countries does the association cover? a) ............................................................ b) ............................................................. c) ............................................................. 3. Give a brief description of the objectives or roles of the association in the agriculture and livestock sector a) ................................................................. b) ................................................................. c) ................................................................. 4. What matters of mutual benefit do members of the Association(s) normally deliberate on? a) .............................................................................................................. b) ................................................................................................................ c) ................................................................................................................ 5. What levels of the value chain and types of products is the Association interested in (not restricted to AVCD value chains): a) ............................................................. b) ............................................................. c) ............................................................. 6. Who qualifies to be a member of the Association? Include categories of membership a) ................................................ b) ................................................ c) …………………………………. 7. What are the charges for different categories of membership (KES) a) .................................................. b) .................................................. c) ............................................... 8. State the current number of members in the different categories Membership Category Number registered Number of primary members represented How often do various members meet per year 140 a) Individual farmers b) Individual traders c) Corporate d) Other Category (specify) 9. Was there a difference in membership in your organization for older men and women and male and female youth before inception of AVCD? Yes or No a) If yes, did the difference in organization membership by gender and age change with inception of the project? Yes or No). If yes, what changed? …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… ………………. b) Why did it change or not change? …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… ……………………………………………… 10. Are there key actors in the industry who are not members of your Association? YES/NO................ 11. If YES, what are the reasons for not being members of the Association? a) ................................................... b) ................................................... c) ................................................... 12. Which other Associations / Organizations do you collaborate with? a) ....................................... b) ....................................... c) ....................................... 13. What information are members required to provide regularly? a) ..................................................................................................... b) ..................................................................................................... c) ..................................................................................................... II. STATE OF COMPETITION (RELATED TO PRODUCTION AND MARKETING) 14. How much [insert relevant product I service] did your members produce / sell last year? Product / Service Amount Produced / Sold (tons / KES) Gender ratio of producers / sellers (men>35; men <35; women>35; women <35) 141 15. What THREE TOP issues do the members complain about? a) .......................................... b) .......................................... c) .......................................... 16. Was there a difference in the nature and complaint by men and women of different ages? Give examples. ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ……………………………………………. 17. What challenges/constraints do you face in the discharge of your mandate? a) .................................................................................................................. b) .................................................................................................................. c) ................................................................................................................... 18. How best could the above challenges be addressed? ..................................................................................................................... .................................................................................................................... 19. What is your general satisfaction with the level of competition in the Kenya [insert the relevant name] value chain(s)? Rank from 0 (not satisfied) to 10 (very satisfied) ............. 20. If dissatisfied with the value chain or markets (if ranking in Q17 is less than 5), state any THREE major constraints you face a) ......................................... b) ......................................... c) ......................................... 21. What are the causes of these constraints? a) ........................................ b) ....................................... c) ....................................... 22. The AVCD program has engaged in agriculture and agri-nutrition value chain and market development over the past five years through the promotion of technology and innovation. Has your Association engaged in the activities and interventions of the program? YES/NO 23. If YES to Q20, list key areas of engagement below a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... d) .......................................................................................................................... 24. In your view, how have the activities, interventions or collaboration by the AVCD program influenced your engagement with the agriculture and nutrition sectors? Summarize positive and negative effects or what worked (didn’t work) well. a) ........................................................................................................................ b) .......................................................................................................................... 142 c) ........................................................................................................................... d) .......................................................................................................................... 25. Generally, how would you rate the performance of the AVCD in its role of promoting agricultural and nutrition technology and innovation, considering the direct link of technology and innovation on agriculture value chains and markets? Rate on a scale of 0 (unsatisfactory) to 10 (quite satisfactory) 26. What changes or improvements, if any, would you propose to make the AVCD program more impactful? a) .................................................................................................................... b) ....................................................................................................................... c) ........................................................................................................................ d) ………………………………………………………………………………………. III. POLICIES AND REGULATIONS AND GENERAL BUSINESS ENVIRONMENT 27. The County Government is a major player in agricultural value chains and markets. What was the level of your Association’s involvement in the value chain related (policy) decisions by county government? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… …………………………………………………………………. 28. In your view, how do the roles and functions of County influence operations and conduct of the [insert relevant name] value chain and markets? Summarize both positive and negative impacts/influences below: a) ...................................................................................................................... b) ...................................................................................................................... c) ...................................................................................................................... 29. If the roles and functions of the County Government influence the value chain or markets negatively, what changes would you propose? a) .................................................................................................................... b) ...................................................................................................................... c) ...................................................................................................................... 30. Generally, how would you rate the performance of the County Government? Rate on a scale of 0 (unsatisfactory) to 10 (very satisfactory) .................... 31. Generally, what is your view of the adequacy of the agricultural and value chain regulatory institutions in Kenya? Rate from 0 (inadequate) to 10 (highly adequate) ........................ 32. What is your view of the adequacy of instruments/legislations used by regulatory institutions (e.g. operational and monetary penalties, arrest, prosecution, blacklisting, etc.)? Rate from 0 (inadequate) to 10 (highly adequate) 143 33. In your view, are the taxes and levies charged by government and regulatory institutions commensurate with the services provided to the maize/milling industry? Rank from 1 (excessively high) to 10 (very low) ………. 34. How do you rate the level of Kenyan value chain players’ compliance with contractual arrangements? Rate from 0 (none) to 10 (excellent – most are compliant) ………………………………………… 35. Rate the adequacy of logistics and infrastructure in your members’ areas of operation. Rate from 1 (very poor) to 10 (very good) ………… CONCLUDING REMARKS i). Name any other Associations or Organizations you would recommend we talk to ........................................................................................................................................................ ...…………………………………………………………………………………………………………… ……………………………………………… ii). Please add any comments or observations on topics that we have not covered in the space below. 144 QUESTIONS FOR GOVERNMENT INSTITUTIONS KEY INFORMANTS Interviewer: ____________________ Date: _______________ Respondent name and position: ____________________________________________________ Other respondents’ names and positions [If any]: _____________________________________ I. GENERAL INFORMATION ABOUT THE INSTITUTION 1. Name of the institution _____________________________________________________________ 2. Brief description of the functions or roles of the institution targeting the agriculture, livestock and nutrition sectors a) .................................................... b) .................................................... c) .................................................... 3. Explain how the agriculture, livestock and nutrition sectors fit into the specific policy interventions being pursued by your institution a) ..................................................... b) ..................................................... c) ..................................................... 4. The number of staff of the institution specifically dedicated to performing roles relating to the agriculture, livestock and nutrition sectors (# - separate into men>35; men <35; women>35; women <35) ........................................................................................................................................................... ........................................................................................................................................................... ...................................................... 5. Does your institution have a gender policy and or strategy? Does it have gender equity/ equality positions? Give specific examples: a) Gender policy ………………………. b) Gender strategy ……………………. c) Gender equity / equality ……………. II. INFORMATION ON RESEARCH AND DEVELOPMENT 6. Types of agricultural value chains and products the institution is interested in: a) ......................................... b) ......................................... c) ......................................... 7. List the specific services the institution provides to value chain actors: a) ......................................... b) ......................................... c) ......................................... 145 8. How many times in the last five years have staff of the institution held joint meetings/workshops with agricultural value chain and market players in order to address issues affecting them? ........................To the best of your recollection, please list the joint meetings and workshops below: ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………… …………………………………………………………………………………………………………………….. III. STATE OF COMPETITION (RELATED TO PRODUCTION AND MARKETING) 9. How many companies (including state companies) currently operate in the country / county / value chain (specify)? ____________ 10. According to your records, how much [insert relevant product name] was produced in the past year in the country / county? Product Amount produced in the past year (tons) 11. What THREE key challenges do farmers, traders and service providers in the country / county face? a) ..................................................... b) ..................................................... c) ..................................................... 12. Are there gender and age based constraints? Which ones? 13. What is your general satisfaction level with the Kenya / County agricultural markets? Rank on a scale of 0 (not satisfied) to 10 (highly satisfied) ..................…………… 14. If dissatisfied with the market (if rating above is 5 or below), list THREE major constraints you face as a government institution a) ............................................. b) ............................................. c) ............................................. 15. What are the causes of these constraints? a) ........................................…… b) ........................................…… c) ........................................…… 16. The AVCD program has engaged in agriculture and agri-nutrition value chain and market development over the past five years through the promotion of technology and innovation. How has your institution engaged in the activities and interventions of the program? List key areas of engagement below: 146 a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... 17. In your view, how have the activities, interventions or collaboration of the AVCD program influenced the agriculture and nutrition sectors? Summarize positive and negative effects or what worked (didn’t work) well below. a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... 18. Generally, how would you rate the performance of the AVCD in its role of promoting agricultural and nutrition technology and innovation, considering the direct link of technology and innovation on agriculture value chains and markets? Rate on a scale of 0 (unsatisfactory) to 10 (quite satisfactory) ……………………………………………. 19. What changes or improvements would you propose to make the AVCD program more impactful? a) .................................................................................................................... b) ....................................................................................................................... c) ........................................................................................................................ V. POLICIES AND REGULATIONS 20. List key policies and regulations enacted and implemented over the last three years that affect the relevant value chains and markets a) .................................................................................................................... b) ....................................................................................................................... c) ........................................................................................................................ 21. Did your institution engage with the AVCD program on any policy and regulatory interventions? List key areas of engagement and outcomes below: a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... d) ........................................................................................................................... 22. How would you rate the adequacy of agricultural value chains and markets policy and regulatory institutions? Rate from 0 (inadequate) to 10 (highly adequate) ...................... 23. How would you rate the adequacy of instruments/legislations used by the regulatory institutions? Rate from 0 (inadequate) to 10 (highly adequate) ...................... 24. How would you rate the effectiveness of instruments/legislations used by the regulatory institutions? Rate from 0 (inadequate) to 10 (highly adequate) ...................... 147 25. In your view, how have the activities, interventions or collaboration of the AVCD program influenced the policy and regulatory regime? Summarize positive and negative effects or what worked (didn’t work) well below. a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... 26. Generally, how would you rate the performance of the AVCD program in its role of catalyzing policy, regulation and institutional changes that are supportive of technology and innovation application? Rate on a scale of 0 (unsatisfactory) to 10 (quite satisfactory) ……………………………………………. 27. As an institution, what challenges do you face in discharging your duties? a) ........................................................................................................................ b) ........................................................................................................................ c) ......................................................................................................................... 28. Are there any roles for programs like AVCD in addressing the challenges you face in discharging your duties? List the Top 3 roles in order of importance a) ........................................................................................................................ b) ........................................................................................................................ c) ......................................................................................................................... CONCLUSION 29. What other government institutions would you recommend we talk to? _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ ________________________________ 30. Please add any comments or observations on any important topic or issue we have not had a chance to discuss ______________________________________________________________ 148 QUESTIONS FOR COUNCIL OF GOVERNORS AGRICULTURE SECTOR SECRETARIAT KEY INFORMANTS Interviewer: ____________________ Date: _______________ Respondent name and position: ____________________________________________________ Other respondents’ names and positions [If any]: _____________________________________ I. GENERAL INFORMATION ABOUT THE COUNTY 1. Brief description of Respondent’s county (in terms of major agro-climatic conditions) ____________________________________________________________ 2. List of policies developed by respondent’s county over the last three years for the agriculture and livestock value chains a) ............................................................................................................................................ b) ................................................................................................................................................ c) ............................................................................................................................................... 3. How effective has the County Government been in implementing these policies? Please provide concrete examples in each case ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 4. List of all laws developed by respondent’s county over the last three years for regulating agriculture and livestock value chains a) ................................................................................................................................................... b) ..................................................................................................................................................... c) ..................................................................................................................................................... 5. How effectively are the laws and regulatory frameworks enforced by the County Government? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 6. How has the implementation of the policies and enforcement of laws impacted on the growth of the agricultural value chains in the county? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 7. How much has been allocated by the County Government to the agriculture and livestock sector over the last three financial years? Please indicate how much of this is for recurrent expenditure, and for sector development ........................................................................................................................................................... ........................................................................................................................................................... ........................................................................................................................................................... 149 II. INFORMATION ON THE COUNCIL OF GOVERNORS’ AGRICULTURE SECTOR SECRETARIAT (CGASS) 8. What specific reasons necessitated the formation of the CGASS? d) ..................................................................................................................................................... e) ..................................................................................................................................................... f) ..................................................................................................................................................... 9. What are the functions of the CGASS? a) ..................................................................................................................................................... b) ..................................................................................................................................................... c) ..................................................................................................................................................... 10. What role did the AVCD program play in the establishment of the CGASS? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 11. Other than the AVCD program, which other organizations or agencies support the CGASS? Briefly describe the nature of support provided by these organizations/agencies ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 12. How would you rate the quality of support provided by the AVCD program to CGASS? (On a scale of 1 to 3: Extremely low and inadequate (1), Moderate (2), Adequate and relevant (3) 13. What mechanisms has the Council of Governors developed to ensure that the gains from CGASS are institutionalized and sustained beyond the AVCD program? Please comment on the measures taken with regard to: a. Staffing and operations b. Budgetary allocations c. Knowledge Management d. Others (please specify) 14. How does the COG Agricultural Committee coordinate its functions and activities with? a. The Ministry of Agriculture, Livestock and Fisheries? b. The Senate? c. Parliamentary Committee on Agriculture? d. Development partners and funding agencies supporting county level agriculture and livestock development interventions? e. The private sector in the agriculture and livestock sectors in the county? 15. What role has the CGASS played in facilitating the Agricultural Committee’s engagements with the bodies listed in 14 (above)? 150 ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… III. CAPACITY OF COUNCIL OF GOVERNORS’ AGRICULTURAL COMMITTEE 16. What impact has the Agricultural Committee had in the agriculture and livestock sectors in the counties? Please comment on the following areas by giving specific examples a. Coordination with the national government b. Influencing national laws and policies on agriculture and livestock value chains c. Coordinating agriculture and livestock value chain development activities across the counties d. Influencing dissemination of technologies to farmers e. Improving agricultural productivity f. Creating an enabling environment for markets and private sector growth in the agricultural sector g. Improving food and nutrition security 17. How has the AVCD program supported in the intervention areas mentioned in 18 (above)? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 18. In what specific areas has the AVCD program not offered adequate support to the CGASS? a) ..................................................... b) ..................................................... c) ..................................................... IV. STATE OF AGRICULTURAL SECTOR IN THE COUNTIES – CONSTRAINTS & CHALLENGES 19. What major changes have occurred at the county level in the agriculture and livestock sectors over the last three years? (please highlight positive and negative changes) ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 20. Are there categories of farmers who have been adversely affected by these changes? YES = 1; NO = 0 21. If yes (20 above) which categories of farmers have been affected by the changes? a. …………………………………………………… 151 b. ……………………………………………………… c. …………………………………………………………. d. ……………………………………………………….. 22. What specific constraints do farmers face in the following areas? a. Access to appropriate technology b. Access to agricultural markets c. Access to financial services d. Laws and policies 23. How has the Agricultural Committee dealt with these challenges? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 24. In your view, how have the activities, interventions or collaboration of the AVCD program influenced the agriculture and nutrition sectors? Summarize positive and negative effects or what worked (didn’t work) well below. a) ........................................................................................................................ b) .......................................................................................................................... c) ........................................................................................................................... 25. Generally, how would you rate the performance of the AVCD in its role of promoting agricultural and nutrition technology and innovation, considering the direct link of technology and innovation on agriculture value chains and markets? Rate on a scale of 0 (unsatisfactory) to 10 (quite satisfactory) ……………………………………………. 26. What changes or improvements would you propose to make the AVCD program more impactful? a) .................................................................................................................... b) ....................................................................................................................... c) ........................................................................................................................ 27. Are there any roles for programs like AVCD in addressing the challenges you face in discharging your duties? List the Top 3 roles in order of importance a) ........................................................................................................................ b) ........................................................................................................................ c) ......................................................................................................................... 152 CONCLUSION 28. What other government institutions would you recommend we talk to? _____________________________________________________________________________ _____________________________________________________________________________ _____________________________________________________________________________ 29. Please add any comments or observations on any important topic or issue we have not had a chance to discuss ______________________________________________________________ 153 Annex 11. CHECKLIST FOR INTERVIEWS WITH AVCD STAFF • This interview should be completed by the Chief of Party and senior program staff. • Please expand the paragraphs as necessary. 1. Description a. Name of Chief of Party of the grant contract: XXX b. Name and title of the contact Person: c. Name of beneficiary (ies) and affiliated entity (ies) in the Activity: XXX d. Title of the Activity: e. Contract number: f. Start date and end date of the Activity: g. Target county (ies) or region(s): Final beneficiaries &/or target groups (if different) (including numbers of women and men): Table 1: Target Direct Beneficiaries: Counties Sub counties Wards Male Female Total No. of Households Target Beneficiaries 2. Assessment of implementation of Activity activities 1.1. Executive summary of the Activity Please give a global overview of the Activity’s implementation for the whole duration of the project Referring to the updated final logical framework matrix (see point 2.3. below), describe the level of achievement of the outcome(s) on both the final beneficiaries and/or target group (if different) and the situation in the target county which the Activity addressed. Please explain if the intervention logic has proved to be valid, including with the possible changes and their justifications, comment on the likeliness of reaching the final target(s) related to the impact in a near future (specify); Please indicate any modification that have be brought to the Logframe matrix since the start of the Activity and explain briefly why (complete explanation should be placed in the following section under the relevant level considered (outcomes, outputs, activities). 1.2. Results and Activities 154 A. RESULTS What is your assessment of the results of the Activity so far? Include observations on the performance and the achievement of outputs, outcomes and impact, and whether the Activity has had any unforeseen positive or negative results. Referring to the final updated logframe matrix (see point 2.3. below) please comment on the level of achievement of all the results on the basis of the corresponding current value of the indicators and all the related activities implemented during the reporting period. Outcome (Oc) – "" Comment on final status of indicators associated to Oc and explain any changes, especially any underperformance; refer to the indicators and assumptions in the Logframe: (possibly) intermediary Outcome 1 (iOc1) – "<Title of intermediary Outcome 1>" (…) Output 1.1. (Op 1.1.) – "Title of Output 1.1. " (…) Following the above assessment of results, please elaborate on all the topics/activities covered. B. ACTIVITIES Activity 1.1.1. Please explain any problems (e.g. delay, cancellation, postponement of activities) which have arisen and how they have been addressed (if applicable) Please list any risks that might have jeopardized the realization of some activities and explain how they have been tackled. Activity 1.1.2. (…) 1.3. Describe if the Activity will continue after the support from USAID has ended. Are there any follow up activities envisaged? What will ensure the sustainability of the Activity? 1.4. Explain how the Activity has mainstreamed cross-cutting issues such as promotion of gender and social inclusion, environmental sustainability and combating malnutrition. 1.5. How and by whom have the activities been monitored/evaluated? Please summarize the results of the feedback received from the beneficiaries and others. 1.6. What has your organization or any actor involved in the Activity learned from the Activity and how has this learning been utilized and disseminated? 1.7. Please list all materials (and number of copies) produced during the Activity in whatever format. Please state how the items produced are being distributed and to whom. 1.8. Please list all contracts (works, supplies, services) awarded for the implementation of the Activity since 2015, if any, or during the extension period, giving for each contract the amount, the name of the contractor and a brief description on how the contractor was selected. 1. Beneficiaries/affiliated entities and other Cooperation 155 1.1. How do you assess the relationship between the beneficiaries/affiliated entities of this Activity (i.e. those having signed an MOU or an affiliated entity statement)? Please provide specific information for each beneficiary/affiliated entity. 1.2. Is the above agreement between the signatories to continue? If so, how? If not, why? 1.3. How would you assess the relationship between your organization and State authorities in the Activity counties? How has this relationship affected the Activity? 1.4. Where applicable, describe your relationship with any other organizations involved in implementing the Activity: ● Associate(s) (if any) ● Contractor(s) (if any) ● Final beneficiaries and target groups ● Other third parties involved (including other donors, other government agencies or local government units, NGOs, etc.) 1.1. Where applicable, outline any links and synergies you have developed with other programs, projects, activities or interventions. 1.5. If your organization targeted areas and value chains that received previous USAID grants in view of strengthening the same target group, in how far has this Activity been able to build upon/complement the previous/ongoing one(s)? (List all previous/ongoing relevant USAID programs). 1.6. How do you evaluate cooperation with the services of the Contracting Authority? 1. Visibility How is the visibility of the USAID contribution being ensured in the Activity? Name of the contact person for the Activity: …………………………………………… Signature: ………………………………Location: …………………………………… Date report sent: ……………………………… 156 Annex 12. Focus Group Discussion Tool Performance Evaluation of the FTF Accelerated Value Chain Development Program 1. Authority This evaluation is conducted under the Statistics Act 2006. The information provided in this interview will be treated in strict confidence and used for research and informational purposes only. Please be assured that none of your responses to the questions will be shared publicly and that any information you provide will be used exclusively for the purpose of the evaluation. 2. Objective The USAID/Kenya and East Africa Mission has recently commissioned a final performance evaluation of the Kenya Feed the Future Accelerated Value Chain Development (FTF-AVCD) Program, a five-year activity implemented by a consortium of Consultative Group on International Agricultural Research (CGIAR) Centers led by the International Livestock Research Institute (ILRI). The other members of the consortium include the International Crops for Research Institute for Semi-Arid Tropics (ICRISAT) and the International Potato Center (CIP). The purpose of the evaluation is to assess the design and implementation of the AVCD program to inform the design of future activities which harness agricultural technologies and innovations within the CGIAR and other research organizations. It will also provide lessons on the innovative approach of partnering with the CGIAR centers to implement development interventions and how partnership with the national and county governments, and the private sector can contribute to Kenya’s development. The evaluation is conducted by LINC LLC, in partnership with ICF Macro. I would like to introduce to you the team which will be conducting the evaluation and request your assistance in responding freely to their inquiries on the subject. To ensure the success of the evaluation, they will need to conduct brief focus group discussions with the project staff and amongst community groups that have interacted with AVCD. We respectfully request your assistance in facilitating the successful scheduling of meetings. 3. Instructions The following notes are useful in completing the FGD. Read carefully and understand before completing. More specific notes are provided as filling instructions to the relevant sections for your reference. i). Who to complete this form Evaluation team moderators. ii). Period covered The information required is for the years 2015 to 2021, where applicable. iii). Completion and submission of the Form The respondent is encouraged to generate and retain a copy of the completed interview. All sections should be completed. Any parts deemed inapplicable should be indicated as "not applicable". 4. Secondary Sources Some of the information required for this enquiry can be obtained from published material – reports or data. Kindly attach or link copies of any relevant supporting documents. 5. Other Comments In case you have any comments or information relating to the data provided, please insert them in the comment box found on page 5. For any further inquiries regarding this evaluation, please contact Patrick Lohmeyer (plohmeyer@linclocal.org). We value your cooperation in this activity and thank you for your support. Sincerely, Hezekiah Agwara Team Leader, AVCD Performance Evaluation 157 • Available resources (Assets or capital within the community participating in the AVCD projects, namely: Human capital, Social capital, Natural capital, Physical capital and Financial capital.) I. Human Capital II. Social Capital III. Natural Resources IV. Physical Capital V. Financial Capital • Vulnerability context of the community (Factors affecting stability of community food systems as well as undermining agricultural productivity). • Who is vulnerable (categorize by individuals/households; seasonality of vulnerability)? • Adaptation and mitigation measures the community has adopted to cope with unstable or insecure food systems and other vulnerabilities. (Probe which of these are a result of the AVCD project intervention.) • Improved agricultural and nutrition practices adopted by the community and the general perception as compared with traditional practices. (Probe which of these are a result of the AVCD project interventions.) • Changes in behavior in community nutrition practices and daily diets. (Probe which of these are a result of the AVCD project intervention.) • Change in engagement of women and men of different ages in economic activities (Probe which of these are a result of the AVCD project intervention.) • Notable shortcomings, gaps, challenges or weakness in the AVCD intervention and implementation approaches and suggestions for improvement I. Shortcomings / Gaps II. Improvements • External influences on the interventions that enhanced or limited their potential impact I. Enhancers II. Limiting factors 158 • Positive and negative effects of the interventions undertaken by the AVCD program I. Positive effects II. Negative effects • How best should similar project (s) address the named challenges, limitations or negative effects in future? (Probe for what similar projects should start/keep/stop doing and adjustments or improvements to be made.) • In what ways can agriculture benefit from lessons learned from the AVCD project? • Please give any other views concerning the agriculture development and nutrition for this area? • 159 FGD WITH AVCD FIELD STAFF Role of group ………………………………………... Location …………………………… Characteristics of group: ……... Age distribution of group: …………. Gender distribution of group: M ………...; F…………; Under 35 (M……...; F………) Number of discussants lasting to the end of FGD …. Counties and Value chains…………………………………………………………………… • Available resources (Assets or capital within the community participating in the AVCD projects, namely: Human capital, Social capital, Natural capital, Physical capital and Financial capital.) I. Human Capital II. Social Capital III. Natural Resources IV. Physical Capital V. Financial Capital • Vulnerability context of the community (Factors affecting stability of community food systems as well as undermining agricultural productivity). • Who among the beneficiaries is vulnerable (categorize by individuals/households; seasonality of vulnerability i.e. if vulnerability is constant all year or if it is seasonal)? • Adaptation and mitigation measures the community has adopted to cope with unstable or insecure food systems and other vulnerabilities. (Probe which of these are a result of the AVCD project intervention.) • Improved agricultural and nutrition practices adopted by the community and the general perception as compared with traditional practices. (Probe which of these are a result of the AVCD project interventions.) • Changes in behavior in community nutrition practices and daily diets. (Probe which of these are a result of the AVCD project intervention.) • Change in engagement of women and men of different ages in economic activities (Probe which of these are a result of the AVCD project intervention.) • Notable shortcomings, gaps, challenges or weakness in the AVCD intervention and implementation approaches and suggestions for improvement I. Shortcomings / Gaps 160 II. Improvements • External influences on the interventions that enhanced or limited their potential impact I. Enhancers II. Limiting factors • Positive and negative effects of the interventions undertaken by the AVCD program I. Positive effects II. Negative effects • How best should similar project (s) address the named challenges, limitations or negative effects in future? (Probe for what similar projects should start/keep/stop doing and adjustments or improvements to be made.) • In what ways can agriculture benefit from lessons learned from the AVCD project? • Please give any other views concerning the agriculture development and nutrition for this area? 161 FGD WITH COMMUNITY AND VALUE CHAIN BENEFICIARIES Name of group ………………………... Location of group (village, [sub] county, County) ……………………... Characteristics of FGD group: ……... Age distribution of group: …………. Gender distribution of group: M ………...; F…………; Under 35 (M……...; F………) Number of discussants lasting till end of FGD………. Value chain (s) …………………………………………………………………………………… Proportion of discussants that were direct or indirect beneficiaries of the AVCD project interventions (proportion piling): Direct beneficiaries (………%.); Indirect beneficiaries (………%.); non beneficiaries (…………%) • Available resources (Assets or capital within the community participating in the AVCD projects, namely: Human capital, Social capital, Natural capital, Physical capital and Financial capital.) Please indicate the availability of the following assets in your community and to your gender if different from the rest of the community. I. Human Capital II. Social Capital III. Natural Resources IV. Physical Capital V. Financial Capital • Vulnerability context of the community (Factors affecting stability of community food systems as well as undermining agricultural productivity). • Who among you is vulnerable (categorize by individuals/households; seasonality of vulnerability i.e. if vulnerability is constant all year or if it is seasonal)? I. Proportion of respondents that are continuously vulnerable (…………%) II. Proportion of respondents that are periodically vulnerable (…………%) • Explain what causes the periodic vulnerability _________________________________________________________________________________ • Adaptation and mitigation measures the community has adopted to cope with unstable or insecure food systems and other vulnerabilities. (Probe which of these are a result of the AVCD project intervention.) I. Adaptation strategies used in this community: 162 II. Mitigation strategies used by community: • Improved agricultural and nutrition practices adopted by the community and the general perception as compared with traditional practices. (Probe which of these are a result of the AVCD project interventions.) • Changes in behavior in community nutrition practices and daily diets. (Probe which of these are a result of the AVCD project intervention.) • Change in engagement of women and men of different ages in economic activities (Probe which of these are a result of the AVCD project intervention.) • Notable shortcomings, gaps, challenges or weakness in the AVCD intervention and implementation approaches and suggestions for improvement I. Shortcomings / Gaps II. Improvements • External influences on the interventions that enhanced or limited their potential impact I. Enhancers II. Limiting factors • Positive and negative effects of the interventions undertaken by the AVCD program I. Positive effects II. Negative effects • How best should similar project (s) address the named challenges, limitations or negative effects in future? (Probe for what similar projects should start/keep/stop doing and adjustments or improvements to be made.) • In what ways can agriculture benefit from lessons learned from the AVCD project? • Please give any other views concerning the agriculture development and nutrition for this area? 163 FGD WITH MOTHERS OF CHILDREN UNDER 2 YEARS ON NUTRITION BCC Name of group ………………………………………... Location …………………………… Purpose of group: ……. Age distribution of group: …………. Gender distribution of group: M ………...; F…………; Under 35 (M……...; F………) Number of discussants starting the FGD …. Number of discussants lasting to the end of the FGD …. County / Value Chain…………………………………………………………………………………… 1. How many among you hail from households that are part of the BCC for nutrition intervention? …………… out of ……... 2. Who mainly receives these messages among HH members? Proportion piling. Mother of child (……%); Father (……%); grandmother (………%); grandfather (……%); Aunt ( ……%); Uncle ( ……%); Other (specify) ( ……%). 3. What messages? Give examples of messages you received. 4. How were the messages delivered to you? (e.g. at a baraza or scheduled meeting, paper handouts, announcements in public places like markets, video clips etc. List up to 4) 5. How were the messages given before the introduction of the AVCD project? (Probe for effectiveness.) ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… 6. Were the messages given under the project new to you and your relatives/ family or had they been given before? Circle the right New / Not New a. If New, what of the aspects did you find to be new? …………………………………………………………………………………………………… …………………………………………………………………………………………………… b. If not new, what had you and your family learnt before the project? …………………………………………………………………………………………………… …………………………………………………………………………………………………… c. Who told you? ……………………………………………………………………………………….. 7. Was there a difference in the way nutrition BCC was provided for older men and women and male and female youth? Yes or No a. If yes, did the difference in provision change with inception of the project? Yes or No. If Yes, what changed? ………………………………………………………………………………………………………… …..……………………………………………………………………………………………………… b. Why did it change or not change? …………………………………………………………………………………………………… 164 8. Was BCC for nutrition service targeted? a. Who was targeted? …………………………………………. b. Why? ……………………………………………………………. c. Was targeting beneficial? Yes or No. If Yes, what were the three key benefits of targeting? 9. What shortcomings/challenges / weakness did you observe in the BCC for nutrition intervention? (Probe for the causes and what the project should continue/stop doing) a. At project initiation phase …………………………………………………………………………………………………… …………… b. At project implementation phase …………………………………………………………………………………………………… ………… 10. How did you solve the challenges/shortcomings named in 11? a. Solutions at initiation phase ………….……………………………………………………………………………………… ………… b. Solutions at implementation phase …………………………………………………………………………………………………… ………… 11. How best should similar project (s) address the named challenges in future? (Probe for what the project should start / keep / stop doing and adjustments and improvements to be made.) ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… …………………… 12. In what ways can the community benefit from the gains and lessons learned from the BCC for nutrition component of the AVCD project? (Probe for what categories of (people/households) in the community benefited from the BCC-nutrition intervention) ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… Please give any other views concerning BCC-nutrition situation and interventions for this area …………………………………………………………………………….…………………………… …………………………………………………………………………………………