February 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Molly Hageboeck and Felix M’mboyi of Management Systems International (MSI) Inc. under the Kenya Program Support Project. Evaluation of the USAID-KARI Partnership for Increased Rural Household Incomes (2004-2013) ii EVALUATION OF THE USAID-KARI PARTNERSHIP FOR INCREASED RURAL HOUSEHOLD INCOMES (2004-2013) March 1, 2013 Contracted under No. AID-623-I-12-00001 Kenya Program Support Services DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS ACRONYMS ....................................................................................................................................................................... v GLOSSARY ........................................................................................................................................................................ vi EXECUTIVE SUMMARY ................................................................................................................................................ vii EVALUATION PURPOSE, QUESTIONS AND METHODS .................................................................................. 1 Evaluation Purpose ........................................................................................................................................................ 1 Evaluation Questions .................................................................................................................................................... 1 Evaluation Methods ....................................................................................................................................................... 1 Study Limitations ........................................................................................................................................................... 4 PARTNERSHIP PROGRAM BACKGROUND ........................................................................................................... 5 Overview of the USAID-KARI Partnership ............................................................................................................ 5 Funding for the USAID-KARI Partnership (2004-2012) ...................................................................................... 5 FINDINGS ........................................................................................................................................................................... 6 Question 1.a ................................................................................................................................................................... 6 Capacity Building Investments and Immediate Results of KARI Research ...................................................... 7 Down-stream Outcomes and Impacts of KARI Research ...................................................................... 8 Question 1.b: ................................................................................................................................................................ 17 Overview of M&E at KARI ...................................................................................................................17 The USAID-KARI Performance Monitoring Plan (2009-2012) .............................................................17 Tracking USAID-KARI Downs Stream Outcomes and Impacts ............................................................18 Question 2 .................................................................................................................................................................... 20 Maize Program Component ................................................................................................................20 Food Security Program Component ....................................................................................................22 Dairy Component ................................................................................................................................23 Question 3 .................................................................................................................................................................... 24 Administrative/Financial Management Challenges............................................................................26 Personnel System Challenges .............................................................................................................26 CONCLUSIONS AND RECOMMENDATIONS....................................................................................................27 Conclusions .................................................................................................................................................................. 27 Recommendations for KARI ..................................................................................................................................... 27 Recommendations for USAID.................................................................................................................................. 29 iv ANNEXES .........................................................................................................................................................................30 ANNEX 1: Evaluation statement of work ........................................................................................................... 30 ANNEX 2: Documents reviewed .......................................................................................................................... 36 ANNEX 3: People Interviewed .............................................................................................................................. 38 ANNEX 4: Summary of seed company interview results ................................................................................ 41 ANNEX 5: Agro-Survey – Methodology .............................................................................................................. 42 ANNEX 6: Agro-Survey – Descriptive Statistics ............................................................................................... 46 ANNEX 6: Participatory Stakeholder Meeting Summary & Participants ...................................................... 52 v ACRONYMS ADSP Agricultural Development Support Project ASCU Agriculture Sector Coordination Office CGA Cereal Growers Association CIMMYT International Maize and Wheat Improvement Center ECF East Coast Fever FAO Food and Agriculture Organization FIPS Farming Inputs Promotion Services FCRP Food Crops Research Project FTC Farmers Training Centers FtF Feed the Future GLS Grey Leaf Spot GOK Government of Kenya GNCP Good Neighbors Community Project ILRI International Livestock Research Institute ITM Infect-and-Treat Delivery Mechanism KARI Kenya Agriculture Research Institute KARO Kenya Agriculture Research Organization KAVES Kenya Agricultural Value Chain Enterprises KBS Kenya Bureau of Standards KDDP Kenya Dairy Development Program KEPHIS Kenya Plant Health Inspectorate Service KHCP Kenya Horticulture Competitiveness Project KMDP Kenya Maize development Program KSC Kenya Seed Company KSU KARI Seed Unit MoA Ministry of Agriculture MSI Management Systems International MSV Maize Streak Virus NARP National Agricultural Research Project NEPAD New Partnership for Africa's Development NGO Non-Governmental Organization OPVs Open Pollinated Varieties PMP Performance Management Plan SO7 Strategic Objective Seven SOW Statement of Work USAID United States Agency for International Development vi GLOSSARY Open Pollinated Seed – is where pollination occurs by insects, birds, wind, or other natural mechanisms. The quality of the seed is less controlled, often quality-contaminated, lower yielding and more disease and drought susceptible. For Open Pollinated Varieties it is recommended farmers buy seed every three seasons. Hybrid Seed – In hybrid seed production, the crosses are specific and more controlled, producing a higher quality seed (often giving higher yields and being more disease and drought resistant). To produce hybrid seed, elite inbred varieties are crossed with well-documented and consistent phenotypes and the resulting hybrid seed is collected. Hybrid seeds cannot easily be produced by small-scale farmers and therefore if hybrid seeds are to be used, they have to be bought each time by farmers. Seed Production System Development – The production of a new plant variety undergoes a number of stages. The first stage involves biological or genetic plant breeding or engineering and produces the desired plant attributes. In an economy where technical-capacity is well developed, this stage is usually conducted by the private sector; where technical-capacity in the seed industry is not yet developed these tasks are often carried out by public-sector agricultural research and development institutions. Commercialized Technologies – a private company is only likely to want to invest in a technology (investing in manufacturing a salable commodity and manufacturing systems, developing distribution and retailing systems and management systems, etc.) if secure-ownership of rights to the technology can occur. Legalized ownership of the technology – intellectual property rights that can be legally and cost￾effectively enforced. Practical property-ownership – e.g. hybrid-seed-technology cannot easily be copied due to the bio￾technology and nuclear seed not being further disseminated and held only by this company; whereas open pollinated varieties are less easy to hold the property rights over and be competitive. Therefore a technology is only likely to become ‘commercial’ if money can be made by the development, sale and secure ownership of the technology. Few Open Pollinated varieties are sold by commercial firms. Transgenic – relating to an organism whose genome has been altered by the transfer of a gene or genes from another species or breed. Un-Commercialized Technologies – some technologies a public research institute produces can sometimes be justified by a public demand due the fact the technology does not have a commercial opportunity. vii EXECUTIVE SUMMARY EVALUATION PURPOSE USAID’s Statement of Work (SOW) for this final performance evaluation of its current partnership with the Kenya Agricultural Research Institute (KARI), under Grant Agreement No. 615-007, identified its primary purpose as learning. In discussions, the staff of KARI expressed a similar view, in light of KARI’s anticipated absorption, in 2013, into the Kenya Agricultural Research Organization (KARO), a newly created entity under Government of Kenya’s 2012 Agriculture, Livestock, Fisheries and Food Authority Bill. EVALUATION QUESTIONS In its Statement of Work (SOW) for this evaluation, USAID identified three main questions and 17 sub￾questions for the evaluation team to address. With USAID concurrence, MSI concentrated on the main questions for the evaluation, as listed below, working in information on sub-questions wherever possible. 1. How has the project performed in terms of achieving projected results and impact, and how effective is the M&E system in measuring them? 2. What are the adoption rates of the agricultural technologies proposed to farmer beneficiaries, and is there evidence that there is diffusion of those technologies in the target areas and scaling up and market replication? 3. What factors beyond the control of KARI played a significant role in affecting, positively or negatively, the underlying challenges the project sought to address, and what was done to adjust the project design to those factors? EVALUATION METHODS MSI’s proposed methodology for this evaluation was presented to USAID/Kenya in an inception report submitted by the evaluation team in December 2012 and approved by USAID. The methodology MSI submitted at that time reflected its understanding of the evaluation questions and USAID’s SOW expectations concerning the implementation of a “survey of 200 agro-dealers and agro-vets, in targeted areas, to “market test” the degree of diffusion and scale-up of three (3) KARI innovations from among the six components of the project”. Consistent with its proposed methodology, MSI’s evaluation team gathered data using five methods: (a) document review, (b) an agro-dealer survey, (c) key informant/stakeholder interviews, (d) partner site visits at KARI locations, and (e) a stakeholder collaborative group meeting. MSI analyzed data collected through these methods using a simple conceptual framework for organizing information from the evaluation about KARI research results and down-stream outcomes and impact that is consistent with frameworks for analyzing agricultural research processes and outcomes. viii STUDY LIMITATIONS Several factors impeded the work of the evaluation and required a two-week delay in submission of the draft, including difficulties encountered in collecting key project documents; lack of information on adoption related in part to survey focus on agro-dealers but not farmers; and some confusion, which was subsequently resolved, about KARI technologies to be included in case studies under Question 2. FINDINGS Under this partnership program, KARI has selected serious agricultural, livestock and natural resource management issues facing Kenya on which to focus its research attention. In all of the main program areas, KARI scientists have produced research products ranging from vaccines, to seeds and other plant materials that have clear advantages over predecessor solutions. Certification is an important step on the path towards adoption over which KARI has little control. KARI’s maize, nutribusiness and dairy components show that these processes can be expensive and sometimes so slow that market windows of opportunity may have been missed. Across the board, evidence from the evaluation shows that KARI research products are not consistently being taken up on the scale needed to significantly reduce the incidence of diseases or create the resistances for which they were designed. Farmer habits as well as their education levels and awareness of and access to new and better agricultural technologies may play an important role in adoption. What farmers like and feel comfortable with (maize to eat and older/known seed and fertilizer varieties) – may be a distinct “factor” in the adoption equation even if, as a general rule, commensurate with their understanding and resources, farmers make economically sound choices. CONCLUSIONS Many of the problems that impede the up-take of KARI research solutions lie downstream in distribution channels and farmer awareness. It is not, however, in KARI’s interest to simply observe these impediments. KARI knows this and is actively involved in a range of efforts aimed at expanding knowledge about research products and their effectiveness. KARI and partner organizations are also working to develop pathways for the commercial production and distribution of as large a portion of its products as possible, compensating with other approaches only where no commercial mechanism is found. What is missing from this picture, the evaluation revealed, is enough evidence to sort out what will spark action along the production distribution chain and ignite farm level uptake, and how to trigger the adoption of better farming products and practices faster and on a much larger scale. KARI has an immense talent for thinking big and finding solutions. It is time to turn that spotlight on what it takes to stimulate demand and use for those solutions on Kenyan farms. Accordingly the evaluation team’s recommendations for KARI focus almost entirely on the biggest challenge it faces: breaking through impediments to utilization, using the same set of skills it uses to break through scientific barriers – hypothesis generation, testing, and following the evidence that emerges. ix RECOMMENDATIONS Recommendations for KARI  Discuss progress toward performance indicator targets in quarterly and annual reports, one product at a time.  Add performance indicators in order to more regularly and reliably understand the production, distribution and adoption of research results.  Develop and implement business-oriented research product distribution plans, with clear targets and milestones, for one product at a time.  Use the period before KARI joins KARO, or the period just after, to internally review KARI's strengthens, weaknesses, and opportunities to improve the extent to which KARI research solutions that could solve important problems in Kenya, actually do so.  Identify and test more “retail” mechanisms for delivering evidence-based information about KARI products to farmers.  Organize headquarters based “field days” that include “listening sessions” with commercial firms and others who work downstream from KARI to build their “ownership” of next generations of KARI research products.  Modify commercialization approaches that are not generating enough product to plausibly change the status of the problem the product was designed to solve.  Identify and test various approaches that mix commercial and non-commercial disseminator approaches, such as the “One Acre” program that subsidized seed sales for an in-kind return. Recommendations for USAID  Encourage KARI to adopt a more robust, learning-oriented performance monitoring system for the new partnership arrangement – one that includes measures that provide insights into or predict the likelihood that KARI research results will be adopted at the farm level.  Help KARI develop a clear vision and plan for enhancing the adoption focus and effectiveness of downstream organizations – without trying to become those organizations.  Foster efforts among its other USAID agriculture sector partners to promote and apply KARI agricultural research solutions, in unison, by multiple project teams – to build a critical mass of adopters.  Work with KARI to find solutions to administrative and financial matters affecting the partnership and KARI’s work. 1 EVALUATION PURPOSE, QUESTIONS AND METHODS EVALUATION PURPOSE USAID’s Statement of Work (SOW) for this final performance evaluation of its current partnership with the Kenya Agricultural Research Institute (KARI), under Grant Agreement No. 615-007, identified its primary purpose as learning. In discussions, the staff of KARI expressed a similar view, in light of KARI’s anticipated absorption, in 2013, into the Kenya Agricultural Research Organization (KARO), a newly created entity under Government of Kenya’s 2012 Agriculture, Livestock, Fisheries and Food Authority Bill. The evaluation was designed to assist as a timely vehicle for helping the Mission and KARI: 1. Examine the extent to which the project’s objectives and goals – at all results levels – have been achieved; 2. Understand if research results, training, and outreach have yielded sustained impact in the use of new technologies and on agricultural yields; and 3. Capture lessons that can be applied to current and future agricultural research and technology. EVALUATION QUESTIONS In its Statement of Work (SOW) for this evaluation, USAID identified three main questions and 17 sub￾questions for the evaluation team to address. The Mission’s evaluation questions are provided in the text box below as well as in the full Evaluation SOW in Annex 1. Based on discussions with the Mission and with written approval from USAID’s Contracting Officer, the evaluation team focused the evaluation around the three main questions working in, where feasible, information that addresses as many of the sub-questions under each main evaluation question as possible. In working with the Mission’s questions, the evaluation team found that their work proceeded more smoothly when they split each of the three main questions into their two distinct parts. In some places in the report main questions are separated into parts in this manner. EVALUATION METHODS MSI’s proposed methodology for this evaluation was presented to USAID/Kenya in an inception report submitted by the evaluation team in December 2012 and approved by USAID. The methodology MSI submitted at that time reflected its understanding of the evaluation questions and USAID’s SOW expectations concerning the implementation of a “survey of 200 agro-dealers and agro-vets, in targeted areas, to “market test” the degree of diffusion and scale-up of three (3) KARI innovations from among the six components of the project”. Consistent with its proposed methodology, MSI’s evaluation team gathered data using five methods: (a) document review, (b) an agro-dealer survey, (c) key informant/stakeholder interviews, (d) partner site visits at KARI locations, and (e) a stakeholder collaborative group meeting. Each of these methods is described more fully in a text box in this section. MSI data collection instruments associated with these methods are included in annexes to this report. 2 USAID’s SOW: Key Evaluation Questions PROJECT IMPACT: How has the project performed in terms of achieving projected results and impact, and how effective is the M&E system in measuring them?  How well did the project achieve the intended results as defined within the project’s Performance Management Plan?  What additional evidence, qualitative or quantitative, exists that link project activities to improved agricultural output, increased uptake of improved agricultural technologies and practices, or reduced incidence of plant and animal diseases or parasites?  What evidence exists to show that KARI research or activities have increased food security or economic security among beneficiaries or in the broader community? IMPACT SUSTAINABILITY AND SCALE-UP: What are the adoption rates of the agricultural technologies proposed to farmer beneficiaries, and is there evidence that there is diffusion of those technologies in the target areas and scaling up and market replication?  To what extent have new technologies developed or promoted by KARI demonstrated a strong cost-benefit for farmers, with the initial cost of investment in new technologies offset by increased production or lower costs in other areas?  What are the adoption rates of the agricultural technologies proposed to farmer beneficiaries?  How well/how much have beneficiaries/farmers continued to use new technologies developed or promoted by KARI?  What barriers exist for continued use or dissemination of improved technologies and practices developed or promoted by KARI?  What relationships, with agro-distributors, agricultural service providers, relevant government ministries and district offices, community organizations, financial institutions, and/or NGOs, has KARI established that will support the continued use of improved technologies?  Were the diffusion rates of the proposed agricultural technologies sustained over time among targeted zones?  Did household food security increase in the targeted zones as a result of the project’s activities? PROJECT DESIGN AND LESSONS LEARNED. What factors beyond the control of KARI played a significant role in affecting, positively or negatively, the underlying challenges the project sought to address, and what was done to adjust the project design to those factors?  What unintended/unexpected outcomes, positive or negative, resulted from project activities?  How have socio-cultural factors, such as culture, economic status, gender, ethnicity, geography, contributed to or undermined the adoption of improved technologies? o How has this information changed KARI’s approach to its research? o Is this information relevant to other agriculture or economic growth projects? How can it be shared  What factors beyond the control of KARI played a significant role in affecting, positively or negatively, the underlying challenges the project sought to address?  What changes were made or might have been made to the project design to increase its impact?  What could the project have done to increase ownership of project goals and activities among stakeholders, such as individuals, communities, government, business service providers, etc?  How well did funding levels, and staff competency and capacity, facilitate achievement of program objectives? 3 Summary of Evaluation Data Collection Methods DOCUMENT REVIEW The evaluation team undertook a literature review of various project documents including baseline surveys, work plans and budgets for the various components, quarterly and annual project reports. Literature reviews were also extended to information sources from stakeholder reports. A number of other project￾external reports were also consulted such as project documents of previous projects supporting KARI and reports looking at agricultural research in other developing economies. The historical context explaining how KARI has evolved as an institution has contributed to a deeper understanding of the issues. TECHNOLOGY COMMERCIAL-DISSEMINATION SURVEY (AGRO-DEALER SURVEY) An Agro-Dealer/Agro-Vet survey was also conducted for 229 outlets. The survey sought to collect information on types of commodities stocked and the awareness of the KARI initiated seed technologies and fertilizers. This survey has highlighted aspects of technology commercial dissemination. KEY INFORMANT / STAKEHOLDER INTERVIEWS The study employed a key informant interview tool to collate and collect information from key stakeholders in the agriculture sector and seed industry in the country. In addition, several KARI research centers engaged in research activities funded in this project were also visited and a discussion held with relevant scientists who participated in the project activities. Over 19 institution interviews were conducted, plus KARI and KARI project component personal interviews and long-term and short-term training recipients. These interviews have illustrated issues surrounding KARI and USAID funding to the six research components. PARTNER (KARI) SITE VISITS Site visits were conducted at various KARI research centers by the evaluation team including KARI Biotech Centre in Nairobi, KARI Thika in central Kenya, KARI Katumani in Machakos, KARI Kitale and KARI Kakamega in western Kenya, and KARI Kibos in Nyanza/lake region. The physical visits were meant to familiarize the evaluators with the research activities on the ground. Similar visits were also made to seed companies including Kenya Seed, Western Seed, East Africa Seed and Dryland Seed companies. The visits were meant to familiarize evaluators with the actual working environment of these companies. Also visited were KARI’s other partners including the Red Cross, FIPS Africa, CGA, WVI, KMDP, USAID-KHCP, STAK, ASCU, ILRI, MOA, MOLD, and KEPHIS. These visits have contributed to understanding the nature of institutions receiving, demanding and commercializing technologies produced by KARI. STAKEHOLDER COLLABORATIVE GROUP MEETING A stakeholder meeting with participants from KARI, the private sector and industry-support-agencies discussed what and how the project can better work to develop the industry. This forum allowed KARI personnel and industry stakeholders to discuss and analyze in greater depth how the industry better develop and how KARI can develop as an institution to support this. It was held on the 15th January 2013 and a total of 15 persons attended of which 53% came from KARI, 27% from other government organizations, 13% from USAID projects and 7% from the private sector. 4 MSI analyzed data collected through these methods using a simple conceptual framework for organizing information from the evaluation about KARI research results and down-stream outcomes and impact as shown below. This model is consistent with frameworks for analyzing agricultural research processes and outcomes.1 Figure 1. Conceptual Framework for organizing information Under this model, KARI scientists, and other staff in the field and headquarters, produce the results in the first, or left hand box on this diagram. On the far right, farmers make decisions to try products and other technologies and to persist in using them if they find doing so will provide value relative to their cost. In the middle box are a wide variety of actors who play critical roles in transferring KARI research findings and products to farmers. The final box in the diagram represents adoption and use of KARI research results by Kenyan farmers. In the evaluation, a case study approach was used to examine three KARI technologies consistent with expectations set forth in USAID’s SOW with respect to Question 2. STUDY LIMITATIONS Several factors impeded the work of the evaluation and required a two-week delay in submission of the draft, such as the following:  Despite efforts by USAID and KARI to assemble relevant documents prior to the start of the evaluation in early December, 2012, the team found that it needed to ask for additional foundation documents, including KARI proposals for this effort, which it did not receive until late in January 2013.  In its evaluation SOW, USAID expressed its interest in understanding the adoption of KARI￾based products by farmers, yet the survey it asked the evaluation team to implement explicitly identified agro-dealers and agro-vets as the preferred respondents.  A few of the sub-questions included in the Evaluation SOW turned out to be not sub-questions, but rather separate and distinct inquiries which the evaluation team had neither the LOE nor other resources to undertake. One of these sought a farmer cost-benefit analysis in relation to KARI technologies and the other sought broad evidence concerning food security results.  Some confusion arose with respect to a SOW instruction calling for KARI to identify three technologies for the evaluation team to examine through case studies and agro-dealer survey results, resulting in one choice not being viewed as an appropriate example of KARI-specific innovations. Adjustments were made to the set of cases examined. 1 W. Peterson, G. Gijsbers, and M. Wilks, An Organizational Performance Assessment System for Agricultural Research Organizations: Concepts, Methods, and Procedures, ISNAR Research Management Guidelines No. 7. The Hague: International Service for National Agricultural Research,2003. available at: ftp://ftp.solutionexchange.net.in/public/food/resource/res03011201.pdf 5 PARTNERSHIP PROGRAM BACKGROUND OVERVIEW OF THE USAID-KARI PARTNERSHIP USAID’s support to Kenya agricultural research began in 1964, primarily in the development of cereals. 2 USAID sponsorship of the Food Crops Research Project (FCRP-615-229) started in 1967. This gave birth to most of Kenya’s maize hybrids. The next phases of support created a new National Agriculture Research Project (NARP), which evolved after the formation of KARI in 1979. Phase I began in 1987 and Phase II ran from 1992 to 1998. The National Agribusiness Development Support Project (NARP III), a follow on to Phase II, was a continuation of past USAID investments in agricultural research. The KARI component of Agriculture Development Support Project (ADSP) aimed to increase participation and efficiency of the private sector in supplying agricultural inputs to smallholders and providing output market services. While progress was made under these programs, the Food and Agriculture Organization (FAO), writing shortly before KARI’s 2003 proposal for new partnership investments in agricultural research in 2002, under the New Partnership for Africa’s Development (NEPAD), characterized its findings about the situation at the farm level, in Kenya and beyond, in alarming terms: “Significantly, yields of most important food grains, tubers and legumes (maize, millet, sorghum, yams, cassava, groundnuts) in most African countries are no higher today than in 1980. Cereal yields average.” The current USAID-KARI partnership, which built on past collaborations, was initiated in a 2003 proposal to USAID that focused on biotechnology and followed up, in 2004, with a second proposal that broadened the scope to include maize, dairy, soil fertility and horticulture. In 2007, a nutribusiness component was added and in 2009 a food security dimension was incorporated into the scope of KARI’s work with USAID under USAID Strategic Objective Seven (SO7) which focused, at the highest level, on “increased rural incomes”. FUNDING FOR THE USAID-KARI PARTNERSHIP (2004-2012) Funding for the USAID – KARI “improved rural incomes” partnership under SO 7, which is the focus of this evaluation, averaged around $415,000 per year for the past eight years, for a total, according to documents shared with the evaluation team, of $3,328,166. Table 1 below provides a breakdown by component. Table 1. USAID Funding for KARI’s Program in Support of SO 7 (2004-2012) Program Component 2004 / 2005 2005 / 2006 2006 / 2007 2007 / 2008 2008 / 2009 2009/ 2010 2010 / 2011 2011 / 2012 Total % of Total Dairy Development 34,688 62,562 17,300 26,483 18,021 15,251 79,500 39,829 293,6 34 8.82 Maize Development 51,331 60,802 10,000 48,410 8,140 56,402 50,265 48,880 334,2 30 10.04 Soil Fertility 32,929 16,122 4,999 6,709 19,715 20,238 32,179 21,584 154,4 75 4.64 2 Kenya Agricultural Research Institute (March 2012) “Integrated Agricultural Research and Development: Enhancing Food and Nutrition Security Through Increased Production and Agribusiness”. Presented to USAID. 6 Program Component 2004 / 2005 2005 / 2006 2006 / 2007 2007 / 2008 2008 / 2009 2009/ 2010 2010 / 2011 2011 / 2012 Total % of Total Food Security 0 0 0 0 0 26,717 52,206 31,989 110,9 12 3.33 Nutribusiness 29,100 5,900 0 9,984 5,584 2,825 28,559 23,531 105,4 83 3.17 Biotechnology 207,651 179,227 107,822 151,933 239,996 226,238 103,670 30,558 1,247, 095 37.47 Horticulture 27,392 0 0 0 0 0 0 0 27,39 2 0.82 Evaluation & Audit, M&E, Program Management 155,586 73,759 186,464 1 62,453 194,687 90,288 80,100 111,608 1,054, 945 31.70 Totals 538,67 7 398,37 2 326,58 5 405,97 2 486,14 3 437,95 9 426,47 9 307,97 9 3,328, 166 100.0 0 Source: KARI data, MSI analysis Grants and donor-funded programs, averaging $12,121,708 per year, make up roughly 30% of KARI’s budget. USAID’s annual funding for KARI represents about 3.4 percent of KARI’s grant and donor programs funding annually, which is a significantly smaller fraction of KARI’s total annual budget. Table 2. KARI Total Financing 2006-2012 (US$)3 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 Average Percentage From GOK agriculture budget for recurrent expenses 17,125,000 20,625,000 21,875,000 22,575,000 24,800,000 25,025,000 53% From GOK agriculture budget for project funding 3,587,444 3,100,407 3,131,425 2,999,375 3,456,500 3,704,805 8% KARI Incomes from sales, royalties etc. 7,307,038 5,807,613 2,703,950 3,463,650 3,434,525 4,730,913 10% Grants, Development Partners 15,035,663 16,221,700 5,448,725 8,102,625 13,515,025 14,404,731 29% Source: KARI data, MSI analysis FINDINGS MSI’s findings from this evaluation are organized by evaluation question. Question 1 is split into two parts, with achievements treated first and KARI monitoring and evaluation of the program addressed second. QUESTION 1.A HOW HAS THE PROJECT PERFORMED IN TERMS OF ACHIEVING PROJECTED RESULTS AND IMPACT? In addressing this question, MSI distinguishes between the immediate results produced by KARI’s 3 Accounts department KARI (January 2103) with evaluation team analysis and calculations. 7 research program, on which KARI’s performance monitoring plan for the partnership focuses, and down-stream outcomes and impact, which depend on the actions of other organizations and farmers as well as on KARI outreach efforts. Each of these is reported on separately below. CAPACITY BUILDING INVESTMENTS AND IMMEDIATE RESULTS OF KARI RESEARCH Under the current partnership with USAID, KARI monitors five performance indicators pursuant to a performance monitoring plan for the project agreed upon with USAID in 2009. Two of these performance indicators focus on capacity building through investments in long and short term training while the other three focus on the status of KARI’s research program, on a component by component basis. This section summarizes KARI’s performance on these indicators. a. Capacity Building Investments In 2003, at the end of a World Bank funded NARP, KARI reported that project training funds had allowed it to double its number of PhDs. Under the current partnership, begun in 2004, KARI has again invested in project training resources with long term capacity in mind. Two staff were trained to the PhD level and eight to the Master’s level, equally divided between men and women at each level. Using short term training funds, six men and one woman have completed a University of Nairobi microbiology course; five men and five women each completed biotechnology/biochemistry courses and two men and one women finished courses in scientific writing. b. Immediate Results of KARI Research Research under the USAID-KARI partnership is organized around program components, four of which focused on agricultural products: maize, dairy, other staple crops and horticulture. The remaining three elements of the program cut across product lines. These include soil fertility, biotechnology and nutribusiness, as shown in the matrix below. Table 3. Overlapping Aspects of the KARI Partnership Research Program Agricultural Product Focus Foundational Elements Transformative Food Security: Soil Biotechnology Nutribusiness Dairy Disease prevention Maize Liming Transgenic lines Food Security: Staple Crops Liming Transgenic lines Weaning foods Horticulture Liming Transgenic lines Source: KARI Data, MSI analysis KARI reports program results separately for maize, dairy and biotechnology components of its USAID partnership; other research lines combined for reporting purposes in a food security cluster. Two of KARI’s research program indicators have been used since 2009, namely one that measures the number of technologies “under research” and another than tracks those that are “ready for transfer” to the private sector and farmer beneficiaries. A final indicator which measures technologies that are “in research trials” was added in 2012. Performance on each one is shown below (target/performance). Table 4. Indicator: Technologies/management practices “under research” – exceeded target (+10) Component 2009 2010 2011 2012 Total Maize 5/6 6/6 19/19 55/55 85/86 Dairy 7/8 7/7 7/5 0/2 21/23 Biotechnology 3/0 5/5 5/5 5/6 18/16 Food Security -- 6/6 6/10 0/6 12/22 Total 15/14 24/24 37/39 60/69 136/146 8 Source: KARI Data, MSI analysis Table 5. Indicator: Technologies/management practices “in research trials” – recently added (even) Component 2009 2010 2011 2012 Total Maize 10/16 10/16 Dairy 5/0 5/0 Biotechnology 1/1 1/1 Food Security 0/0 0/0 Total 16/17 16/17 Source: KARI Data, MSI analysis Table 6. Indicator: Technologies/management practices “ready to transfer” – below target (-10) Component 2009 2010 2011 2012 Total Maize 6/6 9/3 6/13 3/3 24/25 Dairy 2/1 2/0 6/0 0/0 10/1 Biotechnology 2/0 1/0 3/0 0/1 6/1 Food Security 2/0 1/0 3/0 0/4 6/4 Total 12/7 13/3 18/13 3/8 41/31 Source: KARI Data, MSI analysis Down-stream Outcomes and Impacts of KARI Research While the primary results of the USAID￾KARI partnership investments in agricultural research are new agricultural technologies and farm management practices that address known problems, the true value of these investments is realized only when these innovations are adopted on a broad scale in areas for which they were designed. Moving from research outcomes to farm-level adoption along the model shown earlier requires the participation of additional actors, as illustrated in the figure on this page. In this section of the evaluation, MSI traces KARI research results forward to determine whether and to what degree they have been disseminated and adopted. Each component of the USAID-KARI partnership is broadly examined from this perspective. In addition, under Question 2, below, a more in-depth look is taken at adoption for specific research products in a set of three case studies. Before turning to each of these components individually, it is worth noting several findings that are cross-cutting in nature emerged from MSI’s component-by-component examination of the KARI partnership program. They are highlighted here as they may help readers to recognize the common threads when reading about specific components.  Research takes time. There is a degree of unpredictability that cuts across much of KARI’s Figure 2: Research outcomes to farm-level adoption model 9 work. For KARI staff, this is a given, and it explains why targets for what is “under research” or “in trials” are better aligned than targets and performance on its “ready to transfer” indicator.  Certification is an important step on the path towards adoption over which KARI has little control. KARI’s maize, nutribusiness and dairy components show that these processes can be expensive and sometimes so slow that market windows of opportunity may have been missed.  Expansion and Scaling Up are Different. Across several program components evidence attests to the adoption of KARI technologies in some places, but these data, for the most part, do not indicate that solutions are moving quickly to scale, i.e., catching on at the pace or at levels that will solve national problems, even when KARI technologies have the horsepower to do that, as discussed under Question 3.  Dissemination mechanisms vary by research product and extent/quality of coverage. This is particularly true for last mile linkages between both commercial and non-commercial distribution systems for transferring new technologies and farmers. One in four stockists carries solutions that KARI knows will work – but they may not have been delivered recently.  Farmer habits as well as their education levels and awareness of and access to new and better agricultural technologies may play an important role in adoption. What farmers like and feel comfortable with (maize to eat and older/known seed and fertilizer varieties) – may be a distinct “factor” in the adoption equation even if, as a general rule, commensurate with their understanding and resources, farmers make economically sound choices. a. Enhancing Food Security Component – Soil Fertility Work on soil fertility came into the USAID-KARI partnership through KARI’s 2004 proposal, along with maize, dairy and horticulture into the program. While reported as a separate component for most of the project period, KARI now considers soil fertility to be a key element of its broader food security emphasis. USAID funding for Soil Fertility under the KARI partnership program for 2004-2012 totaled $154,475, or 4.64 percent of the program total. This component did not have a performance target, but nevertheless reports on its research progress. Table 7. New Technologies or Management Practices as a result of USG assistance KARI Program Element: Enhancing Food Security – Soil Fertility Number of New Technologies or Management Practices Under Research as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Soil and Nutrients 1 1 Source: KARI Data, MSI analysis The primary focus of the soil fertility component has been on soil acidity as a limiting factor on farm production and on the application of lime to raise soil pH. Current research under the component includes approaches for improving soil fertility, including blending fertilizer use with the use of organic products such as manure, which cuts farmer costs as it benefits the soil. KARI research has linked low productivity on Kenyan farms to several practices, including the use of inorganic fertilizers that contain ammonium over long periods (Di-Ammonium Phosphate (DAP) and Calcium Ammonium Nitrate (CAN); continuous cropping; soil erosion on sloping lands, and the absence of water conservation practices. With USAID funding, KARI developed a pH baseline for the Western region and introduced practices that improve soil fertility including planting trees and grasses and adding 10 lime to the soil that farmers use for production. When KARI undertook research trials that involved delivering lime and fertilizer together, including through commercially available products, the results showed that Mavuno, an existing fertilizer that does not contain ammonium, outperformed other lime delivery systems KARI tested. It produced a mean grain yield (tons per hectare) of 3.64 compared to 2.6 with DAP alone, the most frequently utilized fertilizer, product sales showed improvement. 4 MSI’s survey of agro-dealers indicates that 33 percent of them stock Mavuno. b. Biotechnology Component Biotechnology is one of KARI’s newest programs. It was introduced as the first component under the current USAID-KARI partnership in a 2003 proposal. Due to a long gestation period, research in this area is just now beginning to generate work that is “in trials” or “ready to transfer”. Biotechnology, with a “life of project” budget of $1,247,095 represents 37.47% of USAID’s total investment in KARI research. KARI’s biotechnology program focuses on opportunities that lie beyond what can be achieved through conventional plant, animal and microbial research. It focuses on two broad research streams: Table 8. New Biotechnology Technology or Management Practices Under Research KARI Program Element: Biotechnology Number of New Technologies or Management Practices “Under Research” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Animal 2 2 2 2 2 2 Maize 1 2 2 2 2 2 CBSV 1 1 1 1 1 Source: KARI Data, MSI analysis Table 9. New Biotechnology Technology or Management Practices “Made available for Transfer” KARI Program Element: Biotechnology Number of New Technologies or Management Practices “Made Available for Transfer” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Animal 2 1 1 1 Maize 1 CBSV 1 Source: KARI Data, MSI analysis Transgenic agricultural products, KARI explains, have the potential for breaking through current limits, such as on maize yields, and for enhancing crop resistance to drought, diseases, and pests, nutritional content, with dramatic implications for farm output. KARI’s first transgenic maize and cassava varieties have recently moved into a research trial phase. While KARI is optimistic about the future of transgenic products, with both legislation and enabling regulations in place, staff recognize that some reservations among politicians and in the populace persist, albeit at a relatively modest level, not withstanding, negative campaigns by anti-biotech groups funded locally by foreign organizations. Vaccine and diagnostics kits for animals represent the second biotechnology research stream under this program component. KARI is using a “kit” approach to this field, meaning that it will package a vaccine or diagnostic test it develops, to ensure that veterinarians and other downstream providers of 4 David S. Mbakaya, J. R. Okalebo, M. Muricho and S. Lumasayi, “Effects Of Liming And Inorganic Fertilizers On Maize Yield In Kakamega North And Ugunja Districts, Western Kenya”, KARI. (http://www.kari.org/conference/conference12/docs/EFFECTS%20OF%20LIMING%20AND%20INORGANIC%20FERTILIZERS% 20ON%20MAIZE%20YIELD%20IN%20KAKAMEGA.pdf) 11 these products have everything they need for each delivery. KARI is currently working on five diagnostic kits; the product furthest along is a diagnostic kit for Contagious Caprine Plero-pneumonia (CCPP) for goats, which is awaiting trademark approval. A kit for Contagious Bovine Plero-pneumonia (CBPP) is also close to reaching this stage. While arrangements for commercial production of these kits have not yet been established, KARI anticipates that it will work with Kenya’s national veterinary diagnostic testing laboratories to demonstrate and disseminate these kits. c. Dairy Component The Dairy Component came into the USAID-KARI partnership through KARI’s 2004 proposal, which also brought in maize, soil fertility and horticulture. USAID funding for Dairy Component under the KARI partnership program for 2004-2012 totaled $293,634 or 8.82% percent of the program total. Table 10. Number of new Dairy Technologies or Management Practices under Research KARI Program Element: Dairy Number of New Technologies or Management Practices “Under Research” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual ECF 1 2 1 1 1 Helminth 1 1 1 1 1 Lucerne 5 5 5 5 5 5 Dual Purpose Maize Source: KARI Data, MSI analysis Table 11. Number of new Dairy Technologies or Management Practices “Made Available for Transfer” KARI Program Element: Dairy Number of New Technologies or Management Practices “Made Available for Transfer” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual ECF 1 1 1 Helminth 1 1 1 Lucerne 5 Dual Purpose Maize 1 1 Source: KARI Data, MSI analysis Research under KARI’s dairy component focuses on two sets of needs, one related to the production of animal feeding material and the other focuses on animal health. Livestock feed products address scarcity problems in Kenya, particularly during the dry season. Lucerne, which has traditionally been imported, was selected for attention for its potential to improve quality fodder at the farm level as well as to enhance milk quality. KARI research trials, initiated in 2006- 07, were undertaken in collaboration with the Kenya Dairy Development Program (KDDP) through Land O’Lakes. KARI identified eight Lucerne varieties that are “best bets” for specific agro-ecological zones. KARI’s research has also led to the development of local Lucerne varieties that are currently undergoing National Performance Trials through KEPHIS. Animal health research has included, among other priorities, parasitic worms (helminth) which have long been a problem for Kenyan farmers. Responding to a request for help from a Kenyan dairy cooperative, KARI sought low cost solutions. This resulted in the development and dissemination of medicated helminth feed blocks (with molasses as primary raw material) that not only attack parasites, but also provide supplementary nitrogen, minerals and energy to the animals. KARI reports it has 12 worked successfully for farmers, whom it has taught how to use simple materials including molasses to create these blocks. KARI is encouraging commercial production of the blocks and is also making it possible for farmers to create their own by providing the recipe (science protocol) in a small pamphlet that is readily available on the Internet. It notes that “in areas with continuous rainfall, use of these blocks prevent constant re-infection with nematode parasites which would otherwise necessitate regular and frequent use of anthelmintics.” KARI’s work on animal health also includes the development of a vaccine that protects cattle against East Coast Fever, a devastating disease. A MSI case study on this important KARI research product is include under Question 2. d. Maize The maize component of the KARI program was introduced through KARI’s 2004 proposal, and also covered the start-up of work on soil fertility, dairy, and horticulture under the current partnership. USAID funding for the maize component of the program (2004-2012) totaled $334,230 or 10.04 percent of the program total. The KARI maize component has yielded the program’s largest number of products “under research” and products “ready for transfer.” Table 12. Number of new Maize Technologies or Practices Under Research KARI Program Element: Maize Number of New Technologies or Management Practices “Under Research” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Grey leaf spot 2 2 2 2 6 6 25 25 MSV 2 3 3 3 12 12 30 30 Striga 1 1 1 1 1 1 Source: KARI Data, MSI analysis Table 13. Number of new Maize Technologies or Practices “Made Available for Transfer” KARI Program Element: Maize Number of New Technologies or Management Practices “Made Available for Transfer” as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Grey leaf spot 3 3 2 1 2 3 6 MSV 2 2 7 2 4 10 10 10 Striga 1 1 Source: KARI Data, MSI analysis Maize in Kenya is often equated to ‘food security,’ and its unavailability causes the country to officially declare a hunger situation. Kenyan maize farmers often experience production losses as a function of crop diseases. Grey leaf spot, MSV and Striga are particularly damaging to maize yields in Kenya and are a specific focus of work under the partnership. KARI development of new disease resistant maize focuses on both mid-altitude (KH 500 series) varieties and on highland (KH 600 series) varieties. Between 2006 and 2012, fourteen seed varieties in these two clusters received KEPHIS certification. Of these, eight were KH 500 series, mid-altitude seeds, while three were KH 600 series, highland varieties. In terms of the yield, KARI staff estimated for the evaluation team (and the teams subsequent analysis of published KEPHIS data confirmed) that the average yield for KARI’s KH 500 mid-altitude series was between 3-5 tons per hectare while for the KH 600 yields are closer to seven tons per hectare in KARI trials. On farm trials, however, as KARI staff explain, yields can be 40% to 50% lower, depending on farming practices and soil conditions. KARI research trials normally involve a rigorous comparison to an existing commercial variety. However data from these trials, which would help USAID understand the merits of specific technologies that KARI reports that are “ready to transfer”, are not found in KARI quarterly or annual reports. 13 For new hybrid seed varieties, including maize, the path from KARI’s trials to widespread adoption involves multiple actors and steps, including certification, commercial production, demonstrations, and commercial and other forms of distribution. The Kenya Plant Health Inspectorate Service (KEPHIS) conducts two types of research trials at a cost of $1,800 for both and, based on the results, recommends certification with the MoA. According to the KEPHIS website, roughly 100 seed certification applications are received per year and 20 of those are recommended for certification. To date, KARI has been the most prolific breeder of new seeds on the continent, after South Africa. In 2008, Commerce & Industry, a Kenyan business magazine, featured KARI on its cover when the MoA certified the release of 29 KARI seeds in a single year, of which 18 were maize varieties. While KARI remains the top breeder, numerically, the International Maize and Wheat Competitiveness Improvement Center (CIMMYT) estimates that roughly 75% of all seeds registered are now registered by private sector entities, including several of Kenya’s roughly 90 registered seed merchants. As an MSI reanalysis of KEPHIS maize seed certification data for two time periods (2000-2006 and 2007-2012) demonstrate, KARI continues to lead in the number of new seed varieties it releases, but both Western Seed and Kenya Seed Company are stretching to surpass it. Figure 3. Number of Seeds Certified by Owner over Two Time Periods Source: KEPHIS data, MSI analysis Among the maize varieties KARI released in the second period shown in this chart (2006-2012), ten were KH 500 series mid-altitude varieties while three were KH 600 highland varieties. Nevertheless, when commenting on KARI’s maize research program during a stakeholder meeting MSI organized, industry representatives and other stakeholders indicated they perceived KARI as overinvesting in highland varieties. Those who spoke on this topic said that mid-altitude varieties, needed by a much larger percentage of Kenyan farmers, should have higher priority. KARI staff concurred with this assessment and indicated that over the last few years it has been shifting its emphasis and agreeing that the need is greater at mid-altitude, where drought, pests, and disease are all problems. It was the view of KARI and also some industry stakeholders that advances in mid altitude varieties will come through advanced transgenic research currently underway to develop varieties that meet ecological challenges in the medium altitude regions of the country. To facilitate the seed commercialization, the KARI Seed Unit prepares foundation (breeder seed) that 14 can be delivered to private sector firms that KARI licenses, on either an exclusive or non-exclusive basis, who will produce and sell certified seed.5 In each year, the majority of this seed is retained by the KARI Seed Unit as stock, while the remainder is disseminated to private sector firms. KARI Seed Unit figures prepared for the evaluation team show that a very small percentage of some varieties, such as the KH 600 series went to the MoA and NGOs for demonstration purposes. Under licensing arrangements with KARI, seven firms produced certified seed for 15 KARI maize varieties between 2009 and 2012.6 Their combined production of certified maize seed from KARI varieties included a low of 259,366 kg (or roughly ¼ metric ton) in 2010 and a high of 1,568,163 kg (1.5 metric tons) in 2011, which MoA staff indicate corresponded to the Government’s popular “One Acre Fund” subsidy program. This was implemented in the wake of the previous year’s drought, through which certified seeds, training and fertilizer were provided in selected districts below cost and farmers paid back with a share of their harvests. These figures are similar, though not identical to KARI seed sales figures obtained from the East Africa Seed Company in connection with a case study under Question 2. Putting this into perspective, the highest reported volume of certified KARI seeds for these years, 1.5 metric tons, in 2011, represents but a fraction of the 25,000 metric tons of certified maize seed that all Kenyan seed companies together produce each year, on average. The small share of this market represented by KARI-certified maize seed grown over the past four years speaks to current certified seed-production capacity along the KARI research-to-farmer chain. This has significant implications for the production of breeder seed if any single KARI disease resistant maize variety is to be brought to scale, i.e., produced in sufficient volume to arrest a maize disease or pest problem nationally. Hybrid maize seeds reach farmers through commercial agro-dealers, or stockists. A current KEPHIS figure for the number of licensed agro-dealers in Kenya, including both seed and fertilizer stockists is around 4,000. Notably this figure is roughly 80% higher than the number of stockists KEPHIS reported licensing in 2005/2006. These data are consistent with a Tegemeo Institute study that found that the distance to a registered hybrid maize seed stockist had declined from six km on average to three km on average between 2000 and 2007.7 This expansion provides support to Kenyan government initiatives, including stockist training, who view strong commercial distribution as an asset. In addition to commercial channels for distributing maize varieties it develops, KARI provides some of the seed it produces to the MoA and NGOs for demonstration purposes.8 Other USAID projects, particularly the Kenya Maize Development Program thorough ACDI/VOCA, have also been a vehicle for collaborative, non-commercial efforts to raise awareness, conduct demonstrations, and organize Field Days for farmers. In addition, KARI has a small community-based grant program called the Agricultural 5 The KARI Seed Unit also sells, at close to cost, open pollinated variety (OPV) maize varieties which for the most part commercial seed firms, KARI reports, are not interested in handling them. In interviews, one commercial firms reported to the evaluation team that it viewed the practices of the KARI Seed Unit as being in competition with the private sector, but this view was not voiced by other firms that the evaluation team met with. (See Annex 4). 6 Dryland Seeds, Freshco International Ltd, Leldet Ltd, Oil Crop Dev Ltd, East African Seed Co Ltd, Oldrai Ltd, Veterinary & Agronomic E. 7 Kibaara, Betty et. al., “Trends in Kenyan Agricultural Productivity: 1997-2007”, Tegemeo Institute of Agricultural Policy and Development. http://www.tegemeo.org/documents/work/Tegemeo-WP31-Trends-Kenyan-Agricultural-Productivity.pdf. This same study reported that between1997 and 2007, the percentage of farmers growing hybrid maize had risen from 70% to 74%, albeit with a good deal of regional variation and in some areas a decline. Some of the areas with strong increases in the percentage planting hybrid seeds included Western Kenya (rising from 74% to 87% over that period and the High Potential Maize Zone, which rose from 89% planting hybrid seeds in 1997 to 94% in 2007. 8 When it attends farmer Field Days, KARI reports that it tries to pick up and bring along MoA extension service personnel, and return them to their offices. Various KARI staff indicated that these agents do not have the funds needed to operate their vehicles. These observations are consistent with the findings of a more extensive study extension services in Kenya by the Tegemeo Institute. (Muyanga, Milu and T.S. Jayne. Agricultural Extension in Kenya: Practice and Policy lessons, Tegemeo Institute of Agricultural Policy and Development. Working Paper No. 26 (2006). Nairobi, Kenya) 15 Technology and Information Response Initiative (ATIRI) through which it provides assistance, including seed in some instances to communities that want to improve their agricultural performance. e. Enhancing Food Security Component – Staple Crops The Food Security component of the USAID-KARI partnership was added through a 2009 KARI proposal. USAID funding for Food Security totaled $110,912 or 3.33% percent of the program total. Table 14. Number of new Food Security Technologies or Practices “Under Research” KARI Program Element: Enhancing Food Security – Crops Number of New Technologies or Management Practices Under Research as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Cassava & Sweet Potato 1 African Leafy Vegetables 6 6 6 6 6 Sorghum & Millets 1 Cowpea 1 Source: KARI Data, MSI analysis Table 15. Number of new Food Security Technologies or Practices “Made Available for Transfer” KARI Program Element: Enhancing Food Security - Crops Number of New Technologies or Management Practices Made Available for Transfer as a Result of USG Assistance 2009 2010 2011 2012 Planned Actual Planned Actual Planned Actual Planned Actual Cassava & Sweet Potato 1 African Leafy Vegetables 2 Sorghum & Millets Cowpea Source: KARI Data, MSI analysis The Food Security element of KARI’s partnership with USAID reflects growing concerns in Kenya about the country’s deteriorating food security situation. KARI encourages the production of Traditional High Value Crops (THCVs) that are considered to have been “orphaned” as maize popularity grew, by making available improved varieties and cuttings. Much of the work in this program component focuses on non-commercialized crop varieties that have been produced by scientists but which do not attract commercial interest from seed companies. Accordingly, traditional crop-planting materials are being channeled to farmers through the KARI Seed Unit, as well as through an arrangement with the MoA. MSI’s case study on sweet potato vines and cassava cuttings under Question 2 is illustrative of this work. Data on acreage under traditional high value crops are showing strong results indicating positive trends in the adoption of the varieties KARI is making available and, with others, working to demonstrate and disseminate. During a stakeholder workshop MSI conducted to examine findings from this evaluation, various stakeholders presented their views on how KARI might further enhance its food security program under the partnership. These included: (a) do more research and produce technologies that respond to new challenges: climate change, disease resistance, drought tolerant, and early maturing; (b) increase the supply of non-commercialized seed stocks because the demand is currently outstripping the supply in virtually all the seed varieties that KARI seed unit supplies; (c) produce more Open Pollinated Varieties (OPVs) so that smallholders have to cheap access to technology which they recycle over a period of 16 Sustainable partnerships mean higher incomes for French bean farmers. French beans are a major horticultural export crop in Kenya and a potential income earner for smallholder farmers. However, the sub-sector faces many challenges including seed shortages, uncoordinated plantings, and poor weather. This has led to major imbalances in supply and demand for quality produce and significant fluctuations in farm gate prices. USAID-Kenya Horticulture Competitiveness Project (KHCP) is intervening at a national level, providing innovative technical solutions to recover production volumes in order to meet market requirements, minimize the usage of agrochemicals, and ensure full compliance with recognized international standards. Through technical assistance offered by USAID-KHCP partner Good Neighbors Community Program (GNCP), farmers in Rift Valley and Western regions have adopted French beans as a product to diversify their crop base. The farmers have been trained in good agricultural practices and postharvest handling of French beans to generate export quality produce. In addition, they were linked to a secure buyer - USAID-KHCP commercial partner Canken International Ltd. The combined effort by GNCP and Canken is proving to be a success in terms of improving farmers’ productivity, backed by a steady market. As a result, 38 farmers supported by GNCP sold 5,480 kg of French beans valued at Ksh 274,000 ($3,300) to Canken International for export this month. Through Canken, farmers are accessing new seed varieties, receiving advanced technical trainings in pest and disease control and agrochemical management, with the added bonus of having a ready buyer for their crop. GNCP is providing complimentary basic skills training in good agricultural practices and nursery management. The collaboration has resulted in French beans prices firming up at 50 Ksh/kg ($0.61/kg) and the expansion of Canken’s production base to Bungoma and Trans-Nzoia counties. Grown Kenya Monthly, Issue No. 22, July, 2012 USAID: Kenya Horticultural Competitiveness Project time; (d) regularly carry out awareness programs and field days including demonstrations in Farmer Training Centers (FTCs) so that new technologies become familiar and accessible to farmers; (e) train private extension staff from relief organizations and NGOs on new technologies and extension methods; and, (f) carry out baseline surveys and other demand pull surveys to determine the cropping needs of farms so that KARI can undertake research that is more informed about farmers’ needs. f. Horticulture Component KARI’s work on horticulture preceded the start of the current partnership program by a number of years. By the time the current program started, in 2004, the KARI Horticulture Unit, through the National Horticulture Research Centre in Thika, had already embarked on various horticulture research projects. It used early USAID funding for various activities, including research on tissue cultures that resulted in tissue culture advances with bananas, snow peas, tomatoes and French beans. KARI research under a previous USAID project resulted in the 2000 release and certification by KEPHIS of Kutuless (J12), a rust resistant French bean variety with extra fine pods. The French bean story, in the text box below, illustrates the way in which KARI research, when transferred to farmers through commercial distributors, yields economic benefits. KARI also credits USAID for investing early in KARI research on roses, noting that flowers are now Kenya’s third largest export product, with roses comprising 70 percent by value of all exported flowers. Horticulture was the smallest component of the current USAID-KARI partnership project on which this evaluation focuses, and was in operation for only the first year of the project (2005/2005), with a funding level of $27,392, or 0.82 percent of the project total. Upon early termination of the program (MSI was told that KARI staff say they never fully understood), KARI’s Horticulture Research Centre was reportedly in a temporary crisis because various farmers groups and farming communities had already been drafted to help undertake project activities. When the horticulture component terminated, it put the research center on the spot by farmers who had become overly enthusiastic about the project prospects. 17 g. Nutribusiness Component The Nutribusiness Component of the KARI-USAID partnership came into the program through KARI’s 2007 work plan. While relatively small, this component has a strong linkage to downstream commercial success. Funding for this project component was $105,483 or 3.17 percent. Nutribusiness was added as a dimension of the KARI partnership program for the specific purpose of absorbing the results and ongoing activities of a terminating USAID project, under which rural women, working together, had developed nutritious food products from local crop materials. The original project, a University Linkages activity involving the University of Nairobi, Tuskegee and the University of Pennsylvania involved rural Kenyan women in a series of workshops that led to the formulation of a variety of products, including porridge-type mixes for infants, and subsequently helped build production facilities for these food products in Bomet and Murang’a. KARI completed this work and designated the two products as being “ready for transfer” in 2006. As of 2013, those products are still awaiting certification from the Kenya Bureau of Standards (KBS) certification process. In the meantime, both products (NIMIX and MAMIX) are in use by households in the locality close to the production areas of Makueni and Mbeere. This signifies acceptability in KARI’s view, and as an endorsement of their nutritional value by households utilizing these products. While NIMIX and MAMIX have not yet received certification, another version of a weaning food produced has received. According to a published story by Audrey Maretzki, the original University Linkages project director, it is being distributed Azuri Nutri-Mix which is looking to a Murang’a women’s group to raise their production capacity in order to respond to high demand in leading Nairobi supermarkets. QUESTION 1.B: HOW EFFECTIVE IS THE KARI M&E SYSTEM IN MEASURING RESULTS AND IMPACT? Overview of M&E at KARI Over the past four to five years, KARI reports that it has made a significant investment in improving its M&E capacity. KARI has created a separate department dedicated to M&E, drawing in staff from KARI’s Socio-economics and Biometrics division. This M&E office has developed a performance indicator system keyed to KARI’s strategic plan which is used to gather quarterly data. A harmonized monitoring system that can meet all donor needs as well as KARI’s own need to monitor performance against its Strategic Plan is a system goal. KARI reports that data on performance measures it tracks are being collected quarterly, which allows KARI to organize reports by calendar or fiscal years, as needed. KARI has invested in an electronic database for managing this system and reports that it is gradually shifting staff requirements away from traditional trip reports and towards reporting on a planned and actual basis against indicators. KARI's top management, in a MSI interview, also indicated that in 2013 the institute plans to roll out internet connectivity and a modern Wide Area Network [WAN] that will link all regional research centers and the head office to improve efficiency and facilitate seamless reporting for all research indicators. The USAID-KARI Performance Monitoring Plan (2009-2012) MSI’s review of KARI documents, including proposals dating back to 2003, indicates that KARI has a clear understanding of performance management concepts. KARI proposals for the current partnership program included elements of a Logical Framework for each component and both short and long term performance indicators aligned with project activities. Some of these indicators were direct measures of KARI results, while others focused on downstream results and impacts such as the number of farmers 18 adopting new varieties and the land area on which specific crops, or even varieties are being grown. While downstream results indicators were proposed by KARI, the performance monitoring plan (PMP) indicators USAID asked KARI to report on, beginning in 2009, does not include them. As discussed under Question 1.a, the indicators on which KARI reports focus on (a) short and long term training and (b) numbers of items at different stages in KARI’s research process, i.e., “under research”, “in trials” and “ready for transfer”. Since 2009, KARI has set targets on and reported against these quantitative indicators. In terms of producing the numbers that USAID’s PMP for KARI requires, the KARI system is effective. Questions posed by the evaluation team allowed KARI to demonstrate that behind each number it reports to USAID, information about what is being counted, e.g., a seed variety that is moving through the process, or a type of diagnostic test, is being collected. Beyond keeping these kinds of counts, it was not clear to MSI what USAID and KARI were learning from this performance monitoring system. The evaluation team’s review of KARI quarterly and annual reports on the partnership are not visibly linked to the PMP system. None of these reports, for example, included or reported on PMP targets, or identified what research products were associated with each numerical PMP target in the quarterly and annual report narratives. The reports also did not show how KARI was progressing through a given year with respect to meeting expectations that, for example, a specific type of plant material or vaccine would move from the laboratory into trials that year. It is possible, the evaluation team found, to read through a number of KARI’s quarterly and annual reports on the partnership program components and research efforts and gain a narrative sense of progress on elements of the program (a specific diagnostic test, or fodder variety) and what the next step will be, but it is not necessarily easy, nor did narratives closely correlate with the three progress stages the PMP use to monitor the progress of KARI’s research program. Tracking USAID-KARI Downstream Outcomes and Impacts USAID’s SOW for this evaluation, like KARI proposals that drew program components into the partnership arrangement, displayed a strong interest in the utilization of KARI’s research and requested information on yields, adoption rates and other types of evidence of the value and application of this agricultural research program. To this end, MSI not only sought data on these matters, it tried to determine whether USAID and KARI could access this kind of information on a more routine basis by including tracking indicators for downstream outcomes and impacts in the program’s performance monitoring system. The team’s inquiry focused on five results measures: yield, certification, production for sale/distribution, farmer adoption an application (acres planted, goats tested). For some of these measures, the evaluation team found either KARI already has data on specific result measures or data is routinely collected by other government agencies. Evidence in this regard was collected primarily for maize, but in most cases discussions with other KARI program staff indicted that parallel information for their research products also exists. For maize, the evaluation team found that KARI has yield data for every variety it develops and that in most case they obtain, through field trials, data for a commercial check as well, but these data are not included in KARI reports on the USAID partnership. The team also found that KEPHIS publishes yield data and seed company production data for every variety it tests on its website, as the small table below for an example. In addition, the evaluation team was able to determine, and help KARI request, information on the annual level of commercial seed each of the firms that KARI licenses to produce and sell maize. While KARI, if not USAID, is well aware of the yields for seed varieties it develops, information on how much firms it licenses to produce each year is not something KARI research staff appeared, from interviews, to know. Seed production data, moreover, is a measure that could be useful as a “leading indicator” for understanding adoption, as low levels of production automatically constrain effects further downstream. 19 Table 16. National Maize Variety List (1964-2012) - SELECTED INFORMATION National Maize Variety List (1964-2012) Official Release Name Year of Release in Kenya Owner(s) / Licensee Maintainer and seed source Grain yield (t ha-1 ) Special attributes KH500-49A 2010 KARI KARI-Muguga South 6-7 Resistant to MSV, GLS and Turcicum blight Dual purpose (49% DM above commercial hybrids) Intermediate –dent grains Medium size cobs Grown in medium altitude transitional areas Source: KEPHIS Website: Crop Variety List Table 17. Seed Maize –Volumes Produced for Commercial Sale - SELECTED INFORMATION Seed Maize –Kari Varieties-Volumes Produced For Commercial Sale (2009 – 2012) Year MERCHANT SPECIES VARIETY WEIGHT(Kg) 2009 Dryland seeds Zea mays KH 500-21A 10,000 2009 Freshco International Ltd Zea mays KH 500-31A 2,000 Source: KEPHIS, at KARI’s request In addition to identifying sources of information on yield (both with KARI and from a third party) and for commercial seed production, the evaluation team discussed with USAID and KARI its capacity and plans for collecting indicator information on the farmer adoption and land/animal application of its research products to be included in KARI’s partnership project proposals. MSI’s discussions with USAID indicated that they had not seen a need for KARI to track and report on these kinds of indicators. Interviews with KARI staff, on the other hand, revealed a strong interest in these kinds of data, along with a recognition that, in the past, KARI socio-economic research had not had this focus. In this vein, KARI Planning, Monitoring and Evaluation and KARI Socio-economics and Biometrics Division staff described a new effort they have initiated that will, for the first time, produce these kinds of downstream results and impact data for specific programs. Earlier research, such as KARI’s 2003 beneficiary assessment survey, described in KARI’s 2005-2015 Strategic Plan, which revealed that awareness of KARI technologies in the surveyed districts varied from low to moderate (15-60%) have been more organization-wide in character. The new study KARI staff described will be a combined effort to develop a baseline for an anticipated new partnership program with USAID focused on enhancing food security with a retrospective effort to trace the impact of previous USAID-funded agricultural research. The first planning meeting for KARI’s outcome/baseline study was held on November 19-20 at AIRC, Kabete, which Dr. Mille Gadbois helped to launch on behalf of USAID. Discussions with KARI staff working on this effort indicate that they recognize that critical challenges for the design of its upcoming outcome/baseline study include choosing the right outcome indicators to measure in order to capture information not only for FtF but also on other outcomes important to KARI’s Strategic Plan. It is important to do so in a way that both adequately characterizes the FtF’s target population and can be replicated. 20 QUESTION 2: WHAT ARE THE ADOPTION RATES OF THE AGRICULTURAL TECHNOLOGIES PROPOSED TO FARMER BENEFICIARIES, AND IS THERE EVIDENCE THAT THERE IS DIFFUSION OF THOSE TECHNOLOGIES IN THE TARGET AREAS AND SCALING UP AND MARKET REPLICATION? To provide USAID and KARI with an in-depth understanding of diffusion and adoption in relation to specific KARI technologies, the evaluation team examined three innovations which, pursuant to USAID’s evaluation SOW (Annex A), were selected by KARI as being representative of research results across the program’s components.  Maize Program: Two maize seed varieties, one mid-altitude, Maize Streak Virus (MSV) resistant variety (KH500 43A) and the other a Grey Leaf Spot (GLS) resistant highland variety (KH 600 15A)  Food Security Program: Two traditional high value crops (THVC): sweet potato and cassava, with support from the USAID-KARI partnership Biotechnology Component  Dairy Program: The development of a vaccine to prevent East Coast Fever among cattle. The technologies KARI selected include both commercialized and non-commercialized technologies, as described in the case studies below. Maize Program Component: Commercialized Disease Resistant Hybrid Maize Varieties Under the Maize Component of the USAID-KARI partnership, as described above, research focuses on mid-altitude and highland varieties that are resistant to three specific problems (GLS, MSV and Striga). The two varieties KARI selected for examination from an adoption perspective are described below. Evaluation findings with respect to their dissemination, however, were gathered through common processes, including a large scale survey of agro-dealers and agro-vets.  KH500-43A, a Mid-Altitude, MSV Resistant Variety Maize Streak Virus (MSV) is common in Kenya and can be devastating to farm yields. The most serious outbreak in recent memory occurred in 1988/89 and caused an estimated yield loss of 40 percent of the maize crop in central Kenya. MSV spreads by leafhopper parasite. Maize leaves become chlorotic, turning yellow or whitish due to a decreased amount of chlorophyll that leads to stunting of plant growth and the maize cobs. KARI research for the KH500 series of maize, initiated in 1997, focuses on hard grain mid-altitude varieties. KH 500 43A, in addition to being MSV resistant was also bred for dual purpose use – grain for humans and leaf and stem for animal fodder. Average yield for the KH 500 43A variety, as posted on the Kenya Plant Health Inspectorate Service (KEPHIS) website, which officially tests and certifies seed varieties, is 6.5 tons per hectare.  KH 600-15A, a Highland, GLS Resistant Variety GLS was first reported to have affected Kenya’s maize crop in 1995. In 1998, maize yield losses in Kenya caused by GLS were estimated to be in the range of 30-50% in some areas. According to a baseline survey of GLS prevalence, 75% of Kenyan maize farmers were reported to have experienced some losses from this disease. When certified for release by KEPHIS, the yield for KH 600-15A was officially reported to be between 7 and 8 tons per hectare. KARI public statements give the yield for this variety as 7.43 tons per hectare. 21 These two hybrid seed maize varieties are sold by the East African Seed Company, through an exclusive licensing arrangement, and are the only maize varieties this firm sells.9 Data from this seed company indicate that between 2009 and 2012, national sales of KH 500 43A increased, while sales of KH 600 15A were more erratic, as the graph below shows. These data are reasonably consistent with KEPHIS figures for the volume of each of these seed varieties produced over this period.10 KEPHIS data further show that KH 50043A and KH 600 15A represented 11 percent and 21 percent, respectively, of the 2,318,877 kilos of certified KARI seed produced by firms during 2009-2012. Figure 3: Tons of Selected KARI Maize Seed Varieties Sold by the East Africa Seed Company Source: East Africa Seed Company Data, MSI analysis Seeds sold by the East African Seed Company reach farmers through local agro-dealers, or stockists, who number about 4,000 nationally, as discussed under Question 1 above. To understand the dissemination of these two KARI maize varieties, MSI contracted Research Solutions for Africa to carry out a survey of agro-dealers and agro-vets as indicated in USAID’s evaluation SOW. This survey covered 224 respondents, of whom 195 reported that they sell maize seed.11 Of the 195 respondents that sell maize seed, 17 (8.7%) reported that they currently carry KH 500-43A and 30 (15.4%) reported that they currently carry KH 600-15A. A small fraction of stockists who sell KH 600 15A (3 of 55) indicated that this was their bestselling maize seed, and that sales accounted for roughly 56% of their maize seed sales. Nevertheless, the clear message in this data is that most agro￾dealers are not stocking either variety. This statistic alone dramatically shrinks the pool of potential KH 500 43A and KH 600 15A adopters. On another survey question, stockists who used to carry one or both of these KARI seeds identified the 9 In contrast, Kenya Seed Company (KSC), the largest of the 87 licensed seed merchants in the country, East African Seed Company carries 23 varieties in total, of which 21 are hybrid varieties and two are open pollinated varieties. KSC has a distributor network covering 21 sites around the country on its website. Western Seed Company, another large maize seed producer, sells 16 varieties, of which 11 are hybrids and five are open pollinated maize varieties. 10 KEPHIS data for KH 500-43A shows that for this variety, East African Seed Company produced 7,550 kg of commercial seed in 2010 and 236,182 kg in 2011. For KH 600-15A, production was around 150,000 kg in 2009 and 2010, declining to 119,900 in 2011 and 48,800 in 2012. 11 The survey also captured information on a fertilizer product that the KARI soil fertility program component had identified in field trials as having positive effects on both soil acidity and farm. 0 50 100 150 200 250 300 2009 2010 2011 2012 KH 500 43A KH 600 15A 22 year when they first carried these seeds and spoke about why they do or do not carry them today.  Among the top reasons given for not continuing to sell these KARI products were: unavailability (38%); low demand (29%) and product cost (19%). A confirming insight on data indicating that some stockists found that KARI products were not available from their supplier was provided in an interview with a senior representative of East African Seeds Company who stated that seed￾demand for these two varieties is ‘stronger than they can supply,’ and suggested that KARI needs to strengthen its ability to meet ‘breeder-seed’ demands from East African Seed.  Among stockists who said they have continued to stock KARI varieties, the most common reasons for continuing to stock KH 500 43A were: high demand (29%), high quality (20%) and ready availability (17%). For KH 600 15A, a key reason for stocking this variety, in addition to high demand, was heavy rains (25%). Against the backdrop of these reports on the two KARI varieties examined, it is important to note that the bestselling variety of maize according to an agro-dealer interviewed was H614 produced by the Kenya Seed company, a high altitude hybrid variety sold by 61 of the agro dealers interviewed. This accounted for an estimated 74 percent of their mean average maize seed sales. While popular, KARI data show that this variety yields 30 percent less than KH 600 15A, and slightly more than KH 500 43A. Notably, the popular Kenya Seed Company H614 was characterized to have a sweet taste, is tolerant to weevil attack, is hard-grained, tolerates heavy rains, and sells at KShs10 less than KH 600 15A. An important lesson from this maize seed dissemination case study is the arrangements KARI makes with seed merchants represent only the first step along the path to commercialized dissemination and farmer uptake. The extent to which a seed merchant has penetrated available distribution channels, which for maize is the stockists system, is also critical when KARI's aim is broad application of a seed variety in the areas where it is appropriate. Food Security Program Component: Non-Commercialized Traditional High Value Crops: Sweet Potato and Cassava Among the traditional high-value crops on which the Food Security Component of the KARI partnership focuses, sweet potato and cassava are of particular interest. Both of these crops have been highlighted as food security priorities by the African NEPAD, under the Comprehensive African Agriculture Development. KARI is involved in regional activities focused on these two crops through the Association for strengthening Agricultural Research in Eastern and Central Africa. Maintaining adequate disease-free stock of plantings for future multiplication and protecting the gene pool for each variety are central concerns with traditional high value crops, as diseases can easily spread between farmers, since cuttings are frequently shared. KARI’s work on these crops since 2009 has focused more on disease control than on increasing yield, and has included laboratory processes; green housing and a field hardening processes took a total of six months before planting materials were ready for multiplication and distributed to the regional KARI centers. To foster farm-level planting of disease-free cassava and sweet potato varieties, KARI reports that it is working with MoA extension staff and other local agricultural communities to select farmers to grow clean-planting materials. Farmers then receive planting materials at subsidized prices and KARI receives a portion of income from the contracted farmers. This practice is reportedly helping KARI maintain a clean stock for future multiplication and enable program sustainability. For the period 2007/08 until 2015, the Ministry of Agriculture is making payments to the KARI Seed Unit for the multiplication of 23 sweet potatoes and cassava cuttings. KARI data on this arrangement show that MoA payments of KShs 150 million (US$ 1.9 million, per year, are translating into KARI making available, on average, 2,299,911 sweet potato vines and 2,502,679 cassava cuttings, per year). KARI’s Seed Unit also works with the Red Cross, FIPS Africa, World Vision International, Action Aid International - Kenya, FAO of the UN, Catholic relief Services and the Anglican Church Food Security NGOs. While KARI is the main player in research and development of cassava and sweet potato varieties and plant materials, other new companies including Genetic Technologies International Limited and Jomo Kenyatta University of Agriculture and Technology’s Enterprise Limited are undertaking the same activity, but on commercial scale. Data on farmer adoption and planting of sweet potatoes and cassava in the semi-arid Eastern region targeted by USAID’s funding for the KARI Food Security Component had not yet been collected. However, KARI’s 2010/2011research on adoption in the Makueni and Machakos districts in the Eastern Region will serve as a baseline to which data collected in the future can be compared. With respect to adoption, KARI will also be able to compare data for efforts in Eastern Kenya under its USAID partnership to adoption studies in other parts of Kenya in 2001 and 2007 that showed positive responses when the availability of disease-free sweet potato and cassava planting materials was increased. Dairy Component: East Coast Fever Vaccine (ECF) Development East Coast Fever (theileriosis), which is spread by ticks, is estimated to kill one cow every 30 seconds in East, Central and Southern Africa, over a million per year in all. Once contracted, mortality can be as high as 100%, devastating herds within a month. Research on East Coast Fever at KARI started decades ago and first yielded a vaccine in the late 1970s. Further research and the complexity of the vaccine’s infect-and-treat delivery mechanism (ITM) delayed KARI’s formal release until late in 2012. Conceptually ITM delivery is easy to understand. It involves giving cattle a subcutaneous dose of tick￾derived parasites and, at the same time, providing the antidote, a long-acting tetracycline. In practice, it is more difficult, and requires vaccine specific training and veterinary expertise. Other factors that have made it difficult to bring this vaccine to market included the “need for a liquid nitrogen cold-chain, lack of awareness of its existence, the unit cost of providing it to cattle and its still-limited availability.” 12 This same source, GALVmed, captures some of the drama of the vaccine’s long development process, as paraphrased below: “Through the KARI Field Station at Muguga North in the 1990s, the Food and Agriculture Organization contracted the production of a commercial batch and in 2007 this became exhausted after it was sold. As a result, an NGO (GALVmed) asked that a second batch be made, which is now half used. In the 1990s KARI took a strain of the tick-parasite to make a make an additional vaccine and as a result of collaborative efforts (ILRI, FAO, GALVmed) in early December 2012, after KARI had also trained a number of veterinary service providers.” Pre-release adoption sites in Tanzania indicate that the vaccine is 95% effective under non-trial conditions. Estimates from the 1990s suggest that the vaccine will save affected regions at least $186 million annually. In practice, the cost of adopting this treatment must also be considered. In 1990, a cost analysis for the vaccine, as it was understood at the time, suggested that the unit cost of a lifetime immunization might be as low as $2.37 per cow.13 Published reports on the current actual cost of 12GALVmed, “GALVmed and East Coast fever”, GALVmed.org, http://www.galvmed.org/activities/east-coast-fever [accessed 12 February, 2013] 13 A.W. Mukhebi, et al, “Cost analysis of immunization for East Coast fever by the infection and treatment method”, Preventive Veterinary Medicine, 9 (1990) 207-219. http://pdf.usaid.gov/pdf_docs/PNABL839.pdf 24 delivering the ECF vaccine to Maasai pastoralists’ herds in Tanzania put the unit cost closer to $10 per head in this relatively remote area.14 Yet, even at $10 per treatment this report indicates that 80 percent of calves in some wards have been vaccinated. Further experience will show what the average cost in Kenya and across the region turns out to be and whether that fosters or impedes adoption rates. Commercialization that results in large scale production and delivery of the East Coast Fever vaccine is a major challenge. Steps toward commercialization in Kenya, as a DFID report indicates, are evolving: “Commercial validation of ITM was carried out in Kenya [by] Coopers Kenya Limited, a private pharmaceutical company… but due to geographical and production system restriction the company was unable to sustain the delivery. More recently…Veterinares sans Frontiers Germany (VSF-G) in collaboration with KARI and ILRI has established a successful commercial delivery in the Maasai pastoral areas.” 15 Discussions regarding commercialization options are underway at KARI and a private partnership arrangement is currently being sought. KARI staff indicated to the evaluation team that release and commercialization might have occurred earlier if KARI had identified and engaged the right mix of private sector production and delivery actors, as Coopers turned out to not be as interested as KARI expected in such vaccine development. The ECF vaccine story provides useful lessons on the importance of early efforts, and possibly multi￾pronged, efforts to engage actors who represent all of the elements of a commercial delivery chain. For research products with the potential of the EFC vaccine, achieving scale may require the participation of a variety of firms that can address cold chain requirements, and an even larger number of firms and NGOs capable of developing training programs for staff and veterinary intermediaries and delivering a complex treatment to thousands of farmers and herders across a large number of African countries. QUESTION 3: WHAT FACTORS BEYOND THE CONTROL OF KARI PLAYED A SIGNIFICANT ROLE IN AFFECTING, POSITIVELY OR NEGATIVELY, THE UNDERLYING CHALLENGES THE PROJECT SOUGHT TO ADDRESS, AND WHAT WAS DONE TO ADJUST THE PROJECT DESIGN TO THOSE FACTORS? KARI’s vision, and its stature as Kenya’s flagship agricultural research enterprise, is reflected in the scale of the problems that dominate its research agenda under its long term USAID partnership. If cattle are dying from ticks every 30 seconds, or if maize is dying from a virus or a weed, finding a solution is on the KARI agenda. In all of its proposals to USAID, as well as in its own Strategic Plan (2005-2015), KARI describes itself as research institution. In fact it is a network of 44 research centers around the country, many of which have specialized expertise and mandates. That being said, KARI also envisions itself as an active participant in efforts that transformed research results into farm-level benefits, including through “ensuring timely and continuous access to knowledge and emerging technologies” for crops, livestock and natural resources management” and “coordinating with parties in value chains to ensure sustainable commercialization” of its research products. In this 14 “Fighting East Coast fever - Lessons from Maasailand”, New Agriculturist. http://www.new-ag.info/en/developments/devItem.php?a=1709 15 DfID, Pro-poor vaccine-based control of East Coast fever, Research Into Use Program, 2008. http://www.dfid.gov.uk/r4d/PDF/Outputs/ResearchIntoUse/AHP14.pdf 25 vein, KARI staff point to Field Days and other awareness raising activities it undertakes in each of the USAID partnership program areas; licensing agreements reached with commercial firms, and where it has step in when commercial firms were not interested, to make open pollinated seed varieties and plant materials more widely available through the KARI Seed Unit. At the same time, they are quick to say other actors, including other USAID programs, have a greater capacity for, and therefore bigger role to play, than does KARI does in ensuring that all of the steps required to facilitate farm level adoption of research-based solutions occurs. As the forgoing suggests, KARI, like most research institutions, sees itself as playing a unique role in coming up with innovative solutions to critical agriculture sector problems, and “doing its share” to disseminate knowledge and promote commercialization and adoption. One-on-one, however, KARI staff will say that they recognize that, despite their efforts and those of others who share their desire to see research results applied, awareness and adoption of KARI technologies are sometimes discernible only over relatively limited areas, often where work first started or demonstration projects were mounted. The limited role played by Kenya’s extension service, which is well documented by Tegemeo Institute analytic studies, as well as in abundant anecdotes, functions as an easy scapegoat in the face of data on low awareness from KARI’s own research, MSI’s agro-dealer survey for this evaluation, and other studies. Kenya’s situation in this regard is not unique. The Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA) in its Strategic Plan for Knowledge Management and Scaling-Up characterizes agricultural research knowledge management throughout Eastern and Central Africa as being “hampered by challenges” including poor understanding of stakeholder communications mechanism, “their knowledge needs, attitudes and practices," gaps in “knowledge products and services for the agriculture sector," ineffective use of media and weak monitoring of communications interventions. ” 16 These are problems, ASARECA points out, that while they involve and have implications for research institutes, are shared challenges for a wider range of organizations in the sector. Beyond knowledge management, the evaluation suggests that there is a wider array of issues along the research to farmer path to be addressed. Working the adoption problem backwards, farmer awareness plays a role, but so does the absence of product on stockist’s shelves, as the evaluation survey demonstrated. This in turn appears from KEPHIS production data to link to an inability or lack of firm interest in producing/delivering product in sufficient volume to keep stockist’s shelves reliably full. In a final turn, the level of breeder seed the KARI Seed Unit produces could affect this entire chain. KARI staff add to this picture, citing additional issues they view as impediments to adoption including inadequate community participation in demonstrations and other technology transfer activities; high levels of poverty affects uptake on technologies that require some outlay of cash; and, poor infrastructure, which affects marketing and input supply, and mechanization problems. Interview data indicate that KARI engages on many of these issues through: adaptive research, the KARI￾Agricultural Technology & Information Response Initiative; farm field research schools and farmer￾researcher groups, and ICT based technology transfer approaches. One view held by KARI staff on how to improve adoption focuses on starting with what you already know about adoption, and building from there. Several staff were able to cite numbers of farmers they believed were already using KARI seeds and planting materials. KARI has also recently established an Outreach Department that it expects will strengthen its capacity on the non-research aspects of what it takes to raise awareness and demand, pick commercial partners and ensure supply and distribution, and listen to customers at all levels. New functions, KARI indicates, would add to what the organization already does, ranging from interacting 16 Association for Strengthening Agricultural Research in Eastern and Central Africa. 2010. Turning Knowledge into Action. Strategic Plan for the Knowledge Management and Upscaling Programme, 2009–2014.ASARECA, Entebbe. 26 with existing Government of Kenya knowledge management initiatives (KAINnet) to supporting community technology gain initiatives through its small grants program. While all of the steps KARI has been taking and intends to take will likely stimulate improvement, small incremental improvements may not yield dramatic changes at the level of national problems. Beyond significant problems in the research-to-farmer chain that KARI is already addressing, and may find more effective ways to address, KARI staff also raised more basic issues with the evaluation team, on more than one occasion. These included both funding and personnel issues, with funding issues being the ones that KARI most hopes can be addressed as a new partnership arrangement is established. Administrative/Financial Management Challenges  Some KARI scientists reported that they faced a slow and bureaucratic funds-release process for their research activities from the USAID Coordination office and this hampered research efforts. KARI scientists in most of the regional centers visited were of the view that the USAID funding should directly send research funds to centers and transmit administrative funds to the KARI head office to reduce time wasted in waiting for funds transfer and release.  A large number of KARI staff, meeting as a group with MSI to discuss research results, focused on one aspect of the administrative challenge above – namely monthly instead of quarterly funding releases and reporting. Quarterly, they collectively informed MSI, is a better option in terms of planning, time management and for a range of other reasons. KARI’s one budget cycle is quarterly as are those of other organizations KARI works with. Options for shifting KARI to a quarterly system were viewed as being highly desirable to KARI’s scientist/administrators.  Some indicted that scarce government funds for research activities makes KARI over-reliant on grants. Some described disruption in the flow of donor funding even when research was going on, completely disrupted and even stopped on-going research because KARI did not have the capability to release any ‘standing in’ funds to sustain research activities. It was suggested that KARI create a division of research funds, which government budget should complement to align with donor funding in order to sustain research and innovations with limited disruptions. Personnel System Challenges  It was reported in one of the discussion forums by some KARI scientists and other stakeholders that senior researchers who joined KARI had ended up as administrators at the head office thus diminishing research capability at the regional centers. This hampered effective research and led to slow generation of research technologies at the center station levels.  Some comments received during interviews suggest that some KARI scientists perceive that they are working in less-than-ideal conditions and find this to be a disincentive. They reported as disincentives: very low salaries and remunerations and being placed under the ordinary’ civil servants salary scales despite their higher academic qualifications. Some reported that they, or others, try to fill in the income gap by consulting or unofficially working as breeders or researchers for other organization. 27 CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS Under this partnership program, KARI has selected serious agricultural, livestock and natural resource management issues facing Kenya on which to focus its research attention. In all of the main program areas, KARI scientists have produced research products ranging from vaccines, to seeds and other plant materials that have clear advantages over predecessor solutions. Across the board, however, evidence from the evaluation shows that KARI research products are not consistently being taken up on the scale needed to significantly reduce the incidence of diseases or create the resistances for which they were designed. Many of the problems that impede the up-take of KARI research solutions lie downstream in distribution channels and farmer awareness. It is not, however, in KARI’s interest to simply observe these impediments. KARI knows this and is actively involved in a range of efforts aimed at expanding knowledge about research products and their effectiveness. KARI and partner organizations are also working to develop pathways for the commercial production and distribution of as large a portion of its products as possible, compensating with other approaches only where no commercial mechanism is found. What is missing from this picture, the evaluation found, is enough evidence to sort out what will spark action along the production distribution chain and ignite farm-level uptake, and how to trigger the adoption of better farming products and practices faster and on a much larger scale. KARI has an immense talent for thinking big and finding solutions. It is time to turn that spotlight on to what it takes to stimulate demand and use for those solutions on Kenyan farms. Accordingly the evaluation team’s recommendations for KARI focus almost entirely on the biggest challenge it faces: breaking through impediments to utilization, using the same set of skills it uses to break through scientific barriers – hypothesis generation, testing, and following the evidence that emerges. RECOMMENDATIONS FOR KARI To better track and communicate KARI research program results to USAID and more broadly:  Unpack current practices for reporting on USAID performance targets. Specify which products numeric targets are linked to and discuss progress toward these performance indictor targets in quarterly and annual report, one product at a time. To better learn about and communicate the downstream effects of KARI research products:  Add performance indicators that will be monitored, but not targeted, to more regularly and more reliably understand the production, distribution and adoption of research results, one product at a time – using information that KARI and other agencies already collect, e.g., KEPHIS data on the volume of certified seed produced by KARI licensees.  Use the opportunity presented by KARI’s planned baseline/impact study to establish a replicable methodology that can be applied on an affordable schedule to generate trend data to help guide 28 the KARI research program, e.g., every five years. To improve the likelihood that each KARI research product will become available to as many appropriate farmers as possible:  Develop and implement business-oriented research product distribution plans, with clear targets and milestones, one product at a time.  Use the period before KARI joins KARO, or the period just after, to internally review KARI’s strengths, weaknesses, and opportunities to improve the extent to which KARI research solutions that could solve important problems in Kenya actually do so. Modify and make improvements on all the other recommendations below based on what KARI itself knows about what works and what doesn’t. To expand KARI’s ability to deliver evidence-based change message to farmers:  Identify and test more “retail” mechanisms for delivering evidence-based information about KARI products to farmers. For example:  Target stockists directly with the appropriate level of information on KARI product science and advantages – help them help customers make evidence based decisions.  Cull best practices – determine what worked and has been sustained from KARI’s small grant program and communicate with communities and farmers to gather stories about technology adoption among their peers – access to radio is increasing and might be a good test option.  Develop and test the effectiveness of a community grant follow-on program – where a community that KARI helped to adopt a technology turns around and helps another community do the same. This can also help transfer best practices from KARI’s model. To stimulate optimal levels of commercial production and distribution of KARI research products:  Organize headquarters-based “field days” that include “listening sessions” with commercial firms and others who work downstream from KARI to bring research results to farmers and build their “ownership” of next generations of KARI research products.  Modify commercialization approaches that are not generating enough product yield to plausibly change the status of the problem the product was designed to solve. For example, where licensing approaches are part of a commercialization package and production and/or distribution is less than KARI would expect, test alternative licensing terms to better achieve adoption/utilization aims, e.g.: o Make annual license renewal contingent on meeting specific product production targets, e.g., certified seed, vaccine rounds o Make annual license renewal contingent on meeting specific targets for the number of outlets through which the product is available to farmers (and possibly the regions in which those outlets are found) 29  Waive the royalty fee if a high target for production and/or distribution outlets is accepted/met.  Also identify and test various approaches that mix commercial and non-commercial approaches. For example, review net costs to government of the “One Acre” program that subsidized seed sales for an in kind return. The seed delivery effect was to more than double the volume of certified seed produced and presumably used. To determine what awareness raising/commercialization/distribution approaches actually work, and which work better than others:  Develop working relationships with the Tegemeo Institute and other entities that know how to apply the kinds of rigorous methods needed for determining effect, as well as awareness and dissemination experiments that KARI uses to test seed varieties. RECOMMENDATIONS FOR USAID Over many years, USAID has provided KARI with strong support and encouragement, and KARI has flourished from this assistance, building human capacity for the future as well as a track record of research breakthroughs. USAID is now planning to move forward with KARI as it moves into KARO and becomes part of a larger enterprise dedicated to advancing agriculture and food security in Kenya. Encouraging KARI to try new things, learn and get better has long been part of USAID’s role in this partnership. It is along this line that the MSI team recommends to USAID that it:  Encourage KARI to adopt a more robust and learning-oriented, performance monitoring system for the new partnership arrangement – one that includes indicators KARI only monitors, as well as indicators it both targets and monitors. The focus for non-targeted indictors should be on measures that provide insights about or predict the likelihood that KARI research results will be adopted at the farm level.  Help KARI develop a clear vision and plan for enhancing the adoption focus and effectiveness of downstream organizations, whose production, demonstration and distribution efforts are all essential for farmer adoption of research results – without trying to become those organizations.  Foster efforts among its other USAID agriculture sector partners to identify high value KARI agricultural research solutions that can be promoted and applied, in unison, by multiple project teams, to build a critical mass of adopters.  Work with KARI to find solutions to administrative and financial matters affecting the partnership and KARI’s work. 30 ANNEXES ANNEX 1: EVALUATION STATEMENT OF WORK AID-623-TO-13-00008 Under Contract AID-623-I-12-00001 A.1. BACKGROUND The USAID program ‘Support to Agricultural Research in Kenya’ partnerships with KARI under Grant Agreement No. 615-007 is aimed at achieving “Increased Household and Incomes.” A total of six research components are targeted through this effort: Kenya Maize Development Program (KMDP) The prime goal of KMDP is to develop maize varieties that will make a change in on-farm productivity from the current 1.5 – 2 metric tons per hectare to 4.0 metric tons per hectare by the end of the program. Soil Fertility Improvement Program (SFIP) The goal of SFIP is to increase levels of soil phosphorous and other nutrients, and to improve use of economically viable, environmentally sound, gender sensitive, integrated soil fertility management strategies through farmer field schools in Kakamega District, Western Kenya. Kenya Dairy Development Program (KDDP) The goal of KDDP is to provide 800,000 smallholder dairy farmers with improved technologies in dairy health and dairy production, with approximately 3.5 million dairy cattle capable of producing 2.8 million MT milk per day. Enhancing Food Security This component of the program aims at increasing yields and income of smallholder sorghum, millet and cowpea producers. Targeted farmers are encouraged to upgrade to improved technologies capable of significantly increasing crop yields. Kenya Agricultural Biotechnology Program (KABP) The objective of the KABP is to facilitate the transfer of feathery mottle virus resistance traits to Kenyan sweet potato varieties using genetic engineering technologies. Nutribusiness Project The Nutribusiness project was started in Kenya in 1992 among the rural communities in Bomet and Murang’a. The University of Nairobi, in collaboration with U.S.-based University of Pennsylvania and Pennsylvania State University, took the leading role in the implementation process. The program’s objective is to develop and to propose to target households the two following flour formulations: (1) the Mamix formulations using alternatively sorghum, sweet potato, cowpea, amaranthus, and (2) the Nimix formulations using cassava, pigeon pea, millet, sorghum and dried cowpea. USAID has supported KARI in the research topics described above from December 2003 to March 2012. The core program activities are managed through a “Program Coordination Unit.” 31 Program Activities: In order to achieve program’s objectives, KARI is carrying out the following research activities: Maize: Develop maize varieties resistant to important biotic stresses: gray leaf spot, maize streak virus, and striga. Investigate dual purpose maize varieties for fodder and grain. Promote improved maize seed varieties. Soil fertility: Conduct on-station and on-farm research on locally appropriate fertilizer products and soil fertility practices. Train farmer field schools in Kakamega District on soil fertility best practices. Dairy: Promote immunization against East Coast Fever (ECF) Identify distributors and stockists Test ECF vaccines for viability Research minimum effective dose to reduce costs Train animal health workers and AI providers in ECF immunization Helminth Control: Train farmers in Helminth control Investigate effective treatment methods Food Security: Conduct research into causes of low adoption rates of proposed agricultural technologies among target farmer populations. Create farmer maintained sweet potato planting material multiplication plots at Matiliku, Mukuyuni, Kibwezi (Kwakyai), and Mutituni. Sensitize farmers on the importance of appropriate soil and water management techniques, and soil testing uses on crop yield gains. Conduct trials and organize farm level demonstration plots using several crop varieties. Biotechnology: Assess the effects of important Cassava viruses on farmer preferred Cassava cultivars. Develop Maize cultivars resistant to storage pests. Develop and disseminate animal vaccines and diagnostics. Nutibusinesses: Develop composite nutrimix flours (Mamix for Mbeere, Nimix for Makueni, Bascot and Muranga Construct production units at Mbeere and Makueni A.2. STATEMENT OF WORK A.2.1. Overview Under this task order, MSI shall conduct a final performance evaluation of the KARI partnership, primarily for learning. The final evaluations submitted by MSI must allow USAID to accomplish three primary purposes: 1)to examine the extent to which the project’s objectives and goals – at all results levels – have been achieved; 2)to understand if research results, training, and outreach have yielded sustained impact in the use of new technologies and on agricultural yields; and 3)to capture lessons that can be applied to current and future agricultural research and technology dissemination partnerships. USAID/Kenya has recently entered into a new agreement with KARI, making the outcome of this evaluation particularly relevant to improving the relationship and increasing the impact of KARI’s 32 activities. The methods and the final evaluation must be consistent and meet the standards of USAID’s Evaluation Policy (especially Appendix I), and the USAID Forward Quality Evaluation criteria. A.2.2. Key Evaluation Questions The evaluation must address the following questions and sub-questions: 1. Project Impact: How has the project performed in terms of achieving projected results and impact, and how effective is the M&E system in measuring them? How well did the project achieve the intended results as defined within the project’s Performance Management Plan? What additional evidence, qualitative or quantitative, exists that link project activities to improved agricultural output, increased uptake of improved agricultural technologies and practices, or reduced incidence of plant and animal diseases or parasites? What evidence exists to show that KARI research or activities have increased food security or economic security among beneficiaries or in the broader community? 2. Impact Sustainability and Scale-up: What are the adoption rates of the agricultural technologies proposed to farmer beneficiaries, and is there evidence that there is diffusion of those technologies in the target areas and scaling up and market replication? To what extent have new technologies developed or promoted by KARI demonstrated a strong cost￾benefit for farmers, with the initial cost of investment in new technologies offset by increased production or lower costs in other areas? What are the adoption rates of the agricultural technologies proposed to farmer beneficiaries? How well/how much have beneficiaries/farmers continued to use new technologies developed or promoted by KARI? What barriers exist to continued use or dissemination of improved technologies and practices developed or promoted by KARI? What relationships, with agro-distributors, agricultural service providers, relevant government ministries and district offices, community organizations, financial institutions, and/or NGOs, has KARI established that will support the continued use of improved technologies? Were the diffusion rates of the proposed agricultural technologies sustained over time among targeted zones? Did household food security increase in the targeted zones as a result of the project’s activities? 3. Project Design and Lessons Learned. What factors beyond the control of KARI played a significant role in affecting, positively or negatively, the underlying challenges the project sought to address, and what was done to adjust the project design to those factors? What unintended/unexpected outcomes, positive or negative, resulted from project activities? How have sociocultural factors, such as culture, economic status, gender, ethnicity, geography, contributed to or undermined the adoption of improved technologies? How has this information changed KARI’s approach to its research? Is this information relevant to other agriculture or economic growth projects? How can it be shared? What factors beyond the control of KARI played a significant role in affecting, positively or negatively, the underlying challenges the project sought to address? What changes were made or might have been made to the project design to increase its impact? What could the project have done to increase ownership of project goals and activities among stakeholders, such as individuals, communities, government, business service providers, etc? How well did funding levels, and staff competency and capacity, facilitate achievement of program objectives? The questions shall be considered both at a partnership / management level across KARI activities and 33 also at the activity level for each distinct focus area in Maize, Dairy, Soil Fertility, Food Security, Biotechnology and Nutibusiness. A.2.3. Evaluation Design and Data Collection Methods It is expected that the contractor shall work with the KARI Agreement Officer’s Representative (AOR) to define the most appropriate evaluation methods based upon the goals and questions of the evaluation. However, the COR is only person authorized to give technical direction besides the CO. The contractor is expected to complete the evaluation using a combination of quantitative and qualitative analytical methods. The contractor will review data and documentation related to the project and the development challenge. The evaluation should include interviews with USAID technical staff and implementing partner staff. It should also include interviews, surveys, and/or focus groups with project participants and beneficiaries. The contractor should work with the KARI AOR and the COR, as needed, to ensure that the methods and the final evaluation are consistent with the standards required in the IDIQ and Section A.2.1 of this task order. The contractor shall: review data and documentation related to the project and the development challenge, including review of KARI’s baseline survey, PMP plan, quarterly reports. (Note: Baseline reports are available for three research activities i.e. Maize, Soil Fertility and Dairy, and the evaluation team will have access to those reports. However baseline data will have to be collected for Food Security, Biotechnology and Nuti-business by the team during the evaluation period.) develop a detailed inception report that will frame the methodological approach to be utilized to evaluate the evaluation questions in Section A.2.2. The following is an illustrative example of the kinds of methodological approaches they are expected to recommend. a. Evaluation Design Question 1: Program Impact. Baseline surveys, performance monitoring plans (PMPs) and databases will be reviewed to determine how each component performed in terms of achieving its intended results, both in terms of the achievement of those results and the adequacy of the PMP in measuring those achievements. The evaluation team will conduct interviews with implementing partner staff member involved in project implementation to determine the successes of and lessons learned about the project, taking into account any findings and recommendations from previous evaluations of USAID support to KARI, interviews with key stakeholders, and site visits to selected beneficiaries and their representatives. Question 2: Impact Sustainability and Scale-up. As a public sector agricultural science and technology institute, KARI’s innovations may take considerable time to be conceived, developed, tested, demonstrated to be effective, released to the market, adopted by farmers and farm communities and scaled up by market forces. Accordingly, to examine the sustainability and scale-up of KARI innovations in terms of addressing food insecurity in Kenya, the evaluation team will ask KARI to select up to three innovations from among the six components of the project that have actually been released to the market with the potential to be scaled up by market forces. To examine the diffusion of the three selected innovations, the evaluation team will conduct a representative, large-sized sample of agro-dealers and agro-vets in the target areas, using a very short questionnaire designed to determine growth in demand for the innovations among their client base. Examples of the three innovations to be tracked might include drought resistant maize seed, improved or adapted sweet potato varieties, or specialized animal medicines. Question 3: Project Design and Lessons Learned. Factors, beyond the control of KARI, that may have played a significant role in affecting the underlying challenges the program sought to address, and their impact on project design, will be assessed. For example, in response to the recent drought / climate change patterns in Kenya, farmers in semi-arid regions have been encouraged to switch from maize cultivation to drought-resistant crops such as cassava, only to discover that the available cassava varieties were subject to disease. How KARI responded to these unanticipated setbacks will be explored. The degree to which there were unintended/unexpected outcomes – positive or negative – resulting from program activities, will also be assessed. Finally, the ability of KARI to adapt to unanticipated changes in 34 the implementation environment, and whether changes might have been made to program design that would have increased its impact, will be evaluated. b. Data Collection Methods The following data collection methods are expected to be utilized by the Evaluation team: Review of KARI’s baseline survey, PMP plan, quarterly reports, M&E databases, and any previous evaluations or other reports. Structured interviews of KARI project technical and administrative staff, as well as key stakeholders at both national and district levels, to be identified in coordination with USAID. A survey of 200 agro-dealers and agro-vets, in the targeted areas, to ‘market-test’ the degree of diffusion and scale-up of three (3) KARI innovations from among the six components of the project that have actually been released to the market with the potential to be scaled up by market forces. Data collected from secondary sources c. Data Analysis Question 1: Program impact. The evaluation team will review all performance management information available from the KARI, including a baseline survey, the performance monitoring plan (PMP) and the M&E database. The evaluation team will also review any previous evaluations that have been completed for USAID support to KARI, and the degree to which the recommendations of those evaluations have been followed up. Question 2: Impact Sustainability and Scale-up. The evaluation team will analyze the results of the rapid survey of agro-dealers and agro-vets in target areas, utilizing simple cross-tabs analysis to assess the proportions of respondents who, for example, state that one or more of the three innovations developed by KARI are increasing rapidly in sales to their client base. Qualitative questions about the benefits of the technologies released by KARI will be coupled with questions about unanticipated events, such as drought / climate change, disease strains, or other barriers. The results of this survey will be supplemented by interviews with key stakeholders. Question 3: Project Design and Lessons Learned. The evaluation team will participate in interviews of KARI staff, and conduct interviews of key stakeholders at both national and district levels, to make a qualitative determination of lessons learned in implementation about responding to unanticipated factors or developments, the manner in which KARI responded to them, and what lessons may have been learned relating to improved design of future activities. B.1. KEY PERSONNEL Team Leader: Monitoring and Evaluation Specialist – Senior Expatriate The Team Leader must have: A minimum of a Master’s Degree in monitoring and evaluation, research methodology, or related field. Ten years of professional experience in implementing, monitoring and evaluating development programs in developing countries, particularly in Africa, with extensive experience leading evaluations – preferably with experience on USAID economic growth evaluations relating to agricultural research and development projects. Strong background in social and cultural issues affecting rural development, including gender. Demonstrated written communications skills, especially in drafting evaluations, assessments and reports, required. Familiarity with USAID Forward quality evaluation standards and requirements. Senior Agricultural Analyst/Agriculture Economist: Agricultural Analyst – Senior Local The Senior Agricultural Analyst/Agricultural Economist must have: A minimum of a Master’s Degree in agricultural economics, agribusiness, or related agricultural field. At least ten years’ experience in Africa developing, managing and/or evaluating agricultural research and/or value chain projects. Thorough grasp of how agriculture research contributes to competitiveness. 35 Knowledge of the social and cultural factors in agricultural development projects. Extensive experience managing agriculture sector programs through donor or NGO-funded activities. Experience designing and implementing household or firm-level surveys in developing countries. Knowledge of and experience with gender issues. Fluency in both Swahili and English required. C.1. CONTRACT DATA REQUIREMENTS LIST # Deliverable Due Date 1 Initial meeting between COR, relevant USAID personnel, and contractor evaluation team key personnel. o/a December 3, 2012 2 Inception Report which provides a projected timeline and describes in detail the final evaluation methodology and data collection methods (including draft interview questions and data collection tools) to be used. USAID shall review and approve or require changes no later than December 14, 2012. No later than 5 days after #1. 3 Briefings of progress to date, including any issues or problems encountered. May be conducted in person or via email as agreed at the initial meeting. Weekly 4 Presentation of preliminary findings and conclusions of the evaluation to USAID/Kenya and key stakeholders. No later than 6 weeks after acceptance of 5 Draft Final Evaluation Report for review. USAID will provide Inception Report. comments on the draft report within 10 business days of receipt. No later than 3 days after #4. 6 Final Evaluation Report. No later than 7 days after receiving comments from the COR on #5. Note: All days are calendar days unless otherwise noted. C.2. APPLICABILITY OF THE BASE IDIQ All terms and conditions from the basic IDIQ apply to this task order. C.3. ACCOUNTING AND APPROPRIATION DATA BBFY 2003 EBFY 2004 Fund DV OP Kenya Prog Area 615-007 Dist Code 615-M Prog Elem 6150007.05 SOC 4100301 Amount $46,044 BBFY 2009 EBFY 2010 Fund DV OP Kenya Prog Area A26 Dist Code 615-M Prog Elem A140 SOC 4100301 Amount $68,573 36 ANNEX 2: DOCUMENTS REVIEWED 1. Association for Strengthening Agricultural Research in Eastern and Central Africa. 2010. Turning Knowledge into Action. Strategic Plan for the Knowledge Management and Upscaling Programme, 2009–2014.ASARECA, EntebbeDfID, Pro-poor vaccine-based control of East Coast fever, Research Into Use Program, 2008. http://www.dfid.gov.uk/r4d/PDF/Outputs/ResearchIntoUse/AHP14.pdf 2. European Union Statistics Department (2011) Research and Development Expenditure 3. GALVmed, “GALVmed and East Coast fever”, GALVmed.org, http://www.galvmed.org/activities/east-coast-fever [accessed 12 February, 2013] 4. “Fighting East Coast fever - Lessons from Maasailand”, New Agriculturist. http://www.new￾ag.info/en/developments/devItem.php?a=1709International Fund for Agricultural Development (2005) A Review of Kenyan Agriculture 5. Journal of Agricultural and Development Economics (FAO) (2005) The Maize Green Revolution in Kenya Revisited 6. Kenya Agricultural Research Institute (2012) Integrated Agricultural Research and Development: Enhancing Food and Nutrition Security Through Increased Production and Agribusiness. Presented to USAID 7. Government of Kenya (2012)The Kenya Agricultural Research Bill 8. Kenya Agricultural Research Institute (2009) Strategic Plan and Implementation Framework Kenya Agricultural Research Institute (2010) Annual Performance Report, September 2009 - 2010 9. Kenya Agricultural Research Institute (2009) Annual Performance Report, September 2008 – 2009 10. Kenya Agricultural Research Institute (2007) Annual Performance Report, September 2006 - 2007 11. Kenya Agricultural Research Institute (2003) Increased Rural Household Incomes; Three Year Proposal. 12. Kenya Agricultural Research Institute (March 2012) “Integrated Agricultural Research and Development: Enhancing Food and Nutrition Security Through Increased Production and Agribusiness”. Presented to USAID. 13. Kenya Producers Coalition & Centre for Governance and Development (2010) Resource Allocation and Utilization in Kenya’s Agriculture Sector : A Public Expenditure Review 14. Kibaara, Betty et. al., “Trends in Kenyan Agricultural Productivity: 1997-2007”, Tegemeo Institute of Agricultural Policy and Development. 15. Mbakaya, David S., J. R. Okalebo, M. Muricho and S. Lumasayi, “Effects Of Liming And Inorganic Fertilizers On Maize Yield In Kakamega North And Ugunja Districts, Western Kenya”, KARI. 16. Ministry of Agriculture Statistics Department (2011) 17. Mukhebi, A.W., et al, “Cost analysis of immunization for East Coast fever by the infection and treatment method”, Preventive Veterinary Medicine, 9 (1990) 207-219. 18. National Agricultural Research Policy (July 2012) Agricultural Sector Coordination Unit 19. Nambiro E, H de Groote & W Kosura (2001) Market Structure and Conduct of the Hybrid Seed Industry 20. Tegemeo Institute (2006) Agricultural Extension in Kenya: Practice and Policy Lessons and (2008) Agricultural Policy-Making in Sub Saharan Africa: Kenya’s Past Policies with data gained from Public Expenditure Review 2005. 21. Tegemeo Institute (2006) Agricultural Extension in Kenya: Practice and Policy Lessons 22. Tegemeo Institute (2004) Competitiveness of Kenyan and Ugandan Maize Production: Challenges for the Future 23. USAID Kenya (June 2012) Request for Proposal (RFP) # SOL-615-12-000006 Kenya Agricultural 37 Value Chain Enterprises (KAVES) Project 24. USAID Kenya (2011) Feed the Future FY 2011–2015 Multi-Year Strategy 25. USAID (2004) End of Term Evaluation of the Agribusiness Development Support (ADSP) Project 26. USAID (2004) End of Term Evaluation of the Agribusiness Development Support (ADSP) Project 27. W. Peterson, G. Gijsbers, and M. Wilks, An Organizational Performance Assessment System for Agricultural Research Organizations: Concepts, Methods, and Procedures, ISNAR Research Management Guidelines No. 7. The Hague: International Service for National Agricultural Research, 2003. 28. World Bank (2009) Kenya Agricultural Policy Review: Current Trends and Future Options for Pro-Poor Agricultural Growth 29. World Bank (2008) Project Performance Assessment Report. National Agricultural Research Project-Phase II, with this data originally taken from Ministry of Agriculture(2006) Public Expenditure Review. 30. World Bank (2008) Project Performance Assessment Report. National Agricultural Research Project-Phase II 31. World Bank / (2007) Review of Public Spending to Agriculture: a Joint Study 32. World Bank (2004) Implementation Completion Report. Second National Agricultural Research Project (NARP II) . 38 ANNEX 3: PEOPLE INTERVIEWED Name Division Title Email/Cellphone Contacts Interview period KARI STAFF MEMBERS INTERVIEWED Dr.Ephraim Mukisira KARI HQTs Director, KARI eamukisira@kari.org / director@kari.org 0733621729 December 2012 Dr. Joseph Gichane Mureithi KARI HQTs Deputy Director Research & Technology, KARI jgmureithi@kari.org/ mureithijg@gmail.com 0722830308 December 2012 Dr. Andrew Mailu KARI/USAID Programs Program Coordinator ammailu@kari.org 0720876134 December 2012 January 2013 February 2013 Ms. Jayne Gathii KARI/USAID Programs Program Manager jgathii@kari.org 0722788669 December 2012 January 2013 February 2013 Dr. Joseph Ochieng KARI HQTs Assistant Director, Staple Crops jawochieng@kari.org 0721548243 December 2012 February 2013 Dr. Stephen Kimani KARI Muguga Research Officer, Natural Research Management 2skimani@gmail.com 0729312472 December 2012 Mr. Samuel N. Sinkeet KARI HQTs Program Officer, Livestock snolesinkeet@kari.org 0721980101 December 2012 Dr. Simon Gichuki KARI NARL-Biotech Head of Biotechnology Centre stgichu@yahoo.co.uk 0722813687 December 2012 Dr Soi KARI NARL-Biotech Principal Scientist December 2012 February 2013 Dr. Monicah Waiganjo KARI-Thika Entomologist-Deputy Centre Director monicahwaiganjo@yahoo.com 0733595182 December 2012 February 2013 David Mbakaya KARI Kakamega Soil Scientist dsmbakaya@yahoo.com 0721917583/ 0735448336 January 2013 Dr Dickson Libeyo KARI Kitale Principal Research Officer dicksonlibeyo@yahoo.com January 2013 Henry Ngesa KARI Kibos Plant Breeder hngesa@yahoo.com January 2013 39 Cheran Ariithi KARI Katumani Outreach Officer cariithi@yahoo.com December 2012 Dr. Charles W. Kariuki KARI Machakos Centre Director￾KARI Katumani Research Centre cwkariuki@jambo.co.ke 0722830575 December 2012 Dr. David Miano Maingi KARI-HQs Assistant director Animal Production Research/ National Coordinator ASAL dmmwangi@kari.org/ kasalkenya@gmail.com 0727781127/ 020 3568991 January 2013 KARI Biotechnology Centre [Trained Officers] Catherine Taracha Anderson Wambugu John Kemboi Sitienei John Irungu USAID Trained Officers under SO7 December 2012 February 2013 SEED COMPANY STAKEHOLDERS INTERVEIWED Dr. Yash Bhargava East African Seed Co. Ltd Head, Research & Development yash@easeed.com/ info@easeed.com 0718983033 December 2012 Jesse Onsando East African Seed Co. Ltd Business Development Manager onsando@easeed.com 0734333161/ 0722701610 December 2012 Saleem Esmail Western Seed Company Ltd CEO& Breeder ceo@westernseedcompany.com 0722514236 January 2013 Samuel Gicheru SIMLAW-NRB Asst. Marketing Officer-Seed Shop samuelgacheru@yahoo.com 0734811861/ 0721425602 January 2013 Dryland Seed Limited Kimotho Ngila Director info@drylandseed.com 254-44-21449/722829287 December 2012 GOVERNMENT INSTITUTIONS Ministry of Agriculture Dr Johnson Irungu Director of Crops irungu_waithaka@yahoo.co.uk December 2012 February 2013 Clement C. Muyesu Ministry of Agriculture Assistant Director of Agriculture muyclem@yahoo.com 0733888844 January 2013 ASCU – Ministry of Agriculture Dorcus Mwakoi Technical Officer dmwakoi@ascu.go.ke January 2013 Mr. Abed Kagunda Mathagu KEPHIS Head, Biosafety & Phytosanisary Services akagundu@kephis.org /director@kephis.org 0721354269 January 2013 40 Dr. Esther Kimani KEPHIS General Manager, Phytosanitary Services ekimani@kephis.org / ekimaniw@gmail.com 0722516221/ 0733874274 January 2013 DEVELOPMENT ORGANIZATIONS INTERVIEWED Dr. Paul Seward FIPS-Africa Managing Director fipsafrica@yahoo.com 0724700007 January 2013 Dr. Erick G. Mworia Kenya Red Cross Food Security Manager Mworia.eric@kenyaredcross.org /erickgiri@yahoo.co.uk 0703364565 January 2013 Antony Kioko Cereal Growers Association Deputy Director akioko@cga.co.ke January 2013 Seed Traders Association of Kenya - STAK Dr Evans Sikinyi Executive Director esikinyi@stak.or.ke December 2012 Farming Inputs Promotion Services FIPS Dr Paul Steward Dr David Priest Director Technical Officer fipsafrica@yahoo.com 0724700007 January 2013 Kenya Maize Development Program KMDP Steve Collins Director webmaster@acdivoca.org January 2013 International Livestock Research Institute ILRI Phil Toye 0725546883 December 2012 41 ANNEX 4: SUMMARY OF SEED COMPANY INTERVIEW RESULTS Interviews were conducted with five commercial seed companies  Kenya Seed Company – publically owned (Kenyan)– breeds and sells seeds (controls about 70% of seed market in Kenya and produces averagely 30,000 tons of maize seed.)  East Africa Seed Company – privately owned (Kenyan)– breeds and sells seeds (produces about 5,000 tons of maize seed annually and heavily relies on the foundation seed – breeder seed supplied by KARI though it also has its own breeders.)  Western Seed Company – privately owned (Kenyan) – breeds and sells seeds (produces about 3,000 tons of maize seed annually entirely from its own breeders.)  Dryland Seed Company – privately owned (Kenyan)– breeds and sells seeds (produces seed both from commercialized technologies including hybrids and non-commercialized technologies including OPV and THVC seeds. It’s a new company.)  Monsanto – privately owned (foreign-owned) - breeds and sells seed. Of these:  3 described their relationships with KARI as being excellent  2 identified areas where they hoped to collaborate with KARI in the future, e.g., transgenic seeds  1 had a problem with the amount/timing of a seed delivery from KARI  1 had a problem with the royalty payments KARI requires  1 had a problem with KARI’s direct sale of seeds and thought that KARI should not compete with the private sector With respect to KARI’s research output:  3 said that KARI ‘s maize research program has overemphasizes highland varieties relative to mid-altitude where there is a great need for new/better varieties – mid altitude regions reportedly presents an opportunity for the country to meet food security challenges if a new frontier is opened through new seed for these regions that are vast in acreage  1 said that while it was a breeder, KARI was ahead on breeding highland varieties and it was buying two of them two from KARI  1 said that a more consistent stream of new releases is needed 42 ANNEX 5: AGRO-SURVEY – METHODOLOGY Overview Scope: a quantitative survey of 200 Agro-Dealers and Agro-Vets with electronic data capture. Geography: focus on Kitale, Highlands, Central Highlands and midlands regions; possibly other regions with high agricultural activity. Objective: ‘Market-test’ the degree of diffusion and scale-up of three KARI innovations from among the six components of the project that have actually been released to the market with the potential to be scaled up by market forces. Selected technologies: three technologies were selected for the survey in a meeting between MSI and KARI on 4 December 2012. On 3 December 2012, USAID had accepted this approach of having KARI propose the three technologies. 1. Maize variety KH 600 15A, introduced to the market in 2008 and sold by East African Seed, mainly in Highlands and Kitale areas. The maize is for human and animal consumption. There are about 500-1,000 stockists, according to East African Seed. 2. Maize variety KH 500 43A, introduced to the market in 2008 and sold by East African Seed, mainly in Central Highlands and Midlands areas. The maize is for human consumption, leaf and stem are animal fodder. There are about 500-1,000 stockists, according to East African Seed. 3. Fertilizer (NPK) and soil nutrient (calcium, sulphur, lime) is sold all over Kenya since 2005, under the Mavuno Fertilizer brand name by Athi River Mining. The majority of the 200+ dealers and retailers of Mavuno are found in Nairobi and the Mount Kenya, North-Rift, and South Rift regions. Sampling: Target regions for the survey are Nairobi, South Rift (including Kitale) and Mt. Kenya /Highlands. Despite the relatively high incidence of outlets in Nairobi, we propose to split a sample of 140 outlets as follows: 42 (30%) in Nairobi, 49 (35%) in South Rift and 49 (35%) in Mt. Kenya/Highlands. These outlets will be located in 3-5 different locations within each of the 3 regions (so 9-15 locations in total), depending on actual incidence of AD/AVs. Locations will be sampled randomly and from there survey implementers will snowball to other outlets to manage the expected low incidence of outlets. We propose to conduct the remaining 60 interviews with AD/AVs of whom the producers of the technologies have advised us that they are selling the technologies. These will be purposely sampled from the producer’s client lists. We aim for at least 20 outlets per technology with a balanced regional distribution (pending a list of retailers for the Maize varieties). The first sample of 140 will answer questions, such as market awareness, general availability, substitutes, unexpected shocks, and more. The second sample of 60 will tell us how reliable the producer information is and provide feedback from outlets as to whether the targeted product it is at least known and if they have sold the product in the past or are currently selling it. Workplan A kick-off meeting between Research Solutions Africa (RSA) and MSI, as well as a kick-off meeting between USAID, MSI and RSA were held on 3 December 2012. During these meetings and subsequent communications, the following timelines were agreed upon: 6 December: Submission of RSA inception report contribution and proposed questionnaire outline (pending detailed information on the three selected technologies 8 December: Submission of revised outline questionnaire (paper version) for comments 10 December: Finalization of paper questionnaire 43 11 December: Small field trial of the questionnaire in or near Nairobi 12 December: Public holiday; approval by MSI of revised survey instrument 13 December: Programming of the (revised) final questionnaire and device testing 14 December: Training of enumerators 15 December: Field trial (half day) 17 December: Survey kick-off 17-19 December: Daily analysis by MSI of emerging data, in partnership with RSA supervisors 20 December: Survey complete 22 December: Clean data submission. The enumerators and team leaders are recruited from the RSA database and have a long standing relationship with RSA. They have performed to RSA’s full satisfaction on similar projects. The team will be gender balanced. The remainder of the team is full-time staff of RSA. The training will be facilitated by RSA’s research and field teams at the RSA offices, with MSI staff observing the training. The training agenda includes a general introduction to the project, an interactive discussion of the questionnaire, instructions on sampling, instructions on quality control measures, details on logistics and payment, role plays, as well as a short field trial. We expect to work with 15 enumerators and 3 team leaders to ensure a completion of the survey well within the deadline. The relative sizes of teams are to be finalized once the sampling regions are finalized and the first trial is completed on 11 December. The teams will meet each morning with their team leader to plan the routing for the day. The data will be collected on Hand Held Devices (Huawei Ideos smart phones) with dedicated survey collection software installed. RSA will program and test the software before commencing training. The software will capture the time and GPS location for each interview, as well as an image of each outlet in addition to the answers to the survey questions. After each interview (connection permitting), the interviewer will send the collected data to project servers. When the teams encounter any issues, they will promptly liaise with the team leaders. Otherwise, the teams meet again in the evening to hand over the phones to the team leader for a cross-check that all interviews have been sent and take care of recharging the devices overnight. We carry spare batteries for phones discharging during the survey operation. RSA will share emerging data with MSI each day and report any problems and propose solutions to address them. A field supervisor will ensure efficient logistics and resolve any delaying issues. The GPS co-ordinates will be projected onto a map after the survey as an overview of the sampling locations. Quality control On Day 1 of the survey, RSA will continuously monitor electronically the incoming data for any arising issues. On the other days, RSA will download the previous day’s collected data early in the morning for progress and quality analysis. First, RSA will check that it has received data from all phones in field (and therefore the teams were complete). RSA will check the number of interviews per enumerator and promptly flag any obvious quality issues. All issues raised at this stage are relayed by phone to the responsible team leader to share with the specific enumerators before they deploy to field in the 44 morning. RSA then analyzes the data in more detail to spot any other quality issues or suspicious data that will need correction. In addition to its online analysis of progress, work ethic (e.g. time spent in field), diligence and quality, RSA will also employ proactive field quality control measures. The team leader will accompany a portion of each enumerator’s visits in order to observe whether the interview is applied as it should. In addition, the team leaders will back-check a proportion of the outlets visited to cross-check the answers given. Questionnaire Below is the first outline of the questionnaire. Section 1: Respondent and outlet details Purpose or random sample (see above); name of respondent; sex of respondent; phone number of the respondent; name of AgroDealer; and location of AgroDealer. Does the outlet sell the following product categories: Fertilizers, seed varieties, vet supplies, machinery, non-farming supplies (multiple choice, multiple option)? Section 2: Product range 1. Which maize seed brands do you sell? (multiple choice) 2. Which is your bestselling maize seed brand? (multiple choice) 3. What share of your maize seed sales does your bestselling brand account for? (%) 4. What is your bestselling maize seed variety? (open answer) 5. What is your second bestselling maize seed variety? (open answer) 6. Which fertilizer brands do you sell? (multiple choice) 7. Which is your bestselling fertilizer brand? (multiple choice) 8. What share of your fertilizer sales does your bestselling brand account for? (%) 9. What is your bestselling fertilizer variety? (open answer) 10. What is your second best-selling fertilizer variety? (open answer) Section 3: Awareness and presence of KARI varieties 1. Are you aware of any of the following technologies: Maize KH 600 15A, Maize KH 500 43A, Mavuno Fertilizer? (multiple choice, multiple options) 2. For those that the respondent is aware of: Have you ever stocked any of these technologies? (multiple choice, multiple options) 3. For those that the respondent has stocked [all questions will first be asked about technology A, then about B, then about C, if affirmative above]: 4. When have you started to stock (technology A, B, C)? (4 digit answer – year) 5. Are you still stocking (technology A, B, C)? (yes/no) 6. If no, why have you stopped stocking (technology A, B, C)? (multiple choice: e.g. low demand, price, quality, availability, superior products available, subsidized distributors, others – specify) 7. If yes, why do you stock (technology A, B, C)? (multiple choice: e.g. low demand, price, quality, availability, no superior products available, good margins, others – specify) 8. In what package sizes do you offer (technology A, B, C)? (multiple choice [if information on available package sizes is available]) 9. How much of (technology A, B, C) have you sold during the past 12 months? (Estimate volume by weight in kg) 10. Do you currently have (technology A, B, C) in stock? (yes/no) 11. For those in stock, which volume of (technology A, B, C) do you have in stock? (volume by weight) 12. What is your retail price per [determine practical unit] for (technology A, B, C)? 45 13. Where do you buy (technology A, B, C) from? (Name of wholesaler, where possible.) 14. Where have you obtained information about (technology A, B, C) from? (multiple choice) For all that the respondent is aware of: 1. Are you aware of any non-market sources of the same product, e.g. government or NGO distributions? (yes/no) 2. Name these sources (open) 3. Are there any products that you consider superior to (technology A, B, C) or similar products that you sell higher volumes of? (yes/no) 4. Name these brands (open) [unless a list of substitutes is available for a multiple choice question] 5. Why are they superior? (Multiple choice) 6. Where do you source these alternatives from? (Name of wholesaler, where possible.) 7. Who do you sell to most of your products? (male/female) 8. Who do you think makes the decision in the household on which seed to buy? (male/female) 9. In addition: an outside photograph of the outlet, GPS location and time stamp for the interview, will occur. 46 ANNEX 6: AGRO-SURVEY – DESCRIPTIVE STATISTICS A. DEMOGRAPHIC DATA Responses by Gender Respondents Frequencies Percentage Males 112 50.7 Female 109 49.3 Total 221 100 Regions Frequencies Percentage Mt. Kenya & Highlands 65 28.8 Nairobi 66 29.2 South Rift+ Kitale 95 42 Total 226 100 Outlet Types Frequencies Percentage Retail 154 67.87 Wholesale 63 28.05 Both 7 4.08 Totals 226 100 Years Business Existence Frequencies Percentage 0-5 years 108 48.40 6-10 years 52 22.80 11-15 years 36 16 16-20 years 13 5.50 >20 years 17 7.30 Totals 226 100 Clients Served Frequencies Percentages Subsistence Farmers 221 100 Institutions 40 17.84 Big Commercials 53 24.43 Other agrovets 73 33.33 NGO 18 8.45 Companies 13 5.63 Cooperatives 2 10.32 47 B. RESPONSES TO SURVEY QUESTIONS Stocks Frequencies Percentage Seed Varieties 108 48 Veterinary Supplies 154 67.90 Fertilizers 196 86.43 Machinery & Equipment 54 23.98 Stockists of Maize Seeds Frequencies Percentage Yes 197 89 No 24 11 Totals 221 100 SEED COMPANY SUPPLIES Seed Company Types/Variety Frequencies Percentage Proportion 1 East Africa Seed Co. 44 21.7% 2 Kenya Seed Co. 141 71.0% 3 Syngenta 19 9.5% 4 Duma 26 12.7% 5 Pioneer 28 13.6% 6 Seedco 30 15.4% 7 Dekalb 18 9.0% 8 Monsanto 9 4.5% 9 Western Seed Co. 8 3.6% 10 Olerai Co. 10 5.0% 11 Leidet Co. 8 3.6% 12 Pannar 34 16.7% 13 Elgon Kenya Ltd 3 1.3% 14 Simlaw Seeds 6 3.2% 15 KARI seeds 4 1.8% 16 Farm Chem 2 0.9% 17 Fresh Co. 16 7.7% 18 Others (Oil crop, Agriseed, Griffton etc) 9 4.1% FERTILIZER FOR PLANTING TRADED Fertilizer Type Frequencies Percentage Proportion 1 Chapa Meli DAP 96 45.0 2 Chapa Meli NPK 5 3.0 3 MEA DAP 91 43.0 4 Ruiru DAP 4 2.0 5 Turbo DAP 4 2.0 6 DML 1 0.67 7 MAVUNO DAP 9 4.0 8 Others 1 0.33 48 Total 212 100 Fertilizer Types Fertilizer Type Frequencies Percentage Proportion 1 Chapa Meli CAN 89 41.4 2 Chapa Meli Urea 16 7.4 3 MEA CAN 76 35.2 4 MEA Urea 1 1.9 5 Ruiru CAN 1 1.9 6 CAN Turbo 1 1.9 7 MAVUNO CAN 9 4.3 8 Others 13 6.0 Totals 206 100 KARI TECHNOLOGICAL UPTAKE Have you ever stocked any of these technologies? Technology Type Proportions Percentage Tec A 56 31.5 Tec B 39 21.9 Tec C 83 46.6 Total 178 100 Current Stockings of Kari Technology Technology Sold Stocklists Percentage Tec A 30 13.30 Tec B 17 7.50 Tec C 40 17.70 Missing 139 61.50 226 100 Characteristics of Stock Lists Varieties Stocked Frequencies Percent 1 MAIZE KH 500 43A 3 1.3% 2 MAIZE KH 500 43A, MAVUNO FERTILIZER 4 1.7% 3 MAIZE KH 600 15A 14 6.2% 4 MAIZE KH 600 15A, MAIZE KH 500 43A 11 4.9% 5 MAIZE KH 600 15A, MAIZE KH 500 43A, MAVUNO FERTILIZER 20 8.8% 6 MAIZE KH 600 15A, MAVUNO FERTILIZER 9 4.0% 7 MAVUNO FERTILIZER 49 21.7% 8 MAIZE KH 600 15A, MAVUNO FERTILIZER, 1 0.4% 9 MAIZE KH 600 15A,MAIZE KH 500 43A 1 0.4% 49 10 Never stocked any. 114 50.6% Grand Total 226 100% Buyers of Products by Gender Frequencies Percentage Males 167 76 Females 54 24 Totals 221 100 Who makes decision in the HH to purchase Seeds? Frequencies Percentage Males 118 53.5 Females 36 16.2 Both 67 30.3 Totals 221 100 Technology Combinations Frequencies Percent A 8 3 A,B 28 12 A, B,C 58 24 A,C 21 9 B 7 3 B,C 6 3 C 63 26 Not Aware 47 20 Totals 238 100 Information Sources on Seeds Frequencies Percentage Suppliers/ dealers 13 12.4 Other Agrovets 10 9.5 Customers/Farmers 21 20 Sales people by EASEED 32 30.5 Media 8 7.6 KARI/KARI leaflets/posters 9 8.6 ASK show/Field Days 5 4.6 KEPHIS 1 0.9 Agricultural Office 1 0.9 KFA 2 2 Other sources 3 3 Total 105 100 Sources of the Information Frequency Percentage 50 Suppliers/ dealers 15 12.4 Other Agrovets 12 9.5 Customers/Farmers 25 20 Sales people by EASEED 38 30.5 Media 10 7.6 KARI/KARI leaflets/posters 11 8.6 ASK show/Field Days 6 4.8 KEPHIS 1 0.9 Agricultural Office 1 0.9 KFA 2 2 Other sources 4 3 Total 127 100% Reasons for Current Stocking Percentage High Demand 28 29% High Quality 14 20.3% Readily Available 12 17.4% No better alternative/ Highly competitive 4 5.8% Convenient Packaging 3 4.3% Highly Profitability 10 14.5% Low Cost 4 5.8% Shortage of other varieties Kenya Seed H614 2 2.9% Total 69 100% Reasons for Current Stocking of Tec B Frequency Percentage High Rainfall 9 25% Shortage of Kenya H513 4 11.1% Great yield 2 5.5% High Demand 8 22.2% New Product thus Customers Enthusiastic to try 4 11.1% Adaptive to Weather/ short rains 7 19.4% Faster Maturity 1 2.8% Readily Available 1 2.8% Total 36 100% Superior Products or Similar to A/B that fetch High Sales Seed Type Frequency Percentage Kenya Seed H614/ six series 52 57.9% Kenya Seed H624 2 2.6% Duma 43 17 19.4% Kenya Seed HB6213 2 2.6% Dekalb DK8031 2 2.6% Kenya Seed H516 6 5.3% 51 DH01 3 2.8% Pannar 3 2.8% Freshco 3 2.8% Monsanto 3 2.8% Total 93 100% 52 ANNEX 6: PARTICIPATORY STAKEHOLDER MEETING SUMMARY & PARTICIPANTS Participatory Meeting of Stakeholders over KARI’s Agricultural Research Supported by USAID --------------------------------------------------------------------------------------------------------------------- 15 January 2013, Management Systems International (MSI) Office, Westlands, Nairobi Workshop facilitator: Dr. John Recha Introduction The workshop took 4 hours (9:00 am – 1:00 pm) and began with welcome remarks from John Recha who invited Ami Henson (USAID Support Project Chief of Party) to give an opening statement. This was followed by introduction of the participants. Nick Shirra (evaluation team leader) then gave a presentation titled “Developing the Agri- Research Sector”, that led into a participant question and answering session. Summary of the Participant Question and Answering Session In response to a question on the implication of agricultural research reforms being undertaken in Kenya, the KARI Director discussed various issues as followed: - Following a consultative process, KARI will be transformed into the Kenya Agricultural Research Organization after the Agricultural Research Bill was signed into law by the president of Kenya on 14th January 2013. - Under KARO, a total of 10 research institutions will be created. These are Food Crops , Livestock, Horticultural, Industrial Crops, ASAL, Biotech, Water, Gene Bank, Forest, and Fisheries Research Institutes - KARO will be headed by a Director general, with a governing body that will have representatives from farmers, universities, the private sector, government and international community - The 10 research institutes will be headed by Directors and each will have an advisory body - Adaptive Research centres will be established to undertake activities that link up the 10 KARO research institutions. Astrong linkage with extension will be encouraged to increase technology adoption - KARO’s structure aims at strengthening agricultural research and bring about results more results oriented agricultural technology production - Other Issues to clarify - KARI had created departments that would drive research. These include Outreach and Partnerships, Technology Transfer, and Information Management divisions - A need to invest in ICT had been identified and investment in more ICT was on the way, that would cost 300 million shillings - The KARI Seed Unit (KSU) is growing strong, and will be boosted by the newly created Agricultural Research and Development Fund. The fund will help capture patents, rights/royalties, and competitive grants. The Fund is run by an assistant director and programme officer - Capacity building for agri-business department and other departments is key and 6 agri-business officers will be hired to strengthen the unit. This is an opening to USAID to provide input and as a key partner in assisting KARI manage this challenge - In order to link with industry, the managing Director of Kenya Seed Company now sits on the board of KARI, and likewise the KARI Director sits on the board of Kenya seed Company Participants from the seed industry raised the following concerns: 53 - There is a need to start doing business in a different way. For example, elsewhere in the world, national agricultural research organizations work with the private sector. The KARI business format will not work well if there are no joint ventures with international research organizations that undertake cutting edge science, and the private sector (e.g. seed industry) - The government of Kenya shouldn’t interfere in the seed sector the way it does now. For example they take critical germplasm and dish it out to farmers under the seed distribution program - KARI should be more efficient in production and maintenance of research materials - Researchers who join KARI end up as administrators thus diminishing research - Maize varieties take about 7 years in the market before a new variety appears, and there is a need to allow the new improved varieties to thrive for the benefit of the farmers. For example the H610 series has now been overtaken by H620 series that have an ability to double the yield. Yet the H620 series is not being promoted - Important to work on new germplasm that can double yield and are disease resistant - The Kenya Seed Company is a giant that controls the seed industry in Kenya and the region. The Kenyan government owns nearly 60% of Kenya Seed Company and having the Kenya Seed Company MD in the KARI board will not inject any new ideas and Kenya Seed Company will use the opportunity to strangle the other seed companies. There is a need to integrate into the KARI board, private sector players for real changes to be seen - KARI also has opportunity to work with privates sector to produce breeder/foundation seed for the orphan crops (now called traditional high value crops) program which can become well commercialized once it picks up Participants from government institutions (KARI and KEPHIS) raised the following issues: - KARI Seed Unit is detached from the farmers - There is need for more funds to help KARI undertake research demonstrations and funds for more breeder seed multiplication - The KARI Seed Unit would wish that next project provides resources for maintaining the current seed varieties before moving on to new research priorities - There is a need to roll out high quality seeds faster (e.g. 1991 breeders seed is being produced now (year 2013) after 22 years), and maintain high quality seed already produced - There is a need to have linkage between the researchers and industry - Seed production co-ordination by KARI is poor and needs to be strengthened - Kenya Seed Company return to the KARI board will ‘swallow’ the rest of the industry players, leading to low quality and expensive seed. It is the right time now to delink KARI from Kenya Seed Participants from the Ministries of Agriculture (MoA) and Livestock Development (MoLD) gave the following input: - MoA is one of the biggest buyers of the KARI seed since 2006. The Ministry pays KARI Seed Unit for orphaned crop (now called traditional high value crops) seed, that have been grown and are more visible now especially in ASALs - MoA program on enhancing agricultural productivity was introduced in 2010/11 in collaboration with 4 seed companies and 4 KARI research centres - The different seed companies are now producing seed for traditional high value crops, while the 4 KARI centres are bulking the seed - Over time these seed have resulted into production of more food - There is need to support seed bulking by farmer groups. These farmer groups need to be certified by KEPHIS Challenges were raised by participants from the KARI-USAID coordination office a listed: 54 - There has been very little private sector involvement in KARI coordination and research activities. What are the reasons behind this state of affairs? - What is the extent to which KARI involves private sector players in their research activities? For example when a variety is developed, what is the consultation level and active involvement of the private sector? - There is a feeling that KARI has just at about only 30% involvement with private sector. How does this change in order to enable for example seed companies to make profitable ventures that will benefit KARI? - Why are old crop varieties still popular yet KARI has many new varieties that have not been fully taken up? A participant mentioned that the government of Kenya hardly puts sufficient funds to research programmes and should step up research funding to be complimented by development partners Break-out into two groups then occurred to discuss ‘how can KARI better work with the private sector’ Summary from the Maize Group In the breakout session, the maize groups had the following list of ideas:  Strengthen KARI variety maintenance programme to meet the private sector demands  Strengthen the government’s capacity on promotion, dissemination and licensing of new technologies e.g. it takes a month for the private sector to release a technology into the market while it takes KARI 5 years to do the same  KARI to delink itself from being a seed merchant to being a seed breeder only. Then it needs to put in place an efficient mechanisms to collect royalties from the seed companies  KARI need to prioritize research needs eg LGB and on Yellow maize as livestock feeder and fast track variety release and registration  Breeders need more funding and flexibility to multiply, therefore needs support of donors  USAID should commission a survey on adaptation of the new technologies released by KARI and compare with technologies released by other organizations  KARI should re-evaluate variety licensing. Accordingly, the seed companies should have a period in which to market new technologies from KARI. When the time frame expires, then the technology should be withdrawn. Royalties collection needs to improve in KARI.  Promote yellow maize and produce suitable varieties. Kenya Seed Company doesn’t have yellow maize, its therefore an opportunity for KARI to work with private sector to initiate this. Yellow maize can also be used by livestock industry whose demand is ever growing  The emerging seed industry challenge now is to getting big pieces of land to produce the certified seeds. The current production is tied to pieces of land owned by KARI & ADC and doesn’t meet the demand. Irrigated land can be used, but this is more costly and it is only suitable for producing high value crop seeds  KEPHIS – KARI partnership cannot work with the existing seed production work by KARI. KEPHIS is a regulator of all seed companies including KARI Seed Unit. If KARI release breeder seed to the private sector (to pick up from there), it will be easy to regulate and get high quality produce.  Therefore it will be better for the KARI Seed Unit to handle breeder seed only and to maintain. A review of KSU needs to occur to improve efficiency. A KEPHIS representative reported that KSU was registered as a seed company able to sell seed.  Newcastle Vaccine also needs to be better commercialised by KARI Summary from the Other Staple Group The staple identified were: 55 - Sorghum - Pearl millet - Finger millet - Rice - Wheat - Legumes – peas, cow peas, green grams, pigeon peas, dolichos, - Irish potato - Sweet potato - Cassava There is a need to focus on the key crops. The following ideas were discussed by the group - Important to link their production in semi arid areas; therefore a need for suitable varieties - Wheat and rice are already well catered for by other institutions in terms of research support and development - ABVs value chain has attempted to prioritize the various crops - The MoA is moving out of millet and green grams seed distribution because the private sector is taking up commercialization of the seed - According to Dryland Seed Company Ltd, most farmers are planting sorghum more than any other variety due to demand from Kenya Breweries Ltd; green grams are also in high demand from most hotels; cow pea which is also in high demanded but from households - Staple crops like sorghum and millet need to attention from nutritionists/food technologists to improve popularise their nutritive value and how the foods can be prepared for meals - It is important for KARI to increase demonstration plots for these crops to increase awareness Therefore priorities should be put on other staple that are: - Early maturing/drought tolerant. The current market leaders include sorghum, green grams and cow pea - Highly nutritious. They can have value addition and be marketed aggressively. These applies to -Millets- pearl, finger millet etc -Potatoes, sweet potato, cassava -Other crops including rice and wheat The group attempted to answer the question “At what stage should private sector and KARI engage”? In response, a group member mentioned that it is important to look at the agriculture sector value chain. It is important to identify the key stakeholders including the private sector, which should be propped in finding out the right crops that are key in value chains? Developing a business strategy for KARI is key in commercialization in terms of business orientation of the organization as this will enhance visibility. The emerging cross cutting issues on the other staple crops were: - Agriculture Product Value Chain; there is a need to identify the entry points - Capacity building needed within the value chains – e.g. Newcastle vaccine used by groups at the farm level; ration formulation etc; of KARI in business strategy - Commercialization is important as soon as possible - Engage the private sector at the initial stage of technology generation - Promotion and outreach of technologies is key in adoption and use - Awareness creation on highly nutritious of crops is needed - Integrated crop-livestock production system to optimize production decisions - Improving existing varieties is a key pillar for KARI in tandem with private sector needs. Commercialization of KARI technologies to private sector should be an outlook to consider, and communication channels need to be opened 56 - USAID should also support promotion of technologies for private sector to know the existence of such technologies. Awareness about technologies helps sort certain marketing challenges - KEPHIS Improving existing varieties should be key to KARI in tandem with private sector. Commercialization of these varieties in conjunction with PS should also be key. - MOA: is promoting commercially oriented agriculture for smallholder farmers. Farmer groups can be incubated at an Farmers Taining Centres to become commercially oriented in the production of value added products - To bring out the nutrition and health fact on these corps, partnerships among stakeholders is needed - A strategy for better commercialization needs to be put in place Emerging Issues from the Plenary Presentation by the Two Groups - To improve the KARI Seed Unit. - There is a need for an incubation department within KARI. This should be linked to specific areas within the private sector and in that way, KARI can commercialize its emerging technologies. - KARI has technology innovation units in 12 centres. These are one-stop shops where one can access information on research outputs. There is overwhelming demand for KARI technologies and these units are also being used to train farmers and provide them with information. The centres will be used to engage the private sector - MoA sometimes assumes a market and then produces large quantities and the market is not always present. 57 List of Workshop Participants Name Organization Contact cellphone Email address Mary Karanja Ministry of Agriculture Traditional High Value Crops 0722 254986 marykaranja@yahoo.com Abed Kagundu Kenya Plant Health Inspectorate Service 0721 354269 akagundu@kephis.org Ian Chesterman USAID-KHCP 0732 761764 ichesterman@fiatrac.com Patrick Maluku Kenya Plant Health Inspectorate Service 0720 242368 pmaluku@kephis.org Charles Bett Kenya Agricultural Research Institute (KARI), Katumani Station 0713 241421 cbet_chembett@yahoo.co.uk Lawrence Mragwa KARI Headquarters 0717 111022 Lfragwa2011@gmail.com Stepehn Kimani KARI 0729312472 2skimani@gmail.com Jayne Gathii KARI-USAID program 0722 788669 jgathii@kari.org Angela Wokabi Minstry of Livestock Development 0724710632 awokabi@gmail.com Milcah Mutisya Dryland Seed Ltd 0726781867 mutisyamunyiva@drylandseed.com Dickson Ligeyo KARI 0722 770108 ligeyootieno@yahoo.co.uk Ephraim Mukisira KARI 0726 987156 eamukisira@kari.org Catherine Taracha KARI 0722 931158 tarachac@gmail.com Andrew Mailu KARI-USAID Program 0720 876134 ammailu@kari.org Steve Collins ACDI-VOCA 0722 701104 scollins@acdivoca-kenya.or.ke Fred Opundo MSI 0722 572142 jopundo@msi-kenya.com Molly Hageboeck MSI DC 202 484 7170 mhageboeck@msi-inc.com Ami Henson USAID Support CoP ahenson@msi-kenya.com Nick Shirra MSI 0705 685380 n.shirra@gmail.com Felix M’mboyi MSI 0726 897102 director@cerdsconsult.com John Recha Facilitator 0721264936 John.recha@gmail.com