KENYA INNOVATION ENGINE (KIE) FINAL PERFORMANCE EVALUATION FINAL REPORT JUNE 2018 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, a Tetra Tech Company. KENYA INNOVATION ENGINE (KIE) FINAL PERFORMANCE EVALUATION FINAL REPORT June 21, 2018 IDIQ No. AID-623-I-12-00001 Award No: AID-615-TO-17-00006 Prepared by Management Systems International (MSI), A Tetra Tech Company 200 12th St South, Suite 1200 Arlington, VA, USA 22202 DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of the Management Systems International and do not necessarily reflect the views of USAID or the United States Government. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT iii CONTENTS Acronyms........................................................................................................................... iv Executive Summary ..........................................................................................................1 Project Background................................................................................................................... 1 Findings and Conclusions.......................................................................................................... 1 Recommendations..................................................................................................................... 5 Introduction........................................................................................................................8 Project Background................................................................................................................... 8 Purpose, Audience and Intended Use and Evaluation Questions....................................... 10 Evaluation Methodology, Design and Limitations ................................................................ 11 Findings and Conclusions ................................................................................................13 Question 1 ................................................................................................................................ 13 Question 2 ................................................................................................................................ 24 Question 3 ................................................................................................................................ 26 Question 4 ................................................................................................................................ 28 Question 5 ................................................................................................................................ 29 Recommendations...........................................................................................................30 Program Objectives, Design and Indicators ......................................................................... 30 Annex I: Bibliography......................................................................................................34 LIST OF TABLES Table 1: Stakeholders Interviewed ..............................................................................................................................11 Table 2: Counties and Innovations Visited ................................................................................................................12 Table 3: KIE Performance on Select Indicators........................................................................................................15 Table 4: FTF Indicator 4.5.2-5: Innovation Adoption..............................................................................................16 Table 5: Food Security, Sales and Job Creation Indicators....................................................................................17 Table 6: Innovation Companies Interviewed...........................................................................................................178 Table 7: KIE QIV FY2017 Quartery Report Indicator Results.............................................................................22 Table 8: Performance on FTF Indicators 4.5.2-38 and 4.5.2-12 ...........................................................................29 LIST OF FIGURES Figure 1: KIE Results Framework .................................................................................................................................. 9 Figure 2: Innovation Systems and Relationships to Indicators..............................................................................10 KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT iv ACRONYMS ACSU Agricultural Sector Coordination Unit AVCD Accelerated Value Chain Development ADPP Animal Draft Power Program AI Artificial Insemination ASAL Arid and Semi-Arid Lands BDS Business Development Services CGA Cereal Growers Association CIP International Potato Center COP Chief of Party COR Contracting Officer’s Representative DEC Development Experience Clearinghouse DIV Development Innovation Ventures FIRM Financial Inclusion for Rural Microenterprises FTF Feed the Future HHI Household Income HR1 High Rainfall Area 1 IAC Investment Advisory Committee IBLI Index-Based Livestock Insurance IP Implementing Partner IPM Integrated Pest Management ISC Innovation Screening Committee KAPAP Kenya Agricultural Productivity and Agribusiness Program KAVES Kenya Agricultural Value Chain Enterprises KENAFF Kenya National Farmers’ Federation KHCP Kenya Horticulture Competitiveness Project KIE Kenya Innovation Engine KIM Kenya Investment Mechanism KLMC Kenya Livestock Marketing Committee KSP Kenya Support Project LOL Land O’Lakes M&E Monitoring and Evaluation MoA Ministry of Agriculture MSI Management Systems International KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT v OEG Office of Economic Growth PMP Performance Management Plan PREG Partnership for Resilience and Economic Growth RA Rigorous Assessment SA1 Semi-Arid Area 1 SA2 Semi-Arid Area 2 SMEs Small and Medium-Sized Enterprises SOW Statement of Work STTA Short-Term Technical Assistance TA Technical Assistance TIA Takaful Insurance of Africa USAID United States Agency for International Development USG United States Government WAO Ward Agricultural Officers KENYA INNOVATION ENGINE: FINAL DRAFT EVALUATION REPORT 1 EXECUTIVE SUMMARY Project Background The Kenya Innovation Engine (KIE) was a five-year (2012–2017), $18,324,873 million program implemented by Land O’Lakes (LOL). An innovation incubator and business accelerator, KIE was designed to support private sector agriculture innovations that would increase productivity and incomes of smallholder farmers. Funded under Feed the Future (FTF), the increases in productivity and income were intended to have flow-down outcomes for improved food security and nutrition, and reduced poverty. KIE was designed as a business incubator of sorts, whose activities included providing grant funding and direct technical assistance (TA) to 191 entrepreneurs and other institutions with “innovative” agriculture products and services that had potential to be self-sustainable and scalable. KIE issued $4.2 million in grant funding and $2.7 million in technical assistance, with Stage 1 grants ranging from $30,000 and $100,000 and Stage II grants ranging between $300,000 and $500,000. Although KIE activities did not include direct interventions for food security, nutrition and poverty reduction, its original Performance Monitoring Plan (PMP) included FTF indicators for these development objectives. The evaluation will inform implementation of the upcoming Kenya Investment Mechanism project that is due to begin in early 2018. Findings and Conclusions The evaluation responded to five questions forming the core evaluation requirement: Question 1 To what extent did KIE achieve its goals and objectives? While KIE benefits were intended to flow down to smallholder farmers in areas with the greatest density of poverty to address FTF goals of food security and nutrition, KIE’s measurements of success were inherently contradictory: performance was measured by sales and growth of private sector companies to an impoverished population that had minimal funds, farms with economic potential, sufficient number of animals, access to markets, general education level and intellectual capacity to understand the cost/benefits of how innovators’ products could benefit them. As a result, innovators sold their products to farmers who were already doing relatively well in terms of food security and nutrition. Nonetheless, 33 percent of farmers interviewed felt that they had increased household food security and nutrition as a result of increased agricultural production or increased income from using KIE-supported innovation products. Sales of innovations were considerably stronger when innovators had personal networks and established reputations within farming communities (such as agro dealers and cooperatives), and with local governments. As cost was often cited as a reason for farmers not adopting innovation, it is possible that farmers simply do not understand how to calculate cost/benefit. One innovator observed that adoption was higher among farmers who had begun using another product sold by a KIE-supported innovator for farm recordkeeping, as they had a better understanding of the cost/benefit. The need to demonstrate 1 While KIE awarded 20 grants, two of these grants were awarded to the University of Nairobi. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 2 product/service cost/benefit and utility for near-term benefit in ways that farmers can understand was a recurring theme among stakeholders. While 16 innovations supported by KIE meet the established criteria of innovation and viability, three are clearly not innovations and were not based on viable business models: subsidized artificial insemination (AI) training at a government institute; importation of organic fertilizer; and farm recordkeeping. The “innovation”-related expenses of these grantees were minimal in comparison to their grant funding of $400,000 to $500,000. Funding at this level may be excessive, surpassing typical angel investments for U.S.- based entrepreneurial ventures. While six others supported by KIE do not meet the PIRS definition of “new technology,” five of those are relatively innovative for Kenya. Although KIE had systems to vet appropriateness of innovations to be supported, the only conclusion is that these systems were not adhered to, allowing support to some entrants that were not particularly innovative, desirable, sustainable or viable. Takaful, a firm that offers pastoral livestock insurance, stands out as meeting the criteria for innovation and viable business model. Although KIE operations and grants manuals reflect systems for proper vetting, the 2012–2016 Lessons Learned Report alludes to the contradiction of innovation with KIE’s highly iterative and amorphous process, and the highly documented and strict requirements of the program. Turnover of KIE staff from project inception to conclusion resulted in a loss of institutional knowledge as to why support was granted to products that did not meet innovation criteria described in the PIRS. Most innovations have had or currently have support from other development programs, either in grant form or other types of assistance. While no KIE-supported innovations appear to be catalytic game-changers for smallholder farmers, defined by the evaluation team as having provided strong and measurable results toward improved household income and food security, five of the 19 companies supported show signs of sustainability and viability and are contributing positively to agricultural incomes and productivity, and have succeeded in achieving adoption abroad: Amtech (Ethiopia, Rwanda), Kenya Biologics (Morocco, Kazakhstan), Takaful (Ethiopia, Somalia), iProcure and Real IPM. The common characteristics of these companies are a track record of business success or technical expertise, commitment to rural communities, hands-on field focus and innovations that offer near-term benefits to farmers in a way that they can understand. The remaining innovations supported are in various stages of continuing operations either through income or donor support, and three innovations (mentioned above) had ceased operations or were winding down. KIE fully expresses the inherent tension between pure private sector development and social development, as highlighted in one stakeholder’s comment that KIE’s design is incompatible with a thriving and innovative start-up ecosystem. The expectation for innovator companies to apply the same systems and processes for monitoring and evaluation (M&E), documentation and reporting as donor projects is not only unrealistic but also unwise. The nature of innovation and entrepreneurship is inherently risk￾taking and defies the bureaucratic process of monitoring. In fact, firms that are genuinely engaged in entrepreneurial endeavors cannot afford staff and activities that do not contribute to the innovation’s success. Conversely, entrepreneurial firms that are saddled with staffing, learning and processes to meet reporting requirements are likely to be distracted from their core business needs. Even though KIE grant funding amply covered the costs of additional staffing, grant funding of $400,000 to $500,000 may encourage wastefulness and attract “grantpreneurs” who have become experts in securing donor funding. While it is not possible to verify the accuracy of innovators’ reporting, the significant overachievement of targets, motivation to inflate numbers to achieve milestones that trigger payments and comments by stakeholders interviewed that numbers were inflated leads to the conclusion that reported target numbers are not accurate. The notion of innovator companies reporting on milestone achievements that in turn trigger funding reflects an inherent conflict of interest. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 3 Looking at the activity through a gender lens, the evaluation team found that KIE had female field representatives in Makueni, Machakos, Meru, Bomet and Nandi. Among innovations supported through KIE grants, three were led by women. However, KIE did not take specific measures toward FTF outcomes for social inclusivity of women and youth. For example, KIE sex-disaggregated performance data reflects a disproportionately low percentage of female beneficiaries (25 to 50 percent) compared with the high percentage of female farmers in Kenya (80 percent). Furthermore, KIE did not take steps to include women or youth as agents of change by encouraging their participation as innovators supported by KIE Question 22 To what extent did the program (USAID and the IP) adapt and change over time to meet the varying needs of smallholder farmers to achieve program objectives as they evolved over time? KIE was neither designed nor implemented to directly address the needs of smallholder farmers; it was designed to identify, foster and bring to scale innovative market-driven solutions and enable innovations to become fully sustainable in the market. Although KIE originally had indicators for food security and nutrition, it did not collect data on these indicators and, by the end of the project, discontinued including them in performance data reporting. Frequent changes in KIE staff (almost 100 percent turnover by project’s end) and management (two chiefs of party [COPs]) and within USAID (five contracting officer’s representatives [CORs]) contributed to delays in grant approvals, thus truncating implementation time for innovators to promote product adoption. Similarly, administrative processes and reporting required by USAID and KIE sometimes negatively impacted innovators’ ability to progress. Question 3 To what extent did KIE collaborate with other Office of Economic Growth (OEG) programs and with what results? KIE had successful collaboration with two other OEG programs. Its work with the Kenya Horticulture Competitiveness Project (KHCP) to issue a joint call for proposals at the start of the project resulted in seven innovators (six at Stage 1 and one at Stage II) receiving KIE support. KIE worked with the Kenya Agriculture Value Chain Enterprises (KAVES) project to help innovators access farmers’ cooperatives and set up demonstration plots that showcased innovator products and services, addressing some challenges faced by farmers with whom KAVES was working. Innovators were also invited to participate in farm field days organized by KAVES and its partners. Nonetheless, some opportunities for collaboration were missed. For example, KIE-supported innovators KLMC and Takaful could have contributed to building resilience among the vulnerable pastoralist communities in northern Kenya through the Partnership for Resilience and Economic Growth (PREG) and Resilience and Economic Growth in Arid Lands (REGAL) projects. A missed opportunity to work with the Financial Inclusion for Rural Microenterprises (FIRM) project could have been instrumental in its provision of soft loans for AI kits to technicians trained through a KIE innovator. FIRM also could have been instrumental in building a more sustainable market mindset with iProcure, which was interested in commercial borrowing and moving away from grant financing. 2 In a meeting with USAID on November 9, 2017, it was agreed that the answer to this question should focus on changes in KIE processes, internal operations, and program objectives, as KIE was not designed to meet the needs of smallholder farmers to achieve objectives. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 4 Question 4 To what extent has the technical assistance (TA) provided by the Business Advisory Services been effective in transitioning innovation champions into the private sector? The majority of innovators found the TA to be highly valuable. Entrepreneurs clearly most valued TA that was tailored to innovators’ needs, and it most likely assisted their expansion for securing investment. More than 90 percent of the innovators interviewed stated that TA helped in transitioning their businesses to the private sector. Specific examples include TA on intellectual property rights protection (Maseno University), remote sensing (Takaful), setting up the factory and operations, and developing a mobile app for ordering (Kenya Biologics). A number of KIE innovators cited information technology training and ways to best adapt their products to an ever-changing technological landscape as top priorities because they consider the most lucrative business opportunities in Kenya to involve data and computer technology. iProcure cites the STTA provided directly to farmers for recordkeeping through innovator ALIN as benefiting iProcure adoption: as farmers increase their knowledge of financial management, they are able to understand the cost/ benefits of iProcure, which bolsters sales. While TA was provided at no cost to innovators, several indicated that they would be willing to contribute to the cost of assistance that was specifically tailored to their needs. Generic TA, particularly when a trainer had little knowledge of an innovator’s specific business, was unappreciated and unlikely to contribute to the innovator’s business growth. STTA provided directly to farmers that demonstrated how to use innovator products, near-term benefits of innovations and cost/benefits of innovations was recognized as contributing to adoption of innovators. Question 5 To what extent has the program induced private investment, and how likely are incubated projects to grow without future support from KIE and USAID? KIE may be credited with expediting innovators’ attractiveness to investors through a no-cost capital injection that enabled them to mitigate financial risk while entering challenging geographic markets and experimenting with business expansion without eroding their financial base and company valuations. A number of innovators cited tailored TA in governance, financial and operating systems as highly valuable in bolstering their knowledge and ability to approach investors; when combined with early success and a degree of financial liberty, entrepreneurs could negotiate investment with credibility and confidence. Seven of the 19 KIE-supported innovators have attracted outside investment. These companies include Kenya Biologics (Elephant Vert), iProcure (BSP Novaster, Safaricom), Maseno University (Agri Seed Co.), Lachlan (Unifert), Wanda Organics (Village Capital), Amtech (Microsoft) and Real IPM (BioPest). These companies demonstrate the combination of competent management and the right products to address farmer needs clearly and in the near term. However, growth potential, profitability and sustainability depend on continued demand and lack of competitors, financial management and growth. An inherent contradiction exists in the KIE objective of encouraging innovators to be self-sustainable while supplying them with large grants and cost-free TA over an extended period. Given their broad base of assistance from development organizations, innovators may continue to enjoy the lower-stress support from development programs rather than pursue pure commercial business expansion. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 5 Recommendations The following recommendations are intended for USAID as it designs and implements future programs related to supporting market-driven solutions to development challenges, and may be particularly informative for the upcoming Kenya Investment Mechanism (KIM) project. Program Objectives, Design and Indicators 1. Establish a single identity for the project to assist management, staff and USAID to remain on track. Examples might be: agribusiness incubator, agribusiness self-sustainability, venture capital/angel investor or something broader, such as access to finance with a menu of mechanisms appropriate to different needs. 2. Clearly identify objectives for funding such as agriculture innovation, scaling up adoption of (non￾innovative) agriculture products and services in target areas, support for innovations that address the most critical problems, next-stage business expansion for existing businesses or products. Various options could be made available, depending on the extent to which the business meets development objectives. For example, an existing business looking for next-stage expansion may be assisted with angel funding, while an innovation that solves a critical agriculture problem but needs to be brought to market may initially receive grant funding and mentoring. 3. Establish indicator targets that can be directly measured and met through the project activities, rather than requiring flow-down, over which the IP has no direct influence or control. 4. Establish sets of selection performance criteria, support and funding models appropriate for different objectives. For example, if the objective is to foster innovations by people who have catalytic ideas but no experience with bringing a product to market, identify mechanisms to pair them with an entrepreneur or mentor who has experience. If the objective is to maximize benefit to smallholder farmers, then support should be directed to companies whose products are proven to already have made an impact. Financing Criteria 5. Work with banks to devise loan products that take into account the high-risk nature of innovative agribusiness companies. Work with rural banks and savings and credit cooperatives (SACCO) to create loan products for reputable cooperatives and farmers’ associations for onward lending to their members to purchase products. 6. Provide grants only when other funding options are not available, when innovations address the most critical development needs and where direct and lasting benefits will accrue to intended beneficiaries. In-kind grants are preferable to cash grants. 7. Seek to identify amounts and nature of applicants’ other sources of donor funding to avoid “grantpreneurs,” who may be proficient at writing proposals and conducting presentations that appeal to donors, but who do not have a commitment to delivering on development or business objectives. While funding from other donor sources is not inherently bad and could augment gaps in USAID support, full transparency will assist USAID in decision-making as well as understanding the factors that contribute to success. Other donors may also be a source through which to promote USAID support to agriculture innovations that have early stage acceptance but insufficient means to expand. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 6 Utilize Local Networks 8. Promote project and availability of support through channels as an alternative to donor networks and that will increase reach to women and youth, such as through social networks, agriculture institutes and entrepreneur networks. Require the IP to acknowledge the structural barriers that may limit women and youth from applying and/or being successful applicants; require a written, actionable plan on specific measures that will be taken to overcome barriers; require regular review of progress with the COR and the Mission’s gender lead; and hold the IP accountable for a meaningful level of social inclusion. 9. Establish relationships with local business incubators that can provide localized mentoring support and assist in promoting opportunities to entrepreneurs.3 As non-donor establishments, their reputations and success have been earned by demonstrating results to entrepreneurs and investors. 10. To select applicants that are most likely to succeed by private sector standards, selection criteria should include a review of entrepreneurs’ (or company leadership’s) personal characteristics. Engage local business incubators in the process, as they have insight into local hallmarks of successful entrepreneurs. 11. Commission a reputable private sector firm to conduct thorough market research and feasibility studies on the viability of innovator products prior to supporting them. Studies should include market demand, price sensitivity, direct and indirect competition and product lifecycle. 12. Offer solution-based mentoring support. For example, if adoption is a problem, offer mentoring services to help entrepreneurs identify and address the root cause. If the issue is that farmers don’t understand the cost/benefit, provide assistance in “merchandising” approaches to overcome the problem. If entrepreneurs are unable to penetrate new geographic areas, access assistance to make introductions within farmer cooperatives. If products require demonstration, provide assistance in establishing demonstration plots with co-operatives and extension services. 13. Provide only technical assistance that is tailored to entrepreneurs’ specific needs. Require a contribution by the training recipient to signify commitment and to encourage thoughtful training requests that are specifically tailored to the innovator’s needs. If innovators are without resources, technical assistance could be forgiven when meeting benchmark objectives or through other non￾cash means. 14. Teach entrepreneurs how to prepare financial statements, as investors and banks require it. Learning should occur through a process of doing and coaching, rather than through one-off or generic training. Monitoring Progress 15. Establish a process to monitor companies’ financial performance, progress and use of grant funds. Benchmarks should be established collaboratively with the company/entrepreneur and monitoring of benchmarks should be conducted by a third party, such as an auditing service. The auditing process should not be so onerous or time-consuming for the company/entrepreneur that it detracts from company success. Companies not meeting performance standards should receive 3 https://thefounder.co.ke/top-5-business-incubators-kenya/ KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 7 specific requirements and timelines in which to correct problems. Failure to fix performance issues as required should result in discontinuation of support. 16. Set clear targets for adoption of innovations that are aligned with development objectives, e.g., arid communities, farmers with 5 to 10 hectares, women farmers, etc. 17. Require collaboration among OEG projects when synergy results in a clear and direct benefit toward their indicator targets and objectives. Requirements for collaboration should be written into each project’s design as deliverables so that all parties understand and agree on how best to work together to achieve mutual goals. Development Objectives 18. To maximize the adoption of agriculture innovations, emphasize a rural IP presence, ensure protracted farmer contact and employ an aggressive farmer field day schedule. 19. Develop a mechanism such as vouchers to provide free or subsidized innovation products to farmers for a period sufficient for them to recognize the financial benefit. Subsidize farm recordkeeping to help farmers understand the cost/benefit of agricultural innovation products. 20. If a project objective is to improve food security and nutrition, it should work in areas where food security is lacking, and malnourishment is evident (e.g., arid and semi-arid lands [ASAL] regions). 21. Projects targeting poverty alleviation as a primary goal should work directly with intended beneficiaries and not expect support to entrepreneurs to necessarily pass through to intended beneficiaries in a significant way. 22. Projects targeting small-scale farmers, women and youth as beneficiaries should work directly with those groups and not through intermediaries; in the case of KIE, innovation companies. General USAID Management Considerations 23. Support a centralized database registry for donors to share information on organizations receiving grants and avert supporting grantpreneurs. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 8 INTRODUCTION Project Background Kenya Innovation Engine (KIE) was a five-year (2012–2017), 4 $18,324,873 million program funded through USAID’s Feed the Future (FTF) program. Implemented by Land O’Lakes (LOL), KIE intended to support FTF’s theory of change that envisions agricultural improvements increasing smallholder farmer productivity and income, which will lead to increased food security and nutrition. KIE’s role in FTF was to support entrepreneurs, research institutions and producer organizations5 whose agricultural innovations could increase agricultural productivity and incomes, with grants and technical assistance to achieve broad adoption of the agricultural innovations, therefore incubating these innovations and accelerating business. KIE was also intended to seek and promote approaches that enhance women, youth and other vulnerable groups’ access to productive resources and broad participation in the agricultural economy. KIE awarded 20 grantees; however, USAID directly managed one grantee (Lachlan). This evaluation focuses on the grants that LOL managed. KIE’s statement of work (SOW)6 describes the program goal and objectives as follows. Goal To harness innovation for cutting-edge results through private sector strategies with the potential to generate significant measurable increases in household income and/or nutrition in selected agricultural value chains: maize, dairy, horticulture and staple food crops such as sorghum and millet. Objectives 1. To systematically test, cultivate and adapt high-impact innovations that further the objectives of the USAID/Kenya FTF strategy and promote equal opportunities for women, youth and other vulnerable groups. 2. To support broad adoption of innovative approaches across target populations in the Kenya FTF regions. 3. To generate sustainable and scalable private sector solutions to further FTF goals. 4. Where appropriate, to identify and promote innovations with the potential for cross-national and/or global adoption, which may also be eligible for USAID/Washington Development Innovation Ventures (DIV) support. 5. To identify and support Kenyan for-profit and not-for-profit entities in the agricultural value chain to enable the development and adoption of high-impact innovations. Project Design The FTF theory of change (TOC) is that increasing sustainable agricultural through smallholder farming will lead to increased food security and farmer incomes, which will lead to improvements in nutrition. KIE was intended to support this TOC by “harnessing the power of innovative private sector approaches to leverage resources and integrate new approaches to agricultural solutions, thereby accelerating efforts to 4 While KIE was designed as a five-year project, the actual project received a seven-month extension. 5 As the majority of KIE’s grantees were enterpreneurs, this report largely refers to grantees as such. . KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 9 address poverty and food security needs in Kenya.”7 KIE’s activities focused on providing grant funding of up to approximately $500,000 for “innovations” that contributed to FTF objectives, and graduating the innovation companies through stages of sustainability and scalability via technical assistance. KIE was designed to build on USAID/Washington’s DIV initiative and support innovations through various stages, including “proof of concept” (Stage 1), “rollout” (Stage 2) and “scale-up” (Stage 3). KIE was expected to support 24 innovations for Stage 1, 10 for Stage 2 and four for Stage 3. Stage 1: Proof of Concept: Seed funding to incubate new ideas, supporting the development and experimentation needed to transition innovations from ideas to “proof of concept,” with a transition period of six to 12 months. Stage 2: Rollout: Projects are rolled out to test technical and market viability through rigorous testing and market research to assess impact, adoption rates, distribution and marketing plans, benefits for the farmers and households and how the approach functions at scale. In addition, to meet the criteria for graduating to Stage 3, Stage 2 projects must have the financial controls and organizational structure necessary to compete for and receive a Stage 3 grant from USAID. By design, KIE worked in two geographic sub-regions that the FTF strategy targeted as containing the greatest number of rural poor, highest poverty density, lowest-income households, highest number of malnourished children and highest percentage of female heads of households. 8 KIE’s Results Framework (see graphic) provides an insight into its design hypothesis that is driven by technology development, dissemination and innovation. Intermediate Result 6 (IR6), “Improved access to diverse and quality foods,” does not appear to result from project activities, but instead is shown as a starting point. FIGURE 1: KIE RESULTS FRAMEWORK 7 KIE SOW. 8 Land O’Lakes Award Contract, May 16, 2012, USAID/East Africa, Nairobi. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 10 KIE’s SOW offers little in the way of its own TOC, although its 2012 Performance Monitoring Plan (PMP) acknowledges assumptions relative to government, the environment and external environment. The PMP makes no reference to activities, pathways or assumptions regarding individual behavior change that would lead to adopting new technology and behavior change at the household level. The PMP offers a diagram as “illustration of the project’s complete set of innovation systems and their relationship with performance indicators.” KIE’s initial contract amount was $13 million; through four contract modifications between 2014 and 2016, KIE reached its final budget of $18 million. KIE’s contract also contained financial incentives ranging from $25,000 to $40,000 per innovation per stage, to graduate companies through stages of successful innovation. KIE’s end-of-project budget allocation was approximately $2.7 million in short-term technical assistance (STTA), $4.1 million in grants and $11 million for core services (project administration). Purpose, Audience and Intended Use and Evaluation Questions Purpose, Audience and Intended Use USAID/Kenya’s stated purpose of the final performance evaluation is to assess the efficacy of KIE’s design; determine whether the project successfully met its goals of scaling new innovations for market-based solutions; and discover how it could have been improved to bring new innovations to market. The performance evaluation is intended to provide lessons learned to shape programming. Although not contained in the evaluation statement of work, USAID/Kenya indicated that the evaluation would inform implementation of the upcoming Kenya Investment Mechanism project that is due to begin in early 2018. The key audience for this evaluation is the USAID/Kenya and East Africa (USAID/KEA) Office of Economic Growth (OEG) and other stakeholders. Evaluation Questions The five questions that guided this evaluation are as follows: 1. To what extent did KIE achieve its goals and objectives? 2. To what extent did the program (USAID and the IP) adapt and change over time to meet the varying needs of smallholder farmers in order to achieve program objectives as they evolved over time? 3. To what extent did KIE collaborate with other OEG programs and with what results? 4. To what extent has the technical assistance provided by the Business Advisory Services been effective in transitioning innovation champions into the private sector? 5. To what extent has the program induced private investment, and how likely are incubated projects likely to grow without future support from KIE and USAID? FIGURE 2: INNOVATION SYSTEMS AND RELATIONSHIPS TO INDICATORS KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 11 Evaluation Methodology, Design and Limitations Data Collection Methods Data was gathered from both primary and secondary sources. The qualitative methods provided a means of collecting in-depth information from stakeholders. Through interviews with key informants and beneficiaries, the evaluation team explored the interventions and the context in depth, explored emerging themes and issues and triangulated data among primary and secondary sources. The evaluation team comprised three members: one agriculture specialist (Kenyan), one value chain specialist (Kenyan) and one team leader (American). In most instances, the team split into two sub-teams to cover two locations concurrently and collect more data. The evaluation took place September 6 through November 24, 2017, with data collection from September 22 to October 13. Desk Review The evaluation reviewed project documents and reports produced by LOL and innovators such as activity implementation documents and other periodic progress reports. The evaluation also reviewed online resources and other reports relevant to activity efforts. A full list of documents reviewed is contained in the Bibliography (Annex I). Key Informant and Group Interviews The evaluation team consulted with 152 stakeholders through individual and group interviews in Nairobi, Kericho, Eldoret, Nandi, Bomet, Kisumu, Meru and Isiolo. Key informant interviews (KIIs) were with USAID staff and USAID/OEG implementing partners, KIE implementers, innovator companies, investors, county government and insurance companies. Interviewees discussed their experiences with and perceptions of the various activity components addressed in the evaluation to better understand KIE’s processes and outcomes. TABLE 1: STAKEHOLDERS INTERVIEWED Group interviews typically brought together three to six individuals who have a common interest. These differed from key informant interviews in that the discussion often reflected both individual perspectives and the group’s consensus on issues. The evaluation team selected areas for fieldwork to achieve geographic diversity, covering each of the three FTF zones, as KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 12 Table 2 shows. Similarly, the innovation companies interviewed represent a diverse pool of KIE grantees covering the breadth of innovations that represent the diversity of KIE’s support. 9 In selecting these innovations, the following criteria were considered: funding level (including companies that received both large and small-scale grants10), value chain (including dairy, livestock and horticulture), solicitation wave and selection stage (including some that were selected at proof-of-concept and some at the scaling-up stage). TABLE 2: COUNTIES AND INNOVATIONS VISITED County Feed the Future Zone Innovation Kericho Zone 1: High rainfall Amtech, Indicus, Kenya Livestock, Kenya Biologics Kisumu Zone 1: High rainfall Value Farms, Maseno Meru Zone 2: Semi-arid iProcure, Indicus, ALIN, Wanda Organic, Kenya Biologics Isiolo Zone 3: Arid Takaful, Kenya Livestock Sampling for KIIs and group interviews purposively encompassed a broad range of stakeholders, including small-scale farmers, women and youth, covering a variety of value chains, types of organizations, sizes and types of grants and stages of development achieved in the project (stages 1 and 2). The team did not visit all innovations in each county, but instead selected innovations based on information derived in the desk review and early interviews. Particularly critical in finalizing the selection and balance of stakeholders in each county were the initial discussions with the KIE team, who identified key partners in the area. In some cases, this created space for additional interviews with other key stakeholders. The team aimed to keep the overall number of key informant and group interviews per county in the range of 10 to 15. Methodological Limitations To avoid the potential for positive bias among informants selected by LOL and innovators, the evaluation team selected informants and respondents from comprehensive lists whenever possible. In addition, drawing from a wide and diverse respondent pool, including partners and national and county government representatives, helped offset biases. The team also obtained information from non-project sources, particularly USAID, about key individuals to include. The availability of desired key informants varied and the evaluation team worked with those available at the specified times, which may mean that some gaps in data are unavoidable. The evaluation team had little to no access to key documents such as grant agreements and contracts due to confidentiality concerns. Quantitative data available to the evaluation team was data collected by innovators and subsequently reported by KIE. Verification of this data is beyond the scope of the evaluation. 10 The evaluation team defined grants of USD $160,000 to $480,000 as large, while those less than USD $85,000 were considered small. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 13 FINDINGS AND CONCLUSIONS Question 1 To what extent did KIE achieve its goals and objectives? Findings Findings for this question are organized as follows: 1. Furthering FTF objectives 2. Broad adoption of innovative approaches 3. Sustainable and scalable private sector solutions 4. Cost/benefit of USAID investment11 KIE’s goal of “harnessing innovation for cutting-edge results for significant measurable increases in household income and/or nutrition in selected agricultural value chains: maize, dairy, horticulture, and staple food crops such as sorghum and millet” is addressed through the first three questions. 1. Feed the Future Objectives As reported in the 2016 Feed the Future Global Performance Evaluation Report, there is a lack of clarity on what constitutes desirable programmatic integration of agriculture and nutrition, and most IPs have a stronger focus on agriculture.12 Food Security Among the 93 farmers interviewed, 33 percent (54 percent men, 46 percent women) felt that they had increased food security as a result of increased agricultural production or increased income from using KIE-supported innovation products. However, the regions that were visited and farmers who were interviewed did not appear to have issues of food insecurity (as measured by malnutrition and hunger). In fact, most beneficiaries appeared perplexed by questions about food security, reporting that food scarcity is not an issue for them. Except for the outskirts of Isiolo, areas visited are known for their strength in agriculture, favorable climate and good soil. The majority of farmers visited operated successful farms and had stable supplies of food throughout the year and the economic means to sustain a relatively good quality of life. Nutrition Among the 93 farmers interviewed, 33 percent (71 percent men, 29 percent women) reported an improvement or potential to their nutritional intake as a result of increased income from using KIE￾supported innovation products. Respondents indicated that they had what they considered to be a nutritional diet prior to the use of innovator products and had not changed their diets much, if at all, as a result of using the innovator products. There were no noticeable differences in responses among men and women, geographic areas or innovations adopted. 11 Although this was not part of the original evaluation questions, USAID expressed interest in this at the final presentation of the evaluation. 12 Dexis Consulting, Feed the Future Global Performance Evaluation Report, USAID. 2016. p. iv. http://pdf.usaid.gov/pdf_docs/pbaaf131.pdf KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 14 2. Promote Equal Opportunities for Women and Youth As an FTF-funded program, one of KIE’s program objectives was to “further the objectives of the USAID/Kenya Feed the Future (FTF) Strategy and promote equal opportunities for women, youth and other vulnerable groups.”13 USAID/Kenya’s FTF strategy notes that inequitable access to the factors and benefits of agricultural production by women is an underlying cause of poverty, and KIE’s SOW states that “developing strategies and building [gender-based] differences into the design and implementation of KIE innovations will help lessen their intensity, resulting in not just better performance but also contributing to great social equity and equality.” For each innovation selected for assistance, KIE was supposed to do the following: 14 • Complete a concise gender analysis and youth analysis of target populations in the FTF focus regions to aid planning and design of gender-sensitive innovations. • Where appropriate, promote innovations that reduce gender inequalities and promote youth participation. • Collect gender and youth-disaggregated data, and monitor gender and youth impacts of innovations at all stages. According to KIE’s Grants Manual, selection criteria of grantees will “consider gender inclusion and increasing youth participation” and will be regarded as “a part of ongoing training and monitoring during implementation.”15 The Grants Manual reiterates the requirement for gender and youth analyses for each innovation selected.16 It is unknown if gender and youth analyses were conducted or the extent to which gender and youth inclusion were considered because KIE staff interviewed were more recently hired. According to the KIE Q4 FY2017 Quarterly Report, “KIE sought to identify and provide opportunities for all, with special emphasis on encouraging women and young girls to champion innovations,” although there is no evidence of such emphasis. Grantees such as M-Farm Ltd., Wanda Organic Ltd., Real IPM and the University of Nairobi have women in leadership positions. The report indicates that women held 15 percent of full-time equivalent jobs created during the KIE project implementation.17 Furthermore, Wanda Organic Ltd., iProcure and Indicus are led by people considered to be youth (i.e., younger than 36, based on the Government of Kenya classification).18 In Kenya, as in many other regions of the developing world, women constitute more than 80 percent of the agricultural producers.19 Women manage 40 percent of small-scale farms in Kenya, but account for 75 percent of the labor force in Kenyan small-scale agriculture.20 While women in Kenya reportedly produce more than 70 percent of the food consumed in the country, they face severe constraints in accessing productive resources, including markets, credit, education and training, and support services 13 KIE Section C – Description/Specifications/Statement of Work, p. 9. 14 KIE Section C – Description/Specifications/Statement of Work, p. 13 15 KIE Operations and Grants Manual, 2012. 16 KIE Operations and Grants Manual, 2012. 17 IBID. 18 IBID. 19 Njoki Ndwiga, Julie. Challenges Women Farmers Face in Accessing Agricultural Extension Services. University of Nairobi, Institute of Anthropology, Gender and African Studies, p. 1. http://erepository.uonbi.ac.ke:8080/xmlui/bitstream/handle/11295/76922/Ndwiga_Challenges%20women%20farmers%20face%20in%20accessing %20agricultural%20extension%20services.pdf?sequence=3&isAllowed=y 20 ActionAid, Policy Brief, Feb 2015, “Delivering Women Farmers Rights” pg. 3. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 15 such as agricultural extension services.21 Accordingly, the USAID/Kenya and East Africa Gender Analysis for Regional Development Cooperation Strategy 2016 notes discernable gender-based disparities in access to resources that result in a productivity gap between male and female farmers that can be explained by limited mechanization coupled with a lower use of advanced agricultural technologies, including pesticides and inorganic fertilizers.22 Despite the significantly higher proportion of female farmers in Kenya, KIE performance data reflects a disproportionately lower percentage of women than men reached through KIE activities,23 as Table 4 shows. The Feed the Future Global Performance Evaluation Report explains that “there are often competing incentives, driven by indicator results, to prioritize sales and production￾focused objectives over inclusivity measures unless the activity is designed specifically to target vulnerable populations.” 24 TABLE 3: KIE PERFORMANCE ON SELECT INDICATORS Indicator Men Women M:W Ratio Number of jobs created 4,245 1,810 2.35 Number of farmers who applied new technologies 111,218 94,801 1.17 Number of MSMEs receiving BDS 171 29 5.9 Number of people with savings account or insurance as a result of USG assistance 28,084 22,248 1.26 Number of people receiving STTA 103,196 86,849 1.19 Source: Feed the Future Kenya Innovation Engine Quarterly Progress Report (FY2017 Q4) Regarding the inclusion of youth, KIE reports that 25 percent of farmers who have adopted Kenya Biologics’ Tutrack are youth, as are 70 percent of Amtech Technologies staff and 70 percent of the 200 artificial insemination service providers trained. While instructing KIE to consider gender-based disparities, the language in KIE’s SOW refers to women in terms of beneficiaries of the agricultural innovations rather than as change agent participants (i.e., innovators). Among Stage 2 (rollout) innovators, one has a woman in a leadership position. Information regarding sex and age of the innovator applicants was not available to enable disaggregation of applicants. It is unknown whether KIE took special measures to incentivize applications from female and youth applicants. Women and youth innovators are being engaged through USAID’s East Africa Trade Investment Hub, Mastercard Foundation and IREN. One such innovator, FarmDrive, a firm started by two young women, has developed an innovative approach to facilitating farm credit for smallholder farms using mobile phones, alternative data and machine learning to close the critical data gap that prevents financial institutions from lending to creditworthy smallholder farmers.25 21 Njoki Ndwiga, Julie. Challenges Women Farmers Face in Accessing Agricultural Extension Services. University of Nairobi, Institute of Anthropology, Gender and African Studies. P. 1. 22 Benjamin, Judy, and Meyers, Lis. USAID/Kenya and East Africa Gender Analysis for Regional Development Cooperation Strategy 2016-2020, Gender Analysis Report. Banyan Global, Washington, D.C., January 15, 2016. Pp 21-23. 23 KIE QIV FY2017 Quarterly Report, November 2017. 24 FTF Global Performance Evaluation Report 25 https://farmdrive.co.ke/. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 16 3. Adoption of Innovations Adoption Prevalence A few farmers indicated that they had discontinued using innovator products due to high cost, use of an alternative or lack of availability of the product. Conversely, all farmers who were introduced to ALIN’s farm recordkeeping process intended to continue use. More than half of the farmers interviewed who reported using ALIN’s recordkeeping indicated that they did not pay for the recordkeeping book (which cost approximately USD $5); all of the farmers trained in how to use it found it valuable and said they intended to continue using it. Innovators reported that working through cooperatives and farmers associations had the obvious benefits of reducing marketing costs and time for farmers to make buying decisions, as marketing could be conducted with large groups of farmers, and farmer purchases were influenced through the groups’ leadership. Established relationships within communities, strong distribution channels and working in partnership with county governments were strong determinants of innovators in gaining adoption and penetrating new markets. Assistance from county governments was noted in Kericho, Meru and Isiolo. In Isiolo, the county government staff helped innovators organize the tagging of animals. In Meru, the county government assisted in farmer mobilization for Kenya Biologics, Lachlin and ALIN. In Kericho, the county government organized dairy cooperatives to identify people to receive AI training from Indicus. Performance data indicates that targets for adoption of improved technologies or management practices were far surpassed, with men having higher adoption rates than women. 26 TABLE 4: FTF INDICATOR 4.5.2-5: INNOVATION ADOPTION Sex LOP Target Achievement % Achieved Male 20,000 111,218 556 Female 27,000 94,801 351 Total 95,100 206,019 217 Source: Feed the Future Kenya Innovation Engine Quarterly Progress Report (FY2017 Q4) KIE was reluctant to take the lead in introducing innovators to county government officials, fearing that farmers would get wind of USAID support, which could lead to some requesting free or subsidized innovator products. Similarly, KIE was hesitant to work with county government officials, fearing their requests for per diems and other benefits. Conversely, adoption of innovator products was limited when innovators failed to penetrate farmer networks and build relationships and trust in their products due to the conservative nature of farmers and their aversion to change. Some innovators attributed sales challenges to farmers objecting to prices, however, few interviewed farmers raised the issue of high prices being a deterrent to purchasing.27 Interestingly, iProcure observed that farmers using financial recordkeeping were more likely to use their innovation as they understood the cost/benefit. Adoption was also inhibited by farmers having difficulties selling their produce due to a lack of market. Adoption was further inhibited by difficulties dealing with agro business dealers, who unilaterally raised innovator product prices in contradiction to earlier agreed￾on price points, thus pricing the products out of the market. 26 A discrepancy exists in the reported figures in the Quarterly Progress Report 4Q 2017; the reported sum of male and female numbers is 47,000 and the overall figure is reported as 95,100. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 17 According to the 19 innovators participating in KIE, five (26 percent) have expanded the geographic reach of sales beyond KIE target areas; however, the evaluation was unable to verify this information. These companies are Amtech, Kenya Biologics, Takaful, iProcure and Real IPM. Household Income Among farmers interviewed, over half reported higher incomes as a result of reduced losses or higher yields that they attribute to using innovator products. This can be seen in better pest control resulting in fewer losses (Kenya Biologics, Real IPM), improved insurance products resulting in fewer losses (Takaful), increased efficiencies resulting from better information management (Amtech, iProcure, KLMC, Alin) and better soil management resulting in higher yields (Lachlan). Farmers in Meru who used Alin recordkeeping notebooks reported a 40 percent increase in income; however, the figure is somewhat suspect, as financial recordkeeping was new to them. KLMC farmers in Isiolo reported a 20 percent increase in livestock sales prices. Kendat potato growers in Meru reported a 50 to 70 percent increase in incomes as a result of using farm machinery. Lachlan farmers reported a 100 percent increase in incomes when they used a combination of certified potato seeds and their soil treatment product. iProcure customers reported savings of 20 percent when purchasing inputs through iProcure, resulting in lower expenses and hence higher profits. KIE did not track the amount of increase in household income; however, KIE performance data (as submitted by innovators) reports incremental sales of over $25 million. For lack of valid statistical data, a crude extrapolation of approximately half of the 258,000 rural households reported to benefit from KIE (129,000) divided into incremental sales of $25 million results in an increase of approximately $194/household. Job Creation Improved farmer incomes did not necessarily translate to long-term employment by farmers who adopted technology. Job creation reported by KIE was primarily noted in short-term employment during specific periods, e.g., for the harvest, artificial insemination and livestock tagging. Much of the long-term job creation may be attributed to the innovators themselves. TABLE 5: FOOD SECURITY, SALES AND JOB CREATION INDICATORS Indicator LOP Target Achievement % Achieved Number of food security entities that applied new technologies or management practices as a result of USG assistance 92 1,400 1,522 Value of incremental sales attributed to FTF implementation $8,538,212 $25,511,542 298 Number of jobs created as a result of FTF implementation 348 6,055 1,740 Source: Feed the Future Kenya Innovation Engine Quarterly Progress Report (FY2017 Q4) 4. Sustainable and Scalable Private Sector Solutions Innovator Selection KIE subcontractors Dalberg Global Investment Advisors and IDEO.org were tasked with overseeing selection and support of innovator companies. Dalberg is a subcontractor hired to lead the selection process criteria through a “Rigorous Assessment” methodology based on “Desirability” (Impact and Innovation), “Feasibility” (ease of rollout and organizational capacity) and “Viability.” Dalberg was also responsible for capacity building and mentoring activities to help grantees develop financial and market entry strategies. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 18 KIE defined innovations as “any technology that was first adopted in a previous year should not be included. Technologies to be counted here are agriculture- and nutrition-related technologies. Relevant technologies include any technology that was first applied in a previous year and that continues to be applied will be included as continuing.”28 However, several innovations (as described by the PIRS) that KIE supported do not meet the PIRS definition, as they are already being used globally or in Kenya. Examples include pheromone-based pest traps, portable weighing machines and farm recordkeeping books. According to the SOW, 29 “innovativeness” of applicants was to be determined by conducting analyses of each innovation that were to be submitted to USAID. These analyses were also supposed to determine the level of demand for the innovation; determine the probability that it will be widely adopted and the factors that limit and/or enable widespread adoption; and determine and estimate the potential impact on the FTF objectives. 30 None of the KIE staff interviewed for the evaluation were employed by KIE at project inception and as such were unable to ascertain whether such analyses were conducted. KIE’s Operations and Grants Manual describes KIE’s implementation approach as follows: Stage 1: Proof of Concept: Seed funding will be provided to incubate new ideas, supporting the development and experimentation needed to transition innovations from ideas to “proof of concept.” Illustratively, seed funding will have an average of $50,000 and for most projects, the proof of concept stage will last from six to 12 months. Stage 1 applicants will need to demonstrate a plan for both technical and organizational viability. It will also be critical for Stage I projects to provide a convincing methodology that will lead to wide adoption of the innovation. Stage 1 projects will be independently evaluated to assess the potential proof of concept. Evidence-based methods will be used to assess market demand for the innovation, effectiveness of the design, distribution and marketing plan, uptake by local farmers and private sector, cost-benefit analysis and/or reliability of results. In addition to meeting the criteria for graduating to Stage 2, Stage 1 projects must: • Demonstrate the financial controls and organizational structure necessary to compete for and receive a Stage 2 grant from USAID; • Present a viable business and marketing plan; and • Have objective evidence that demonstrates that the innovation in question has positive agricultural development outcomes. Based on this evaluation and meeting the criteria, the most promising ideas will transition to Stage 2. KIE recognizes that not all Stage 1 applicants will quickly graduate to Stage 2; they may require more resources to become viable and scalable. Stage 2: Rollout: At this stage, once proof of concept is secure, projects will be rolled out to test technical and market viability. Pilots will be accompanied by rigorous testing and market research to assess impact, adoption rates, distribution and marketing plans, benefits for the farmers and households and how the approach will function at scale. In addition, to meet the criteria for graduating to Stage 3, Stage 2 projects must have the financial controls and organizational structure necessary to compete for and receive a Stage 3 grant from USAID. 28 Performance Indicator Reference Sheet (PIRS) for FTF Indicator 4.5.2-5: Number of farmers and others who have applied new technologies or management practices as a result of USG assistance, KIE Performance Management Plan (PMP), August 16, 2012 29 KIE Section C – Description/Specifications/Statement of Work, p.10 (Award/Contract #:623-C-12-00001) 30 Ibid. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 19 Among the 679 innovation applications, 16 were selected for Proof of Concept (Stage 1) and three entered directly at Rollout (Stage 2). All but one of the 19 innovators received upward of $200,000 in grants (the exception being Maseno University that experienced extensive delays in the grant approval process). By contrast, Forbes Magazine cites the typical angel investment in 2015 as ranging from $25,000 to $100,000. 31 TABLE 6: INNOVATION COMPANIES INTERVIEWED Level of Innovativeness and Commercial Potential Grantee Funding Level, $ # of Grants Innovative for Kenya Product Already in Kenya Innovative and Likely Self-Sustaining Kenya Biologics Limited* 400,031 2 yes some Amtech Technologies Limited* 537,118 2 yes some Takaful Insurance of Africa 435,167 1 yes none iProcure Ltd* 434,575 2 yes some Moderately Innovative and Moderate Potential Maseno University 47,475 1 some Kenya Livestock Marketing Committee (KLMC) 405,635 2 no some Lachlan 297,087 1 yes some Neither Innovative nor Financially Viable Indicus EA Ltd 375,602 1 no much ALIN* 411,971 2 no some Wanda Organic Ltd* 414,226 2 no much Five of the 19 innovator companies report that the innovations supported by KIE have succeeded in achieving adoption abroad. These companies include Amtech (Ethiopia, Rwanda), Kenya Biologics (Morocco, Kazakhstan), Takaful (Ethiopia, Somalia), iProcure and Real IPM. (The evaluation team could not independently verify the veracity of the innovators’ self-assessments concerning international adoption.) According to the KIE contract, IDEO.org was originally responsible for coaching and training in the “Desirability” by helping grantees refine their innovation design to better meet human needs, coaching grantees to answer a series of questions such as: How are customers responding to the innovation? What is/is not resonating with end-users? What insights can be garnered from customer feedback regarding an agricultural products’ installation, mobility, wear and repair, durability and performance, where applicable? Answers to these questions were intended to highlight design challenges and constraints to adoption, with an iterative prototyping process designed to improve usability and adoption of the innovative agriculture product or method. According to a “Lessons Learned” report (2012–2016):32 Due to their lack of a local presence in Kenya, IDEO.org did not have the ability to respond to requests for assistance on a short notice, according to the Fund Manager. On the other hand, IDEO expressed their frustration with the KIE design arguing that it was incompatible with a thriving and innovative start-up ecosystem. For IDEO, the partnership highlighted “the inherent tension with the design process, which is highly iterative and can sometimes be amorphous, and the highly documented and strict program needs of partners felt misaligned.” IDEO argued that for such partnership to be successful in the future, the Funding Agency should be prepared to tolerate risk and ambiguity. Regarding their partnership with the innovations, they argued that the enterprises should either give equity or pay for a portion of the services provided by IDEO in order to ensure accountability. 31 https://www.forbes.com/sites/allbusiness/2015/02/05/20-things-all-entrepreneurs-should-know-about-angel-investors/#640b14fc1aa5 32 Makeda Tsegaye, Program Report on Lessons Learned 2012-2016, NetBizImpact, Ltd., Nairobi KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 20 Although IDEO was intended to play a critical role in mentoring innovations to become truly market￾driven, none of the innovators interviewed were familiar with IDEO. Sustainability and Scalability Five of the 19 companies admitted into the KIE program — Amtech, Kenya Biologics, Takaful, iProcure and Real IPM — claimed that they are scalable and self-sustainable. While a thorough review of financial data, business, markets and marketing plans would be required to verify this, anecdotal information and successful attraction of investment partners suggested that these companies are indeed likely to be self￾sustainable and scalable. These companies share some common characteristics: a proven track record of success in high-level endeavors, a field focus, a commitment to rural areas and communities and products that offer near-term benefits to farmers in a way that farmers can visualize and understand. Leadership of these companies also possess qualities that are generally recognized as hallmarks of successful entrepreneurs: adaptive, creative, proactive and persistent. INNOVATOR SUCCESS: TAKAFUL Takaful’s Index-Based Livestock Insurance (IBLI) product is a truly innovative product. Benefits to Farmers: • Herders have increased confidence they will be reimbursed for animal deaths in case of drought, thus enabling more prudent business risk. • Farmers no longer engage in panic selling of animals for below their true market value or hasty killing for food, due to fear the animals will die of dehydration and/or malnutrition. • Increased confidence has led to more reasoned decisions and has increased the likelihood that families will survive with animals through the worst part of inevitable droughts by 36 percent. Characteristics of Management: • Proven track record and subject matter expert — experienced insurance professional. • Commitment to rural areas — He originates from the ASAL region he serves and has a deep commitment to the herder community. • Products offer near-term benefits to farmers in a way they can visualize and understand (after training) • Personal qualities: adaptive, creative, pro-active, intelligent and persistent The remaining 14 innovations supported are at various stages of business expansion and contraction. • Indicus has discontinued activity financed by KIE because it was wholly dependent upon grant funding. • Kendat continues to attempt to be profitable. • Maseno University has not received any income from its product so far. • Lachlan has attracted investors to expand operations. • ALIN is unable to get farmers to pay for its product and will continue operations through donor funding. • KLMC will continue to rely on donor funding. Dalberg opines that beyond the five successful innovations noted above, “The rest are largely unsuccessful, although some have achieved a measure of success and may find a way to stay alive with multiple donor assistance.” This opinion is largely supported by interviews with innovators themselves. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 21 A number of grants were awarded that do not appear to meet the criteria of innovation, sustainable, or scalable. For example, Wanda Organics is a fertilizer importer that received $427,000 in grants from KIE. Importation of fertilizer is not an innovation, and organic fertilizer has been on the market for decades. There are several competitors in the Kenya organic fertilizer space and demand is not currently high for this product. The company is not self-sustaining and has stopped paying its employees in Meru. KIE’s Q4 Quarterly Report states: “The company needs time to streamline their internal organizational functions, articulate risk mitigation strategies and refine their business model to ensure sustainability, and that the model can scale without disruptions.” 33 Another innovation supported by KIE is Indicus, whose grant of $387,000 subsidized 50 percent of the cost of 200 students’ tuition ($50,000) to be trained in AI at the government school, ADC Kitale. Indicus entered the KIE project at Stage 2, allowing it to bypass the filter of Rigorous Assessment conducted by Dalberg Global Development Advisors. Indicus’ gross profit, (exclusive of administrative expenses to enroll students in the AI training) is $337,000 ($387,000 - $50,000). Indicus is no longer involved in arranging AI training. Another firm supported by KIE whose product is not new technology is ALIN￾Sokopepe, whose business was farm recordkeeping. The evaluation was unable to determine how these grantees were able to bypass rigorous assessment, vetting processes, and the multiple sign-offs required. Eighty percent of the Stage II innovations have attracted private capital, while 55 percent of all innovations are exploring different forms of private and public partnership. 5. Donor-Driven vs. Private Sector Profit Motive Grant payments to innovators were based on achieving milestones, some of which were donor-driven — recruiting key personnel, approval monitoring and evaluation plan, contracting training facilitators and developing a procurement plan — and some of which were driven by rates of adoption by farmers. Achievement of milestones and rates of adoption were reported to KIE by the innovators themselves, with verification conducted by two part-time KIE field staff in Eldoret and Meru. While the type of data collection and data management required by KIE is typical for donor-funded organizations, it was not a standard practice among the innovator companies. Not only were the practices of donor-type data collection unfamiliar, but the time required to learn the practices and collect the data detracted from their core business of product sales and distribution.34 KIE field staff reported challenges in verifying innovator data due to remote and dispersed locations, lack of farmer addresses, incorrect contact information, and farmers unreachable by mobile phone during business hours due to the nature of their work. Furthermore, KIE field staff observed that it was in the innovators’ interest to overstate the performance indicators, as achievement of milestones triggered grant payments. Stage 1 grantees received between $35,000 and $85,000 in grants, and Stage 2 grantees received between $350,000 and $485,000 in grants based on milestone achievement. Six companies graduated from Stage 1 to Stage 2, so the aggregate amounts entrepreneurs received increase to well over $500,000 for most Stage 2 companies. In addition to data collection, innovators were required to prepare documents that, while typical of donor projects, are unfamiliar to the private sector and according to innovators, consumed considerable time to understand and prepare. While some reports have private sector value (such as Strategic Business Plan, Business Model Development, Marketing and Distribution Plan and Finance and Procurement Manuals), other required documents (such as Project Activity Plan, Monitoring and Evaluation Plan and Monthly 33 KIE QIV FY2017 Quarterly Report, November 2017. 34 Makeda Tsegaye, Lessons Learned 2012-2016 (KIE) KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 22 Reports) are strictly donor-oriented and, as such, were challenging and time-consuming for innovators to master. A survey of innovators conducted for the 2012–2016 Lessons Learned Report35 states that innovators were expected to deliver on original targets despite unexpected encounters in the field that countered original assumptions. The report suggests that this was contrary to the nature of innovation, which is inherently experimental and therefore prone to change. In addition, innovations were expected to generate specific results such as “number of farmers adopting the innovation,” even if farmers were not the primary target for their innovation. While setting up interviews with farmers identified as beneficiaries of innovator products, evaluators were faced with repeat cases of farmers who reported not having used any KIE innovations or having awareness of them. The evaluators also came across a few instances where passers-by were registered as participants of innovator training sessions or meetings. This suggests that in some cases, innovators may have been motivated to inflate numbers in order to achieve milestones that triggered payment. Seven of the innovators interviewed have, or have had in the past, financial and/or material support from other donors. 6. Cost/Benefit of Results While KIE did not track costs associated with each indicator, some crude calculations may provide insight into cost/benefit. Indicator results are those reported in KIE’s QIV FY2017 Quarterly Report. Unless otherwise indicated, the total project budget of $18,324,874 is used as the denominator for calculations. TABLE 7: KIE QIV FY2017 QUARTERLY REPORT INDICATOR RESULTS Indicator Achievement as of Q4 FY2017 Cost Per Number of rural households benefiting directly from USG interventions 257,814 $71/household Value of new private sector investment in agriculture sector or food chain leveraged by FTF implementation $8,236,834 $2.22/each private sector dollar invested Number of food security private enterprises (for profit), producer organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) that applied new technologies or management practices as a result of USG assistance 1,400 $13,083/enterprise Number of jobs created as a result of FTF implementation 6,055 $3,026/job created Number of individuals who have received USG-supported short-term agricultural sector productivity or food security training 190,045 $14/person* Number of farmers and others who have applied improved technologies or management practices as a result of USG assistance 206,019 $89/person *Calculation using CLIN 003: STTA $2,738,503 Conclusions By all accounts, KIE’s primary focus was on supporting innovations in agriculture to become commercially successful. While these activities may have contributed to farmer income, KIE recognized that measuring 35 Ibid. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 23 a causal link to food security or nutrition as a result of farmers using innovator products was not plausible and consistent with KIE’s Results Framework in which “Improved Access to Diverse and Quality Foods” is not a result of project activities. Given the disproportionately low percentage of female beneficiaries (25 to 50 percent) as compared with the high percentage of female farmers (80 percent), it appears that KIE was less than successful in meeting its objective of encouraging women and young girls to champion innovations. The structural gender disparities noted in the USAID/Kenya and East Africa Gender Analysis, combined with the lack of evidence of specific measures to encourage women, corroborates the conclusion in the Global Performance Evaluation Report that, given the indicator-driven behavior of activities to prioritize sales and production, achieving inclusivity requires activities that are designed specifically to target vulnerable populations. Given the language in FTF and indicator requirements for nothing more than gender disaggregation, it is likely that opportunities were missed to support women as change agents, either as entrepreneurs or as leaders for adoption. While KIE benefits were intended to flow down to smallholder farmers in areas with the greatest density of poverty, KIE’s activities focused on expanding growth and sales of innovator businesses. This reflects a contradiction between activities and intended outcomes. The target customers represent a population with minimal funds for acquiring inputs, and frequently lack intellectual capacity to understand cost/benefits of how innovator products will benefit them. As KIE innovators had a profit motive that depended on selling their product, innovators frequently targeted sales efforts to those who had more available funds: farmers who were already doing relatively well in terms of food supply and nutrition. As such, measuring KIE’s impact on improving food security and nutrition objectives among FTF’s target population of smallholder farmers is extremely difficult. While innovator products clearly have flow-down benefits to smallholder farmers, KIE’s focus was firmly fixed on the innovators themselves. Because cost was often cited as a reason for farmers not adopting innovation, it is possible that farmers simply do not understand how to calculate cost/benefit. Given ALIN’s introduction of farm recordkeeping, and iProcure’s observation that farmers who kept financial records were more likely to use the innovation, assisting farmers to understand cost/benefit either through recordkeeping or product trial could increase adoption. While farm recordkeeping is not particularly innovative, it may be considered as a valuable tool to increase adoption of products and services that can enhance agricultural incomes and production. While none of the KIE-supported innovations appear to be catalytic game-changers for smallholder farmers, defined by the evaluation team as having provided strong and measurable results toward improved household income and food security, five of the 19 companies show signs of sustainability and viability and are contributing positively to agricultural incomes and productivity. The common characteristics of these companies are a track record of business success or technical expertise, commitment to rural communities, hands-on field focus, and innovations that offer near-term benefits to farmers in a way that farmers can understand. Although KIE had systems to vet appropriateness of innovations to be supported, it may only be concluded that these systems were not adhered to, thus allowing support to some entrants that were not particularly innovative, desirable, sustainable or viable. KIE fully expresses the inherent tension between pure private sector development and FTF’s human and social development objectives, as highlighted in IDEO’s comment that KIE design is incompatible with a thriving and innovative start-up ecosystem. The expectation for innovator companies to apply the same systems and processes as donor projects is not only unrealistic but unwise. The nature of innovation and entrepreneurship is inherently risk-taking and defies the bureaucratic process of monitoring. In fact, firms that are genuinely engaged in entrepreneurial endeavors need cannot afford staff and activities that do not contribute to the innovation’s success. Conversely, entrepreneurial firms that are saddled with staffing, learning, and processes to meet reporting requirements are likely to be distracted from their core business KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 24 needs. Even though KIE grant funding amply covered the costs of additional staffing, grant funding of $400,000 to $500,000 may encourage wastefulness and attract grantpreneurs who have become experts in securing donor funding. While it is not possible to verify accuracy of innovators’ reporting, the significant over-achievement of targets, motivation to inflate numbers to achieve milestones that trigger payments, and comments by stakeholders interviewed that numbers were inflated, leads one to conclude that target numbers reported are not accurate. The notion of innovator companies reporting on their milestone achievements which in turn triggers funding reflects an inherent conflict of interest. Question 2 To what extent did the program (USAID and the IP) adapt and change over time to meet the varying needs of smallholder farmers to achieve program objectives as they evolved over time? Findings In a meeting with USAID on November 9, 2017, it was agreed that the answer to this question should focus on changes in KIE processes, internal operations, and program objectives, as KIE was not designed to meet the needs of smallholder farmers to achieve objectives. Indicators When KIE was established, its PMP included FTF indicators for food security and nutrition, with indicator targets ostensibly achieved by meeting needs of smallholder farmers through agriculture innovations, rather than directly through KIE interventions. As KIE activities exclusively focused on support of entrepreneurial firms and their agricultural innovations, indicators for nutrition and food security were dropped and excluded from reporting, and indicators reflecting innovator support performance that were added between the first Performance Management Plan (PMP) in 2012 and the last Quarterly Report in November 2017. Indicators Added: • KIE Sub IR1.1.: Enhanced human and institutional capacity development for increased agricultural sector productivity • Custom Indicator 1: Number of animals under improved technologies or management practices as a result of USG assistance. • KIE Indicator 4.5.2(7): Number of individuals who have received USG supported short-term agricultural sector productivity or food security training. • KIE Indicator 4.5.2-37: Number of MSMEs including farmers, receiving Business Development Services from USG-assisted sources • Number of jobs attributed to FTF implementation Indicators Excluded or Indicated as “Not Applicable”: • IR 6: Improved access to diverse and quality foods • Number of hectares under improved technologies or management practices as a result of USG assistance KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 25 • Number of people with a savings account or insurance policy as a result of USG assistance • Prevalence of underweight children under 5 years of age* • Prevalence of poverty: percent of people living on less than USD 1.25 a day* • Per capita expenditure (as a proxy for income) of USG-project level, targeted beneficiaries* • Percent change in agricultural GDP* • Prevalence of wasted children under five years of age* • Prevalence of stunted children under five years of age* • Prevalence of underweight women* The indicators with an asterisk are reported as “N/A” (not applicable) in KIE’s Q4 FY2017 Quarterly Report. The evaluation was unable to identify contract modifications that explain why indicators were dropped or added. 1. KIE Identity and Management KIE has alternately been described by USAID and within KIE reports as an innovations engine, business incubator, agribusiness capacity-building project, agribusiness self-sustainability project, and a project inspired by and based on the venture capital fund model. KIE lacked a singular and clearly measurable focus, with a wide range of objectives that included: support sustainable innovators and adoption of agricultural innovations in Kenya and worldwide; provide food security and nutritional benefits; alleviate poverty; benefit women and youth; prepare entrepreneurs to attract investors and receive loans; and link innovators with farmers. During its five years of implementation, KIE had two Chiefs of Parties and five Contracting Officer Representatives (CORs). KIE further experienced significant staff turnover at all levels; few if any of the original staff remained until the end of the project. Both USAID and KIE staff, as well as innovators, point to delays in approvals and implementation resulting from changes in leadership, management, and staffing, as newcomers underwent steep learning curves, and institutional memory was lost. Administrative Delays Misunderstanding with the State Ministry of Agriculture KIE’s start-up was delayed by six months as USAID held discussions with the Agricultural Sector Coordination Unit (ASCU) under the State Ministry of Agriculture (MoA) on the nature of the relationship KIE was to have with the Ministry. This led to delays in selecting the innovations, with the ultimate effect of reducing the time for promotion and adoption of innovations. Partner Vetting There was a six-month delay in project implementation due to the introduction of partner vetting, a requirement for entities partnering and/or receiving benefits from U.S. Government funds. During this period, there were delays in the signing of contracts, which in some cases required innovators to revise implementation plans to reduce the period of implementation. Innovators and IP staff cite the prevalence of lengthy approval processes as a hindrance to meeting targets. Despite KIE streamlining internal approval processes from about five months in the first wave to two-to￾three months in subsequent waves of applicants, stakeholders claim that funding approval periods could take up to 13 months. As a result, the period of project implementation for several innovators was KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 26 reduced, sometimes multiple times, from two years to as little as six months. This proved a challenge to an innovator who had to compress activities into six months in order to achieve the milestones and receive grant funding. KIE was provided a modification to extend the project end date by four months. Partner vetting also affected some innovators in Stage 1, who had to wait longer to go through Rigorous Assessment (RA) to ascertain whether he/she would be allowed to proceed to Stage 2. Kenya Biologics reported losing staff during this period due to job insecurity and uncertainty of moving into Stage 2. Whereas some innovators passed the RA, they did not receive approval to proceed, as the time remaining to the end of the project was too short. Maseno University was affected in this way. Initially, KIE was only allowed to approve entrants to Stage 1, while Stage 2 entry was determined by USAID. A contract modification enabled KIE to manage both Stages. Seasonality and Geography The issue of seasonality was cited by innovators as a major factor in adoption rates, making it difficult for them to reach their milestones and thus causing delays in the completion of their contracts. For example, Lachlan had difficulties in reaching the number of farmers it needed to meet its milestone due to a lack of certified seeds in Bomet County. Despite having trained farmers on the use of its innovative product, Viazi Power Fusion, the innovator could not reach the required numbers in time. Lachlan’s contract was then amended to allow it to implement activities in Meru County during the wet season in order to reach its milestone. A survey conducted for the Lessons Learned Report found that the eight-to-12 month average incubation period was too short to test agricultural innovations and the timing was not appropriately aligned to the crop cycle or activity cycle. As a result, most of the innovations had to be awarded two to eight month no-cost extensions beyond the incubation period. Conclusions Although KIE was established with multiple and often contradictory identities, it appeared to settle into an identity that was firmly focused on supporting entrepreneurs and expanding their products’ sales and distribution. Given the seasonality of agriculture production, it is unreasonable for innovators to be expected to achieve sales and distribution milestones in the often-abbreviated implementation timelines that were required. Question 3 To what extent did KIE collaborate with other OEG programs and with what results? Findings 1. Collaboration Kenya Horticulture Competitiveness Project KIE worked closely with KHCP to issue a joint call for agriculture and nutrition innovations at the start of the project, with an emphasis on horticulture-related interventions. This led to the first open and competitive call for innovations going out in March 2013 (FY2013). The two projects posted solicitations for innovations on their respective websites, sent e-mails to stakeholders, and used an extensive network of partners, including Dalberg, IDEO.org, and USAID/FTF communication working groups to publicize the first call for innovation proposals. This collaboration resulted in 19 finalists (13 at Stage 1, and six at Stage KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 27 II) being shortlisted from a pool of 188 initial applications, with seven innovators (six at Stage 1 and one at Stage II) receiving support from KIE. Kenya Agricultural Value Chains Enterprises (KAVES) From FY2015 onward, KIE introduced some of the innovators to KAVES in order to help them address specific challenges with farmer mobilization, as well as challenges faced by the target farmers working with KAVES, which could be addressed by the innovations being promoted by KIE. In FY2016, KIE reached out to KAVES to hold meetings in Meru County with KAVES implementers — Cereal Growers Association (CGA) and Kilimo Biashara — with the aim of tapping into the 23,000-strong smallholder farmer database housed by the two partners, inherited from the Kenya Agricultural Productivity and Agribusiness Program (KAPAP), to link the innovators working in the region to these farmers. Furthermore, innovators were invited to participate in field days organized by KAVES partners such as in Uasin Gishu, organized in collaboration with the Kenya National Farmers’ Federation (KENAFF), and in Homabay in partnership with Animal Draft Power Program (ADPP), where Kenya Biologics was able to showcase its innovation to the tomato farmers in the two counties. By setting up demonstration plots in various locations following the collaborative efforts with KAVES partners, innovators helped further adoption of their products. One of KAVES contractors, Eldorift, organized introductory meetings with representatives of 44 dairy cooperatives from four counties to inform them of the Indicus AI training opportunities. Accelerated Value Chain Development (AVCD) KIE and AVCD collaborated through different innovators. The International Potato Center (CIP) worked with Lachlan in Kenya’s North Rift region (Nandi, Uasin Gishu and Elgeyo-Marakwet counties) in conducting field trials to test new potato varieties in the region using its innovation, “Viazi power fusion.” It also trained Ward Agricultural Officers (WAO) on how to use the innovation to increase potato production. AVCD sponsored 26 students for the Indicus AI training to enable them to work with farmers in the HR1 and SA1 regions, where they are working on livestock and dairy value chains. 2. Missed Opportunities Collaboration with additional USAID OEG programs could have proven beneficial for innovators and farmers interested in adopting innovations as described: Financial Inclusion for Rural Micro-enterprises (FIRM) Thirty AI technicians trained by Indicus needed linkage to a financial institution to access soft loans (with affordable rates) for AI kits to enable them to carry-out their mandate. Indicus approached FIRM through KIE to assist in forming this linkage, but this came when the FIRM project was nearing closure mode and thus the assistance never came to pass. This slowed the adoption of AI for some dairy farmers, as the people meant to promote this service did not have the right tools to conduct the service. There were preliminary discussions between FIRM and one of the innovators, iProcure regarding the possibility of getting a loan, but interest evaporated once iProcure was admitted into KIE, with its lucrative grant funding. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 28 Partnership for Resilience and Economic Growth (PREG) PREG brings together humanitarian and development partners to build resilience among the vulnerable pastoralist communities in northern Kenya. There was a valuable opportunity for some of the KIE￾supported innovators, specifically KLMC and Takaful, to contribute in the resilience building of these communities by collaborationg with some of these projects. For example use of Takaful’s Index Based Livestock Insurance (IBLI) would have been a good technology for the livestock keepers in Northern Kenya and could have possibly been implemented through the Resilience and Economic Growth in Arid Lands (REGAL)- Increasing Resilience (IR) and REGAL – Accelerating Growth (AG) activities. Conclusions KIE, KHCP and KAVES enjoyed the fruits of collaboration, creating awareness among farmers about KIE￾supported innovations, while in turn, facilitating sales of innovations. Nonetheless, some opportunities were missed to leverage USAID activities to further the interests of the innovators and farmers and improve results for OEG programs. Question 4 To what extent has the technical assistance provided by the Business Advisory Services been effective in transitioning innovation champions into the private sector? Findings More than 90 percent of the innovators interviewed stated that technical assistance (TA) helped in transitioning their businesses to the private sector, particularly the TA that was tailored innovator needs, such as Intellectual Property rights protection for Maseno University and remote sensing for Takaful. Kenya Biologics cites TA for information technology (IT) as enabling them to develop an app for ordering products, and TA in governance as helpful in setting-up the factory and operations. Most innovator companies supported by KIE are early-stage companies with scarce financial resources for training; these innovators reported that TA was extremely useful. Four of the innovators interviewed indicate that they would be willing to contribute to the cost of tailored TA in the future. A number of KIE innovators cited IT training and how to best adapt their products to an ever-changing technological landscape as a top priority, as they consider the most lucrative business opportunities in Kenya to involve data and computer technology. Some of the more generic TA, e.g., financial management/controls/processes, procurement systems, human resources and staffing, corporate governance, IT, investment attraction and safe pesticide use, were considered helpful by innovators, although less so than the customized offering. General textbook training that was not suited to a specific company’s situation was considered the least effective type of TA. Innovators reported mixed perceptions concerning the effectiveness of training on business plan production. Some stakeholders cited a lack of sector expertise among trainers, such as training in business planning that was general in nature and not specific to the product or market in which the innovator works. Innovators claimed that the consultant had only rudimentary knowledge of their business and were unable to train them in exactly what they needed. Initially, innovators were allowed to select consultants; however, it was discovered that some consultants were unqualified. Thereafter, innovators had to work with the appointed TA consultants and were not allowed to select consultants. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 29 iProcure cites the STTA provided directly to farmers for recordkeeping through innovator ALIN as benefiting iProcure adoption: as farmers increase their knowledge of financial management, they are able to understand the cost/ benefits of iProcure, which bolsters sales. Conclusions TA that was tailored to innovators’ needs was clearly the most valued, and most likely to have assisted their expansion and securing investment to the extent that innovators would be willing to contribute to the cost. Generic TA, particularly when trainers had little knowledge of innovators’ specific business, is unappreciated and unlikely to contribute to innovators’ business growth. STTA that helps innovators demonstrate cost benefits to farmers — delivered either directly to farmers or in the form of “merchandising” materials for innovators — could go a long way toward accelerating innovator adoption. Question 5 To what extent has the program induced private investment, and how likely are incubated projects to grow without future support from KIE and USAID? Findings Investment Attraction Seven of the 19 KIE-supported innovators have attracted outside investment. These companies include Kenya Biologics (Elephant Vert), iProcure (BSP Novaster, Safaricom), Maseno University (Agri Seed Co.), Lachlan (Unifert), Wanda Organics (Village Capital), Amtech (Microsoft) and Real IPM (BioPest). TABLE 8: PERFORMANCE ON FTF INDICATORS 4.5.2-38 AND 4.5.2-12 Indicator LOP Target Achievement % Achieved Value of new private sector investment in the agricultural sector or food chain leveraged by FTF implementation 4,513,000 8,236,834 183 Number of public-private partnerships formed as a result of FTF assistance 37 56 151.3 Source: FTF KIE Quarterly Report (FY2017, Q4) KIE innovations attracted $3,936,891 in new funding in the last quarter of the project (Q4 FY2017), including Takaful Insurance of Africa (TIA), which attracted $2.6 million in new funding from the National Government of Kenya to enable timely insurance payouts to pastoralists. The funds will enable TIA to continue implementing in the seven counties in which it is currently operating and into the Turkana and Samburu Counties. This brings the total amount of funding leveraged through the program to $8,236,834.36 Some of the investments reported by KIE are not true private sector investments, but rather are social investments or government subsidies, such as the $2.6 million “investment” in Takaful from the Kenyan government. The evaluation was unable to verify investment amounts. 36 KIE QIV FY2017 Quarterly Report, November 2017. KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 30 Innovator Growth Potential Five of the 19 KIE-supported innovators report that they are currently profitable or near profitable without grants. These companies are Kenya Biologics, iProcure, Takaful, Amtech, and Real IPM. Stakeholders interviewed unanimously agreed that that these companies’ products are in demand; that the companies enjoy a favorable reputation; and that they are likely to continue on their growth trajectory. In addition to KIE support, most innovators interviewed have received support in the form of grants, TA, and material support from other donor entities ranging from World Bank to European government donors and international NGOs. One innovator stated that a grant received from another donor was more than that received from KIE. Conclusions KIE may be credited with expediting innovators’ attractiveness to investors through a no-cost capital injection that enabled them to mitigate financial risk while entering challenging geographic markets and experimenting with business expansion without eroding their financial base and company valuations. KIE also supported entrepreneurs through BDS that bolstered their knowledge and ability to approach investors; when combined with early success and a degree of financial liberty, entrepreneurs were able to negotiate investment with credibility and confidence. The five companies that have attracted investment thus far have competent management, and the right products to address farmer needs clearly and in the near term. However, growth potential, profitability, and sustainability are dependent upon continued demand and lack of competitors, financial management, and growth. There is an inherent contradiction in the KIE objective of encouraging innovators to be self-sustainable while supplying them with large grants and cost-free TA over an extended period. Given innovators’ broad base of assistance from development organizations, innovators may continue to enjoy support from non￾USAID sources. RECOMMENDATIONS The following recommendations are intended for USAID as it designs and implements future programs related to supporting market-driven solutions to development challenges, and may be particularly informative for the upcoming Kenya Investment Mechanism (KIM) project. Program Objectives, Design and Indicators 1. Establish a single identity for the project to assist management, staff and USAID to remain on track. Examples might be: innovations engine, business incubator, agribusiness solutions, agribusiness self-sustainability, venture capital/angel investor or something broader, such as access to finance with a menu of mechanisms appropriate to different needs. 2. Clearly identify objectives for funding such as agriculture innovation, scaling up adoption of (non￾innovative) agriculture products and services in target areas, support for innovations that address the most critical problems and next-stage business expansion for existing businesses or products. Various options could be made available depending on the extent to which the business meets development objectives. For example, an existing business looking for next-stage expansion may KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 31 be assisted with angel funding, while an innovation that solves a critical agriculture problem but needs to be brought to market may initially receive grant funding and mentoring. 3. Ensure that indicators are targets that can be directly measured and met through the project activities, rather than requiring flow-down, over which the IP has no direct influence or control. 4. Establish sets of selection criteria, performance criteria, support and funding models appropriate for different objectives. For example, if the objective is to foster innovations by people who have catalytic ideas but no experience with bringing a product to market, identify mechanisms to pair them with an entrepreneur or mentor who has experience. If the objective is to maximize benefit to smallholder farmers, then support should be directed toward companies whose products are proven to already have made an impact. Start-ups need to be given time to prove their merit and value in the competitive marketplace as well as clear impact on targeted beneficiaries in the fulfillment of FTF goals before potentially receiving support from USAID. 5. Teach entrepreneurs how to prepare financial statements, as this is what investors and banks require. Learning should be through a process of doing and coaching, rather than through one-off or generic training. Financing Criteria 6. Work with banks to devise loan products that take into account the high-risk nature of innovative agribusiness companies. 7. Provide grants only when other funding options are not available, when innovations address the most critical development needs, and where direct and lasting benefits will accrue to intended beneficiaries. In-kind grants are preferable to cash grants. 8. Seek to identify amounts and nature of applicants’ other sources of donor funding to avoid “grantpreneurs,” who may be proficient at writing proposals and conducting presentations that appeal to donors, but who do not have commitment to delivering on development or business objectives. While funding from other donor sources is not in itself bad, and could augment gaps in USAID support, full transparency will assist USAID in decision-making as well as understanding the factors that contribute to success. Other donors may also be a source through which to promote USAID support to agriculture innovations that have early stage acceptance but insufficient means to expand. Utilize Local Networks 9. Promote project and availability of support through channels that are alternative to donor networks and will increase reach to women and youth, such as social networks, agriculture institutes and entrepreneur networks. Require the IP to acknowledge the structural barriers that may limit women and youth from applying or being successful applicants; require a written, actionable plan on specific measures that will be taken to overcome barriers; require regular review of progress with the COR and the Mission’s gender lead; and hold the IP accountable for a meaningful level of social inclusion. 10. Establish relationships with local business incubators that can provide localized mentoring support and assist in promoting opportunities to entrepreneurs.37 As non-donor establishments, their 37 https://thefounder.co.ke/top-5-business-incubators-kenya/ KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 32 reputations and success have been earned by demonstrating results to entrepreneurs and investors. 11. Commission a reputable private sector firm to conduct thorough market research and feasibility studies on the viability of innovator products prior to supporting them. Studies should include market demand, price sensitivity, direct and indirect competition and product life cycle. 12. To select applicants most likely to succeed by private sector standards, selection criteria should include a review of entrepreneurs’ (or company leadership) personal characteristics, including previous commercial experience or technical expertise, commitment and motivation of the entrepreneur, and networks within communities that the company or products target. Engage local business incubators in the process, as they have insight into local hallmarks of successful entrepreneurs. 13. Offer solution-based mentoring support. For example, if adoption is a problem, offer mentoring services to help entrepreneurs identify and address the root cause. If the issue is that farmers don’t understand the cost/benefit, provide assistance in “merchandising” approaches to overcome the problem. If entrepreneurs are unable to penetrate new geographic areas, access assistance to make introductions within farmer cooperatives. If products require demonstration, provide assistance in establishing demonstration plots with co-operatives and extension services. 14. Provide technical assistance that is tailored to entrepreneurs’ specific needs only. Require a contribution by the training recipient to signify commitment and to encourage thoughtful training requests that are specifically tailored to the innovator’s needs. If innovators are without resources, technical assistance could be forgiven through meeting benchmark objectives or through other non-cash means. Monitoring Progress 15. Establish a process to monitor companies’ financial performance, progress and use of grant funds. Benchmarks should be established collaboratively with the company/entrepreneur, and a third party, such as an auditing service, should monitor benchmarks. The auditing process should not be so onerous or time-consuming for the company/entrepreneur as to detract efforts from company success. Companies not meeting performance standards should receive specific requirements and timelines in which to fix problems. Failure to fix performance issues as required should result in discontinuation of support. 16. Set clear targets for adoption of innovations that are aligned with development objectives, e.g., arid communities, farmers with 5 to 10 hectares, women farmers, etc. 17. Require collaboration among OEG projects when synergy results in a clear and direct benefit toward their indicator targets and objectives. Requirements for collaboration should be written into each project’s design as deliverables so that all parties understand and agree on how best to work together to achieve mutual goals. Development Objectives 18. To maximize the adoption of agriculture innovations, emphasize a rural IP presence and protracted farmer contact, and employ an aggressive farmer field day schedule. 19. Develop a mechanism such as vouchers to provide free or subsidized innovation products to farmers for a period sufficient for them to recognize the financial benefit. Subsidize farm KENYA INNOVATION ENGINE: FINAL EVALUATION REPORT 33 recordkeeping for farmers in the target areas so that they can better understand the cost-benefit of products designed to solve agricultural problems. 20. If a project objective is to improve food security and nutrition, it should work in areas where residents lack food security and malnourishment is in evidence (e.g., ASAL regions). 21. Projects targeting poverty alleviation as a primary goal should work directly with intended beneficiaries and not expect support to entrepreneurs to necessarily pass through to intended beneficiaries in a significant way. 22. Projects targeting small-scale farmers, women and youth as beneficiaries should work directly with those groups and not through intermediaries (in the case of KIE, innovation companies). General USAID Management Considerations 23. Support a centralized database registry for donor information, similar to a collateral or loan registry, whereby donor organizations share information on who they are supporting and the nature of the support. This may help avoid provide support to “development entrepreneurs” who are in the business of getting grants while providing little return on donor investment. KENYA INNOVATION ENGINE FINAL EVALUATION 34 ANNEX I: BIBLIOGRAPHY ActionAid. “Delivering Women Farmers Rights” pg. 3, Policy Brief, Feb 2015. Benjamin, Judy and Lis Meyers. USAID/Kenya and East Africa Gender Analysis for Regional Development Cooperation Strategy 2016–2020, Gender Analysis Report. Banyan Global, Washington, DC, January 15, 2016. Pp 21-23. Donald L. Sexton and Hans Landström (2007): “Six Steps to Heaven: Evaluating the Impact of Public Policies to Support Small Businesses in Developed Economies,” in D. Sexton and H. Landström (eds.), Handbook of Entrepreneurship, Oxford, Blackwell. Makeda Tsegaye: Program Report on Lessons Learned, 2012–2016. NetBizImpact, Nairobi. Njoki Ndwiga, Julie. Challenges Women Farmers Face in Accessing Agricultural Extension Services. University of Nairobi, Institute of Anthropology, Gender and African Studies. p. 1. KIE: Endline Rigorous Assessment Reports prepared by Dalberg International. Availed by Land O’Lakes. KIE: Environment Compliance Update Reports. Availed by Land O’Lakes. KIE: Innovator Snapshot documents. Availed by Land O’Lakes. KIE: Operations and Grants Manual, July 2, 2012. Availed by Land O’Lakes. KIE: Proposal document. Availed by Land O’Lakes. KIE: Quarterly Reports covering the years 2014–2017. Availed by Land O’Lakes. KIE: STTA Report. Availed by Land O’Lakes. KIE: Work Plans covering the years 2012–2017. Availed by Land O’Lakes. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523