EVALUATION MID-TERM PERFORMANCE EVALUATION OF THE SCALING SEEDS AND TECHNOLOGIES PARTNERSHIP IN AFRICA October 26, 2017 MID-TERM PERFORMANCE EVALUATION OF THE SCALING SEEDS AND TECHNOLOGIES PARTNERSHIP IN AFRICA Prepared for the United States Agency for International Development under USAID Contract Number AID-OAA-TO-16-00008, AID-OAA-I-15-00024 Submitted by: Catherine Longley (Team Leader) Lewis Kamiri (Seed Specialist) Thomas Remington (Agronomist) Contractor: Mendez England & Associates (PEEL) 1020 19th Street NW, Suite 875 Washington, DC 20036 Tel: 240-762-6296 www.mendezengland.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS EXECUTIVE SUMMARY............................................................................................................................................. I 1.0 EVALUATION PURPOSE AND QUESTIONS ...................................................................................... 1 1.1 EVALUATION PURPOSE .................................................................................................................... 1 1.2 EVALUATION AUDIENCE.................................................................................................................. 1 1.3 EVALUATION QUESTIONS ............................................................................................................... 1 2.0 PROJECT BACKGROUND........................................................................................................................ 3 3.0 EVALUATION METHODS AND LIMITATIONS................................................................................... 5 3.1 QUALITATIVE DATA COLLECTION AND ANALYSIS....................................................................... 5 3.1.1 Review of SSTP Project Documentation ..................................................................... 5 3.1.2 Review of Relevant Literature ....................................................................................... 5 3.1.3 Face-to-Face Interviews and Direct Observation ...................................................... 5 3.1.4 Skype/Telephone Interviews.......................................................................................... 6 3.1.5 FGDs and Round Table Meetings.................................................................................. 6 3.1.6 Qualitative Data Analysis................................................................................................ 6 3.2 QUANTITATIVE DATA COLLECTION AND ANALYSIS.................................................................... 7 3.2.1 Online Self-Administered Questionnaire for Grantees............................................. 7 3.2.2 Data on Seed Production and Sales.............................................................................. 7 3.2.3 Data on Grantee Selection and Performance ............................................................. 7 3.2.4 Quantitative Survey for Smallholder Farmers............................................................. 7 3.3 LIMITATIONS OF THE EVALUATION................................................................................................. 8 4.0 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ............................................................ 8 4.1 OVERALL PERFORMANCE................................................................................................................. 8 4.1.1 Lessons Learned.............................................................................................................10 4.2 MANAGEMENT AND ADMINISTRATION........................................................................................12 4.2.1 Overview of Current Grants Portfolio......................................................................12 4.2.2 Grant Selection Process................................................................................................12 4.2.3 Effective Support to Partners.......................................................................................14 4.2.4 Coordination and Communication .............................................................................15 4.2.5 Data Management Systems...........................................................................................16 4.3 ENGAGEMENT OF THE PRIVATE SECTOR......................................................................................19 4.3.1 Models Used to Make Seed of Improved Varieties Available to Farmers............19 4.3.2 Increased Availability of Seed of the Focal Crops ....................................................20 4.3.3 Barriers to Increasing Seed Production of Improved Varieties..............................23 4.3.4 Barriers to Distributing Seeds to Local Markets......................................................25 4.3.5 The Roles of the Private Sector in Seed Value Chains............................................25 4.3.6 Value Chain Segments Suitable for Private Sector Investment ..............................27 4.3.7 Effectiveness of the Private Sector in Creating Demand ........................................28 4.4 EVALUATION FINDINGS: SCALING AND ADOPTION OF TECHNOLOGIES .................................30 4.4.1 Use of Improved Varieties by Direct and Indirect Beneficiary Farmers...............30 4.4.2 Strategies to Promote Adoption vs. Application......................................................31 4.4.3 Take-off Points................................................................................................................42 4.5 EVALUATION FINDINGS: ENABLING ENVIRONMENT...................................................................43 4.5.1 Progress to Date............................................................................................................44 4.5.2 Additional Policy Reforms Needed.............................................................................45 ANNEXES ..................................................................................................................................................................47 ANNEX A. EXPRESSION OF INTEREST........................................................................................................48 ANNEX B. TRAVEL ITINERARY....................................................................................................................59 ANNEX C. REFERENCES CITED AND BIBLIOGRAPHY OF DOCUMENTS REVIEWED ...............................61 ANNEX D. DEFINING AND IDENTIFYING FARMER BENEFICIARIES...........................................................66 ANNEX E. DATA COLLECTION METHODS ................................................................................................69 ANNEX F. DATA COLLECTION INSTRUMENTS.........................................................................................77 ANNEX G. ADDITIONAL DATA TABLES....................................................................................................99 ANNEX H. INCREASED AVAILABILITY OF SEED OF THE FOCAL CROPS ................................................108 ANNEX J. FARMER SURVEY REPORT – WAVE 1 ......................................................................................128 LIST OF TABLES Table 1. SSTP Results Framework............................................................................................................................ 3 Table 2. Strengths and Weaknesses of the Grants Selection Process.............................................................14 Table 3. Examples of Technology Availability Models Used by SSTP...............................................................19 Table 4. Comparison of Seed Planting Rates of Crops Observed During Evaluation ...................................26 Table 5. Number of Sample Beneficiary Farmers Who Have Adopted or Applied SSTP-Promoted Varieties......................................................................................................................................................32 Table 6. Main Motivation Factor That Persuaded the Farmer to Plant the Improved Variety for the First Time by Crop and Country ...........................................................................................................33 Table 7. How Farmers Acquired the Improved Varieties When They First Used It/Them (by Crop and Country)..............................................................................................................................................34 Table 8. From Whom Farmers Acquired Improved Varieties When They First Used It/Them (by Crop and Country)...................................................................................................................................35 Table 9. Average Distance to Agro-Dealer (km.)................................................................................................35 Table 10. Radio, TV, and Mobile Phone Ownership Among Beneficiary Farmers.........................................41 ACRONYMS AGRA Alliance for a Green Revolution in Africa ARI Agricultural Research Institute ASA Agricultural Seed Agency BFS Bureau for Food Security BMGF Bill and Melinda Gates Foundation CABI Commonwealth Agricultural Bureau International CAPI Computer-Assisted Personal Interviewing CBO Community-Based Organization CBS Crop Bioscience Solutions Ltd. CBSD Cassava Brown Streak Disease CDM Cervejas de Moçambique CGIAR Consultative Group for International Agricultural Research CIAT CGIAR’s International Center for Tropical Agriculture CLA Collaboration, Learning, and Adaptation CMD Cassava Mosaic Disease COMESA Common Market for Eastern and Southern Africa CPPs Crop Protection Products CRI Crop Research Institute DADTCO Dutch Agricultural Development & Trading Company BV DDL USAID’s Development Data Library DEC USAID’s Development Experience Clearinghouse ECoSIB Entrepreneurship for Commercial Seed Incubation Business ECOWAS Economic Community of West African States EGS Early Generation Seed EMMP Environmental Mitigation and Monitoring Plan EOI Expression of Interest EQ Evaluation Question ESRF Economic and Social Research Foundation ET Evaluation Team FARA Forum for Agricultural Research in Africa FGD Focus Group Discussion FIPS Farm Input Promotions Africa Ltd. FISFAP Financial Inclusion for Smallholder Farmers in Africa Project FRI Farm Radio International FTF Feed the Future GLDB Grain Legume Development Board GPS Global Positioning System IARC International Agricultural Research Center ICT Information and Communications Technology IFDC International Fertilizer Development Center IIAM Agricultural Research Institute of Mozambique/Instituto de Investigacao Agraria de Mocambique IITA International Institute of Tropical Agriculture IPR Intellectual Property Rights IPTT Indicator Performance Tracking Table IVR Interactive Voice Response IVSC Innovations Village Seed Company KII Key Informant Interview KNUST Kwame Nkrumah University of Science and Technology LCIC Legacy Crop Improvement Center M&E Monitoring and Evaluation MAP Modern African Productions MOFA Ministry of Food and Agriculture MT Metric Tons NARI National Agricultural Research Institute New Alliance New Alliance for Food Security and Nutrition NGO Non-Governmental Organization OPV Open-Pollinated Varieties PABRA Pan African Bean Research Alliance PASS Program for Africa's Seeds Systems PEEL Program Evaluation for Effectiveness and Learning PPI Project Performance Index PPP Public-Private Partnership QDPM Quality Declared Planting Material QDS Quality Declared Seed RFP Request for Proposals SAGCOT Southern Agricultural Growth Corridor of Tanzania SAI Sustainable Agriculture Intensification SBCC Social Behavior Change Communication SEI Sustainable Enterprise Index SMS Short Messaging Service SRI Sugarcane Research Institute SSTP Scaling Seeds and Technologies Partnership SWET Story Workshop Educational Trust TASAI The African Seed Access Index ToC Theory of Change UPOV International Union for the Protection of New Varieties of Plants USAID United States Agency for International Development. USG United States Government VBA Village-Based Agent GLOSSARY OF TERMS Adoption (of a variety or other technology) Adoption takes place when a farmer decides to incorporate a new variety or technology into their long-term planting repertoire or management practices. Adoption usually occurs after a period of testing or “application” (see below). In general, there is a blurred line between application and adoption, but when a farmer has grown a variety for more than three years, s/he can generally be considered to have adopted it. Application (of a variety or other technology) The “trying out” or testing of a new variety or technology by a farmer to determine whether or not to adopt it. Application generally takes place over two or three seasons, allowing a farmer to test it under different agro-ecological conditions. Application may or may not lead to adoption (see above). Basic seed This is the original source of all foundation and certified seed. It is produced and maintained by the breeder who developed the variety. It is genetically pure. Also referred to as breeder seed. Breeder seed This is the original source of all foundation and certified seed. It is produced and maintained by the breeder who developed the variety. It is genetically pure. Also referred to as basic seed. Certified seed This is produced by registered seed growers through the multiplication of foundation seed to meet quality requirements regarding genetic and physical purity, as prescribed by a Seed Certification Authority or Agency. Certified seed is recognized by the label issued by the Certification Authority. Complementary technologies Improved agricultural technologies and management practices other than improved varieties. Complementary technologies are also referred to as “other,” “associated,” or “alternative technologies” in the SSTP documentation. They can include fertilizer, crop protection products (see below), mechanization, irrigation, etc. Crop protection products (CPPs) Active substances and compounds used to control pests, diseases, and weeds. CPPs include pesticides, fungicides, herbicides, weedicides, etc. The term is most commonly used in reference to pesticides. Also known as plant protection products. Direct farmer beneficiaries Smallholder farmers who have participated in grantee activities (e.g., demonstration plots and/or short-term trainings) and/or received or purchase inputs or services (e.g., seed, fertilizer, mechanized services) directly from the grantee. Direct beneficiary farmers may or may not be a member of a farmer group or community-based organization. The quantitative farmer survey identified direct farmer beneficiaries as those who had heard of improved varieties and recalled having taken part in activities promoting these varieties within the last three years. Fake seed Seed which is packaged and sold as certified seed, but has either not been produced under the prescribed conditions and/or does not meet the required quality and purity standards. Foundation seed This is produced by highly specialized and strictly controlled seed growers through the multiplication of breeder or basic seed. Foundation seed is genetically pure and provides the source of certified seed. Indirect farmer beneficiaries Smallholder farmers who have not had direct contact with SSTP￾supported activities such as demonstration plots, trainings, or input/service provision, although they may have seen the variety/technology on a neighbor’s farm or obtained it from sources other than the grantee. The quantitative farmer survey identified indirect farmer beneficiaries as those who had heard of improved varieties but had not taken part in activities promoting these varieties within the last three years. Non-beneficiary farmers Smallholder farmers who reside within the target areas of SSTP￾supported activities but are not aware of the varieties or technologies being promoted. Non-beneficiary farmers were not included in the farmer survey. Quality Declared Seed (QDS) or Planting Material (QDPM) A legally recognized class of seed produced by farmers which is not subject to the stringent control and certification procedures of certified seed but has sufficient standards to ensure quality. QDS/QDPM is generally only applicable to self-pollinating or vegetatively produced crops (e.g., bean, rice, cowpea, potato, cassava) for which there is less likelihood of loss of genetic quality. Released variety Varieties that have been officially approved by the National Varietal Release Committee through processes of varietal registration and performance testing. (Varietal) Commercialization program The strategy used by SSTP to produce and make available quality seed of improved varieties that have been released but are not yet available on the market. (Varietal) Scaling program The strategy used by SSTP to produce and make available quality seed of improved varieties that are already available on the market. ABSTRACT The Scaling Seeds and Technologies Partnership (SSTP) is a five-year, $46.8 million grants-based program implemented by the Alliance for a Green Revolution in Africa in Ethiopia, Ghana, Malawi, Mozambique, Senegal, and Tanzania (2013–2018). The evaluation of SSTP addressed overall performance of the Program and questions about: Management/Administration, Engagement of the Private Sector, the Scaling and Adoption of Technologies, and the Enabling Environment. Evaluation methods included: review of data and documentation; farmer and online surveys; key informant interviews; and focus group discussions. The evaluation concluded that SSTP has made good progress, with the main focus on improved varieties rather than complementary technologies. Partnerships, coordination, and communication have been very successful, and additional microfinance partners should be encouraged. Effective data management systems are in place and can be improved by greater attention to learning. Private sector engagement led to increased production of quality seed for the four focal crops in all six countries, though capacity constraints for seed production remain a major challenge. SSTP has been effective in increasing the use of improved varieties by both direct beneficiaries and indirect beneficiaries. Formal seed distribution strategies can be enhanced through improved marketing strategies and strengthened relationships between seed companies and agro-dealers. Linkages with output markets have been shown to create demand for seed of improved varieties. Demand can also be increased by improving the effectiveness of demonstration plots and by engaging multiple stakeholders, particularly specialized communication and promotion organizations who use an integrated range of communication channels. Promotional methods that incorporate farmer-to-farmer learning will have considerably greater impact. SSTP has had positive impacts on country-level policies and regulatory frameworks by working through national seed traders’ associations to put existing seed laws into practice. i EXECUTIVE SUMMARY EVALUATION PURPOSE This is a report of the mid-term performance evaluation of the Scaling Seeds and Technologies Partnership (SSTP), a 5-year, $46.8 million program implemented by the Alliance for a Green Revolution in Africa (AGRA). The purpose of evaluating SSTP is to assess the Program’s effectiveness and gauge ways to ensure it better meets its objectives—both in the current phase and in a potential follow-on phase (referred to here as “Phase 2”). The evaluation will also provide robust empirical evidence in response to the evaluation questions (EQs) included in the Expression of Interest (EOI). This evidence will generate learning about scaling seeds and technology adoption through the partnership approach implemented by SSTP. The evaluation’s findings are expected to contribute to AGRA’s future work on advancing agriculture-led growth in Africa. The learning from the evaluation will be of particular interest to SSTP/AGRA staff, the SSTP Advisory Committee, country teams, sub-grantees and partners, the AGRA Board of Directors, agricultural advisers from the Bill and Melinda Gates Foundation (BMGF), and United States Agency for International Development (USAID) staff at the Bureau for Food Security (BFS) and the country Missions. PROJECT BACKGROUND SSTP began in July 2013 and is being implemented in six countries: Ethiopia, Ghana, Malawi, Mozambique, Senegal, and Tanzania. The Program was originally designed to contribute towards the commitments agreed upon by the New Alliance for Food Security and Nutrition.1 The SSTP Grants program accounts for almost half of the total budget and is the primary mechanism through which SSTP aims to achieve the following three objectives: 1. Improve capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers; 2. Increase the use of quality seeds and other technologies by smallholder farmers; and 3. Improve regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers. EVALUATION METHODS The evaluation used a mixed methods approach, including: documentation review; interviews and focus group discussions (FGDs); the compilation of seed production and sales data for focal crops; an online questionnaire for grantees; and a farmer survey. Site visits were carried out in three countries: Ghana, Mozambique, and Tanzania. Within these three countries, the evaluation focused on the value chains of the following crops: maize in Tanzania and Ghana; cassava in Mozambique; and potato and beans in Tanzania. The farmer survey was conducted in two waves: the first wave was implemented in the three countries in March-May 2017; the second wave will be implemented in early 2018. The evaluation focused on the overall performance of the program plus the four specific themes of the evaluation questions. The evaluation questions, together with the key findings, conclusions, and recommendations are summarized below. SUMMARY OF FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 1 The New Alliance was initiated in 2012 by ten African governments, the private sector, and G8 members to focus, accelerate, and coordinate their joint efforts aimed at reducing poverty and hunger in Africa. One of the New Alliance commitments was “taking innovations to scale” and involved several enabling actions including: establishing 10-year targets for sustainable yield improvements in national priority value chains; identifying core sets of technologies that would contribute to achieving those targets; and ensuring access to those technologies at sufficient scale. By the time of the evaluation, the New Alliance was no longer active, although some countries had incorporated their commitments into other strategies. ii Findings Conclusions Recommendations Overall Performance The SSTP Year 3 Annual Report provides data for 17 out of the 29 indicators: • Fourteen indicators out of the 17 are “on target;” and • Three indicators are not expected to meet their target by program end. There is a mismatch between the actual focus of the Program (largely on seed production and promotion (41 out of 60 grants, 68 percent) and the Results Framework. Despite an emphasis on “associated” or “complementary” technologies in the Results Framework, only 6 out of 60 grants (10 percent) were explicitly supporting such technologies (soil fertility, crop protection, mechanization). The SSTP Program does not have an explicit Theory of Change (ToC) framework. It is not clear how the various activities contribute to outcomes described in the SSTP Results Framework, and it is not clear how the outcomes lead to the goals. Based on the indicator data available at the end of Year 3 and the achievements noted by the evaluation team (ET), SSTP has made good progress. The main focus of the Program (as evidenced by the allocation of grant types) is to increase the availability and use of improved varieties. For maximum impact on productivity (and hence the overall goal of improved food security and reduced poverty), improved varieties must be promoted together with complementary technologies. A ToC is an effective coordination tool. By involving project staff and partners in generating a ToC, everyone develops a more detailed understanding of the program and how their own activities contribute to making change happen. Increased attention and investment should be allocated to complementary technologies such as integrated soil health, integrated crop protection, and labor-saving mechanization. For Phase 2, it is recommended that AGRA/SSTP staff develop a program-level ToC, and that in-country staff and partners develop nested country-level ToCs. Such ToCs will ensure the internal logic of the Program, and can help check that activities undertaken are leading towards intended outcomes. Evaluation Theme 1: Management/Administration EQ 1.1: What are the strengths and weaknesses of AGRA’s grants selection process as a way to select the sub grantees with the highest potential? In what ways could it be improved? Two different grant selection mechanisms were used: competitive requests for proposals and solicited or sole source grants. Applications through both mechanisms were closely based on the priorities in the Country Roadmaps. Solicited and sole source applications skipped the concept note stage. The Grants Committee evaluated all grant proposals. Grants over $300,000 were also reviewed externally. The decision of the Grants Committee was based on verbal consensus and did not involve any written feedback from individual Committee members. Two out of the three most senior SSTP officers felt that the AGRA selection criteria were not in line with the SSTP’s objectives. At least one The grant selection process has its strengths and weaknesses. Examples of strengths: ● Competitive grants are complemented by strategic solicited grants to ensure that potential gaps are addressed. ● The Grants Committee ensures that, in principle, AGRA standards are met, strategy is adhered to, and approved grantees have the required capacity. Examples of weaknesses: ● The AGRA selection criteria used by the Grants Committee were not revised by It is recommended that the Grants Committee adopt a simple, written individual scoring and comments system to make the grants selection process more transparent and ensure all members of the Committee are fully engaged. It is also recommended that all members of the Grants Committee are fully conversant with the proposed SSTP ToC (if not involved in its development) and that the grants selection criteria are revised to ensure clarity and consistency with the Phase 2 rationale and proposed ToC. iii Findings Conclusions Recommendations senior SSTP officer felt that the Committee’s use of verbal consensus lacked transparency. SSTP, yet the objectives of SSTP are different from AGRA. ● The verbal consensus that the Grants Committee uses to review proposals creates a potential risk of bias. ● Solicited proposals skip the concept note stage of the application process, possibly resulting in weaker applicants going through to the proposal stage. EQ 1.2: How effective has the Partnership been in supporting the role of the private sector and other key stakeholders as engines for sustainable development in their countries? EQ 1.3: How effectively has the project management facilitated communication and coordination amongst the different stakeholders involved in the Partnership, including AGRA and its country offices, USAID, grantees, and beneficiaries? Are information transfers among sub-grantees, sharing of lessons learned, and collaborations evident? In what ways could communications and coordination be improved as a way to advance project goals and objectives, meet reporting requirements, and encourage additional relevant partners? Fifty-seven out of 58 grantees who responded to the online survey (98 percent) strongly agreed that the SSTP grant allowed their organization to contribute towards sustainable development. All private sector grantee interviewees were able to provide examples of the same. The ET found numerous examples of SSTP facilitating collaborative linkages between grantees and other partners and donor-funded projects. One gap in the range of partnerships was with providers of credit to farmers. SSTP has recently started working with AGRA’s Financial Inclusion for Smallholder Farmers in Africa Project (FISFAP) in Ghana and Tanzania to address this. Grants were based on the priorities identified by the Country Roadmap, thus ensuring good coordination among the grants. The Early Generation Seed (EGS) studies similarly fostered effective communication and coordination among grantees, partners, and stakeholders. The Partnership has been very effective in supporting the role of the private sector and other stakeholders as engines for sustainable development in their countries. The SSTP Country Teams have been critical in facilitating effective partnerships, linkages, coordination, and communication at the country level. There are opportunities to encourage additional relevant partners, particularly those from related sectors such as microfinance, food security, resilience, and nutrition. Coordination was effectively achieved through the Country Roadmaps and the EGS studies, as well as regular meetings of grantees, partners, and stakeholders. It is recommended that AGRA and SSTP staff reach out to additional relevant partners in the microfinance, food security, resilience, and nutrition sectors. This can be done through existing in-country donor and sectoral coordination platforms. Under Phase 2, it is recommended that the Country Annual Review and Planning meetings continue to take place and that – in addition – the lessons emerging from these discussions are documented and shared to help promote learning across countries and encourage subsequent implementation. Before the end of Phase 1, it is recommended that an internal, participatory lesson-learning exercise be conducted and documented at country- and program-levels. This will help determine what has and hasn’t regarding types of partnerships, approaches iv Findings Conclusions Recommendations Country Annual Review and Planning meetings were reported by meeting participants to be effective in information sharing. The lack of documentation from these meetings suggests that sharing and application of lessons learned is limited. Annual Review and Planning meetings can be made more effective in systematically capturing and sharing the broader lessons of what is working and what is not. Missing is the “big picture” of which types of partnerships, models, technologies, and innovations are working and which are not. to capacity development of seed companies and agro-dealers, models for seed production, marketing for seeds produced, strategies for promoting farming adoption at scale, etc. The current evaluation report provides a starting point to such an exercise. EQ 1.4: Are data management systems in place to collect, track, and report on AGRA and Feed the Future (FTF) indicators, outputs, outcomes, and impacts from the beneficiary level and are they comparable across grantees? Does the collection and reporting conform to USAID requirements? SSTP uses a document management system based on DropBox, which allows SSTP to conform to USAID reporting requirements and environmental regulations. Ten Feed the Future indicators, complemented by SSTP custom indicators, are tracked and reported. Information from the Indicator Performance Tracking Tables (IPTTs) reveal that commercial companies are required to report on twice as many indicators as other grantees. The seed companies do not have the capacity for this level of reporting. SSTP has faced challenges in tracking farmer beneficiaries. Grantees struggle with collecting data on beneficiaries’ purchase of SSTP seed sold by agro-dealers. The current emphasis is on data for reporting, with less attention paid to data for learning, not only for SSTP as a whole, but also for grantees. In Phase 2, it is recommended that SSTP manage data in ways that support two types of learning: 1) learning by grantees (notably commercial companies) involves the joint identification of indicators and how these can be applied in ways that will allow their businesses to develop, i.e., indicators must be useful to commercial growth; and 2) systemic or programmatic learning, which can be guided by a program-level ToC and up-front articulation of questions to be addressed, both within countries and across countries. Evaluation Theme 2: Engagement of the Private Sector EQ 2.1: What models has SSTP used to make technologies promoted by SSTP available? To what extent has the Partnership increased the availability of promoted technologies in markets in target areas? What are the most important barriers that AGRA and the sub-grantees must yet address while trying to increase availability of improved seeds and technologies in local markets? Different models for making seeds available to farmers were distinguished according to the number of constraints2 addressed and the number of partners involved. The report presents data to quantify the actual increases in seed production for the different seed classes (foundation, The availability of quality seed increased for all seed classes and for all focal crops in all countries. For a greater efficiency, quantities of foundation seed produced could have been tied to forecasted quantities of certified seed required. SSTP should develop transparent seed requirement forecasting methods and procedures. These could be piloted by a broad base of partners and stakeholders in Phase 2 in order to ease planning for 2 The constraints identified by SSTP and the EGS studies were: 1) the unavailability of adequate quantities of breeder/foundation seed to produce certified seed or planting materials on a timely basis and in large enough quantities; 2) the capacity of private seed companies to produce and distribute certified seed produced to smallholder farmers; and 3) value chain linkage to output market for farmer-produced commodities. v Findings Conclusions Recommendations certified), for the focal crops (maize, bean, potato, cassava) in each of the six SSTP countries. Eight categories of barriers to increasing the availability of improved seeds in local markets were identified; four of these categories include or relate to capacity constraints. SSTP is already addressing all but two of the barriers identified. For bean and cassava, limited data available suggest that a public-private partnership (PPP) model leads to considerably bigger increases in seed availability than the single organization models. Plans must be in place to ensure the sustainability of PPPs when donor funding is completed. Barriers that have yet to be addressed include: 1) the lack of effective forecasting methods and procedures; and 2) the lack of workable quality management systems for seed production. Capacity constraints for seed production within private and publicsector organizations remain a major challenge. multiplication of seed of the different classes.3 SSTP should support its grantees to develop and implement a workable quality management system for seed production in Phase 2. Ongoing efforts to develop capacity for seed production within private and public sector organizations should be expanded and strengthened in Phase 2. In the report, there are examples of how this might be done. EQ 2.2: What types of technologies/crops are more likely to be produced and made available on a wide scale by the private sector and why? Which segments of the seed or technology value chain is the private sector likely to take on? Which segments are likely to be more difficult for the private sector and why? How can the Partnership improve its approach to scaling through the private sector in light of these differences? EQ 2.3: What has been the private sector’s experience creating demand under the Partnership? How could the private sector’s effectiveness at creating demand be improved through the Partnership? Seeds likely to be made available by the private sector depend on the crop and its profitability potential, as defined by the seeding rate, bulking rate, cost of seed production, and the nature of demand by farmers. Based on these criteria, hybrid maize is most likely to be produced and made available on a wide scale by the private sector. As such, hybrid maize forms the “Private Sector Dominant” archetype in the EGS model. 4 Other EGS archetypes suitable for private sector investment include “Private￾Public Collaboration Archetype” [e.g., open-pollinated variety (OPV) maize] and the “Niche Private Sector Archetype” (e.g., potato). Seed value chains suitable for private sector investment depend largely on the characteristics of the crop itself. The EGS archetypes provide a good indication of the types of crops likely to be produced and made available by the private sector. Crops with strong IPR protection are more likely to be taken on by the private sector. The private sector can specialize in all, one, or a few segments in the value chain provided SSTP should identify and promote the development of financing strategies appropriate to private sector seed companies and facilitate the development of linkages with appropriate financial institutions for credit to expand scaling up activities. This activity can be implemented through the national seed traders’ associations. SSTP should place greater emphasis on assisting grantee seed companies to develop their sales and marketing skills. 3 Although governments theoretically have teams to conduct annual seed forecasting, the lack of clear forecasting methods in the countries visited suggest that there is a need for SSTP to support the development of more effective procedures. 4 The Early Generation Seed (EGS) model was commissioned by the Bill and Melinda Gates Foundation and was applied by SSTP in four countries. vi Findings Conclusions Recommendations Interviews with private sector grantees revealed that the following value chain segments are suitable for private sector investment, provided that there is appropriate intellectual property rights (IPR) protection for the crops concerned: foundation seed multiplication; certified seed processing; certified seed out-growing; germplasm development and licensing; and the production of clean planting materials through tissue culture. Grantees were involved in different partnership arrangements involving private sector companies. In the case of one PPP in Mozambique, the decision not to extend the SSTP grant threatened to end the partnership because alternate financing arrangements had not been identified. Five out of the six interviewed private sector grantees involved in certified seed production lacked field-based marketing agents and relied on government extension agents, farmers and agro-dealers, and/or partnerships with NGOs or others to help promote their seeds/varieties. Grantee seed companies used traditional methods to create demand for seed of improved varieties. By the end of Year 3, 59,706 demonstrations had been implemented, but it is not known how many of these were effective. Four out of the six interviewed private sector grantees involved in certified seed production reported challenges in marketing and in their relationships with agro-dealers. All interviewed private sector companies involved in certified seed production recognized that farmers are more likely to adopt a variety if they can sell the output. Two grantees were developing linkages to such output markets. that an opportunity to derive value from the activity exists, and that financing is available. Lack of access to financing by seed companies, agro-dealers, out-growers, and farmers is one of the main barriers to scaling the multiplication, distribution, and adoption of improved seed. There is a need to improve formal seed distribution strategies through improved marketing strategies and the strengthening of sustainable, working relationships between seed companies and agro-dealers. The selection criteria of crops and varieties to be produced, marketed, and sold by private sector companies should consider the market potential of the end products as well as their suitability to the local agro-ecological conditions. It is necessary to link seed value chains with associated product or output value chains to ensure that inputs provided meet the needs of existing associated output markets. Linkages with output markets have been shown to create demand for seed of improved varieties. The private sector’s effectiveness in creating demand can be enhanced by improving effectiveness of demonstration plots and by engaging multiple stakeholders (including ICT Challenge partners) working in the seed and output value chains to create customer￾SSTP/AGRA should facilitate in-country discussions with crop value chain actors to foster seed companies’ and farmers’ linkages to viable output markets, which demand quality seeds of specific improved varieties. SSTP/AGRA should also seek linkages with other government and donor-funded programs working on expanding agricultural productivity through agricultural market development. SSTP grantee seed companies must be more systematic in ensuring that the varieties selected for multiplication and marketing meet the needs of the final consumers, e.g., specific maize varieties for use in the poultry feed industry; improved cassava varieties for beer brewing; or potatoes suitable for frying chips. SSTP should develop good practice guidelines for demonstration plots (based on SSTP experience and existing literature) so that they are as effective as possible and reach more farmers. SSTP should facilitate linkages and support partnerships between private seed companies and specialized, experienced communication organizations for effective demand creation (see below). vii Findings Conclusions Recommendations driven demand for seed of improved varieties. Evaluation Theme 3: Scaling and Adoption of Technologies EQ 3.1: To what extent have AGRA and its sub-grantees been able to increase the use of quality seeds and improved technologies by smallholder beneficiary farmers? By indirect beneficiary farmers? EQ 3.2: What types of strategies, innovations, or support have been the most effective in encouraging use of improved seeds and technologies for just one season? Over multiple seasons (adoption vs. application)? What important barriers remain by: Direct project beneficiaries? Indirect beneficiaries (defined as, those in “catchment areas” of activities who aren’t directly targeted by the activity)? Early adopters? Majority adopters? Late adopters? The poor, women, and socially marginalized groups? What other types of approaches and methodologies could the Partnership incorporate in its projects to encourage indirect beneficiaries to both try out and continue to use new agricultural technologies and practices? Survey results show that across the three countries: • 28.5 percent of sampled beneficiary farmers are reported to have adopted SSTP varieties more than three years ago (i.e., prior to the start of the SSTP Program); • 34.9 percent of beneficiary farmers adopted or applied5 the varieties since the start of the SSTP Program; • 44.4 percent of direct beneficiaries and 21.5 percent of indirect beneficiaries have adopted or applied SSTP varieties since the start of the SSTP Program; and • 36.6 percent of all sampled farmers are reported not to have applied or adopted any SSTP varieties at all. In all countries, neighbors or relatives provided the main motivation that persuaded beneficiaries (both direct and indirect) to plant the improved variety for the first time. A farmer’s decision to adopt or discontinue a particular variety relates to the particular varietal characteristics, the presence or absence of output markets, and the cost and availability of seed. There was no correlation between The SSTP Program has been effective in increasing the use of improved varieties by both direct beneficiaries and indirect beneficiaries. Promotional methods that are able to draw on the influence of neighbors and relatives will have considerably greater impact than those which do not. Adoption of new varieties can best be encouraged by promoting varieties with preferred characteristics, for which output markets are present, and by making seed available at an affordable cost. Specific efforts are required to link farmers to output markets. Effective technology promotion uses an integrated set of methods and communication channels (e.g., radio, SMS, IVR, TV, drama, As above, it is recommended that SSTP supports efforts to identify and link farmers to output markets for the sale of products from the cultivation of improved varieties. Grantees should promote varieties in ways that encourage farmer-to-farmer learning and/or by working with experienced communication partners who have the capacity to encourage farmer-to-farmer learning. Examples of farmer-to-farmer learning already being used by SSTP grantees are provided in the report. SSTP should expand support to technology promotion by experienced and specialized communication organizations, which use an integrated set of mutually reinforcing methods and communication channels. Partnerships between such organizations and private seed companies should be encouraged. 5 A farmer typically tests or tries out a new variety for two or three seasons before deciding whether or not to adopt it into their planting repertoire. Within the SSTP Program, this period of testing by a farmer is referred to as “application” and will often lead to adoption. In general, there is a very blurred line between application and adoption, but a farmer that has been growing a variety for more than three years can be considered to have adopted it. viii Findings Conclusions Recommendations decisions to adopt/discontinue particular varieties and the strategies used to promote them or the ways in which farmers acquired the seed. Affordability and the absence of output markets emerged as the key barriers to the adoption of improved varieties for both direct and indirect beneficiaries. Additional barriers for indirect beneficiaries included not knowing where to get inputs and a lack of knowledge about improved varieties. A comparison of pre-SSTP adopters, SSTP adopters, and non-adopters with gender and socio-economic status does not show any significant correlations. Three particularly innovative and successful approaches to technology promotion implemented by SSTP communication partners were identified. All rely on integrated methods and communication channels that mutually reinforce each other. Each involves a highly specialized and experienced organization applying or adapting approaches honed over many years. mobile cinema, group events, village-based extension agents, farmer-to-farmer, etc.) that mutually reinforce each other. These approaches are most successful when implemented by specialized and experienced organizations. Seed companies would do well to partner with such organizations to promote their technologies among farmers. The presence of output markets can encourage farmers to apply and continue to use improved varieties. It cannot be assumed that farmers are able to tap into these markets on their own. EQ 3.3: At what level of market penetration6 of potential market demand for targeted technologies is there likely to be spontaneous adoption (over 2+ growing seasons) by indirect beneficiaries (e.g., what is the take-off point at which technologies could “go viral”)? Analysis of take-off points will be possible when the second wave of survey results are available. N/A N/A Evaluation Theme 4: Enabling Environment EQ 4.1: What progress has the Partnership made in supporting country-level and regional policy and regulatory mechanisms with the aim of increasing the production and delivery of improved seeds and technologies? What progress has there been in contributing to New Alliance objectives of improving the enabling environment, as elaborated by the country-specific New Alliance Cooperation Frameworks within countries and across countries? What other opportunities are there for the Partnership to support attainment of New Alliance objectives of enabling environment improvement? EQ 4.2: What additional policy reforms, in order of priority, should the Partnership support (including AGRA, the Advisory Committee, sub-grantees, donors, other stakeholders, etc.) to: ● Raise the supply of improved seeds and technologies in markets in target areas? 6 Market Penetration is defined as the number of people who buy a specific technology at least once in a given period, divided by the size of the relevant market population. ix Findings Conclusions Recommendations ● Increase the production and delivery of improved seeds and technologies on a national and regional level? The New Alliance itself is no longer on the policy agenda. The lack of an effective New Alliance framework has not impeded the progress made in improving the enabling environment in each of the countries visited. SSTP’s role has been to help in “translating” existing seed laws into practice. In Tanzania and Mozambique, the National Seed Traders’ Association, the Southern Agricultural Growth Corridor of Tanzania (SAGCOT), and national platforms for crop value chains played an important role in improving the enabling environment. Effective and well-articulated seed laws already exist in the SSTP countries and do not need to be reformed. One potential gap in most countries (apart from Tanzania and Ethiopia) is the provision for Quality Declared Seed (QDS) and Quality Declared Planting Material (QDPM). SSTP has commissioned detailed studies to identify issues for future policy engagement in specific countries. These include The African Seed Access Index (TASAI) studies and studies by New Markets Lab and SAGCOT in Tanzania. At a regional level, SSTP has supported the development of the Common Market for Eastern and Southern Africa (COMESA) seed catalogue of released and registered varieties, which enables private and public sector actors to know which varieties are approved, and allows them to make business decisions based on data and information. The issues being addressed by SSTP to support country-level policy and regulatory systems are similar in each country visited, though each country is at a different level of development and implementation of seed policies. Policy and regulatory development is a slow and difficult process. By working through national seed traders’ associations and other lobbying platforms, SSTP has had positive impacts. Continued support is needed to put existing laws into practice through the development of guidelines and protocols for implementation. There is also a continued need for awareness creation and capacity development to make seed sector actors aware of existing regulations and newly developed protocols. One possible reform to consider is one that allows for Quality Declared Seed/Planting Material (QDS/QDPM); this seed category currently only exists in Tanzania and Ethiopia. SSTP should continue to strengthen and work through national seed traders’ associations and other lobbying platforms. It is recommended that SSTP commissions a review of the practical application of QDS and QDPM laws and regulations in Tanzania and Ethiopia. This would help determine whether QDS and QDPM might be applied in other countries. Such a review should be undertaken in a participatory manner, involving policy-makers from all SSTP countries (similar to the EGS study) so that it stimulates in-country dialogues on the advantages and disadvantages of QDS and QDPM for different crops (notably cassava, potato, beans, and rice), opening up the possibility of developing QDS and QDPM laws and regulations in countries where they do not currently exist. 1 1.0 EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE This is a mid-term performance evaluation report of the Scaling Seeds and Technologies Partnership (SSTP), a 5-year, $46.8 million program, implemented in Ethiopia, Ghana, Malawi, Mozambique, Senegal, and Tanzania from July 2013 – July 2018 by the Alliance for a Green Revolution in Africa (AGRA). The purpose of evaluating SSTP is to assess the Program’s effectiveness and gauge ways to ensure it better meets its objectives—both in the current phase (ending in July 2018), and in a potential follow-on phase (referred to here as “Phase 2”). In addition, the evaluation is expected to provide robust empirical evidence in response to the evaluation questions (EQs) included in the Expression of Interest (EOI), which will generate learning about scaling seeds and technology adoption through the partnership approach being implemented by SSTP. 1.2 EVALUATION AUDIENCE The evaluation's findings are expected to contribute to AGRA’s future work on advancing agriculture￾led growth in Africa, and will be of particular interest to SSTP/AGRA staff, the SSTP Advisory Committee, country teams, sub-grantees and partners, the AGRA Board of Directors, United States Agency for International Development (USAID) staff [both at the Bureau for Food Security (BFS) and the country Missions], and agricultural advisers from the Bill and Melinda Gates Foundation (BMGF). 1.3 EVALUATION QUESTIONS The EQs cover the four themes presented below: Management/Administration 1. Grant selection process: What are the strengths and weaknesses of AGRA’s grants selection process as a way to select the sub-grantees with the highest potential? In what ways could it be improved? 2. Effective support to partners: How effective has the Partnership been in supporting the role of the private sector and other key stakeholders as engines for sustainable development in their countries? 3. Communication and coordination: How effectively has the project management facilitated communication and coordination amongst the different stakeholders involved in the Partnership, including AGRA and its country offices, USAID, grantees, and beneficiaries? Are information transfers among sub-grantees, sharing of lessons learned, and collaborations evident? In what ways could communications and coordination be improved as a way to advance project goals and objectives, meet reporting requirements, and encourage additional relevant partners? 4. Data management systems: Are data management systems in place to collect, track, and report on AGRA and Feed the Future indicators, outputs, outcomes, and impacts from the beneficiary level and are they comparable across grantees? Does the collection and reporting conform to USAID requirements? Engagement of the Private Sector 1. Availability of improved seeds and technologies: What models has SSTP used to make technologies promoted by SSTP available? To what extent has the Partnership increased the availability of promoted technologies in markets in target areas? What are the most important barriers that AGRA and the sub-grantees must yet address while trying to increase availability of improved seeds and technologies in local markets? 2 2. The roles of the private sector in seed/technology value chains: What types of technologies/crops are more likely to be produced and made available on a wide scale by the private sector and why? Which segments of the seed or technology value chain is the private sector likely to take on? Which segments are likely to be more difficult for the private sector and why? How can the Partnership improve its approach to scaling through the private sector in light of these differences? 3. Effectiveness of the private sector in creating demand: What has been the private sector’s experience creating demand under the Partnership? How could the private sector’s effectiveness at creating demand be improved through the Partnership? Scaling and Adoption of Technologies 1. Use of quality seed and improved technologies by farmers: To what extent have AGRA and its sub￾grantees been able to increase the use of quality seeds and improved technologies by smallholder beneficiary farmers? By indirect beneficiary farmers? 2. Encouraging adoption vs. application7 by different farmer types: What types of strategies, innovations, or support have been the most effective in encouraging use of improved seeds and technologies for just one season? Over multiple seasons (adoption vs. application)? What important barriers remain by: • • • • • Direct project beneficiaries? Indirect beneficiaries (defined as, those in “catchment areas” of activities who aren’t directly targeted by the activity)? Early adopters? Majority adopters? Late adopters? The poor, women, and socially marginalized groups? What other types of approaches and methodologies could the Partnership incorporate in its projects to encourage indirect beneficiaries to both try out and continue to use new agricultural technologies and practices? 3. Take-off points: At what level of market penetration8 of potential market demand for targeted technologies is there likely to be spontaneous adoption (over 2+ growing seasons) by indirect beneficiaries (e.g., what is the take-off point at which technologies could “go viral”)? Enabling Environment 1. Progress to date: What progress has the Partnership made in supporting country-level and regional policy and regulatory mechanisms with the aim of increasing the production and delivery of improved seeds and technologies? What progress has there been in contributing to New Alliance objectives of improving the enabling environment, as elaborated by the country-specific New Alliance Cooperation Frameworks within countries and across countries? What other opportunities are there for the Partnership to support attainment of New Alliance objectives of enabling environment improvement? 2. Additional policy reforms needed: What additional policy reforms, in order of priority, should the Partnership support (including AGRA, the Advisory Committee, sub-grantees, donors, other stakeholders, etc.) to: • • Raise the supply of improved seeds and technologies in target-area markets? Increase the production and delivery of improved seeds and technologies on a national and regional level? 7 The term “application” is unfamiliar to the Evaluation Team, though it is assumed that it refers to farmers’ testing or trying out an improved seed variety or technology for just one season before deciding whether or not to use it again the following season. A brief literature review will be conducted to determine whether this is the most appropriate term to use in this context. 8 “Market Penetration” is defined as the number of people who buy a specific technology at least once in a given period, divided by the size of the relevant market population. 3 2.0 PROJECT BACKGROUND The New Alliance for Food Security and Nutrition (New Alliance), is an initiative of the governments of 10 African countries, the private sector, and G8 members to focus, accelerate, and coordinate joint efforts over the next 10 years to reduce poverty and hunger in Africa. SSTP was designed to contribute to the agreed-upon New Alliance commitments. One of these commitments is “taking innovations to scale,” which involves several enabling actions, including: establishing 10-year targets for sustainable yield improvements in national priority value chains; identifying core sets of technologies to contribute to achieving targets; and ensuring access to those technologies at sufficient scale. SSTP’s objective is to work with 6 of the 10 New Alliance countries to meet 10-year agricultural production targets set by the countries themselves. To do this, SSTP is assisting Ethiopia, Ghana, Malawi, Mozambique, Senegal, and Tanzania to coordinate sector strategies and programs in order to strengthen the seed sector as well as promote commercialization, distribution, and adoption of seeds and related key technologies with the purpose of increasing smallholder yields and resilience to climate change. SSTP outcomes are illustrated in Table 1, and are targeted at increasing the supply of improved varieties of quality seed and other technologies to achieve the following objectives: 1. Improve capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers; 2. Increase the use of quality seeds and other technologies by smallholder farmers; and 3. Improve regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers. Table 1. SSTP Results Framework SSTP Results Framework New Alliance for Food Security & Nutrition Goal: To achieve sustained and inclusive agricultural growth and raise 50 million people out of poverty over the next 10 years SSTP Goal: Improve food security and reduce poverty among smallholder farmers in targeted areas within selected sub-Saharan African countries Objective 1: To improve capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers Objective 2: To increase the use of quality seeds and other technologies by smallholder farmers Objective 3: To improve regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers Outcomes: 1.1 Improved access to business development and financial services by agro-enterprises 1.2 Increased sustainable supply of quality seeds 1.3 Improved capacity of staff for quality seed production and delivery 1.4 Improved infrastructure to support quality seed production and delivery 1.5 Increased public-private partnerships Outcomes: 2.1 Improved smallholder farmers’ access to knowledge and skills 2.2 Increased capacity of farmer organizations to support use of quality seeds and technologies 2.3 Increased use of Information and Communication Technology (ICT)-enabled extension services by smallholder farmers (for New Alliance ICT Extension Challenge Fund Grantees only) Outcomes: 3.1 Increased consultation by private and public sector on key seed production and delivery policy and regulatory issues 3.2 Increased coordination and harmonization of country- and regional-level seed production and delivery 3.3 Increased capacity of producer and consumer associations to engage in seed sector policy formulation and implementation 3.4 Increased capacity of public and private sector to monitor and 4 SSTP Results Framework regulate seed production and delivery 3.5 Increased self-monitoring, measurement, documentation, and communication of progress by SSTP focus country governments The development hypothesis behind SSTP is that more purposeful, effective coordination of technology￾scaling opportunities for seeds and other technologies by a willing private sector within an enabling commercial framework will lead to rapid agricultural productivity increases. For technologies where commercial potential is currently untested or weak, more effective coordination and innovative pilots may stimulate new market development or suggest ways to address these scaling challenges. Better monitoring of commercialization and adoption experiences will provide information on which to base future plans and activities. SSTP intends to support countries in transitioning from state-dominated seed systems to systems that allow private sector provision of key services (e.g., multiplication of foundation seeds), and that strengthen the capacity of the state to carry out critical regulatory functions (e.g., varietal release and seed certification). SSTP aims to support national and regional efforts to revise and implement seed laws, with special focus on: regional harmonization of variety release and seed trade; establishing foundation seed production in core countries; creating or expanding seed company capacities and seed trade associations; and working with agro-dealers and providers of credit to ensure farmers have awareness of and access to improved seeds. As part of their policy commitments under the New Alliance, the target countries pledged to make adopting the best available technologies a top priority. They also agreed to implement the necessary regulatory reforms and assess progress against adoption goals annually. SSTP builds on and continues some of the work of AGRA’s earlier Program for African’s Seed Systems (PASS). SSTP’s objectives are to be met through these seven activities: 1. Developing “Road Maps” of specific public and private sector actions needed 2. Coordinating and aligning efforts among public and private sector actors and donors 3. Providing technical support locally for building the capacity of public and private actors 4. Modeling and piloting priority activities through grant-making 5. Providing regional technical support for finding solutions to cross-cutting issues 6. Monitoring and benchmarking progress toward goals 7. Addressing constraints to regional harmonization The project document does not contain a Theory of Change (ToC) that explains how the above activities and their associated tasks contribute towards the outcomes identified in the Results Framework. SSTP’s start date was July 2, 2013 and the original end date was July 1, 2016. However, stakeholders agreed to two extensions that put the end date at July 1, 2018. The total project duration is five years, with a total budget of $46,769,842—$22 million of which is grants to partners. SSTP will achieve its targets primarily though the Grants Program and, at the time of the evaluation, 60 grants had been awarded. 5 The SSTP program description shows that the overall effect will be a better-aligned and coordinated set of donor and government actions, which leverage greater private sector investment in seed and other technology marketing. AGRA will also convene international and regional groups active in developing seed systems to discuss specific challenges and solutions, and gather momentum at a regional level. Through direct investments to alleviate specific bottlenecks in seed and other input supply chains, SSTP is considered a definitive intervention in resolving barriers to food security—barriers that have for decades caused human suffering and limited Africa’s chances for economic development. 3.0 EVALUATION METHODS AND LIMITATIONS To evaluate the SSTP Program, the evaluation team (ET) used a mix of robust quantitative and qualitative methods. As indicated in the Expression of Interest (EOI) in Annex A, the evaluation covers the entire Program. However, due to budget constraints and the geographic spread of the Partnership, site visits were carried out only in Ghana, Mozambique, and Tanzania. Within these countries, the evaluation focused on the value chains of the crops selected by the EOI—maize in Tanzania and Ghana, cassava in Mozambique, and potato and beans in Tanzania. Data were/will be collected in three phases: 1. Qualitative and quantitative data collection by the ET (January-March 2017), including documentation review, interviews, focus group discussions (FGDs), the compilation of seed production and sales data for focal crops, the online questionnaire for grantees, and remote interviews. 2. Quantitative data collection through a farmer survey. Sub-contracted survey companies collected round 1 survey data from March-May 2017. 3. Sub-contracted survey companies will collect round 2 survey data in 2018. 3.1 QUALITATIVE DATA COLLECTION AND ANALYSIS The ET compiled much of the qualitative data through country field visits to Ghana, Tanzania, and Mozambique—plus a short visit to Nairobi to meet with senior SSTP and AGRA staff and one Nairobi￾based grantee. The travel itinerary is presented in Annex B. The team also reviewed literature and project documentation, and undertook additional remote interviews. 3.1.1 Review of SSTP Project Documentation Documents reviewed included the original project documents, Country Roadmaps, country baseline reports, quarterly and annual reports, minutes from committee meetings, monitoring and evaluation (M&E) reports (including tracking, reporting, and data quality assessment reports), agreements with and proposals from partners and grantees, progress reports from grantees, and the Outcome Indicators’ Annual Assessments reports (see Annex C). 3.1.2 Review of Relevant Literature Where necessary, the broader literature on technology promotion and adoption was consulted and focused primarily on recent publications relating to technology scaling, methods for measuring technology adoption, and impacts of particular approaches used by SSTP grantees, as included in Annex C. 3.1.3 Face-to-Face Interviews and Direct Observation Table G.1 in Annex G summarizes the 95 semi-structured key informant interviews (KIIs) undertaken in January and February of 2017. Annex E includes the list of individuals interviewed, and Annex F presents 6 the interview guides for the various types of key informants. The ET visited the premises of selected grantee seed companies and research stations, agro-dealer hub and retail shops, and farmers’ fields and demonstration plots. Within each country, the ET used three criteria to select grantees producing certified seed: 1) those working on the focal crops; 2) those with the longest and most successful engagement with SSTP activities; and 3) those with office locations that made the logistics of visiting them and others along the same seed value chain easier. After certified seed producer grantees were selected, other grantees and non-grantee/partner key informants working at other nodes along the same value chain were identified. The ET made the selections in consultation with the SSTP Country Coordinators. As such, many of the interviewees and FGD participants were selected – insofar as possible – from the same seed value chain (defined by crop and variety) as seed producers (those involved in developing and producing breeder seed, foundation seed, and certified seed) to seed consumers (farmers). This allowed for data triangulation at each node of the chain. In addition, all of the “communication” grantees involved in promoting farmer knowledge and awareness for the selected crops were interviewed in each country, and visits were made to the beneficiary communities where they undertook activities on the ground. Table G.2 in Annex G provides summary information about the grantees met in each country. As indicated by the table, 16 of the 19 grantees working on the evaluation focal crops for all three countries were interviewed. Site visits were made to 15 out of the 17 grantees with sites (excluding the two policy and capacity development grants which do not have sites). All four different grant types and all four different institutional types were included in meetings (Table G.2, Annex G). 3.1.4 Skype/Telephone Interviews In the cases of 11 key informants that the ET was unable to meet with in person, interviews were conducted via email, Skype, and telephone in late February and March of 2017. These interviews are noted in Annex E, and include SSTP Country Coordinators in countries not visited, USAID staff, and critical Africa seed system actors. 3.1.5 FGDs and Round Table Meetings The ET conducted two different types of FGDs between January and February 2017: one with leaders and members of out-grower groups who had been contracted by one of the selected grantees, and the other with male and female direct and indirect farmer beneficiaries (these were held both with men and women together and with men and women separately), depending on the number of participants. Group sizes averaged 18, but varied from 9 to 32 participants. All but one involved a translator. A total of two roundtable meetings were held, one with partners of the Entrepreneurship for Commercial Seed Incubation Business (ECoSIB) project in Ghana, and another with staff and partners of the Farm Input Promotions Africa Ltd. (FIPS) project in Tanzania. 3.1.6 Qualitative Data Analysis Members of the ET kept written notes for all interviews, meetings, and discussions. These were then summarized and typed into Word using the template in Annex F, and included implications relevant to the EQs. Highlighting the implications of the compiled information from each interview meant that the analysis process effectively started while the interview notes were being written up. Regular, daily discussions among the members of the ET allowed for a cumulative process of cross-analysis with other interviews/data sources. To allow for easy referencing and cross checking, all interview notes for each country were combined into a single Word file. This also made triangulation through word or phrase searches at the report-writing stage easier. To allow for the cross-analysis of findings between the countries, the notes relating to the implications for each of the four evaluation themes (for all three countries) were also compiled into four separate documents. At the end of the country visits to Ghana and Tanzania, the ET jointly discussed and analyzed the findings emerging in relation to each of the EQs. This was summarized as a PowerPoint presentation, 7 which was presented and discussed with SSTP in-country staff, USAID Mission staff, and other SSTP partners and stakeholders. Such a meeting was not possible in Mozambique due to the limited time in￾country and the geographical spread of relevant partners and stakeholders. 3.2 QUANTITATIVE DATA COLLECTION AND ANALYSIS The quantitative approach involved the online self-administered questionnaire for all grantees, the compilation of data on seed production and sales from all six countries for the focal crops, and the ongoing farmer survey. 3.2.1 Online Self-Administered Questionnaire for Grantees In March 2017, the ET conducted an online survey, using Qualtrics software, with SSTP grantees in all six countries and regions. The survey collected data relating to the specific activities undertaken by different types of grantees, their participation in policy processes, and their perceptions regarding effectiveness, profitability, capacity development, and partnerships. All 60 SSTP grantees were invited to participate in the survey. Fifty-eight individuals responded, and data were analyzed in Qualtrics through cross-tabulations and listings. 3.2.2 Data on Seed Production and Sales Country Coordinators in each of the six countries were asked to compile annual data on seed production and sales by grantees for each of the SSTP-supported varieties for the four focal crops. Data were checked and verified against data in Ki-Projects, then used to compile the data tables that appear in Annex H. 3.2.3 Data on Grantee Selection and Performance In each country, Country Coordinators completed a simple form summarizing the grants in their countries, including the selection methods and a rating of their performance for each. These data were analyzed to see if there was any correlation between selection method and performance. 3.2.4 Quantitative Survey for Smallholder Farmers The smallholder farmer survey, a longitudinal panel survey involving two rounds of data collection from the same set of sampled direct and indirect farmer beneficiaries, was carried out in March through May of 2017, and March through April of 2018. The farmer survey includes questions about the farmer’s background, knowledge, and use of any improved varieties and other technologies, knowledge and use of SSTP-promoted varieties of focal crops, and reasons for any discontinuation in use of SSTP-promoted varieties of the focal crops. To address the EQs numbered 3.1 and 3.2, the ET used STATA software to analyze the farmer survey data collected from all three countries. The analyses included descriptive statistics of key variables, and cross-tabulations for key outcome indicators for adoption and application of improved seeds and other technologies. Simple regression analysis was used to determine key contributors to the application and adoption of improved seed varieties across the countries and for each country. In the second round of surveying, the ET will match the two surveys (Wave 1 and Wave 2) and undertake a difference-in￾differences group analysis in order to determine the change of rate in adoption over a year’s time. Table G.1 (Annex G) provides a summary of the data collection methods used and the actual sample sizes per country. 8 3.3 LIMITATIONS OF THE EVALUATION The SSTP Program is implemented in six countries and includes 17 crops. However, it was only possible for the ET to visit three countries and gather data on four pre-selected crops (maize in Ghana; cassava in Mozambique; and maize, Irish potatoes, and beans in Tanzania), as specified by the EOI. Although the broader program documentation has been reviewed, and remote interviews with the Country Coordinators of the three countries not visited have been conducted, the current report is necessarily biased towards the countries visited and their selected crops and technologies. The four selected crops are not necessarily representative of the 17 crops and associated technologies included under the SSTP Program, but do include one grain, one legume, one tuber, and one root crop. Though the evaluation is referred to as a “mid-term” evaluation, it was undertaken approximately 3.5 years into the 5-year program, at a time when some of the grants were ending, some were starting to wind down, and others were being negotiated for extensions. As such, there are relatively few recommendations that can be implemented in the latter part of the current phase of the Program. On the other hand, the 3.5-year time frame is relatively short for assessing outcomes, particularly adoption of new varieties by farmers and increases in seed supply and sales. The lack of data for some of the indicators limited the assessment of the overall Program. This is largely because, as describe in Section 4, the SSTP’s monitoring system is complex and challenging to manage. 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 4.1 OVERALL PERFORMANCE Table I.1 in Annex I provides a summary of progress at the end of Year 3 according to the indicators and targets set by the M&E Plan, which is based on the Results Framework (Table 1). The table shows that data are available for 17 out of 29 different indicators. 9 Of the 17 indicators with data, 14 (82 percent) are seen to be “on target.” The remaining three (17.6 percent) are not expected to meet the target by the end of the Program; in these three cases, the ET considers the target was unrealistically high. Overall, however, the ET found that SSTP has made good progress. Noteworthy achievements include: ● ● Number of crops (17) and varieties/technologies (131) supported. SSTP has been extremely effective, even under identified constraints, in working along the seed value chain to ensure that breeder, foundation, and certified seed of a wide range of crops and varieties is being increased. Number of producer organizations and community-based organizations (CBOs) (465) that applied improved varieties/technologies. Related to this is the number of SSTP extension service events completed (134,221). The fact that both of these indicators exceeded their targets could reflect a larger than originally planned emphasis on extension and outreach. The ET considers this apparent shift in the program design necessary to fulfill Objective 2 (Increased use of quality seeds/technologies by smallholder farmers), which was not articulated sufficiently in the original program design (see Table I.4, Annex I). 9 In some cases, the Annual Report indicates that data are missing because SSTP staff have not been able to authenticate data provided by partners; in other cases, SSTP must rely on another project (i.e., the ICT Challenge Fund) for the data. There are 6 indicators for which either the indicator itself or the target have apparently not yet been agreed. 9 ● ● ● Link with the ICT Extension Challenge Program. Although this linkage was not successful across all SSTP countries, in the countries where the ICT Extension Challenge Program was designed to link with SSTP at the outset, it was considered to have worked extremely well—not only in creating awareness about SSTP-supported varieties and technologies, but also in ensuring that grantees themselves were able to deliver required seed. Developing Partnerships. SSTP was extremely successful in developing and sustaining effective partnerships with a wide range of organizations, including private companies, a multinational company, public bodies, non-governmental organizations (NGOs), universities, and the Consultative Group for International Agricultural Research (CGIAR). Linkages with other donor-funded programs (e.g., Tropical Legumes and Purdue Improved Crop Storage, among others) were formed for collaboration and to leverage outcomes. The Early Generation Seed (EGS) study represents one such linkage, allowing an understanding that different crops need to be supported differently, and a need for greater clarity in the roles the public and private sectors play in different value chains in specific countries. The EGS study authors and other participants at the Addis Convening meeting became knowledgeable advocates for formal seed sector development within their countries, and helped initiate and develop in-country policy debates on key issues. Influencing Policies. Influencing the implementation of seed sector policies in each country is a challenging task, but the range and depth of partnerships, collaborations, and relationships developed has allowed the Partnership and the Country Teams to make excellent progress in three areas: 1) enabling the private sector to play a role in the seed certification process; 2) implementing licensing and royalties for protected public sector varieties; and 3) enabling private sector companies to produce early generation seed of maize and bean. One of the challenges faced by the ET in assessing the overall progress in the implementation of the SSTP Program is lack of consistency in the presentation of the anticipated results or outcomes within the program documentation. Table I.4 (Annex I) summarizes the planned activities and results, as presented in the SSTP Program document, together with the ET’s assessment of the implementation of the activities and the progress towards the achievement of the results. Although the indicators and results of Table H.1 are not well aligned with those in Table H.4,10 there is also an apparent mismatch between the outcomes and objectives presented in the Results Framework and the actual focus of the Program in relation to the attention given to complementary (i.e., non-seed) technologies. Whereas each of the three objectives refers to “seeds and [other] technologies,” only Outcome 2.2 explicitly mentions such technologies; other outcomes refer only to seeds (Table 1). As the paragraphs below show, the SSTP Program itself focuses mainly on seed rather than the complementary technologies, yet these complementary technologies are important for achieving maximum gains in productivity. There are four main types of grants, which correspond to the three SSTP objectives11 (numbers in brackets indicate number of grants to date): 1) those supporting seed enterprises to produce and market seed of improved varieties (41); 2) those supporting associated technologies (soil fertility, crop protection, mechanization) (6); 3) those promoting farmer knowledge about best management practices and awareness of the new varieties and technologies being promoted by the Partnership (9); and 4) those supporting policy and regulatory changes (4). The numbers clearly indicate that the majority of grants (68 percent) are supporting seed production and marketing, and that there are relatively few grants (10 percent) for complementary technologies. Although it is not explicit within the SSTP Results Framework, the marketing efforts by seed enterprises and the awareness raising among farmers (which 10 The Results Framework and the associated M&E Plan were clearly developed after the original proposal had been accepted, and some divergences from the original proposal are to be expected. 11 The first two grant types relate to Objective 1, and the last two grant types relate to Objectives 2 and 3 respectively, as shown in the SSTP Results Framework in Table 1. 10 fall under Objective 1) are both designed to encourage farmer demand for new varieties. This demand then leads to an increase of smallholder farmers who use new varieties (Objective 2). Different grants contribute to the different objectives in more complex ways than might be assumed. However, the fact that complementary technologies play a relatively minor role within the SSTP Program is cause for concern in relation to the Program’s goal of improved food security and reduced poverty. There is a general consensus that improved food security and increased incomes among African smallholders must be promoted through Sustainable Agricultural Intensification (SAI)—a practical pathway to intensify food production while sustaining natural resources (The Montpellier Panel, 2013). Intensified production is based largely on combined use of improved crop varieties and other effective inputs including water (irrigation), labor (mechanization), nutrients (soil conservation and fertilizers), and means of controlling pests, diseases, and weeds [crop protection products (CPP)] 12. For maximum impact, these inputs must be promoted together—on its own, an improved, high-yielding variety does not achieve its full yield potential unless it is used with fertilizer. There is increasing recognition that herbicides and pesticides contribute to the full economic benefits of modern farming by alleviating labor bottlenecks and supporting production potential of other inputs. Virtually all farmers that the ET met in Ghana and Tanzania, both men and women, were purchasing herbicides and insecticides from agro￾dealers. It is encouraging to see that SSTP is: 1) supporting the development of an approach to tailor fertilizer recommendations to the different soil conditions found in farmers’ fields; and 2) providing environmental training to grantees across all six focus countries; however, there are just a handful of grants for non-seed inputs. Increased investment in complementary technologies, particularly on the safe and effective use of crop protection products, will contribute to increased and more stable gains in productivity and hence food security. 4.1.1 Lessons Learned 1. The key lesson that emerges from the above observations about the internal logic of the overall Program is that there is a need for a program-level impact pathway or ToC. A ToC explains how change is expected to occur within complex systems, explicitly linking specific activities with expected results, outcomes, and the overall goals. It helps ensure that activities contribute to outcomes and that the outcomes lead to achievement of the overall goal. A ToC is also a way of ensuring that essential aspects are not left out of the overall program design. The ET believes that a ToC would have made the roles of fertilizer and crop protection products more explicit and given greater attention to these technologies. With the apparent shift in program design from the original proposal to the development of the Results Framework and the M&E Plan, a ToC would have helped ensure internal consistency in the overall Program. Revisiting the ToC on an annual basis bolsters effective monitoring and makes sure the Program delivers on its intended outcomes. This can also help to identify and recognize any unintended outcomes. 2. SSTP has an inherent bias towards seed and varieties, as compared to complementary technologies. This focus either needs to be made explicit in relation to the stated goal, or the Program should be re-oriented to give greater consideration to complementary technologies such as fertilizer, crop protection, and mechanization. With the crops and crop types currently supported by SSTP, there is a bias towards maize (a key staple in many SSTP focus countries) and especially hybrid maize, which, particularly when it is integrated with soil and pest management, has the greatest potential to boost output. 12 Crop Protection Products (CPPs) is the preferred term of industry for products used to control pests, diseases, and weeds. CPPs include pesticides, fungicides, herbicides, weedicides, etc. 11 3. The way SSTP understands and applies the term “scaling” appears to refer only to seed production, not necessarily to the number of farmers reached. The relatively small number of grants for outreach and communication (15 percent of total grants) is evidence of this. Several of these grants were implemented as pilot activities to test so-called new approaches, when, in fact, several of these approaches had already been tried and tested over many years and were known to be able to reach large numbers of farmers. In some cases, the outreach and communication activities were not sufficiently linked to the other grants, which made it difficult to ensure that what was being promoted were SSTP-supported varieties and technologies. 4. Adopting varieties is most effective where there is market demand for the outputs produced, yet there are a limited number of examples of SSTP proactively seeking out and supporting such linkages to output markets. Given that AGRA has a program on output markets, the ET thinks greater efforts could be made to link SSTP’s work with the development of output markets. 5. Many of the private companies that produce certified seed experienced problems in the sale and distribution of their seed. In the case of two grantees with linkages to the ICT Challenge Program, evidence from interviews and a survey implemented by the ICT Challenge partner suggested that the sale and distribution challenges of seeds were alleviated. This was also the case in instances where SSTP outreach and communication grantees worked directly with seed companies in creating knowledge and awareness about the varieties to stimulate demand. 13 However, much of the effort made by seed companies themselves to promote new varieties relied on demonstration plots, but there was no guidance provided by SSTP on “good practice” demonstration plots use and design. While the number of demonstration plots implemented was impressive, it remains unclear how many of these might have been successful or effective in creating demand or providing useful data. Therefore, SSTP should help seed companies establish linkages to outreach and communication organizations to promote and enhance the sale and distribution of their seed varieties. 6. Both the country roadmaps and the EGS studies were seen as effective in creating consensus among stakeholders to support successful seed sector development within the SSTP countries. They also provided a coherent sense of direction to the Partnership, guided the grant solicitation and selection process, and promoted policy debates on particular issues. 7. The composition and skills of the SSTP Country Teams were appropriate and effective in coordinating and managing the SSTP activities in each country visited. In particular, the facilitation and advocacy roles the Country Coordinator and the Deputy played were particularly successful due to their relationships with a range of key stakeholders at the policy level. 8. The SSTP M&E system is complex and challenging to manage. There are a large number of indicators and, in addition to internal program records, data need to be collected and verified from a wide range of sources and a large number of individuals/organizations within each source (farmers, grantees, partners of grantees, ICT Challenge Fund, etc.). Many private sector grantees (the majority of grantees) and their partners do not have the capacity or the incentive to collect these data or report them in the required Ki-Projects software. The definition and identification of direct and indirect farmer beneficiaries remains a source of confusion. There is a need to streamline some indicators and ensure that the data collected by private sector partners is minimized and seen as useful to their own businesses. 13 On the other hand, there were also examples of communication grantees who were less effective because they were not connected directly to the seed companies. 12 9. The overall approach of SSTP is innovative and successful. Many more lessons than have been documented here will emerge at various levels. The intricate and inventive elements make it vital that these lessons are captured, documented, and acted upon so that that the Program continues to evolve internally and so that other donor-funded programs can benefit. While this external evaluation has highlighted a number of lessons, the process of learning and documenting those lessons is most effective when it is internalized within the program. Systems need to be put in place to ensure that this learning is systematically captured and documented for future implementation. CONCLUSIONS Based on the indicator data available at the end of Year 3 and the achievements noted by the ET, SSTP has made good progress. The main focus of the Program (as evidenced by the allocation of grant types) is to increase the availability and use of improved varieties. Increased investment in complementary technologies, particularly on the safe and effective use of crop protection products, will contribute to increased and more stable gains in productivity and hence food security. A ToC is an effective coordination tool. By involving project staff and partners in generating a ToC, everyone develops a more detailed understanding of the program and how their own activities contribute to making change happen. RECOMMENDATIONS • • Increased attention and investment should be allocated to complementary technologies such as integrated soil health, integrated crop protection, and labor-saving mechanization. For Phase 2, it is recommended that AGRA/SSTP staff develop a program-level ToC, and that in￾country staff and partners develop nested country-level ToCs. Such ToCs will ensure the internal logic of the Program, and can help check that activities undertaken are leading towards intended outcomes. 4.2 MANAGEMENT AND ADMINISTRATION 4.2.1 Overview of Current Grants Portfolio At the end of December 2016, there were a total of 60 grants (Table G.3, Annex G), and almost $15 million of the $22 million allocated for grants had been disbursed. Grantee organizations were from the private sector (44 percent); the public sector (14 percent); international organizations (12 percent); and NGOs (30 percent). A breakdown of activities by organizational type reveals that all organizations, except international organizations, which do not undertake foundation seed production, are involved in all types of activities (Table G.4, Annex G). Private sector companies include seed companies and consultancy companies. These are seen to play a major role in certified seed production, creating demand for seed, and strengthening capacity. International organizations and NGOs tend to be more involved in supporting the private sector to create demand for seed and other technologies, and in strengthening capacity. Section 4.3 shows that these are the areas where the private seed sector needs to be supported by the partnership approach promoted by SSTP. On average, private and public sector organizations are involved in more types of activities than are international organizations or NGOs, which focus on smaller numbers of activities (Table G.4, Annex G). 4.2.2 Grant Selection Process STTP issued Requests for Proposals (RFPs) in each country to align with priorities identified by the Country Roadmaps. Country Roadmaps reflected national priorities for food crops and policy commitments made in the New Alliance Country Cooperation Frameworks. Applicants were required to submit concept notes based on a template. The timeline for grant selection was different in each 13 country, and depended on how quickly the Country Team was staffed and operational. Other considerations also play a role in the timeline. In some countries, there was just one RFP (e.g., Tanzania), and in other countries there were multiple requests over time (e.g., Ghana). In Ethiopia, it was not possible to advertise the RFPs because, at the time, AGRA was not legally registered. SSTP staff often reached out to organizations with relevant capacity and encouraged them to apply. By doing so, they made sure that an adequate number of good quality concept notes was received. The only guidance for developing the concept notes provided to applicants were written guidance notes from the application instructions. Both the SSTP Country Teams14 and two or three Nairobi-based SSTP staff independently reviewed the concept notes. They consulted and agreed on which ones to select to go through to the proposal stage. Once the applicants were selected, SSTP country staff provided the written guidance notes, as well as advice and support in developing their full proposals. When the Country Coordinator was satisfied that the proposal met quality standards, it was submitted to the Nairobi office for review by SSTP staff. Where necessary, the SSTP country staff continued to work with the applicant to make any additional revisions requested by the Nairobi-based staff. AGRA’s Chief of Agricultural Transformation also reviewed and approved all proposals for submission to the AGRA Grants Committee. To complement these competitive grants, solicited or sole source grants represented a slightly different selection mechanism. These solicitations were deemed strategic to fully address the priorities the Country Roadmap identified, and to capture opportunities that would not otherwise be funded, including continuing funding for some organizations previously supported by PASS. The decision to solicit applications from PASS grantees was based on the recommendation of PASS staff. In general, solicited and sole source grant applications skipped the concept note stage and went straight to the proposal stage. A solicited proposal had to be accompanied by a written justification by the SSTP Country Coordinator to explain the reasons why the applicant was chosen. The number of SSTP grants awarded to former PASS grantees varied from country to country; in Mozambique, the majority of SSTP grantees were former PASS grantees. In Tanzania, Malawi, and Senegal, the majority of SSTP grants were competitively sourced. A Grants Committee of 8-10 members of senior AGRA staff was responsible for evaluating all grant proposals submitted to AGRA, including all SSTP grants. Grants over $300,000 required an external review by an expert. For this purpose, experts were selected from a roster of individuals with different areas of expertise across different countries. The external reviewer roster was established with experts known to AGRA. The written comments provided by the external reviewers formed part of the documentation considered by the Grants Committee, as did the written justification provided by the SSTP Country Coordinator for solicited grantee applications. Often, the Grants Committee interviewed the Nairobi-based SSTP senior staff member who supported the grant application (according to the country concerned) as part of the review process. This senior staff member provided any necessary clarification and answered questions. The decision of the Grants Committee was based on consensus, which was inevitably influenced by the views of those who were more vocal, expressed opinions early, or had the most influence. Individual Grants Committee members were not required to provide written comments in advance. Had they, the comments would have informed the decision-making process in a transparent manner. It is possible that not all Grants Committee members thoroughly reviewed every application, and perhaps relied instead on views of other members and/or their own knowledge of the applicant. 14 Each Country Team is generally composed of a Country Coordinator, a Deputy Country Coordinator (also responsible for M&E), and an Administrator, plus a Driver. 14 When the SSTP Program first started, the AGRA Grants Review Committee, a sub-committee of the AGRA Board, was expected to review all SSTP grant proposals; however, this was subsequently changed so that they only reviewed grants proposals over $500,000. This second-tier formal review process proved to be too time-consuming within the SSTP timeline, and was abandoned altogether after SSTP was asked to speed up the process. There was still confusion over whether or not USAID was expected to formally approve all grants (actually subgrants) as a final step in the approval process, 15 despite numerous discussions and written agreement to the contrary. The strengths and weaknesses of the grants selection process are summarized in Table 2. It is worth highlighting that AGRA’s ongoing reform process will build on some of these strengths and address some of the weaknesses highlighted below. For example, the Grants Committee will be revamped and the selection criteria revised. 4.2.3 Effective Support to Partners The EQs concerned with effective support to partners relates to USAID’s FORWARD strategy, a strategy that aims to promote sustainable development through high-impact partnerships, and which emphasizes new models for public-private partnerships (PPPs) and support to local organizations. Overall, the SSTP partnerships focused primarily on supporting and strengthening priority seed value chains. This consisted of a set of partners, formally engaged in the specific sub-grant, who received either financial or in-kind support. In most cases, SSTP grantees worked with a range of partners, both other grantees within the same value chain and their own (non-grantee) partners. Table 2. Strengths and Weaknesses of the Grants Selection Process Strengths Weaknesses • • • • • The SSTP grant selection process is informed by and integrated with the larger AGRA strategy, (especially PASS) – this ensures both continuity and complementarity. SSTP issues calls for proposals that are informed by Country Roadmaps – this ensures that the call corresponds to the priorities agreed to in the Roadmaps. SSTP staff draw on deep knowledge of the country landscape, actors, and seed sector – that is used to advise and guide potential grantees and ensures a robust and diverse response. Competitive grants are complemented by strategic solicited grants to ensure that gaps in the different priority seed value chains and emerging problems are addressed, and opportunities exploited. The Grants Committee reviews and ultimately approves all grants and grant extensions – this ensures that, in principle, AGRA standards are met, strategy adhered to, and approved grantees have the capacity to achieve their objectives. • • • • • • AGRA’s two-tiered grant process initially led to a 6-month delay in approving grants and had to be changed to expedite the process. The AGRA selection criteria used by the Grants Committee were not revised for SSTP, yet the objectives of SSTP are different from AGRA. Being based on consensus, there is a potential risk of bias in the way the Grants Committee reviews proposals. Adherence to the roadmap restricts the flexibility of the country teams to select the highest potential grantees. Solicited proposals are allowed to skip the concept note stage of the application process, possibly resulting in weaker applicants going through to the proposal stage. USAID BFS staff feel that they have not been sufficiently engaged in the grant approval process, despite written agreement that USAID approval for grants is not required. 15 Neither BFS nor USAID Mission staff were involved in the grant selection process, and the ET feels there is no need for their involvement. 15 In many cases, SSTP promoted collaborative linkages between grantees and other donor-funded projects, e.g., ICT Challenge Program, Tropical Legumes, Purdue Improved Crop Storage, One Acre Fund, and N2Africa, among others. One gap in the range of partnerships and linkages was noted. Although SSTP project documentation states that the partnership would work with providers of credit to ensure that farmers, especially women, have access to seed, the ET did not find any examples of this. Subsequently, the Team was informed that AGRA is implementing the Financial Inclusion for Smallholder Farmers in Africa Project (FISFAP) https://agra.org/initiatives/fisfap/ and that SSTP is developing linkages with this project in Ghana and Tanzania, the two countries common to both projects. FISFAP supports partnerships with value chain actors and financial institutions that enable farmers to access purchased inputs, including those promoted by SSTP. While it is too soon to be able to assess actual impacts, the ET believes that the partnerships most likely to achieve impact—measured in terms of influencing farmer technology adoption, increased productivity, and profits—are those that: 1) involve ancillary technologies; 2) provide effective support to technology and varietal promotion; and 3) support a pluralistic partnership. Examples of these types of partnerships are provided in Table G.5, Annex G. The multiplicity of partners involved, and the apparent success of SSTP grantees in managing the relationship with these partners, often with useful facilitation from SSTP Country Teams, is striking in all of these examples. Where SSTP supports increasing capacity for local seed companies, effective long-term support is likely to result in seed sector realignment for greater value chain efficiency and profitability. Exactly what this will look like is difficult to predict, but it may involve the eventual buy-out of successful local seed companies by multinationals. 4.2.4 Coordination and Communication Coordinating Frameworks In general, coordination has effectively been achieved through the Country Roadmaps, the EGS studies, and through regular meetings of grantees, partners, and stakeholders. There was consensus within AGRA that SSTP’s portfolio approach to grants (based on the Roadmaps) represented a significant improvement in ensuring coordination along the seed and commodity value chains. The EQs presented in the SOW refer to coordination in a very general sense (which is also the focus in this section). The SSTP project documentation specifies at least four different types of coordination; 16 coordination is also one of the seven SSTP activities.17 SSTP’s support to the EGS studies in Ghana, Malawi, Mozambique, and Tanzania similarly fostered effective communication and coordination among grantees, partners, and stakeholders with significant support from Africa Lead II to design the methodology.18 However, it should be noted that the EGS studies were funded differently in different countries. Within SSTP, the initial Addis EGS Convening meeting and the EGS framework catalyzed a process of capacity development, consultation, research, and dialogue across and within countries. This process culminated in a country-specific EGS report for 16 The four types of coordination are: 1) coordination among donors and government to leverage private sector investment; 2) coordination of private and of public sector actors; 3) coordination of technology-scaling opportunities; and 4) coordination to stimulate the development of new input markets. 17 According to the SSTP project document, coordination is expected to increase the confidence of each partner, leading to invigorated activity within their respective spheres of influence, engendering a collaborative and risk-taking posture that will drive increased geographic penetration and pace of inputs delivery. While it is good to see this explanation of the ToC relating to coordination activities within the project document, such far-reaching claims for coordination were not apparent in the ET’s interviews with SSTP country staff. Once again, if SSTP had really intended for coordination activities to lead to these changes, then a ToC framework, for which both staff and partners would have been involved in formulating, could have helped to develop this understanding and identify ways in which to achieve such changes. 18 The initial EGS study (2015) was prepared for the Bill and Melinda Gates Foundation and USAID in collaboration with Monitor Deloitte. The 2015 overview report provided “a generalizable framework that enables policy makers and donors to tailor their policies and interventions to the needs of specific crops based on market conditions, which … [were referred to] … as market archetypes” (Lion et al, 2015). 16 each of the four countries mentioned above. In all three countries visited by the ET, interviewees showed an awareness of and interest in the EGS studies. The studies prompted a dialogue and spurred agreement about where bottlenecks might occur and how they could be addressed. In this way, the studies provided an effective strategy for the SSTP programs in each country. Coordination and Communication The SSTP Country Teams have been critical in facilitating effective coordination and communication at the country level. Despite the lack of a ToC, coordination frameworks are effective and regular meetings ensure good communication. In each country, annual meetings involving all grantees, government stakeholders, USAID officials, other key partners, and relevant Nairobi-based senior SSTP staff take place. In each country visited, grantees and USAID staff confirmed that these meetings are effective for information sharing. Two other communication and coordination approaches were observed in the countries visited. In Ghana, quarterly meetings ensured that different grantees met on a regular basis and had the opportunity to share and learn from each other. In Tanzania, rather than focusing on linkages between the different grantees, there appeared to be greater emphasis on creating partnerships/linkages within the same grant/project for greater impact. This is illustrated by some of the examples provided above [i.e., grants led by CGIAR’s International Center for Tropical Agriculture (CIAT), FIPS, Aminata Seed Company, and Kibaha Sugarcane Research Institute]. Various other mechanisms and strategies have also contributed to effective coordination and communication within each country. These include SSTP’s close links with other coordinating bodies like the national seed trade associations and government seed units. In most countries, SSTP Country Teams have made extensive efforts to develop good relationships with USAID Mission staff. This is helpful because these relationships were somewhat weak and strained in the beginning, and USAID staff provides important information sharing and coordination among USAID-funded projects within a country.19 4.2.5 Data Management Systems SSTP uses a data management system based on DropBox to effectively transfer, access, and archive all SSTP grantee information, which includes narrative reports, IPTT (Indicator Performance Tracking Table), financial information and Environmental Mitigation and Monitoring Plan (EMMP). This allows SSTP to comply with USAID reporting requirements and environmental regulations. The SSTP Compliance Officer also maintains a dedicated folder on the AGRA server to store critical documents, which makes it easy to compare and aggregate information across grantees and countries. Ten Feed the Future indicators and additional SSTP custom indicators are agreed to, included in all grant agreement documents, and tracked and reported. Information from the IPTTs reveal that because seed company grantees commit to implementing more activities, they are required to track twice as many indicators. SSTP staff support grantees to complete reports, and also reviews the reports for completeness and accuracy at the country level before sending them to Nairobi. The Project Performance Index (PPI) and Sustainable Enterprise Index (SEI), two additional composite indicators, show promise in tracking grantee and seed company performance. They also allow comparison across grantees and companies, between seed archetypes, and between countries. A contracted survey company also undertakes Annual Outcome Indicator surveys. 19 Initially SSTP started as an exclusive AGRA project, with the focus of coordination between the new SSTP country staff and grantees and existing AGRA staff and programming. This began to change as the USAID Missions started to assume ownership and SSTP initiated monthly meetings and learned how to comply with USAID procedures. Coordination is receiving a boost with the opportunity to integrate with the ICT Challenge Program grantees in each country. Mission support for what is called the 3Cs in Malawi – Collaboration, Coordination, Co-location – presents opportunities for new and wider partnerships with USAID supported organizations and projects working in Sustainable Development. 17 The Evaluation Plan notes that SSTP, like other USAID-funded implementing programs that work with private sector partners, has faced challenges in tracking farmer beneficiaries. Grantees continue to struggle with collecting data on beneficiary participation in extension service events and purchase of seed from SSTP seed companies sold by agro-dealers. Apparently, in the initial grant implementation period, SSTP had to focus on what were considered critical indicators, and determined that recording beneficiary numbers was sufficient, while recording beneficiary names was too time consuming. Although varietal and technology demonstrations are the most popular method for creating farmer awareness, data collected from demonstrations is lacking. This is because of: 1) the decision that the grantees would manage demonstrations independently; 2) the decision that the objective of demonstrations was promotion, so data were not necessary; 3) the lack of demonstration guidance and modifiable protocols with data templates; 4) the lack of budget and staffing for demonstration data collection, analysis, and reporting; and 5) the lack of interest or capacity of SSTP country and regional staff to compare quantitative results across grantees and countries. The CIAT grant in Tanzania and the Concern Universal grant in Senegal, where the grantees have data management capacity and a clear commitment in their proposals to collecting and reporting on the results of their demonstrations, are two exceptions. Regarding a data management system, SSTP introduced Ki-Projects with the objective of shifting from using Word, Excel, and scanned documents in DropBox. Ki-Projects was contracted nearly two years ago. The ongoing transition has been problematic because a few grantees had issues synchronizing their data into the server (now resolved) and others have had high staff turnover, which left untrained staff to support the system implementation. Grantees and country staff continue to rely on the existing Word, Excel, and DropBox system. Although more than 90 percent of the reports are uploaded into Ki￾Projects, it seems unlikely that Ki-Projects will be used to generate the kind of tracking data required for effective monitoring and quarterly program reporting. AGRA has recently decided to go with a company called Synergy for its own data management system (according to their web site, Synergy manages the software for Feed the Future indicator tracking). It seems that the incentive to transition to Ki-Projects may disappear and Ki-Projects may become redundant. In general, the current emphasis is on data for reporting, and there has been less attention on data for learning. This applies to SSTP as a whole and to grantees. It appears that the broader lessons of what is working and what is not are not being systematically captured, shared, and applied. SSTP is complex, with different types of partnerships, different models for scaling up seed production and supply, different types of seed, varieties, and other technologies, and a number of innovative approaches to promoting adoption by farmers. Currently missing is the big picture view of which types of partnerships, models, technologies, and innovations are working and which are not. Similarly, for grantees, the focus on data for reporting has led to missed opportunities for sharing data for learning. For example, data from variety performance in demonstrations can potentially be used to inform future farmer customer demand. CONCLUSIONS The grant selection process has its strengths and weaknesses. Examples of strengths include: 1) competitive grants are complemented by strategic solicited grants to ensure that potential gaps are addressed; and 2) the Grants Committee ensures that, in principle, AGRA standards are met, strategy is adhered to, and approved grantees have the required capacity. 18 Examples of weaknesses include: 1) the AGRA selection criteria used by the Grants Committee were not revised by SSTP, yet the objectives of SSTP are different from AGRA; and 2) the verbal consensus that the Grants Committee uses to review proposals creates a potential risk of bias. Solicited proposals skip the concept note stage of the application process, possibly resulting in weaker applicants going through to the proposal stage. The Partnership has been very effective in supporting the role of the private sector and other stakeholders as engines for sustainable development in their countries. The SSTP Country Teams have been critical in facilitating effective partnerships, linkages, coordination, and communication at the country level. There are opportunities to encourage additional relevant partners, particularly those from related sectors such as microfinance, food security, resilience, and nutrition. Coordination was effectively achieved through the Country Roadmaps and the EGS studies, as well as regular meetings of grantees, partners, and stakeholders. Annual Review and Planning meetings can be made more effective in systematically capturing and sharing the broader lessons of what is working and what is not. Missing is the “big picture” of which types of partnerships, models, technologies, and innovations are working and which are not. The current emphasis is on data for reporting, with less attention paid to data for learning, not only for SSTP as a whole, but also for grantees. RECOMMENDATIONS • • • • • It is recommended that the Grants Committee adopt a simple, written individual scoring and comments system to make the grants selection process more transparent and to ensure all members of the Committee are fully engaged. It is also recommended that all members of the Grants Committee are fully conversant with the proposed SSTP ToC (if not involved in its development) and that the grants selection criteria are revised to ensure clarity and consistency with the Phase 2 rationale and proposed ToC. It is recommended that AGRA and SSTP staff reach out to additional relevant partners in the microfinance, food security, resilience, and nutrition sectors. This can be done through existing in-country donor and sectoral coordination platforms. Under Phase 2, it is recommended that the Country Annual Review and Planning meetings continue to take place and that – in addition – the lessons emerging from these discussions are documented and shared to help promote learning across countries and encourage subsequent implementation. Before the end of Phase 1, it is recommended that an internal, participatory lesson-learning exercise be conducted and documented at country- and program-levels. This will help determine what has and has not worked regarding types of partnerships, approaches to capacity development of seed companies and agro-dealers, models for seed production, marketing for seeds produced, strategies for promoting farming adoption at scale, etc. The current evaluation report provides a starting point to such an exercise. In Phase 2, it is recommended that SSTP manage data in ways that support two types of learning: 1) learning by grantees (notably commercial companies) involves the joint identification of indicators and how these can be applied in ways that will allow their businesses to develop, i.e., indicators must be useful to commercial growth; and 2) systemic or programmatic learning, 19 which can be guided by a program-level ToC and up-front articulation of questions to be addressed, both within countries and across countries. 4.3 ENGAGEMENT OF THE PRIVATE SECTOR The EQs on the engagement of the private sector centered on three themes: 1) availability of improved seeds and technologies; 2) the role of private sector in seed and technology value chains; and 3) effectiveness of the private sector in creating demand. Under each theme was a number of sub￾questions. 4.3.1 Models Used to Make Seed of Improved Varieties Available to Farmers To address constraints faced by private sector seed companies in increasing the availability of quality seed of improved varieties to small farmers, SSTP provided grants to private seed companies and public and international research organizations. The constraints identified by SSTP and the EGS studies were: 1) lack of adequate quantities of breeder/foundation seed to produce certified seed or planting materials on a timely basis; 2) capacity of private seed companies to produce and distribute certified seed produced to smallholder farmers; and 3) value chain linkage to output markets for farmer-produced commodities. Examples of the approaches used in addressing these constraints are provided in Table 3, and can be categorized according to whether or not they address one or more constraints and the number of grantees/partners involved. The first two examples are single organizations (whether public or private sector) addressing a single constraint referred to as the single constraint – single organization model. The last two examples are PPPs addressing all three constraints, referred to as the triple constraint – PPP model. Another model identified by the examples in Table 3 is the double constraint – single organization model. Table 3. Examples of Technology Availability Models Used by SSTP Constraints Addressed Country Grantee/Sub￾Grantee/Partners Type of Organization Crop(s) Model Breeder seed Tanzania Agricultural Research Institute (ARI), Uyole Public Sector Maize and Bean Single constraint – single organization Foundation Seed Ghana Legacy Crop Improvement Center (LCIC) Private Sector Maize Single constraint – single organization Foundation seed certified seed Ghana M&B Seed and Agricultural Services (Ghana) Ltd. Private Sector Maize, Rice, and Cowpea Double constraint – single organization Foundation seed certified seed; linkage to output markets Mozambiqu e Agricultural Research Institute of Mozambique (IIAM), Corredor Agro, International Fertilizer Development Center (IFDC), smallholder farmers, Dutch Agricultural Development & Trading Company BV (DADTCO), and Cervejas de Moçambique (CDM) Public-Private Partnership facilitated by International Organization Cassava Triple constraint – PPP 20 Constraints Addressed Country Grantee/Sub￾Grantee/Partners Type of Organization Crop(s) Model Foundation seed certified seed; linkage to Output Markets Tanzania CIAT/Pan African Bean Research Alliance (PABRA), Agricultural Seed Agency (ASA), Mergro & Tour Company, and Beula Seed Company Public-Private Partnership facilitated by International Organization Bean Triple constraint – PPP 4.3.2 Increased Availability of Seed of the Focal Crops SSTP differentiates two strategies for increasing seed availability, though in practice the ET was unable to identify any differences in the strategies other than the type of variety in question: 1. Scaling up production (quantities) of seed of existing varieties sold by the grantees and their distribution channel partners (agro-dealers and retailers); and 2. Commercializing developed and released varieties that are not yet in the market or available to farmers by producing quality seed that can be sold to farmers through the grantees and their distribution channel partners (agro-dealers and retailers). The number of improved varieties made available varied by crop—cassava, bean, maize, or potato. The source of most of the varieties used by SSTP grantees is either International Research Centers or National Agricultural Research Institutes (NARIs). There were no varieties available for scaling up or commercialization from the private sector, multinational, or local seed companies. All had received the necessary release and registration in the country where they were being grown. It is important to keep in mind that while the SSTP Program was initiated in 2014, actual seed production by the majority of the seed company grantees started in 2015 or 2016. Thus, available data covers only one or two years of production, and consists of different varieties in the same country. Data on the quantities produced and sold are summarized in the tables in Annex H by crop and class of seed produced. Maize Foundation Seed Production and Distribution A total of 10 grantees in Ghana (2), Senegal (3), Tanzania (2), Mozambique (1), and Malawi (2) were given support to increase maize foundation seed production. In these five countries, grantees saw an increase in the quantities of foundation seed produced. Senegal, Malawi, Tanzania, and Mozambique saw, respectively, 59 percent, 60 percent, 28 percent, and 80 percent of the seed produced sold to certified seed producers.20 The companies themselves used some of the foundation seed for in-house production of certified seed. The increases in foundation seed availability in the five countries represent potential for further increases in production of foundation and certified seed. The quantities of foundation seed produced depend on the increasing parental lines and their use either as male or female lines for seed production. It takes several seasons to bulk enough material for certified seed production. It would be more efficient to tie the quantities of foundation seed produced to target quantities of certified seed to be produced in the first year of production and beyond. For example, are the inbred lines being produced the ones that are required for the hybrid(s) to be produced? If the desired hybrid is a three￾way hybrid requiring three different inbred lines, and one of those lines is unavailable in sufficient quantities, then producing certified seeds of that hybrid on time is impossible. In the M&B (Ghana) 20 Foundation seed is not a commodity that is broadly sold to farmers; its commercial value comes from the certified seed produced from it. A company that has exclusive use of an inbred line(s) for the production of foundation seed and certified seed from it/them has 100 percent control of that foundation seed. It is not available to other companies. For publicly bred varieties that are shared among different seed companies, the amounts of foundation seed made available to them will depend on initial quantities available. Thus, their ability to produce certified seed is affected by quantities of available foundation seed for those that need it. When in short supply, few companies get it or each company receives small amounts. 21 example in Table 3, the same seed company is producing foundation seed and certified seed (double constraint – single organization model), which theoretically makes it easier to enhance efficiency by tying foundation seed production to certified seed production. Maize Certified Seed Production and Distribution There were 14 SSTP grantees engaged in certified maize seed production across six countries, and in all six, certified seed production by private seed companies has worked well to increase availability of certified maize seed under both the scaling up and commercialization programs. Under the scaling up program, the production of certified maize seed increased from 2015 to 2016. The varieties in production remained the same in Ethiopia and Malawi and increased in Senegal and Tanzania. In Ghana, however, the number of varieties in production fell from five in 2015 to four in 2016. This was due to one of the grantees (M&B) discontinuing production of one variety because the inbred lines provided by the relevant Research Institute had quality issues. Grantees in Ghana, Ethiopia, and Malawi (98 percent, 84 percent and 74 percent, respectively) reported that most of the certified maize seed produced was sold in 2015, with some carry over into the next season. For most grantees, what was produced in 2016 will be sold in 2017, although some were sold for planting in 2016. The commercialization program was in effect in Ethiopia, Ghana, Tanzania, and Mozambique and involved five SSTP grantees. The program resulted in increased availability of certified seeds of improved maize varieties. Certified maize seed production in Ghana resulted in production of 129 metric tons (MT). In Tanzania, it resulted in production of 68 MT. Of this, 84 percent was sold in Ghana, and 93 percent was sold in Tanzania. In 2016, two varieties were introduced into the commercialization program in Ghana and, of the 25 MT produced, 20 MT (80 percent) were sold. In Tanzania, 99 percent of certified seed produced was sold. Ethiopia and Mozambique produced certified maize seed under this program for the first time in 2016 and produced quantities will be sold for planting in 2017. Going forward, the challenge will be maintaining availability of improved varieties for an expanding market, and ensuring the capacity of the seed companies’ staff and out-growers to manage expanded production, business development, and financial services—all of which are discussed below. Bean Seed Production Bean production was evaluated in Tanzania and Malawi under the scaling up program and involved three SSTP grantees in each country. In Tanzania, the three grantees formed the triple constraint – PPP model. In Malawi, all grantees fell under the single constraint – single organization model (all private seed companies). Table H.3 (Annex H) shows that bean seed production of five varieties increased more than three-fold in Tanzania, and that there was a marginal increase in production of two varieties in Malawi. In both countries, sales figures were good for 2015, and the majority of the 2016 production is available for sale in 2017 based on the cropping cycle. Potato Seed Production In Tanzania, potato seed/planting material was produced by two private sector grantees. Under the SSTP scaling up program, one grantee produced 493 MT of potato seed of three varieties in 2015 and 345 MT in 2016 — shown in Table H.4 (Annex H). As part of the SSTP grant, 365 MT of the quantity produced in 2015 was provided to farmers with other inputs for use in demonstration plots and training programs on good agricultural practices. The expectation was that farmers would experience benefits of planting improved potato varieties, learn good potato production practices, and then buy future seed from Mtanga Farms, the grantee. The balance of 130 MT (26 percent) was sold to potato farmers. Of the 345 MT produced in 2016, 288 MT (83 percent) were sold to potato farmers in 2016. The balance of 57 MT is available for sale for 2017 growing season. The program has increased the quantity of potato seed 22 available to potato farmers in the target areas and, as demonstrated by sales in both 2015 and 2016, farmers are willing to pay for the seed. The second SSTP grantee, Crop Bioscience Solutions Ltd. (CBS), started a commercial tissue culture company and began producing potato plantlets and minitubers of four varieties in 2016. A total of 94,124 potato minitubers were produced in 2016 and will be distributed to contracted (paid) decentralized potato seed multipliers for multiplication of potato seed in the coming seasons. Because of its short duration, the success of this multiplication approach could not be assessed. However, because plantlets can be sold after a short period of time (two months) for use in minitubers production, and several cycles can be produced in the course of a year to meet demand production, rapid bulking up of minitubers for use in seed potato production is possible. Minitubers planted in the field will increase numbers of seed potatoes produced over one or two growing seasons, which will increase the quantity of seed potatoes available for sale to smallholder farmers. Being able to sell both plantlets and minitubers gives private seed company like CBS two potential revenue streams. Given that it is multiplying both foundation seed (in the form of plantlets and minitubers) and certified seed potato, CBS falls under the double constraint – single organization model, though it has plans to form a partnership with another private company for linkages to output markets. These plans set CBS to evolve into a triple constraint – double organization (private sector) model. Cassava (Seed) Cuttings Production The production of cassava seed (cuttings or planting material) was evaluated in Mozambique under the scaling up program, for which production figures are provided by Table H.5 (Annex H). The applicable model is the triple constraint – PPP model. In this model, the public sector (IIAM) provided suitable cassava varieties to private sector companies to multiply. The Mozambique case involved Corredor Agro, working as a contract Hub out-grower, and Oruwera Limitada (an SSTP grantee). Within the same model, the IFDC distributed to farmers cassava cuttings produced by the two private companies, and provided training and other technical assistance. They also facilitated the development of value chain linkages in which smallholder farmers cultivated cassava roots for sale to a processor (DADTCO) who then made cassava cake, which was sold to the beer brewing company, CDM. There were three cassava varieties in the 2015 program, and six in the 2016 program. A total of 687,100 cassava cuttings were produced in 2015. Of this, 659,840 (96 percent) was sold to contract out-growers for multiplication and/or distributed by IFDC free of charge to smallholder farmers to promote the new varieties in 2016. The quantities of cassava cuttings produced in 2016 increased to 4,342,393. Of this, 3,467,390 (80 percent) were sold to out-growers for multiplication, to farmers for planting, and to NGOs for distribution as relief material after drought. Other cuttings were used in farmer training programs like nursery maintenance. All of these activities are necessary in any development project with multiple objectives. Based on the 2015 and 2016 production numbers, the scaling up program increased the availability of cassava planting materials in target areas. Interviews with farmer groups indicated that drought conditions in 2015 led to a shortage of cassava planting materials in 2016. Lead farmers or farmer groups were encouraged to multiply cassava-planting materials by using rapid multiplication techniques on their own cassava plantings and distribute them to farmers who had not received from IFDC the freely distributed materials. Some farmers, however, refused to buy planting materials produced by farmers that had previously received free materials; others refused to contribute free labor in exchange for planting materials; and lead farmers were reluctant to sell planting materials to other farmers as a business transaction since they mistakenly were expecting to receive financial compensation from IFDC. These problems indicate that in the future this type of PPP approach will require better implementation. Furthermore, the closure of the IFDC grant (which ended in February 2017) concerned DADTCO who acknowledged their own lack of capacity to 23 assume the role of training, sensitization, and distributing planting materials to smallholder farmers. Partners in the PPP lacked the understanding that they must plan on self-sustainability after projects end. Nonetheless, farmers interviewed were excited about the fact that DADTCO bought all the cassava roots produced and delivered, paid higher prices than market aggregators, and paid quickly—less than four days after delivery. 4.3.3 Barriers to Increasing Seed Production of Improved Varieties Although most of the approaches described above successfully increased the amount of seed available, a number of barriers remain and need to be addressed in some SSTP countries. In most cases, SSTP is aware of these barriers and is already working to address them. As noted in the recommendations, however, and further described below, there are two issues that SSTP could address to enhance quality seed production capacity across the focus countries: 1) development of a seed demand forecasting system; and 2) insistence on documented seed quality assurance procedures by the SSTP-supported seed producers. Intellectual Property Rights (IPR)21 IPR for plant varieties provide incentives for plant breeders from both the public and private sector to obtain a return on investment in research that produces new varieties. Tanzania is a member of the International Union for the Protection of New Varieties of Plants (UPOV), and has enacted Plant Breeder’s Rights laws. Ghana’s and Mozambique’s Seed Acts do not address IPR and Plant Breeders’ Rights. It seems lack of IPR protection (of specific traits) and Plant Breeder’s Rights (of specific varieties) discourages private sector companies like regional or multinational seed companies, from investing in local breeding activities. The breeding activities would exploit promising traits, introduce new varieties into the seed sector, or place parental lines in the hands of a dedicated foundation seed producer; however, without IPR protection, companies fear loss of IPR through illegal sharing of proprietary materials. Foundation Seed Access to quality foundation seed of improved varieties is critical, and its multiplication requires high technical expertise. For the crops of interest to SSTP, most foundation seeds come from the public sector. Based on interviews with SSTP grantees and stakeholders, the three issues that need to be addressed in relation to foundation seed are: 1. The lack of transparent processes with well-defined terms and conditions for transferring breeder seed to private or public seed sectors for timely production of foundation and certified seed. A breeder generally produces a few grams to a few kilograms of the inbred line (basic seed) that make a new variety. With such limited quantities of breeder seed, it is important that there is a clear process to determine which companies should have access to this seed. Of the countries visited by the ET, this issue has only been addressed in Tanzania, which has a tender system for protected, publicly bred varieties. Once a company is given access to a variety, the company can invest in and plan for inbred line maintenance and foundation seed production for its certified seed production plan demand in line with the terms of the agreement. 2. The lack of knowledge about the seed quantities required leads to an inability to plan for seed production. Inbred lines must be routinely maintained and evaluated for genetic purity. This is what ensures that the varieties produced retain their prescribed characteristics. Commercialization of a new variety requires the limited quantities of seed of inbred lines (basic seed) produced by the breeder be multiplied by foundation seed producers. In order for the breeder to provide required quantities of inbred line seed to multiple requesters, the breeder 21 SSTP had commissioned a report entitled “improving access to publicly bred varieties” which was not yet finalized at the time of the evaluation and was therefore not included in the documentation review. 24 needs to know what those required quantities are. A mechanism is needed to forecast foundation seed requirements. This would give the breeder information to plan basic seed production that would meet the needs of requesters. Similarly, foundation seed producers must estimate the quantities of certified seed needing to be produced to sell to farmers and other customers. This estimate feeds into foundation seed requirements. The lack of forecasting mechanisms potentially results in cases of mismatched varieties supplied, insufficient quantities being produced, inadequate amounts of foundation seed produced, poor timing of availability, inconsistent physiological quality, etc. 3. Lack of capacity for maintaining and multiplying parental lines. Interviews illuminated the fact that most seed companies lack capacity – qualified staff, irrigation facilities, cold storage, etc. – to maintain and multiply adequate quantities of parental lines for foundation seed production. The ability of new and emerging seed companies to recruit, train, and retain qualified staff is a challenge. Some SSTP grantees, e.g., in Ghana, have used short-term contractors/experts during the growing season; however, the availability and responsiveness of such experts during the season is challenging. Regulations The seed laws in Ghana and Tanzania have been changed to allow private seed companies to produce foundation seed, but the two countries have adopted different approaches to implementation. Ghana would prefer that, ultimately, foreign seed companies importing certified seed produce foundation seed in country within two years of importation as a way to build in-country capacity for producing foundation seed. Tanzania, on the other hand, has no such restrictions, and has erected minimal barriers against seed importation. The different approaches impact seed availability in local markets differently. For example, ability to import improved varieties immediately increases their availability (provided that seed is available for import). Insisting on local seed production without adequate intellectual property protection means multinational and regional seed companies resist importing their inbred lines, and limits seed availability in the country. Mozambique has a seed law, but regulations for its implementation have not been approved. The law’s implementation to determine what seed can or cannot be imported, who can produce foundation seed, and how long a variety approved in one country might take before it is approved locally, has not been fully enacted. Additionally, despite harmonization of seed movement laws, issues surrounding phytosanitary certification delay seed movement between countries. Timely seed certification by Seed Inspection Agencies remains a barrier to the availability of improved varieties of seeds to farmers in all three countries visited for this evaluation. Seed Quality and Quality Assurance None of the maize grantees interviewed had any documented operational procedures to improve the quality of certified seed produced, processed, and made available to smallholder farmers. Having well documented procedures can add transparency and hedge against a variety of issues from future staff turnover to quality assurance. For example, although a manual processing operation does not equate to poor quality, it can contribute to inadequate seed drying and inadvertent mixing of different varieties during processing. Poor quality seed in the market can reduce availability of and demand for improved varieties. Technical and Entrepreneurial Capacity of Out-growers and Farmers Compared to Open-Pollinated Varieties (OPV) maize seed production, certified hybrid maize production is a complicated process. Companies and out-growers must be supported to develop the necessary skills. De-tasseling female lines during flowering is an intensive activity, which requires a knowledgeable and dedicated crew every day for two to three weeks. All the SSTP maize seed company grantees interviewed used out-growers to produce certified maize seed. Without adequate supervision, 25 inexperienced out-growers cannot be expected to successfully manage many acres of maize seed production during their first year. Due to the prevalence of many smallholder farmers in maize producing areas, another challenge is finding production fields that meet required isolation distances between hybrid maize fields. For instance, in Ghana, inadequate isolation areas contributed to contamination of SSTP-funded foundation seeds. For self-pollinating crops like beans, or vegetatively propagated ones like cassava, it is possible for farmers to effectively produce quality planting materials on their own farms to sell to other farmers as a business. As the case of cassava in Mozambique has shown, farmer willingness to engage in production and sale of planting material is currently lacking. In other cases, such as that of beans in Tanzania, farmers lack the entrepreneurial capacity to successfully engage in profitable, small-scale seed businesses. 4.3.4 Barriers to Distributing Seeds to Local Markets Relationships Between Seed Companies and Agro-dealers The development of strong business relationships between seed companies and agro-dealers is essential. Agro-dealers play a critical role in seed company growth. Seed companies in both Ghana and Tanzania expressed challenges in getting agro-dealers to pay for seed provided on credit, yet a few had started to develop formal business contracts with agreed-to terms and conditions. Terms of seed supply, sale, and return are critical, especially when agro-dealers have unsold seed and inadequate storage facilities, which could impact the quality of seed sold in subsequent growing seasons. Capacity of Agro-dealers Agro-dealers new to the seed business lack knowledge about farmers demand for seed and how to stock and sell seed from different suppliers. They may also lack storage or shelf-space in their shops. These issues make new agro-dealers and retailers hesitant to stock enough relevant varieties of seed for farmers. The selling season for seed to farmers is only at of the beginning of the planting season, which is only few weeks long. Agro-dealers and retailers must balance between ordering too much or not enough seed in a very short time frame. Another barrier is getting agro-dealers to replace the old varieties they know with new varieties that may be better adapted to the changing environmental pressures like drought, heat stress, disease, or increased pests. This barrier can be overcome if seed companies make efforts to create farmer awareness so farmers will demand from the agro-dealers the varieties in question. Most agro-dealers do not track sales and the few that do, do not convey this information to seed company suppliers. A much closer, formal working relationship between seed companies and agro-dealers can help address some of these capacity issues. Access to Financing Lack of access to financing by seed companies, agro-dealers, out-growers, and farmers is a barrier to the availability of improved seed varieties to farmers. Increasing the availability of seed to farmers entails additional investments in product development, production, promotion, distribution, and services to farmers. These, though, are impacted by availability of financial services and by farmers’ lack of cash for buying seed of improved varieties. For example, an agro-dealer who does not buy adequate quantities of seed at the beginning of the planting season because of lack of adequate financing limits the availability of improved varieties of seed available to smallholder farmers. 4.3.5 The Roles of the Private Sector in Seed Value Chains In 2015, USAID and the BMGF commissioned a study on EGS to develop a framework for evaluating and tailoring policies and interventions in the seed sector. The model has been tested and vetted in four of six SSTP countries (and in four non-STTP countries). The ET believes this is a good framework to use to evaluate investment options pertaining to the private sector viability for the four crops discussed in this evaluation – bean, cassava, maize, and potato – and possibly other crops AGRA/SSTP might want to support in the future. The seeds and planting material likely to be made available through the private 26 sector will depend on the crop and its potential for return on investment, profitability, and strategic fit. These terms will be defined by the seeding rate, the nature of farmer demand, and the associated EGS archetypes described below. Table 4 summarizes the relative attractiveness of the four SSTP crops of interest to the private sector based on feedback obtained during the evaluation interviews. Because it takes so long to bulk up, the higher the seeding rate, and the lower the multiplication rate, the less profitable a crop variety is. As the table indicates, beans, potatoes, and cassava have very high seeding rates and lower multiplication rates, therefore seed bulking can take many more seasons than a crop like maize. Maize has a higher multiplication and a lower seeding rate compared to the other crops in the table. It is also planted on many acres and has a high purchasing frequency. Both of these factors influence the level of market demand. Because farmers must buy new hybrid maize seed every growing season, it has the potential to be profitable. OPV maize, on the other hand, has potential to be profitable only depending on how often farmers buy new seed. Table 4. Comparison of Seed Planting Rates of Crops Observed During Evaluation Seed Minimum Purchasing Frequency Seeding Rate/Ha Attractiveness to Private Sector Beans Every 4-6 seasons 80 kg Low Cassava Every 3-5 years 10,000 cuttings Low Potato Every 3-6 seasons 2,000 kg Low to Medium OPV Maize Every 2-3 seasons 22 – 25 kg Low to Medium Hybrid Maize Every season 22 – 25 Kg High Seed potato production is expensive due to low yields (less than 1.0 ton/ha) and high planting rates (2,000 kg/Ha), which mean longer bulking up time. It also has a high production cost because producing disease-free planting materials is necessary. These materials require frequent inspections and use of crop protection products, and both add to production cost. Furthermore, potato seed is perishable, in less than adequate conditions, and it does not store well, which drives demand for quality seed potato. Given these factors, production and sale of disease-free seed potato from tissue culture and minitubers present opportunities for the private sector to engage in seed potato production. Seedling rate, bulking rate, cost of seed production, and level of market demand, are all factors contributing to the classification of the archetypes defined by the EGS study. The “Private Sector Dominant” archetype makes it attractive for the private sector to produce the crops that the market demands. This results in robust private sector investment with minimal public sector involvement. Hybrid maize is an example of a crop that can be made available through the private sector for various reasons that include its potentially high yield; the ability to provide continued improvements and new varieties by stacking multiple desirable traits through breeding; the potential for obtaining IPR for new varieties; and, since hybrid vigor is lost when seed is recycled, there is regular demand through recurrent annual sales. Maize farmers buy new seed every year, which is an assured source of revenue. Private sector companies are, therefore, more likely to invest in the full cycle of hybrid maize development, from breeding, to foundation seed and certified seed production, to distribution, and sale. The “Private-Public Collaboration Archetype” includes crops that have market demand, but the cost of production or demand risk create barriers to private-sector investment and innovation, so public sector involvement becomes necessary. OPV maize falls into this archetype because it might attract some private sector investments for strategic business reasons, such as market share preservation or mixed product portfolio offering. For example, a private sector company may choose to maintain parental lines 27 of OPV maize so they can cross it with a desirable male inbred and produce top cross maize hybrids that meet business or customer needs. Alternatively, important traits like pro-vitamin A traits, drought tolerance, and others in varieties released and registered through the public sector might be introduced through the private sector. For this, suitable licensing arrangements would be necessary to allow the company an opportunity to produce and sell seed and make a profit. The “Niche Private Sector Archetype” includes seed or planting material that is profitable to produce in certain quantities within a vertically integrated private sector, and which requires minimal public involvement. Fitting into this archetype is the Tanzanian production of tissue culture potato plantlets sold to produce minitubers. The minitubers are then multiplied into certified seed and sold to farmers. This system has the potential to be profitable. The private sector might get involved in the production of niche market crops with high demand where a degree of vertical integration is possible, such as potatoes with specific output market linkages, potato seed production, or certain processing potato varieties. The “Public Sector Dominated Archetype” is the least attractive to the private sector. This includes crops such as common beans and vegetatively propagated crops like cassava. Farmers can save and recycle their own seed so, except in cases of national disaster or severe disease outbreak, there is relatively little market demand. Additionally, cassava cuttings are perishable, bulky to transport, and not amenable to distribution through the agro-dealer networks. These factors discourage private sector investments. 4.3.6 Value Chain Segments Suitable for Private Sector Investment The previous section shows that seed value chains suitable for private sector investment depend largely on the characteristics of the crop itself. In addition, crops with strong IPR protection are more likely to be taken on from breeding to marketing and sale by the private sector. It is possible for private sector to specialize in all, one, or a few value chain segments, provided opportunities to derive value from the activity exist, and that financing is available. Certain value chain segments suitable for private sector investment include the following: ● ● ● ● ● In Ghana, most of the SSTP grantees struggled to produce quality foundation seed due to lack of skills and other capacity. Some were able to contract short-term expertise. This suggests possible roles for a private sector foundation seed multiplier who, under contract, can provide this service to other seed companies, and leaving those clients to concentrate on certified seed production, distribution, and sale. SSTP provided a grant to LCIC in Ghana to produce maize foundation seed for other seed companies. Although 2016 was the company’s first time in operation, it is an example worth monitoring in the coming years. Similarly, a private sector company could specialize in processing certified seed produced by other seed companies under contract, or buy and sell seed produced by other seed companies. There are examples from other countries that could inform SSTP/AGRA and private seed companies on about how to structure such businesses and which crops work better than others. Out-growers are a type of private sector. In Tanzania and, to some extent in Ghana, there are private out-growers that specialize in certified seed production. Out-growers could be contracted to produce certified seeds for other companies that lack capacity to produce all of the seed they require, or that do not want to engage in actual seed production in the field. Private sector seed companies that specialize in germplasm development and licensing. LCIC in Ghana was considering this route, but had not developed a business case for it. The rapid production and multiplication of clean planting materials via tissue culture, as an SSTP grantee in Tanzania does to produce potato plantlets and minitubers. The technology reduces the time it takes to bulk up enough planting materials to supply farmers, and ensures that planting materials are disease-free. Technology is not limited to potato—it has been successfully used on such crops as banana, coffee, and sweet potato. The technology is also 28 applicable to cassava, a less profitable crop, and can be applied through a PPP so long as defined roles and responsibilities are in place and a mechanism to ensure sustainability exists. 4.3.7 Effectiveness of the Private Sector in Creating Demand For the most part, grantee seed companies have relied on traditional methods to create awareness and promote demand for seed of improved varieties. The methods used were very similar across the four crops in the evaluation, and included use of demonstration plots, field days, radio programs, printed materials (e.g., leaflets, posters, stickers), small packets of seed, and advertising in local newspapers. The extent to which these methods were used varied depending on company resources and the geographic areas of operation. Most seed companies lacked field-based marketing agents and relied on government extension agents, farmers and agro-dealers, and/or partnerships with NGOs to help promote their seeds/varieties. For the management of demonstration plots, relationships with extension agents tended to be quite informal. In Ghana, an SSTP grant to an outreach company aimed to promote new varieties through demonstration plots managed by government extension agents. However, there was insufficient linkage to the SSTP seed company grantees and their respective seed sales outlets for this to be fully effective. In Tanzania, the ICT Extension Challenge Program included a partnership between SSTP seed company grantees and FRI—described in Section 4.2. In interviews, farmers rated demonstration plots and radio announcements the most reliable sources of information on improved varieties. Such promotional activities may create awareness of the improved variety being promoted, but may not lead to immediate purchase and adoption. Seed companies recognized that sustaining demand creation activities over several seasons is important for effectively creating demand. Provided the variety was seen to be appropriate, farmers who hosted demonstration plots were more likely to quickly adopt a variety from a demonstration plot. Further discussion on the effectiveness of different methods used to create demand and awareness can be found in Section 4.2. At least two seed company grantees were very clear in that the best way to ensure demand for their seed was through linkages to output markets. In Ghana, for example, one grantee seed company is developing a linkage with a chicken feed company for purchasing yellow maize from farmers. In Tanzania, a company producing potato plantlets and tubers has a link with a company that intends to produce frozen chips to sell to restaurants and hotels. Referring to the models presented in Table 3, these companies are effectively shifting to the triple constraint – double organization model. CONCLUSIONS The availability of quality seed increased for all seed classes and for all focal crops in all countries. For a greater efficiency, quantities of foundation seed produced could have been tied to forecasted quantities of certified seed required. For bean and cassava, limited data available suggest that a PPP model leads to considerably bigger increases in seed availability than the single organization models. Plans must be in place to ensure the sustainability of PPPs when donor funding is completed. Barriers that have yet to be addressed include: 1) the lack of effective forecasting methods and procedures; and 2) the lack of workable quality management systems for seed production. Capacity constraints for seed production within private and public sector organizations remain a major challenge. Seed value chains suitable for private sector investment depend largely on the characteristics of the crop itself. The EGS archetypes provide a good indication of the types of crops likely to be produced and made available by the private sector. Crops with strong IPR protection are more likely to be taken on by the private sector. 29 The private sector can specialize in all, one, or a few segments in the value chain provided that an opportunity to derive value from the activity exists, and that financing is available. Lack of access to financing by seed companies, agro-dealers, out-growers, and farmers is one of the main barriers to scaling the multiplication, distribution, and adoption of improved seed. There is a need to improve formal seed distribution strategies through improved marketing strategies and the strengthening of sustainable, working relationships between seed companies and agro-dealers. The selection criteria of crops and varieties to be produced, marketed, and sold by private sector companies should consider the market potential of the end products as well as their suitability to the local agro-ecological conditions. It is necessary to link seed value chains with associated product or output value chains to ensure that inputs provided meet the needs of existing associated output markets. Linkages with output markets have been shown to create demand for seed of improved varieties. The private sector’s effectiveness in creating demand can be enhanced by improving effectiveness of demonstration plots and by engaging multiple stakeholders (including ICT Challenge partners) working in the seed and output value chains to create customer-driven demand for seed of improved varieties. RECOMMENDATIONS • • • • • • • SSTP should develop transparent seed requirement forecasting methods and procedures. These could be piloted by a broad base of partners and stakeholders in Phase 2 in order to ease planning for multiplication of seed of the different classes.22 SSTP should support its grantees in to develop and implement a workable quality management system for seed production in Phase 2. Ongoing efforts to develop capacity for seed production within private and public sector organizations should be expanded and strengthened in Phase 2. In the report, there are examples of how this might be done. SSTP should identify and promote the development of financing strategies appropriate to private sector seed companies and facilitate the development of linkages with appropriate financial institutions for credit to expand scaling up activities. This activity can be implemented through the national seed traders’ associations. SSTP should place greater emphasis on assisting grantee seed companies to develop their sales and marketing skills. SSTP/AGRA should facilitate in-country discussions with crop value chain actors to foster seed companies’ and farmers’ linkages to viable output markets, which demand quality seeds of specific improved varieties. SSTP/AGRA should also seek linkages with other government and donor-funded programs working on expanding agricultural productivity through agricultural market development. SSTP grantee seed companies must be more systematic in ensuring that the varieties selected for multiplication and marketing meet the needs of the final consumers, e.g., specific maize varieties for use in the poultry feed industry; improved cassava varieties for beer brewing; or potatoes suitable for frying chips. 22 Although governments theoretically have teams to conduct annual seed forecasting, the lack of clear forecasting methods in the countries visited suggest that there is a need for SSTP to support the development of more effective procedures. 30 • • SSTP should develop good practice guidelines for demonstration plots (based on SSTP experience and existing literature) so that they are as effective as possible and reach more farmers. SSTP should facilitate linkages and support partnerships between private seed companies and specialized, experienced communication organizations for effective demand creation (see below). 4.4 EVALUATION FINDINGS: SCALING AND ADOPTION OF TECHNOLOGIES Much of the analysis presented in this section comes from the quantitative farmer survey undertaken in the three countries for the four focal crops. The survey was designed for implementation across two years. The findings presented here are based on analysis of the first wave of the survey conducted in March-May 2017. The full survey findings from Wave 1 are in a separate report (see Annex I. Farmer Survey Report), and findings from the second wave will be presented in a combined survey report in 2018. The survey report in Annex I presents a summary on survey methods, sampling, and findings, survey questionnaire, and screening questionnaire used in Wave 1. 4.4.1 Use of Improved Varieties by Direct and Indirect Beneficiary Farmers The survey results show that a relatively high proportion of beneficiary farmers began growing SSTP￾supported varieties prior to the start of the SSTP Program. This was particularly true in Ghana and Tanzania, countries which both have long history of improved varieties. Across all three countries and all four crops, Table 5 shows that 28.5 percent of sampled beneficiary farmers reported adopting SSTP varieties more than three years ago (prior to the start of the SSTP Program), and 34.9 percent of beneficiary farmers adopted or applied23 the varieties since the start of the SSTP Program (within the past three years). When farmers were disaggregated by beneficiary status, data show that 44.4 percent of sampled direct beneficiaries and 21.5 percent of indirect beneficiaries have either adopted or applied SSTP varieties since the start of the SSTP Program. Overall, 36.6 percent of all sampled farmers reported not applying or adopting any SSTP varieties at all. Unlike Ghana and Tanzania, varieties promoted by SSTP in Mozambique do not have a long history. Only 3.7 percent of farmers reported beginning to grow the SSTP cassava varieties before the SSTP Program. The overall rate of adoption/application during the SSTP Program was 21.9 percent of all beneficiary farmers in Mozambique. There was a big difference between direct beneficiaries (38.3 percent) and indirect beneficiaries (5.8 percent) in that period. Qualitative data collected through FGDs and interviews in Mozambique reveal that only association members could access planting material of the SSTP-supported cassava varieties. In general, farmers were keen to adopt the varieties to benefit from the sales of cassava. To allow for this, many new associations were formed. However, access to planting material was a constraint, not because it was not available but because farmers were reluctant to sell planting sticks to other farmers. Apparently, they wanted the project to intervene and act as buyer. Clearly, the model for making cassava-planting material available to farmers in Mozambique requires some adjustment to meet the high level of farmer demand for improved varieties. In Ghana, 35.5 percent of farmers had adopted one or more of the varieties before the SSTP Program started. Almost 30 percent of beneficiary farmers (35.7 percent direct beneficiaries and 23.2 percent indirect beneficiaries) adopted or applied one or more SSTP varieties since the beginning of the SSTP Program. Tanzania shows a similar pattern to Ghana, but with generally higher rates of 23 A farmer typically tests or tries out a new variety for two or three seasons before deciding whether or not to adopt it into their planting repertoire. Within the SSTP Program, this period of testing by a farmer is referred to as “application” and will often lead to adoption. In general, there is a very blurred line between application and adoption, but a farmer that has been growing a variety for more than three years can be considered to have adopted it. 31 adoption/application, especially during the SSTP Program period (51 percent of all beneficiaries adopted SSTP varieties), and with much more subtle difference between direct and indirect beneficiaries. The results for all three crops in Tanzania show similar patterns (Table G.6, Annex G). The fact that almost the same proportion of indirect beneficiaries are planting SSTP varieties as direct beneficiaries might be due to seed availability through agro-dealers, and functional agricultural output markets that stimulate demand for improved varieties. 4.4.2 Strategies to Promote Adoption vs. Application Under certain conditions, a farmer will adopt a variety he or she has tested or “applied” on his/her farm if it is appropriate to the agro-ecology of the farm. The conditions are that: the variety displays the household members’ preferred characteristics; the seed must be accessible; and there is a market for the output (Morris et al, 1999; Doss, 2006; Fisher et al, 2015). The more educated and (usually) better￾off farmers are also more likely to adopt improved varieties than their less educated peers (Bruce et al, 2014; Morris et al, 1999). Adoption depends more on the local farming environment; however, farmers’ learning processes are important in new technology adoption. Learning from their own experiences and from those in their social networks is a proven means to affect farmers’ adoption decisions (Foster and Rosenzweig, 1995; Conley and Udry, 2000; Bandiera and Rasul, 2002, cited by Doss, 2006). The sub￾sections below present the survey findings on farmer learning, motivation to test new varieties, and the extent to which promotional strategies and means of seed acquisition influence adoption vs. application. How Farmers Learn About New Varieties Clearly, for all countries, neighbors and relatives are critical to farmers learning about new varieties and being motivated, either as direct or indirect beneficiaries, to cultivate an improved variety (Table 6). 32 Table 5. Number of Sample Beneficiary Farmers Who Have Adopted or Applied SSTP-Promoted Varieties Adoption Types Ghana (N=1095) Mozambique (N=1180) Tanzania (N=1317) All Countries (N=3592) Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Pre-SSTP Adopters N 234 155 389 33 11 44 439 153 592 706 319 1025 % 42.4 28.5 35.5 5.7 1.8 3.7 45.7 43 45 33.7 21.3 28.5 SSTP Adopters / Appliers N 197 126 323 223 35 258 511 161 672 931 322 1253 % 35.7 23.2 29.5 38.3 5.9 21.9 53.2 45.2 51 44.4 21.5 34.9 Non￾Adopters / Non￾Appliers N 121 262 383 326 552 878 11 42 53 458 856 1314 % 21.9 48.3 35 56 92.3 74.4 1.1 11.8 4 21.9 57.2 36.6 TOTAL N 552 543 1095 582 598 1180 961 356 1317 2095 1497 3592 % 100 100 100 100 100 100 100 100 100 100 100 100 Pre-SSTP Adopters are farmers who started using improved seeds more than three years ago (Pre-SSTP). SSTP Adopters/Appliers are farmers who started using improved seeds two or three years ago or last season (since the start of the SSTP Program). Non-Adopters/Non-Appliers are farmers who have never used improved seeds in their plots. 33 Table 6. Main Motivation Factor That Persuaded the Farmer to Plant the Improved Variety for the First Time by Crop and Country Source of Information Ghana Mozambique Tanzania Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Farmer-managed demonstration N 13 3 16 56 1 57 117 0 117 % 3.0 1.1 2.2 21.9 2.2 18.9 12.3 0.0 9.3 Seed company/Project￾managed demo/Agro￾dealer N 26 10 36 25 3 28 73 14 87 % 6.0 3.6 5.1 9.8 6.5 9.3 7.7 4.5 6.9 Neighbor/Relative N 187 185 372 68 32 100 495 273 768 % 43.4 65.8 52.2 26.6 69.6 33.1 52.1 86.9 60.8 Field Days/Agricultural show N 13 2 15 6 1 7 9 2 11 % 3.0 0.7 2.1 2.3 2.2 2.3 0.9 0.6 0.9 Community events N 9 1 10 26 2 28 89 11 100 % 2.1 0.4 1.4 10.2 4.3 9.3 9.4 3.5 7.9 Government extension agent/Village-based agent (VBA) N 122 32 154 41 1 42 86 8 94 % 28.3 11.4 21.6 16.0 2.2 13.9 9.1 2.5 7.4 Farmer training course N 1 1 23 2 25 36 0 36 % 0.2 0.0 0.1 9.0 4.3 8.3 3.8 0.0 2.8 Radio program N 22 23 45 1 0 1 6 1 7 % 5.1 8.2 6.3 0.4 0.0 0.3 0.6 0.3 0.6 Agriculture Organization/TV/Message on Farm N 14 4 18 0 0 0 1 0 1 % 3.2 1.4 2.5 0.0 0.0 0.0 0.1 0.0 0.1 Others N 24 21 45 10 4 14 38 5 43 % 5.6 7.5 6.3 3.9 8.7 4.6 4.0 1.6 3.4 Total N 431 281 712 256 46 302 950 314 1264 % 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 34 As cited above, this is consistent with the broader literature. Neighbors and relatives are particularly influential among indirect beneficiaries. Extension agents help motivate direct beneficiaries (in all three countries, especially Ghana) who are also motivated by farmer-managed demonstration plots [Mozambique (cassava), Tanzania (potato)]. 24 This suggests that promotional methods that draw on the influence of neighbors and relatives would have considerably greater impact than those that do not. How Farmers Acquire Seed of New Varieties Farmers acquire the seed of SSTP varieties planted in different ways across different countries and across different crop (Tables 7 and 8). In Ghana (maize), the majority of farmers (61.9 percent) purchased seed from seed companies or agro-dealers, extension agents or market traders, at full cost. Almost one third (32.2 percent) acquired the seed for free, mostly from other farmers. In Mozambique (cassava), almost all farmers (95 percent) acquired planting material for free through the farmer association, through another farmer (presumably an association member), or from the extension agent working with the association. In Tanzania, almost all maize and bean seed was purchased at full cost. Maize was purchased mostly from either government or village-based extension workers (86.6 percent). Given the relatively small proportion of maize seed sourced from seed companies and agro-input dealers, it seems likely that government extension workers are either working as agents for seed companies, or that village-based extension workers supply seed according to the FIPS model described below. Bean seeds were acquired from seed companies/agro-input dealers (36.2 percent), market traders (35.8 percent), and other farmers (27.2 percent). Because bean is self-pollinating, it is possible for farmers to produce good-quality seed. There is a well-developed informal regional market for beans for consumption, so farmers are willing to pay for the seed from the informal sector (i.e., farmers and market traders). In the case of potato, almost 67 percent of seed was purchased at full cost, and almost 30 percent was acquired for free. Just over 66 percent was acquired from other farmers. Although most freely acquired seed is from other farmers, the SSTP grantee seed company provided potato seed free to project farmers in the first year of the grant, and then other direct beneficiaries were expected to purchase seed from the project farmers and from the seed company in subsequent years, as confirmed by the data. In general, there was no discernable difference between direct and indirect beneficiaries in relation to the way seed was acquired. Table 7. How Farmers Acquired the Improved Varieties When They First Used It/Them (by Crop and Country) Source of Seeds Ghana Mozambique Tanzania Maize Cassava Maize Beans Potato Purchased at full cost N 441 5 403 411 273 % 61.9 1.7 90.2 85.8 66.9 Purchased at reduced cost N 29 9 15 1 6 % 4.1 3.0 0.03 0.0 0.0 Got it for free N 229 287 8 11 120 % 32.2 95.0 0.0 0.03 29.4 In-kind or any exchange N 13 1 1 6 9 % 1.8 0.3 0.0 0.0 0.0 TOTAL N 712 302 447 429 408 24 It is unclear why demonstration plots and associated field days should have so much more impact for cassava and potato than for other crops; this is something that requires further investigation. 35 Table 8. From Whom Farmers Acquired Improved Varieties When They First Used It/Them (by Crop and Country) Source of Seeds Ghana Mozambique Tanzania Maize Cassava Maize Beans Potato Faith-Based Organizations (FBOs)/NGOs N 24 132 4 0 40 % 3.4 43.7 0.0 0.0 9.0 Seed company/Agro￾input N 219 24 8 177 72 % 30.8 7.9 0.0 36.2 0.0 Gov’t extension agent/VBA N 151 72 380 4 31 % 21.2 23.8 86.6 0.0 0.0 Other farmer or relative N 242 103 35 133 294 % 34.0 34.1 8.0 27.2 66.2 Market trader N 109 1 12 175 7 % 15.3 0.03 0.0 35.8 0.0 Total N 712 302 439 489 444 % 100.0 100.0 100.0 100.0 100.0 Distance to Agro-Dealers Farmers are more likely to be able to purchase seed of improved varieties if the source is within a reasonable distance. Reducing the distance between farmers and agro-dealer shops is among the SSTP indicators. For all cases where farmers acquired SSTP-promoted seed from an agro-dealer, Table 9 presents the average distance farmers reported their homes are to the agro-dealer shop where they purchase maize and beans. 25 The average distance traveled when they first used the seed, which may have been prior to the SSTP Program or during the SSTP Program, is compared to the average distance for the season immediately prior to the survey. These distances have reduced from 11.4 km to 4.6 km in Ghana, and from 3.0 km to 2.7 km in Tanzania. The project target was a reduction in distance from an average of 20 km to 6 km. Table 9. Average Distance to Agro-Dealer (km.) Country Crop Number/Mean Average Distance When Seed of SSTP Variety Was First Purchased Average Distance Last Year (2016) Ghana Maize N 198 198 Mean 11.42 4.57 Tanzania Maize N 369 369 Mean 3.20 2.73 Beans N 154 154 Mean 2.60 2.69 Total N 523 523 Mean 3.03 2.72 Application, Adoption and Discontinuation Farmers will test or try out a new variety over several seasons before deciding to adopt it into their planting repertoire (Okali et al, 1994). It is normal for farmers to test a small quantity of a new variety’s seed, and perhaps even increase the quantity of seed sown each season while it is being tested. Within 25 Cassava and potato have been excluded from the table because their planting material is bulky and tends not to be supplied by agro-dealers. 36 the SSTP Program, this period of testing by a farmer is referred to as “application” and will often, not always, lead to adoption. In general, there is a blurred line between application and adoption, but when a farmer has grown a variety for more than three years, s/he can generally be considered to have adopted it. In all cases where a farmer grew an SSTP-promoted variety, the farmer survey asked whether s/he intended to plant the same variety again the following year. Where farmers answered “yes” they could either mean they are continuing a period of testing/application, or that they have made a decision to adopt the variety. Farmers who answered “no,” clearly decided not to adopt the variety. A decision not to plant a variety again the following year is generally referred to as abandonment or discontinuation. There are two types of cases when a farmer stops growing a particular variety: 1) a farmer decides not to adopt a variety after having tested or applied it for approximately one to three years; or 2) an adopted variety that has been cultivated over several years is replaced by another variety. Data relating to discontinued varieties reveal one of the reasons for discontinuation is low yield. This is often combined with poor rainfall or other weather-related conditions, suggesting that the length of testing time and decisions about adopting are closely related to annual variations in rainfall. Other reasons for discontinuation were a lack of output markets (all countries – see below), the cost of seed (Ghana, Tanzania), the availability of seed (Ghana, Tanzania), particular varietal characteristics such as taste and cooking qualities (Mozambique), and storability (Ghana). With the exception of potato in Tanzania, almost all farmers who acquired seed within the last three years (since the start of the SSTP Program) reported intentions to plant the variety again the following season. However, it is impossible to tell whether these varieties have been adopted or if the application period is merely continuing. In the case of potato in Tanzania, from 219 sampled farmers who reported acquiring seed of SSTP varieties within the past three years, 40 percent said that they would plant it again the following season, and 60 percent said that they would not.26 No correlations could be seen between the means of seed acquisition and the farmer’s intention to plant the variety again. In sum, it appears that a farmer’s decision to adopt or discontinue a particular variety relates to the particular varietal characteristics such as yield often in combination with: weather-related conditions; issues of taste, storability, and other factors; the presence or absence of output markets; and the cost and availability of seed. There seems to be no correlation between decisions to adopt/discontinue particular varieties and either the strategies used to promote them, or the way the farmers acquired the seed. An interesting finding emerging from the survey is that over half the farmers (59.9 percent across all three countries) who discontinued an SSTP-promoted variety, reported that they planned to use the discontinued variety again in future.27 Barriers to Adoption by Different Groups As described below, affordability, availability, and the absence of output markets emerge from both the qualitative and quantitative data as key barriers to improved variety adoption. Affordability Survey findings from Ghana showed that affordability was the main reason for not trying improved seed or other technologies (cited by 44.2 percent of beneficiary farmers). However, in Mozambique, the main reason both direct and indirect beneficiary farmers (52.9 percent) gave for not trying improved seed or technologies was that they did not know where to get the inputs (Table G.7, Annex G). In Tanzania, the overall number of farmers who reported never trying improved seed or other technologies was 26 It is not clear why so many farmers reported that they do not intend to plant the improved potato varieties again; further investigation will be required to determine whether there is perhaps one variety that was found not to be appropriate to farmers’ and market preferences. 27 It is quite possible that this response was perhaps motivated by what the farmer thought the enumerator would want to hear. On the other hand, it is equally possible that the farmer did actually like the varietal characteristics and would be interested in planting the variety again if the seed was available and affordable and if output markets were present. 37 considerably smaller. The main reason they gave was a lack of interest in trying something new (reported by all). Other reasons were affordability (mainly direct beneficiaries), not knowing where to get the inputs (mainly indirect beneficiaries), and lack of knowledge about improved technologies (mainly indirect beneficiaries). These results are presented in Table G.7, Annex G. A comparison of pre-SSTP adopters, SSTP adopters, and non-adopters with gender and socio-economic status (including age, education, household size, farm size, asset ownership, and group membership) does not show any significant differences in adoption based on socio-economic status. Adoption appears to be slightly more likely among male farmers than female farmers in Ghana and Mozambique. Also, those with less education are more likely to be non-adopters in both Ghana and Mozambique (Table J.3.2, Annex J). None of these findings showed any significant positive correlations. Cost of Seed and the Relative Wealth Status of Farmers Survey results show that affordability is a reason for not trying a new variety or technology in Ghana and Tanzania. The cost of seed was cited as a reason for discontinuing the use of SSTP-promoted varieties in both countries. The question of affordability arose in Mozambique but, because the planting materials were provided for free, only a few farmers cited this as an issue. Given these findings, one would expect adoption to correlate with the relative wealth of sampled farmers. The farmer survey collected data on various wealth-related variables, including farm size and the ownership of communications assets like radio, TV, and mobile phones. However, when controlling for other key factors, including beneficiary status, land size was significantly associated with the adoption of improved maize varieties in Ghana. In Tanzania, land size was associated with use of improved seeds for the three focal crops. Only in Mozambique was ownership of radio and television significant predictor of adopting improved cassava varieties. The ET explored the issue of seed cost through interviews and FGDs in Ghana, and concluded that the way farmers perceive the value and cost of seed is an important consideration. Although farmers taking part in the FGDs were generally willing and able to purchase fertilizer and agro-chemicals, they felt that seed was “too expensive.” Information from agro-dealers on the actual costs of hybrid maize seed, agro￾chemicals, and fertilizer for cultivating a one-hectare plot in Ghana revealed that fertilizer costs 10 times what seed costs.28 This suggests that farmers’ perceptions of seed prices are not based on cost alone. Seed of subsistence crops plays an important role in establishing and maintaining social relationships among traditional farming communities, and seed of subsistence crops (as opposed to income-generating crops) is typically not regarded as a commodity with a monetary value (Longley, 2001). Clearly, there is a major challenge in persuading farmers that the investment in non-hybrid seed is worthwhile. Not only is it necessary to illustrate economic advantages in terms of the increased yield quality seed of improved varieties gives, but it is often also necessary to overcome traditional perceptions regarding the social value of seed in creating and sustaining cohesive relationships within farming communities. Output Markets The farmer survey collected data from 461 farmers who reported discontinuing use of SSTP varieties. In each of the three countries, a lack of output markets was among the top three reasons for discontinuation. On the other hand, if there is an opportunity to generate a reliable income from cultivating a particular crop or variety, farmers are likely to take advantage. This is illustrated within the SSTP Program by the recent expansion in the cultivation of potatoes in Tanzania and cassava in Mozambique. In Mozambique, the market for cassava varieties for beer industry use created a very high 28 The total cost of inputs required for one hectare add up to approximately US$100, comprised of hybrid maize seed ($7), agro-chemicals ($16), and fertilizer ($77). 38 level of demand for the SSTP-promoted varieties. Unfortunately, this demand was not satisfied through the grantee’s particular dissemination method. For more subsistence-oriented crops, like maize, the presence or absence of output markets also influences adoption. This is illustrated by the case of a female farmer in the FGD in Amfoe village (Adako District, Volta Region) in Ghana. She was keen to try out abontem maize variety (also known as “yellow corn”) when she saw it on an SSTP-supported demonstration plot. She had eaten yellow maize when she was a schoolgirl and liked it. After growing it for a season, she saw that it was high yielding, but discovered there were no markets for her surplus production, and she decided not to grow it again the following season. In this case, there was in fact a market for abontem (for chicken feed), but the farmer was not aware of this. If the farmer had been linked to the output market she probably would have adopted the variety and continued to cultivate it. This illustrates the need for efforts to link farmers to output markets; it cannot be assumed it will happen on its own. Innovative Approaches Used by SSTP Grantee Allowing a farmer to test or try out a variety and learn about it for themselves is considered to be key in promoting adoption (Fisher et al, 2015). As such, affordable or free sample “tester” packs of small quantities of seed (typically 25-100g) is effective (ICRISAT, 2014). Within the SSTP Program, sample packs were provided for free by FIPS in Tanzania (see below), and by M&B in Ghana. It has also been shown that farmers tend to adopt “packages” of different technologies (e.g., improved varieties, fertilizer, and herbicide) in a step-wise manner, rather than as a package (Byerlee and de Polanco, 1986, cited by Doss, 2006). This has implications for how associated technologies should be introduced and promoted and is a topic for a different study. In each of the SSTP countries there is at least one grantee specializing in communication and promotion of improved technologies. Also, the application of ICT-based extension approaches is supported through SSTP’s link with the ICT Challenge Program. This section describes three particularly innovative approaches encouraging use of improved seeds and technologies that have the potential to be very effective. Unfortunately, none of the areas where these approaches are being implemented were included in the farmer survey, but it is possible to extrapolate relevant findings from the survey to draw lessons on the approaches and methodologies likely effective. Farm Input Promotions Africa Ltd. In Tanzania, FIPS is a grantee working across three Southern Highlands districts. The FIPS approach involves five closely integrated components: 1) when initiating work in a new area, a farmer needs assessment is undertaken with local partners, which includes a representative from the district agricultural office; 2) partnerships are established with the district agriculture office, agricultural research organizations, NGO development partners, and private sector agricultural input companies (for seed, fertilizer, etc.); 3) Farming System Improvement is promoted through mother-baby demonstrations29 using a range of different technologies based on initial needs assessment, and provided through the private sector partners; 4) self-employed VBAs identified by members of the community and trained to promote the technologies and sell inputs to farmers (the VBAs tend to be respected farmers or experienced “organizers” and develop close links with the local government extension officer0; and 5) a Small Pack/Whole Village method that allows everyone to be involved, rather than a single group-based approach. 29 The “mother” demonstration plot includes a range of different varieties and/or technologies, and the associated “baby” demonstration plots are small plots within individual farmer’s fields where a small amount of a single technology or variety is tested. The baby demonstrations are made possible through the provision of small seed packs. 39 The FIPS approach is particularly innovative in relation to scaling and adoption in four ways: 1) links with a range of seed and input companies and research organizations allow for a wider range of varieties and technologies to be promoted; 2) the use of VBAs allows for demo plots in more communities than if relying on government extension workers alone; 3) the mother-baby approach allows for more demonstrations in each community, and potentially reaches more farmers and allows the same variety to be tried over different farm conditions; 4) the free seed samples (for the “baby” demos) allow farmers to learn about a new variety by trying it out for themselves (the assumption is that a farmer is more likely to adopt a variety that they have learned about in practice, not just by seeing it on someone else’s demonstration plot); and 5) where the baby demonstration involves a self-pollinating crop (e.g., rice, beans), this allows for the harvested seed to be shared with other farmers. In relation to scaling, the authors of one study state that, “FIPS-Africa seems to have developed a system that has no limit to scale” (Zaal, van der Lee and Mwongela, 2012: 94). The same authors also highlight the major role of the private sector in FIPS-Africa’s success. Farm Radio International (FRI) Also in Tanzania, FRI – supported through the ICT Extension Challenge Fund – has partnered with SSTP grantees, Aminata Seed Company and Kibaha Sugarcane Research Institute (SRI), to promote improved maize and cassava varieties and associated best management practices. To do so, they are using radio, integrated with and supported by short messaging service (SMS) and interactive voice response (IVR) services. The radio programs consist of a series of 16-20 weekly broadcasts focusing on specific issues or technologies such as use of clean planting material and new varieties, early weeding, pest management practices, and production of clean cuttings. A panel of technical experts from the partner organizations and elsewhere develops the content of the programs and listen to each show before it is broadcast to check for program quality. SMS is used to directly reinforce the content on the radio programs and promote the radio programs and boost series listenership. The SMS service is supported by Commonwealth Agricultural Bureau International (CABI) through Direct2Farm, a service that turns technical factsheets into short SMS and voice messages delivered straight to farmers. Up to September 2016, over 200 individual SMS messages were developed and validated by experts. Over 108,045 SMS messages on cassava, and 13,444 on maize farming had been disseminated to 6,536 and 3,439 cassava and maize farmers, respectively. In one case, an SMS message broadcast on improved cassava varieties sparked immediate SMS requests from farmers in four districts asking where planting material could be obtained. FRI anticipates that similar SMS requests in the future can be resolved by automating the system where keywords trigger a SMS response with the contact details for suppliers within the farmer’s residential zone. Each weekly radio broadcast involves one key question developed by agricultural experts to illustrate the message or topic of the episode. Using call-back, listeners can respond to the question by pressing a number on their keypad, which prompts a series of four or five follow-up questions designed to gather in-depth information. One of these questions might be about local weather conditions (have the rains started in your area?), and one or two of these questions are open-ended, which allows the caller to leave a voice message or a question for the experts. All calls, voice polls, and interactions are hosted and logged through an online ICT platform developed by FRI called Uliza (“to ask” in Kiswahili). Broadcasters select interesting messages to use in “Vox Pop” features in the programs, while technical questions left by listeners are sent to experts who answer them in a Question and Answer session (either live or recorded).30 The information collected through the Uliza platform allows for shifts in the focus of future programs in accordance with seasonal changes on the ground and listener preferences. The information is also used for internal web-based monitoring. 30 One might assume that if farmers can get satisfactory, effective responses to particular concerns through the Uliza platform then they would likely be able to address any potential management problems that would allow them to continue to use an improved variety instead of abandoning it as soon as there is a problem. 40 Results from an outcome evaluation survey conducted in late September 2016 among 632 farmers, revealed that 55 percent of respondents had heard of the programs. Of these farmers, 80 percent had listened to at least one episode. Nearly half of all those who listened to one or more episode started using a practice promoted through the radio programs. Approximately one quarter of all listeners started using two of the promoted practices (FRI and CABI, 2016). Story Workshop Educational Trust Story Workshop Educational Trust (SWET) is a Malawian creative media organization with community mobilization expertise. They use social behavior change communication approaches to promote the adoption of improved technologies for seven crops supported by SSTP. Social and behavior change communication is based on a three-step process of message delivery, internalization, and participation. SWET’s first SSTP grant31 in Malawi started in April 2015 and was implemented across four districts, using radio, television, village cinema, and community mobilization to disseminate information. SMS, phone-ins, and personal visits allowed for audience participation in the radio and TV broadcasts. 32 A series of mobile agricultural clinic “performances” brought together 55,987 farmers in total. This provided them with an opportunity to watch, learn, and apply relevant farming practices as well as meet with agricultural experts, private input suppliers, and output marketing companies. The content of the radio and television programs (each involving 30 episodes) was based on the seasonal activities of eight Master Farmer clubs. These clubs were established by SWET, and their members were trained by agricultural experts and communication specialists at a four-day “bootcamp.” The bootcamp was an opportunity for farmers to meet potential seed suppliers and farm produce processors. So-called “Baby Clubs” were formed by the Master Farmers within the target communities, and 2,010 farmers (three members from each Baby Club) were trained to use SSTP-supported technologies at a series of two-day community-based bootcamps. Master Farmers and others trained at the community bootcamps were expected to pass their knowledge on to other farmers through drama, demonstration plots, and regular group meetings. An evaluation undertaken in early 2017 (Kabuli, 2017) included a survey among 200 beneficiary farmers. The survey revealed high levels of awareness about improved varieties of the different crops (77-99 percent), and relatively high levels of application of the improved varieties for most crops (55-64 percent): 98 percent for maize and 24 percent for rice (Table G.7, Annex G). Of the communications media SWET used to disseminate information, drama was the preferred (cited by 48.5 percent of beneficiaries), followed by radio (23 percent). The boot camps, community cinema, mobile cinema, and TV were each cited by 10 percent or fewer of beneficiaries interviewed.33 The evaluation noted that the three-tier approach of television/community cinema, radio, and community mobilization (including drama, meetings, demonstration plots, and radio listening clubs) complemented each other and produced synergies that effectively triggered behavior change towards adoption of quality seeds and technologies. The evaluation further noted that the three-tier approach also created linkages between farmers and seed suppliers, farm equipment companies, extension services, locally available agro-dealers, and other interested companies (Kabuli, 2017: 24). Despite this, however, most groups experienced challenges in accessing the seed needed for the demonstration plots. Therefore, some plots could not be planted. The 31 SWET has recently been awarded a second SSTP grant to extend the Master Farmer approach to three new districts and address the three challenges faced in the four districts targeted by the first grant, i.e., access to improved seed varieties, access to markets, and long-term adoption of positive behaviors in the face of climate change. 32 It is relevant to note that research conducted by FRI across five countries in Africa has shown that farmers who participate in the design and implementation of radio programming with the help of ICTs are four times more likely than those in passive listening communities to adopt agricultural improvements promoted on the radio (FRI, 2011). 33 Demonstration plots were not among the list of media from which farmers could select their preferences. 41 project stimulated the local extension officers to be more active, improved the marketability of most locally grown crops, and linked farmers to new market opportunities. The project increased demand among targeted farmers for seed of improved varieties, but many farmers experienced challenges in accessing seed, and this led the evaluation to suggest “proper structures” to facilitate networking between farmers and agro-dealers as well as encouraged involvement of more agro-dealers in the project areas. The evaluation noted that there was sometimes a lack of availability of certified seed on the market, forcing farmers to buy “fake seeds” from vendors. Summary and Lessons The three approaches described above are each very different, but all rely on an integrated set of methods and communication channels that mutually reinforce each other to potentially powerful effect. FIPS has adapted and combined key elements of traditional outreach and private sector marketing strategies so that they operate at scale. FRI works closely with agricultural experts to deliver technically sophisticated, integrated radio, SMS, and IVR services based on accurate agricultural information, which is fine-tuned to farmers’ interests and seasonal changes as they happen on the ground. SWET applies a wide range of training and communication approaches, including the use of drama, which has proved to be very popular in the project areas. Each of the three cases involves a highly specialized and experienced organization applying or adapting tried-and-tested approaches developed and honed over many years of practical implementation: in the case of FIPS, over 14 years, and nearly 30 in the case of FRI. In the case of SWET, the conceptual basis for its social behavior change communication (SBCC) approach has been developed and applied within the health sector over the past 20-30 years, and it is only very recently that SBCC has started being used within the agriculture sector. This suggests that there will be many lessons to learn from its utilization. Similarly, one of the SSTP grantees in Ghana, Modern African Productions (MAP), a television production company, has developed a television program called Kuapa. The program shows promise but in the view of the ET, it still has a long way to go in developing and honing its approach. MAP has not previously worked in the agricultural sector or targeted rural populations, and in the views of three different members of the panel responsible for reviewing each episode, it took several episodes before developing an appropriate format for its TV program, and some still felt it could be further improved. Although the program primarily targets rural youth, the ET did not come across anyone, extension agents, agro-dealers, or farmers, who regularly watched the show. Only one or two had even heard of it.34 This suggests that TV alone is insufficient to reach farmers at scale, and must be integrated with other communication channels to be effective. The survey results for the ownership of radios, TVs, and mobile phones (Table 10) show that mobile phone ownership is most widespread (79.0 percent of sampled farmers in all three countries), followed by radio (66.2 percent) and TV (29.3 percent). Although the rate of TV ownership is considerably higher in Ghana than the other two countries, it is still much less available than mobile phone and radio. Table 10. Radio, TV, and Mobile Phone Ownership Among Beneficiary Farmers Ghana Mozambique Tanzania TOTAL N=1205 N=1209 N=1325 N= 3739 Radio N 911 525 1039 2475 % 75.6 43.4 78.4 66.2 TV N 636 151 307 1094 % 52.8 12.5 23.2 29.3 34 This was surprising, given that the ET was talking to many different people closely associated with the SSTP program. Farmers, extension officers, and agro-dealers that the ET met with all agreed that radio is more effective than television in rural areas. 42 Ghana Mozambique Tanzania TOTAL N=1205 N=1209 N=1325 N= 3739 Mobile N 1068 603 1283 2954 % 88.6 49.9 96.8 79.0 Recognizing that farmers will test or try out a new variety over several seasons before deciding to adopt it (Okali et al, 1994), one question is whether the length of time over which new varieties are promoted in a particular community has an impact on adoption. For example, is it sufficient to implement a demonstration plot in a single season, or is it more effective to have repeated demonstration plots over more than one season in the same community? Given that not all farmers will immediately be convinced of the benefits of trying out new technologies (majority adopters and late adopters), the assumption is that repeated demonstration plots over more than one season in the same community would be beneficial. However, given the costs and logistics involved, a seed company or implementing agency might feel that their investments have greater impact among more farmers by increasing the number of demonstration plots geographically instead of concentrating for longer time periods in a fewer locations. The survey findings clearly show that other farmers and neighbors have the greatest impact in influencing farmers’ decision-making. Seed companies or other implementing agencies would do best to tap into these social networks by encouraging farmer to interact and learn from each other. SSTP grantees are currently doing this in at least four different ways: 1. By taking a small pack/whole village approach (as opposed to a group-based approach), FIPS aims to reach as many farmers as possible and provides small quantities of seed for farmers to test new varieties for themselves; by allowing more farmers to test the varieties for themselves, then more neighbors will also potentially see and learn about the varieties as well. 2. SWET uses drama to encourage farmers to interact and learn from each other. 3. FRI, SWET, and MAP all interview farmers on their own farms, allowing them to explain the advantages of the improved varieties and other technologies in their own words so that other farmers can learn directly from them. 4. FIPS uses VBAs who are themselves farmers to influence other farmers; it is important to note that a FGD with VBAs revealed that it took time for them to gain the trust of the farmers in order to influence them. 4.4.3 Take-off Points Further analysis of take-off will be possible when the survey’s second wave of results is available. To determine the level of market penetration where spontaneous adoption of improved varieties by indirect beneficiaries might occur, it is important to note that different crops will have different take-off patterns and time-scales. At least three factors must be taken into consideration when looking at the take-off for different crops: 1) the presence or absence of output markets; 2) the history of a crop in a country or region; and 3) the potential for farmer-to-farmer adoption. Where output markets are present for particular crops and varieties, farmers will be more willing to adopt those crops and varieties to generate income from the sale of outputs. Because they can earn income by selling outputs, they will be more likely to invest in the regular purchase of inputs, particularly if there are product quality standards that must be maintained. Where output markets are present, the take-off point is achieved more quickly than for crops that are mainly grown for subsistence purposes. The history of a crop in a country or region determines the range of varieties available. Where a crop is relatively new and there are no local varieties, farmers must depend on the improved varieties. 43 Finally, self-pollinated crop varieties (e.g., beans, rice) for which quality seed can be effectively maintained by farmers, have the potential to go viral through informal markets and farmer-to-farmer adoption, rather by relying on agro-dealers and other distribution channels of the formal seed sector. For example, DNA analysis of rice varieties grown by sampled farmers in Tanzania shows that 100 percent of varieties analyzed were improved, yet only 6 percent of farmers reported growing improved rice varieties (Oruko et al, 2015). It seems likely that these varieties have taken off because, historically, there was not a wide range of local varieties available to begin with, and improved varieties have spread through farmer-to-farmer diffusion. In the case of beans, the data presented above clearly show that farmers regularly purchase bean seed from the informal market (market traders and other farmers), and it is known that there is a well￾developed regional bean market. For these reasons, one would expect improved bean varieties to take off relatively quickly, though perhaps not as quickly as rice, because beans have a longer history than rice in Tanzania and there is more choice in the range of varieties available. CONCLUSIONS The SSTP Program has been effective in increasing the use of improved varieties by both direct beneficiaries and indirect beneficiaries. Promotional methods that are able to draw on the influence of neighbors and relatives will have considerably greater impact than those that do not. Adoption of new varieties can best be encouraged by promoting varieties with preferred characteristics, for which output markets are present, and by making seed available at an affordable cost. Specific efforts are required to link farmers to output markets. Effective technology promotion uses an integrated set of methods and communication channels (e.g., radio, SMS, IVR, TV, drama, mobile cinema, group events, VBAs, farmer-to-farmer, etc.) that mutually reinforce each other. These approaches are most successful when implemented by specialized and experienced organizations. Seed companies would do well to partner with such organizations to promote their technologies among farmers. RECOMMENDATIONS • • • As above, it is recommended that SSTP supports efforts to identify and link farmers to output markets for the sale of products from the cultivation of improved varieties. Grantees should promote varieties in ways that encourage farmer-to-farmer learning and/or by working with experienced communication partners who have the capacity to encourage farmer￾to-farmer learning. Examples of farmer-to-farmer learning already being used by SSTP grantees are provided in the report. SSTP should expand support to technology promotion by experienced and specialized communication organizations, which use an integrated set of mutually reinforcing methods and communication channels. Partnerships between such organizations and private seed companies should be encouraged. 4.5 EVALUATION FINDINGS: ENABLING ENVIRONMENT Although SSTP was initially developed within the New Alliance for Food Security and Nutrition, established in 2012, the New Alliance itself is no longer on the policy agenda; many of the policy actors interviewed were simply not aware of the New Alliance Cooperation Frameworks, and Forum for Agricultural Research in Africa (FARA) has taken a backseat in its support to the New Alliance. In some 44 cases, this meant donors lacked the leverage to promote change. In Tanzania, however, the issues that formed part of the New Alliance Cooperation Framework have gone ahead under the National Agriculture Strategy. Although there may have been some early challenges in starting up SSTP without an effective New Alliance framework, this does not appear to have impeded progress made in improving the enabling environment for the seed sector. 4.5.1 Progress to Date The focal issues addressed by SSTP to support country-level policy and regulatory systems were all quite similar in countries visited, though each country is at a very different level of development and implementation of seed policy. In general, effective and well-articulated seed laws already existed in the SSTP countries, and SSTP’s role has been to help in “translating” existing laws into practice. The common focal issues being addressed at national levels are described below. • • • Recognizing that seed certification is a major bottleneck, and that existing seed laws allow for private sector involvement in seed certification, SSTP has enabled the private sector to play a role in the seed certification process. In general, this means that private individuals will be accredited to provide seed certification services, and private seed companies will be able to certify their own seed35 with accreditation in accordance to certification regulations and guidelines. The national certification agency will play a regulatory role instead of playing the certification role. Such guidelines were awaiting official approval by the Ministers of Agriculture in the countries visited. Once in practice, it is expected that this will speed up the certification process. Another policy aspect promoted by SSTP is the implementation of licensing and royalties for protected public sector varieties. This was most advanced in Tanzania, where public sector maize varieties had been licensed to three seed companies, though the licensing type and payment methods were still unclear. Overall, 10 grants were awarded to private sector companies for early generation seed production for maize and bean. Again, although this was possible by law, it is not something previously practiced in Ghana and Tanzania, where it had been the mandate of the public sector, i.e., ASA in Tanzania and the Grain Legume Development Board (GLDB) in Ghana. Though the seed policies themselves are very similar, the general level of development and the broader enabling environments of the seed sectors were different in Ghana and Tanzania. Ghana’s seed sector is much less developed, there are many fewer private seed companies, and the National Seed Traders Association (NSTA) has only recently been established and is not yet functional. Among policy-makers, there was a reluctance to allow the importation of certified seed into Ghana. This effectively limited the range of improved varieties available to farmers. In Tanzania, in contrast, there is a long history of international private seed sector investments (e.g., bean seed exports), prompting the government’s recognition of the role of the seed sector in generating revenue. This has provided an incentive for the government to create a favorable enabling environment for both international and Tanzanian private seed companies to develop. Imported certified seed is not regarded as a threat but, instead, is one of the channels to make quality seed available to farmers. More recently, linkages are developing between local and foreign companies (the supply of Dutch potato varieties and foundation planting material). Historical harmonization within the East African Community has supported the more recent regional harmonization processes. NSTA plays a key role in advocating for changes in policy and practice where 35 By agreeing to self-certification, private seed companies will be agreeing to abide with applicable guidelines and laws which carry penalties for non-compliance. They have their business and seed quality reputation to safeguard. Reputable companies with a long-term view on business success will not sell fake/counterfeit seed and would be motivated to find ways to prevent their seed from being counterfeited e.g., through tamper proof packaging, bar codes, etc. 45 necessary, and SAGCOT36 (an SSTP grantee) has also been influential in fostering a more enabling environment in the seed sector. The Tanzanian case illustrates the potential influence of an effective national seed traders’ association and an organization such as SAGCOT. In Mozambique, national platforms for crop-based value chains were seen to play a potentially important role in enabling environment improvement. At a regional level, SSTP has supported the development of the Common Market for Eastern and Southern Africa (COMESA) seed catalogue of released and registered varieties to enable private and public sector actors to know which varieties are approved (and where). This enables them to make business decisions based on data and information. For example, a company may want to introduce a tomato variety into the catalogue in Kenya where it is not available. However, since the variety is released in Ethiopia and Rwanda, COMESA countries, the company knows that it only has to conduct testing in Kenya for one season and then request supporting data from Ethiopia and Rwanda. The challenge is that some COMESA countries do not have developed mechanisms to allow the data transfer. The reason only multi-nationals are interested right now is that most are looking at vegetable seeds that are difficult and expensive for local companies to engage in. As it exists currently, it appears to benefit only big multinational seed companies. 4.5.2 Additional Policy Reforms Needed Formal seed sector laws are in place in the countries visited and, in general, do not need to be reformed. What is needed is continued support to put existing laws into practice through guidelines and protocol development for implementation. In some cases, there is the need for specific protocols to be developed where they do not already exist, e.g., the need for protocols specific to potato and cassava in Tanzania. There is continued need for awareness creation and capacity development to make seed sector actors, officials, and stakeholders aware of the existing regulations and any newly developed guidelines or protocols. This is necessary so that they can support effective implementation. One possible reform that might be considered is one that allows for Quality Declared Seed (QDS) among the SSTP countries. Currently, this seed category only exists in the policies of Tanzania and Ethiopia, and is not allowed under Economic Community of West African States (ECOWAS) regulations. In Tanzania, one of the SSTP grants (for SAGCOT/New Markets Lab) supported the development of a “Legal Guide to Strengthen Tanzania’s Seed and Input Markets,” described as a practical tool to provide information regarding regulation of different aspects of the seed value chain, as well as issues affecting fertilizer and agrochemicals. The Legal Guide identifies key implementation points to enhance the enabling environment for seed and other agro-inputs in Tanzania, and is accompanied by a “Recommendations and Roadmap.” This document details practical steps to implement recommendations outlined in the legal guide. Both the Guide and the Roadmap were developed through processes of widespread consultation and discussion, an important first step in any policy development process. Work currently being undertaken by The African Seed Access Index (TASAI, one of SSTP’s regional grantees) will also help identify the policy issues needing to be addressed in each country. This is a tool for benchmarking the seed system in a given country vs. other countries on the key performance indicators. It enables policy makers and other stakeholders to prioritize work on weak systems. For example, if the indicator “the number of plant breeders working on maize” in a country is too low 36 SAGCOT is the Southern Agricultural Growth Corridor of Tanzania, a public-private partnership that seeks to develop the agricultural sector of Tanzania by fostering responsible agribusiness investments in the country’s southern corridor. The SAGCOT Centre Ltd. works as a broker and catalyst of partnerships among registered partner organizations around inclusive, sustainable, and viable agricultural value chains. 46 compared to others, it is worth understanding why, especially if maize is an important crop for that country, then take corrective measures. Participation in TASAI is voluntary. The goal is to make seeds of improved varieties available to smallholder farmers by addressing the constraints identified by the TASAI tool. It is important to note that some laws and policies that affect seed sector development might lay outside the seed sector itself, e.g., access to finance or subsidies for particular products. In the case of Mozambique, subsidies on wheat flour make it impossible for cassava flour to compete in relation to price, thus limiting the potential markets for cassava products. CONCLUSIONS The issues being addressed by SSTP to support country-level policy and regulatory systems are similar in each country visited, though each country is at a different level of development and implementation of seed policies. Policy and regulatory development is a slow and difficult process. By working through national seed traders’ associations and other lobbying platforms, SSTP has had positive impacts. Continued support is needed to put existing laws into practice through the development of guidelines and protocols for implementation. There is also a continued need for awareness creation and capacity development to make seed sector actors aware of existing regulations and newly developed protocols. One possible reform to consider is one that allows for Quality Declared Seed/Planting Material (QDS/QDPM); this seed category currently only exists in Tanzania and Ethiopia RECOMMENDATIONS • • SSTP should continue to strengthen and work through national seed traders’ associations and other lobbying platforms. It is recommended that SSTP commissions a review of the practical application of QDS and QDPM laws and regulations in Tanzania and Ethiopia. This would help determine whether QDS and QDPM might be applied in other countries. Such a review should be undertaken in a participatory manner, involving policy-makers from all SSTP countries (similar to the EGS study) so that it stimulates in-country dialogues on the advantages and disadvantages of QDS and QDPM for different crops (notably cassava, potato, beans, and rice), opening up the possibility of developing QDS and QDPM laws and regulations in countries where they do not currently exist. 47 ANNEXES 48 ANNEX A. EXPRESSION OF INTEREST 49 PEEL TASK ORDER EXPRESSION OF INTEREST I. BACKGROUND INFORMATION A) Identifying Information 1. Project/Activity Title: Scaling Seeds & Technologies Partnership 2. Award Number: AID-OAA-A-13-00040 3. Award Dates: July 2, 2013 – July 1, 2016 No-Cost Extension Modification to July 1, 2017 Planned No-Cost Extension Modification to July 1, 2018 4. Project/Activity Funding: $46,769,499.00 5. Implementing Organization: Alliance for a Green Revolution in Africa (AGRA) 6. Project/Activity COR/AOR: Mark Huisenga B) Development Context 1. Problem or Opportunity Addressed by the Project/Activity Being Evaluated The G8 Fact Sheet on Food Security and Nutrition, issued by the White House on May 18, 2012, outlined the commitments made at Camp David to launch the New Alliance for Food Security and Nutrition (New Alliance) an initiative by African governments, the private sector, and G8 members to focus, accelerate, and coordinate their joint efforts aimed at reducing poverty and hunger in Africa over the next 10 years.37 The New Alliance builds on the Comprehensive African Agricultural Development Program (CAADP) Country Investment Plans (CIP) and will help accelerate the implementation of key CIP priorities. One of the New Alliance commitments is “Taking innovations to scale” and involves several enabling actions including establishing 10-year targets for sustainable yield improvements in national priority value chains, identifying core sets of technologies that would contribute to achieving those targets, and ensuring access to those technologies at sufficient scale. To meet the commitment to significantly scale access to new technologies, the G8 has called on the Alliance for a Green Revolution in Africa (AGRA) to create a Scaling Seeds and Technologies Partnership (the Partnership). The Partnership will help New Alliance countries coordinate related but separate country, donor-financed and private sector programs to strengthen the inputs sector and to promote the commercialization, availability and adoption of technologies at scale. In particular, the Partnership will help countries transition from state-dominated seed systems to systems that allow the private sector, including local businesses and non-profit organizations to provide key services, but also strengthen the capacity of the state to carry out critical regulatory functions. USAID/Washington has, on behalf of the G8, signed a Cooperative Agreement with the Partnership team to implement the Partnership in Ghana, Ethiopia, Senegal, Malawi, Mozambique and Tanzania. The Partnership activities are targeted at scaling up the supply of improved seed and other technologies to achieve the following objectives: ● To improve capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers; 37 See http://www.whitehouse.gov/the-press-office/2012/05/18/fact-sheet-g-8-action-food-security-and-nutrition 50 ● ● To increase the use of quality seeds and other technologies by smallholder farmers; and To improve regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers. C) Intended Results of the Project/Activity Being Evaluated The objective of the Scaling Seeds and Technologies Partnership activity is to work with New Alliance countries to meet 10-year agricultural production targets set by countries themselves. To meet these targets, AGRA will assist countries to coordinate sector strategies and programs to strengthen the seed sector and promote the commercialization, distribution and adoption of seeds and related key technologies to increase smallholder yields and resilience to climate change. While agricultural productivity has increased in much of the developing world, average grain yields in Sub-Saharan Africa are about one-third of the average yield in developing countries, and one-fifth of the yield of the developed countries. Since 1970, per capita food production in Africa has declined more than 10 percent, while productivity elsewhere in the world has risen significantly. In particular, poor access to and adoption of improved seed varieties and complementary technologies are significant contributors to the region’s relatively lower agricultural yields. Successful adoption at scale in SSA requires addressing regulatory and market barriers and constraints, increasing incentive mechanisms, and strengthening local capacity, with a particular focus on inclusive participation of women and smallholders. Many important seed and other complementary technologies, which are critical to improving agricultural productivity, increasing climate resilience, and enhancing nutrition, already exist in international and national public research institutions and the private sector. Seed companies and farmers are often unaware of, or lack access to, improved seed varieties and other promising technologies. Mechanisms to facilitate the rapid dissemination and commercialization of seeds and other locally tested technologies, focusing on strengthening the capacity of local public and private actors, are needed to improve adoption rates. In countries that are reforming their national seed systems, policies and regulations need to be improved and implemented to respond to local and regional demand, and in a way that allows the private sector to take on increasing functions, while strengthening state agencies so they can provide effective oversight and support. Male and female smallholder farmers, producer organizations, small entrepreneurs and agribusinesses often lack the information, advisory and extension services necessary to guide innovation with improvement in management, marketing production, and adoption of new technologies (management practices, seed varieties, equipment, infrastructure, inputs). Development Hypothesis: The development hypothesis behind this Partnership is that more purposeful, effective coordination of technology-scaling opportunities for seeds and other technologies by a willing private sector within an enabling commercial framework will lead to rapid agricultural productivity increases. For important technologies where commercial potential is currently untested or weak, more effective coordination and innovative pilots may stimulate the development of new markets or suggest new ways to address these scaling challenges. Better monitoring of commercialization and adoption experiences will provide information on which to base future plans and activities. The Partnership also intends to support countries to transition from state-dominated seed systems to systems that allow private sector provision of key services (e.g., multiplication of foundation seeds), and strengthen the capacity of the state to carry out critical regulatory functions (e.g., varietal release and seed certification). The Partnership aims to support national and regional efforts to revise and implement seed laws, focusing especially on regional harmonization of variety release and seed trade, the establishment of foundation seed production in core countries, creating or expanding seed company capacities and seed trade associations, and work with agro-dealers and providers of credit to ensure farmers have awareness of and access to improved seeds. As part of their policy commitments under the New Alliance, the target countries committed to making adoption of the best available technologies a top priority, implementing the necessary regulatory reforms and assessing progress against adoption goals on an annual basis. 51 Key Assumptions: Through their agreement on the New Alliance, the focus countries and G8 donors implicitly recognized that the “old” approach to agricultural development was not fully achieving intended results. Key assumptions of this new approach to bringing seeds and other technologies to scale are: 1) There are technologies (seeds, fertilizer, agricultural chemicals, etc.) that are ready to be taken to scale through commercialization or other means to increase their availability that will lead to sustained increases in productivity, income and food security for smallholders. 2) More effective coordination among focus country governments, G8 donors, and the private sector, particularly related to scaling access to agricultural inputs, will lead to more positive outcomes for smallholder farmers. Better coordination will facilitate follow-through on implementation of key pieces of the reform agenda related to technology scaling and thus to improving the enabling environment for sustained private sector investment, growth of local companies and/or facilitating access by nonprofit organizations. 3) Focus country governments and the private sector will need support to meet market technical requirements (certification, registration, etc.) and adhere to international agreements to which governments have acceded but are not yet in compliance. 4) Efficiency of local and regional markets – particularly trade in agricultural inputs – can be substantially improved through technical support to governments and regional bodies, rationalization and harmonization of trade policies, and private industry monitoring and benchmarking of performance. D) Approach and Implementation The Partnership has three objectives: 1. Improving the capacity of public and private sector groups to produce and deliver quality seeds and other technologies to smallholder farmers; 2. Improving the capacity of smallholder farmers to adopt quality seeds and technologies; and, 3. Improving the policy and regulatory mechanisms for the delivery of quality seeds and technologies to smallholder farmers These objectives will be met by carrying out the following seven activities: 1. Develop “Road Maps” of specific public and private sector actions needed; 2. Coordinate and align efforts among public and private sector actors and donors; 3. Provide technical support locally for building capacity of public and private actors; 4. Model and pilot priority activities through grant-making; 5. Provide regional technical support for finding solutions to cross-cutting issues; 6. Monitor and benchmark progress toward goals; and 7. Address constraints to regional harmonization The overall benefit from the Partnership will be a better-aligned and coordinated set of donor and government actions, which leverage greater private sector investment for the sustainable supply of seed, fertilizer, and other technologies at farmer level. AGRA will also convene international and regional groups active in the development of seed systems to discuss specific challenges and solutions and gather momentum at a regional level. Together with direct investments made by the initiative to alleviate specific bottlenecks in seed and other input supply chains, the Partnership is envisaged as a definitive intervention in the resolution of barriers to food security which have caused human suffering and limited Africa’s chances for economic development for decades. AGRA estimates that these interventions will ultimately contribute to at least an additional $40 million in investment in private sector seed supply in the target countries, including the establishment of 12 seed-or-other technology supply enterprises led by women. At least 50 improved production technologies will be commercialized, and investment in local supply chains will reduce the average distance that farmers need to travel to access agricultural inputs from 20 to six kilometers. These changes in seed and technology markets will contribute to a 45 percent increase in the adoption of improved seed, fertilizers and other production 52 technologies, resulting in an additional 4.5 million metric tons of grain production and improved food security for 7.6 million individuals in four years. Target Areas, Target Crops and Related Technologies Prioritized for Scaling by Country Country Target ecological areas Target crops Related target technologies/interventions Ethiopia North Region, South West region, Southern region, Middle rift valley areas, Central highlands, East Central highlands, North West area, Central west area, North East area Maize, Wheat, Barley, Teff, Sesame, Chickpeas ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Rhizobium inoculants Fertilizer blends and associated technologies Quality seed, including vegetative planting materials, of superior varieties of identified target crops Post-harvest storage Seed processing and testing facilities Vertisol drainage solutions Seed business incubation Capacity building/training Ghana Brong Ahafo, Ashanti, Eastern, Volta, Central Maize, Rice, Cassava, Soybeans, Yam, Cowpeas Fertilizer deep placement Rhizobium inoculants Appropriate mechanization for planting, harvesting, and processing etc. Quality seed, including vegetative planting materials, of superior varieties of identified target crops Post-harvest storage Malawi Machinga, Mzuzu, Karonga, Kasungu, Lilongwe, Blantyre, Shire Valley and Salima Agricultural Development Divisions Cassava, Orange￾fleshed Sweet Potato, Rice, Maize, Pigeonpeas, Beans, and Cowpeas Fertilizer blends SRI technology, quality seed, including vegetative planting materials, of superior varieties of identified target crops Mozambi que Beira, Nacala, Zambezi, Limpopo Maize, Cassava, Soybean Fertilizer blends Rhizobium inoculants Quality seed, including vegetative planting materials, of superior varieties of identified target crops Senegal Groundnut Basin, Casamance, Senegal River Valley, Southern Groundnut Basin, Eastern Senegal Groundnut Basin, Sylvo pastoral zone, Eastern Senegal, Northern Groundnut Basin, Pastoral zone Groundnuts, Rice, Pearl Millet, Maize, and Cowpeas Appropriate mechanization - rice planters/direct paddy seeder and weeder, fertilizer deep placement applicator, and conservation agriculture (ripper) Two-wheel tractors Rice harvesting and threshing machines Hand operated millet threshing machine Fertilizer deep placement Fertilizer blends Quality seed, including vegetative planting materials, of superior varieties of identified target crops Post-harvest storage Tanzania Southern Highlands, Eastern Zone, Northern Zone Maize, Sorghum, Cassava, Round Potatoes, Beans, Fertilizer blends Rhizobium inoculants 53 Country Target ecological areas Target crops Related target technologies/interventions Soybeans and Pigeonpeas ● Quality seed, including vegetative planting materials, of superior varieties of identified target crops Modifications since inception: Mod 1) Increase incremental funding from $1.0 million to $17.0 million Mod 2) Change DUNS number, update standard provisions, change Agreement Officer, revise Program Description, update Key Personnel, revise “substantial involvement,” revise budget, insert geographic code, insert capital equipment summary Mod 3) Extend award date by one year to July 1, 2017 Mod 4) Change payment method, amend limitation on light construction activities Mod 5) Provide incremental funding of $8.0 million to $25.0 million, revise budget, update standard provisions Mod 6) Planned: to extend Cooperative Agreement at no cost by one year to July 1, 2018 E) Documents Key documents will be provided by BFS including: Cooperative Agreement, project work plan, PMP, M&E plan including baseline data, quarterly and annual reports, annual implementation plans, and other important documents. II. EVALUATION RATIONALE A) Evaluation Purpose This Mid-Term Performance Evaluation is intended to provide robust empirical evidence responding to evaluation questions designed to support learning and continuous improvement for the Activity and BFS. The evaluation will assess what is working well and what is not working well in implementation, assess progress toward activities objectives and outcomes, as well as unintended outcomes, and provide evidence based findings and recommendations that AGRA and BFS can use to improve activity effectiveness and better achieve objectives. B) Audience and Intended Uses The results of this evaluation will be used by the Agreement Officer, the Agreement Officer’s Representative, and AGRA to provide guidance on any mid-course corrections and direction during the final years of implementation. Results will be shared with USAID, both in the Bureau for Food Security to inform other scaling activities and Mission Activity Coordinators which are expected to help them coordinate in-country activities more effectively with the Partnership. Evaluation findings will support AGRA’s decision making with regards to its final funding years, and may be shared with other G7 donors and the Bill & Melinda Gates Foundation. C) Evaluation Questions Evaluation questions are grouped according to the specific interests of this Evaluation. Management/Administration 1. What are the strengths and weaknesses of AGRA’s grants selection process as a way to select the sub￾grantees with the highest potential? In what ways could it be improved? How effective has the Partnership been in supporting the role of the private sector and other key stakeholders as a way of meeting the objectives of USAID’s FORWARD strategy – notably as the engines for sustainable development in their countries? 2. How effectively has the project management facilitated communication and coordination amongst the different stakeholders involved in the partnership, including AGRA and its country offices, USAID, grantees, and beneficiaries? Are information transfers amongst sub-grantees, sharing of lessons learned and collaborations evident? In what ways could communications and coordination be improved as a way 54 to advance project goals and objectives, meet reporting requirements and encourage additional relevant partners? 3. Are data management systems in place to collect, track and report on AGRA and Feed the Future indicators, outputs, outcomes and impacts from the beneficiary level and are they comparable across grantees? Does the collection and reporting conform to USAID requirements? Engagement of the Private Sector 1. In what ways has the Partnership increased the capacity of public and private sector groups to produce and make quality seeds of improved varieties and other technologies available to small holder farmers? To what extent has the Partnership increased the availability of promoted technologies in markets in target areas? What are the most important barriers that AGRA and the sub-grantees must yet address while trying to increase availability of improved seeds and technologies in local markets? 2. What technologies/crops are more likely to be produced and made available on a wide scale by the private sector and why? Which segments of the seed or technology value chain is the private sector likely to take on? Which segments are likely to be more difficult for the private sector and why? How can the Partnership improve its approach to scaling through the private sector in light of these differences? 3. What has been the private sector’s experience creating demand under the Partnership? How could the private sector’s effectiveness at creating demand be improved through the Partnership? Scaling and Adoption of Technologies 1. To what extent have AGRA and its sub-grantees been able to increase the use of quality seeds and improved technologies by small holder beneficiary farmers? 38 By indirect beneficiary farmers? 39 2. What types of strategies, innovations or support have been the most effective in encouraging use of improved seeds and technologies for just one season? Over multiple seasons (adoption vs application)? What important barriers remain? by: a. Direct project beneficiaries? b. Indirect beneficiaries? (those in “catchment areas” of activities but aren’t directly targeted by the activity) c. Early adopters? Majority adopters? Late adopters? d. The poor, women, and socially marginalized groups? 3. What are other types of approaches and methodologies that USAID could incorporate in its projects to encourage indirect beneficiaries to both try out and continue to use new agricultural technologies and practices? At what level of market penetration40 of potential market demand41 for targeted technologies is there likely to be spontaneous adoption (over 2+ growing seasons) by indirect beneficiaries? (e.g., what is the take-off point at which technologies could “go viral”?) Enabling Environment 1. What progress has the Partnership made in supporting country-level and regional policy and regulatory mechanisms with the aim of increasing the production and delivery of improved seeds and technologies? What progress has there been in contributing to New Alliance objectives of improving the enabling environment within countries and across countries? What other opportunities are there for the Partnership to support attainment of New Alliance objectives of enabling environment improvement? 2. What additional policy reforms, in order of priority, should the Partnership support (including AGRA, the Steering committee, sub-grantees, donors, other stakeholders, etc.) to: 38 An individual is a direct beneficiary if s/he comes into direct contact with the set of interventions (goods or services) provided by the activity. Individuals who receive training or benefit from activity- supported technical assistance or service provision are considered direct beneficiaries, as are those who receive a ration or another type of good. The intervention needs to be significant, meaning that if the individual is merely contacted or touched by an activity through brief attendance at a meeting or gathering, s/he should not be counted as beneficiary. 39 An indirect beneficiary does not necessarily have direct contact with the activity but still benefits, such as the population that uses a new road constructed by the activity, neighbors who see the results of the improved technologies applied by direct beneficiaries and decide to apply the technology themselves (spill-over), or the individuals who hear a radio message but don’t receive any other training or counseling from the activity. 40 Market Penetration is defined as the number of people who buy a specific technology at least once in a given period, divided by the size of the relevant market population. 41 Market potential is the entire size of the market for a technology at a specific time and represents the upper limits of the market, measured by either the value or volume of sales. 55 a. Raise the supply of improved seeds and technologies in markets in target areas? b. Increase the production and delivery of improved seeds and technologies on a national and regional level? III. EVALUATION DESIGN AND METHODOLOGY A) Evaluation Design Scale and sustainability means achieving a critical mass of adopters of new technologies. Once this critical mass of adopters is reached, further adoption will continue to grow organically (“go viral”). The Partnership emphasizes the role of the private sector for achieving its goals; and the private sector is the preferred pathway for scaling as well. At the same time, the Partnership recognizes that partnerships with the public sector or civil society (e.g., NGOs, universities) may play an important role. This performance evaluation should use a mixed methods approach employing a time series data collection design to allow for robust quantitative and qualitative data collection to assess performance of the Partnership intended to provide insight into what the “take-off point” is for achieving scale of selected technologies and may shed light on innovative growth models for bringing new technologies to scale. Optimally, through this methodology we might better understand what it take to achieve this take-off point for USAID’s planning, budgeting, cost/benefit analysis, and future monitoring and evaluation of Feed the Future programs. The challenge for this Evaluation is to estimate the potential market demand of an innovation and determine the number in a population that represents the early and majority adopters who are and aren’t the direct beneficiaries of the activity. Sampling Parameters: The performance evaluation will cover the entire activity, however, due to budget constraints and the geographic spread of the Partnership, site visits will be carried out in three (3) countries: Ghana, Mozambique, and Tanzania. Countries: The Evaluation Team is expected to conduct site visits with between a quarter and a third of sub-grants in Ghana, Mozambique and Tanzania. Specific commodities that should be examined include maize in Tanzania and Ghana; cassava in Mozambique; Irish potato and beans in Tanzania. The selection criteria will be included as part of the evaluation design. Key informant interviews and/or surveys will be carried out with a majority of grantees (via phone, Skype, Internet, etc.). Contracts: The Partnership has made some additional sub-contracts, and these should be examined for their contribution to the evaluation questions, such as the COMESA Seed Catalog and upcoming PERUSAP training (in 2017). Timing: The first data collection phase should occur in the Summer of 2016 and the second data collection phase in Summer of 2017. B) Data Collection Methods This evaluation is expected to collect data through both qualitative and quantitative methods from a variety of stakeholders. The evaluation should assess impacts on four levels: Enabling Environment: The evaluation is expected to look at the Partnership’s support to policies and regulations in the seed sector at the national and regional level, on the way to fulfilling the commitment to the New Alliance Primary direct beneficiaries: Enterprise-level data will focus on the seed companies with which the Partnership works directly. Data on the numbers of employees (or members), the gender of the employees and 56 leadership/management, and information on the type of organization is being collected by the Partnership for each performance indicator, including the production of seeds, and the value/volume of seeds sold. Secondary direct beneficiaries: Smallholder farmer data is being collected by the sub-grantees, in order to assess impact on applying new technologies and varieties. These farmers are expected to participate in grantee activities such as extension services (e.g., demonstration plots), capacity training (e.g., short-term trainings), and other services and facilitation activities. The assumption is that these farmers would have adopted the improved seeds and technologies and demonstrated impact. Indirect beneficiaries: Effects on smallholder farmers who have bought the Partnership’s supported, improved seed varieties and/or associated technologies by non-grantee agro input dealers (where AGRA has pre￾existing relationships built through other programs or who purchase/receive Partnership-supported improved seed varieties and/or improved technologies and/or improved management practices) will also be examined to look at what elements affect farmer demand, bring about uptake and have an impact on productivity and income. A selection of potential data collection methods is provided for consideration below. The Concept Note, and subsequent Performance Evaluation Protocol (which will include a data collection plan) will be finalized by the Evaluation Team, with approval from the BFS AM and COR, in the Evaluation Plan at the beginning of the assignment. The Evaluation Team should collect data disaggregated by gender and will ensure that women are included and able to actively voice their perspectives throughout data collection. Additionally, for all data collection methods the team will aim to include farmers from different socio-economic backgrounds as determined by proxies such as land size and income. Together this will make sure the evaluation accounts for potential disparate effects related to these respondents’ different characteristics. Sample-based farmer survey. A multi-stage sampling design could be used to determine impact on direct and indirect beneficiaries, perhaps utilizing adaptive sampling (a technique used to amplify responses from difficult to find groups while retaining the ability to generalize). A large initial random sample conducted in target areas, intended to cover both direct and indirect farmer beneficiaries, will generate information on proportions of direct versus indirect beneficiaries, critical demographic information and farmer behavior and use of technologies. The second data collection phase would focus the sampling more on direct beneficiaries with a smaller proportion of indirect beneficiaries interviewed. Key informant interviews: Key informant interviews (KIIs) could be used to collect information from stakeholders. KIIs will be employed in answering all evaluation questions. The Evaluation Teams will use structured or semi￾structured interview guides for KIIs to ensure key questions are systematically answered. These guides will also allow interviewers the freedom to ask spontaneous questions to uncover important and unanticipated information. The Evaluation Team will design a separate key informant interview guide for each stakeholder group. Focus group discussions: FGDs could be carried out with key stakeholders, and direct and indirect beneficiaries. FGDs should allow researchers to collect information about more people in a shorter amount of time and provide valid ways of identifying trends or conclusions with respect to counterfactuals or external factors. Focus group discussion sampling plan: Stakeholders should be identified in consultation with the Partnership’s COP and staff, taking into consideration the limitations on LOE and travel. As mentioned above, the sampling plan for selection of focus group participants will be included in the evaluation plan and in consultation with USAID/AGRA to minimize potential bias. Secondary data collection: Secondary data collection could be from three sources: project performance reporting and data, other project related research including baseline studies, and a literature review of approaches to measuring technology scaling (some provided by USAID). C) Data Analysis Methods 57 Analysis of data on adoption/scaling and data on other dimensions of the Partnership’s performance will require different methodologies. Analysis will entail triangulating data from different sources to reduce bias and provide robust results. The methods for analyzing both quantitative and qualitative data will be elaborated more fully in the Evaluation plan. IV. EVALUATION PRODUCTS A) Deliverables 1. Evaluation Plan (at least 2 revisions, pending USAID approval) 2. Mission Outbriefs – Short presentation of country-wide findings for USAID Mission for each country where site visits were conducted following both phases of data collection 3. Presentation of findings (2): following each data collection phase, short presentation of initial findings/results for USAID and AGRA (utilizing web based interface) 4. Interim evaluation report (2 revision, pending USAID approval) following initial data collection phase, including evidence based findings and short-term actionable recommendations, relating to the evaluation questions and activity implementation, with the intention of information AGRA’s work plan. 5. Evaluation report (at least 2 revisions, pending USAID approval) following second data collection phase, including evidence based findings and short-term and longer term actionable recommendations, relating to the evaluation questions and activity implementation. on the Partnership’s management and administration, private sector engagement, technology scaling and adoption progress, and enabling environment influence 6. Data sets, cleaned and uploaded according to USAID’s open data policy. 7. S-curve models, plotting the Partnership’s progress on beneficiaries’ adoption of selected technologies. 8. All deliverables as specified in the PEEL-TO contract V. TEAM COMPOSITION Describe the intended size of the Evaluation Team, the roles and responsibilities of team members, specific qualifications that team members are expected to possess, and the manner that their qualifications will contribute towards the evaluation effort. The Evaluation envisions a team of three Key Persons - an Evaluation Specialist, an Agriculture Economist, and an Agronomist, but will need to be supplemented by additional specialists possibly including a statistician, a geographer/GIS expert and a marketing/communications expert. Evaluation Team Lead: A senior-level evaluator with a minimum of 10 years of experience managing and/or evaluating multifaceted international development teams, involving farmers, private sector, and public sector involvement. The candidate will also have: a) a demonstrated capacity to conduct independent program evaluation; b) an understanding of USAID’s foreign assistance goals, and its particular objectives related to agricultural development and food security; and c) the ability to analyze issues and formulate concrete recommendations orally and in writing. Experience in Sub-Saharan Africa is necessary. Technical team members (2): Must be experienced experts (5-10 years or more) in international development related to agriculture adoption and scaling of agricultural technologies; in particular, team members must have experience working with African seed systems. Technical team members will also have demonstrated the following: a) the capacity to conduct independent program evaluation; b) a thorough understanding of research methodology; c) experience in effectively conducting outreach and dissemination to policy makers, development practitioners and/or the private sector; and d) the ability to analyze issues and formulate concrete recommendations orally and in writing. Disciplines of all members (4): The team members need familiarity with Sub-Saharan Africa’s agricultural systems with the following required composition of skill sets among them: quantitative and qualitative 58 evaluation, organizational and capacity development, gender analysis, demand and supply economics, agronomist/agricultural systems background of staple crops (i.e., maize, rice, tubers, grain legumes), agriculture economics background, private sector/seed distribution and marketing systems, value-chain orientation, seed regulatory systems. VI. TIMELINE • Timeframe for Evaluation: Approx. July 2016–July 2017 • Final draft: July 2017 59 ANNEX B. TRAVEL ITINERARY 60 First trip: Ghana (January 21–February 1, 2017) Date Country Location Jan. 21 Travel to Accra Jan. 22-23 Ghana Accra Jan. 24 Ghana Ejura Jan. 25 Ghana Kumasi Jan. 26 Ghana Accra Jan. 27-28 Ghana Ho Jan. 29-30 Ghana Accra Jan. 30/Feb. 1 Travel from Accra Second trip: East Africa (February 10–28, 2017) Date Country Location Feb. 10/11 Travel to Nairobi Feb. 12-13 Kenya Nairobi Feb. 14-16 Mozambique Nampula Feb. 16-18 Mozambique Maputo Feb. 19-20 Tanzania Dar es Salaam Feb. 21-22 Tanzania Arusha Feb. 23 Tanzania Travel from Arusha to Mbeya Feb. 24 Tanzania Mbeya Feb. 25 Tanzania Njombe Feb. 26-27 Tanzania Dar es Salaam Feb. 27/28 Travel from Dar es Salaam 61 ANNEX C. REFERENCES CITED AND BIBLIOGRAPHY OF DOCUMENTS REVIEWED 62 REFERENCES CITED Bandiera, O. and Rasul, I., (2002). Social networks and technology adoption in Northern Mozambique. CEPR Discussion Paper no. 3341. London, Centre for Economic Policy Research. Blackie, M. and Albright, K. 2005. Lesson learning study of the Farm Inputs Promotions (FIPS) Project in Kenya. DFID Crop Protection Programme http://www.fao.org/docs/eims/upload/agrotech/1988/Lessonlearning2FIPS.pdf (Accessed March 18, 2017) Bruce, Abel, Kwaku, K., Donkoh, S.A., and Ayamga, M. (2014). Improved rice variety adoption and its effects on farmers’ output in Ghana. Journal of Development and Agricultural Economics, Vol. 6 (6), pp. 242- 248. Conley, T.G. and Udry, C.R. (2000). Learning About aNewTechnology: Pineapple in Ghana. Unpublished Paper. Yale University, Economic Growth Center, Center Discussion Papers. Doss, C.R. (2006). Analyzing technology adoption using microstudies: limitations, challenges, and opportunities for improvement. Agricultural Economics, 34: 207-219. Farm Radio International, 2011. The new age of radio How ICTs are changing rural radio in Africa. http://farmradio.wpengine.netdna-cdn.com/wp-content/uploads/farmradioictreport20111.pdf (Accessed March 18, 2017) Farm Radio International and CABI, 2016. New Alliance ICT Extension Challenge Fund: Up-scaling of interactive information and communication technologies to increase uptake of agricultural innovations in Tanzania: Annual Report submitted to USAID and IFAD (December 2016). Fisher, M., Abate, T., Lunduka, R.W., Asnake, W., Alemayehu, Y., and Madulu, R.B. (2015). Drought tolerant maize for farmer adaptation to drought in sub-Saharan Africa: Determinants of adoption in eastern and southern Africa. Climatic Change, 133: 283–299. Foster, A.D. and Rosenzweig, M.R. (1995). Learning by doing and learning from others: human capital and technical change in agriculture. J. Political Econ. 103(6), 1176–1209. ICRISAT. 2014. Inclusive Market-Oriented Development: Action towards benefiting the poor. ICRISAT IMOD Exemplars - Volume I. Patancheru 502 324, Telangana, India: International Crops Research Institute for the Semi-Arid Tropics. Kabuli, Amon (2017). Final Evaluation Report: Master Farmer (Grant No. 2014 SSTP 013). Kirk Development Research and Training Consultants. Lion, K.D., de Boef, W.S., Huisenga, M., and Atwood, D. 2015. Convening report: Multiple Pathways for Promoting the Commercial and Sustainable Production and Delivery of Early Generation Seed for Food Crops in Sub-Saharan Africa, 23 March 2015, London. Bill & Melinda Gates Foundation, Seattle, WA, and United States Agency for International Development, Washington, DC. Longley, C.A. (2000) A Social Life of Seeds: Local management of crop variability in north-western Sierra Leone. Unpublished PhD Thesis. University of London. 63 The Montpellier Panel, 2013. Sustainable Intensification: A New Paradigm for African Agriculture, London. Morris, M.L., Tripp, R., and Dankyi, A.A. (1999). Adoption and Impacts of Improved Maize Production Technology: A Case Study of the Ghana Grains Development Project. Economics Program Paper 99-01. Mexico, D.F.: CIMMYT Okali, C., Sumberg, J., and Farrington, J. (1994). Farmer Participatory Research: Rhetoric and reality. London: Intermediate Publications on behalf of the Overseas Development Institute. Oruko, L., Diiro, G., Tairo, F., and Amanquah, S. (2015). Diffusion and adoption of improved rice and maize varieties in Tanzania: application of genetic fingerprinting technique. Final Technical Report. Zaal, F., van der Lee, F., and Mwongela, B. (2012) “Best-bet Farm Input Promotions – Africa” (FIPS￾Africa) in P. Gildemacher and R. Mur (2012) Bringing New Ideas into Practice: Experiments with Agricultural Innovation. Amsterdam: Royal Tropical Institute (KIT). http://r4d.dfid.gov.uk/pdf/outputs/ResearchIntoUse/Learning_from_RIU_in_Africa_book2.pdf (Accessed March 18, 2017). AGRA/SSTP DOCUMENTS REVIEWED AGRA Charter of the Grants Review Sub Committee SSTP Program Description SSTP Advisory Committee Meeting Minutes SSTP Quarterly Reports SSTP Indicators at a Glance SSTP PIRS, Final SSTP Implementation Plans (Years 1 to 4) SSTP Data Quality Assessment Reports AGRA SSTP M&E Plan and Annexes SSTP Outcome Indicator Assessment Annual Reports (2015 and 2016) Ghana Anonymous, May 29, 2014. Scaling Seed and Technology Partnership. Ghana Roadmap. 2014 SSTP 007 – IVSC Final Application. May 2014. Enhancing Food Security Through Sustainable Production and Distribution of Certified Seeds in Central Savannah Zone of the Central Region of Ghana. IVSC Q9IPTT Aug 2016-Oct 2016 64 2015 SSTP 029 – LCIC Application. Enhancing Seed Commercialization Through Supply of Foundation Seed. 2014 SSTP 014 – M&B Seed Company Application – Version 2. July 2014. Promotion and Distribution of Certified and Foundation Seeds. 2015 SSTP 019 – FARA Application. Entrepreneurship for Commercial Seed Incubation Business (ECoSIB) in Ghana. 2014 SSTP 019 – Pee Farms Application. 20 August 2014. Increase Access and Use of Quality Seed by Smallholder Farmers. Intermediate IPTT_Dec-Jan_Qtr 10. 2015 SSTP 003 – ASML Application. November 2014. Planting Materials and Associated Technologies Promotion Project (PMATPP). 2014 SSTP 033 – MAP Application. August 2014. Increasing Smallholder Farmers’ Awareness of Improved Seed and New Agricultural Technologies Through a Reality Television Make-Over Series. Juliana Asante-Dartey, Larry Amekuse and Amos Rutherford. July 2016. Early Generation Seed Systems Study, AGRA-SSTP. Final Draft. Kenya 2016 SSTP 002 TASAI Proposal, December 2015. The African Seed Access Index. Mozambique Mozambique Roadmap, October 2013. 2014 SSTP 001 – IIAM Full Application. January 2014. Scaling Production of High Quality Foundation Seed of Improved Cassava Varieties to Small-scale farmers to Meet Market Demand in the Corridors of Nacala and Limpopo in Mozambique. IIAM Interim Narrative Report. July 1, 2016 to September 31, 2016. 2014 SSTP 002 – Oruwera Full Proposal. March 2014. Technical and Financial Application for Scaling Seeds and Technologies Partnership Program (SSTP). 2014_SSTP_IPTT_Oct-Dec 2016_Final. Anonymous. July 2016. Mozambique Early Generation Seed Study. Country Report. Tanzania Tanzania Roadmap. March 2014. 65 Anonymous. July 2016. Tanzania Early Generation Seed Study. Country Report. 2014 SSTP 015 – ARI Application. August 2014. A Sustainable Breeder Seed Production System for Maize and Beans at ARI-Uyole. 2014 SSTP 015 ARI Uyole-IPTT Q9 Final. 2014 SSTP 017 – Aminata Application. August 2014. Production and Dissemination of Quality Seed of Newly Released Improved Varieties of Maize, Beans and Soybean in Tanzania. 2014 SSTP 017 Aminata IPTT Report Nov 2016 – Q9 Final. 2014 SSTP 030 – CIAT PABRA Application. November 2014. Unlocking the Potential of Seed Companies to Reach Smallholders with New Improved Bean Varieties in Northern Tanzania. 2014 SSTP 030 – CIAT – PABRA Grant Narrative Report – Sep-Nov 2016 Final. 2014 SSTP 030 CIAT- PABRA IPTT Q8 Sep – Nov 2016 Final. 2014 SSTP 018 – CBS Application. September 2014. Sustainable and Seed Potato Value Chain in Northern Tanzania. 2014 SSTP 018 CBS IPTT Form Quarter 9 – Final. 2014 SSTP 018 CBS Grant Narrative Interim Q Report 01 October – 30 December. 2016 Final. CBS AGRA Grant Narrative. Qt. Report 10 October – 31 December 2016. 2014 SSTP 026 – SAGCOT – Mtanga Farms. September 2014. Up-scaling Improved Seed Potato Varieties for Smallholder Farmers in Njombe Tanzania. 2014 SSTP 026 SAGCOT – Mtanga IPTT Q8 – Final. 2015 SSTP 013 – MATCC Full Application. Scaling up Local Seed Production and Distribution of Low-N￾and Drought Tolerant Maize Varieties in Tanzania. 2015 SSTP 013 MATCC Quarter VII (Sep-Nov 2016) Narrative Report – Final. 2015 SSTP 013 – MATCC IPTT Form Sep – Nov 2016 Final Malawi 2014 SSTP 013 - The Story Workshop Educational Trust- phase 1. Full proposal and quarterly reports 66 ANNEX D. DEFINING AND IDENTIFYING FARMER BENEFICIARIES 67 Definitions of the different categories of farmer beneficiaries need to be clearly understood so that the sampling design adequately captures the different categories. Based on the EOI (see Text Box 1, below), primary direct beneficiaries are those companies and organizations in receipt of grants from the SSTP project, i.e., grantees. Secondary beneficiaries are smallholder farmers – referred to here as farmer beneficiaries. The three categories of farmers are defined as follows, including how they will be identified by the ET. 1. Direct farmer beneficiaries: those smallholder farmers who participate in grantee activities (e.g., demonstration plots, short-term trainings) and/or receive or purchase inputs or services (e.g., seed, fertilizer, mechanized services) directly from the grantee. A farmer will also be considered as a direct beneficiary if they receive SSTP-related ICT extension advice or are able to recall the name of a specific promotional TV or radio program supported by SSTP. The names of some direct beneficiary farmers are generally included on the beneficiary lists held by SSTP, but the lists are not considered to be accurate. Direct beneficiary farmers may or may not be a member of a farmer group or community-based organization. It can be assumed that many direct beneficiary farmers may have tested the improved seeds and technologies, though whether or not they have adopted them will need to be determined through the monitoring surveys. Due to the inaccuracy of the farmer beneficiary lists, combined with the challenges experienced by SSTP’s M&E sub-contractor in locating individuals from the lists, the ET will identify farmer beneficiaries as those members of a community-based group or farmer group that have received training, inputs and/or services from a grantee seed and/or outreach company/organization. For the purpose of the quantitative farmer survey, direct farmer beneficiaries will be identified as being those who recall having received training, inputs, and/or services from a grantee, or received SSTP-related ICT extension advice, or recall the name of specific TV or radio program supported by SSTP. 2. Indirect farmer beneficiaries: those smallholder farmers who do not have direct contact with SSTP-supported activities such as demonstration plots, trainings, or input/service provision, although they may have seen the technology on a neighbor’s farm. Indirect beneficiaries have tested and/or adopted SSTP-supported technologies (including improved seed varieties and/or management practices) that they may have purchased from an agro input dealer (who may or may not be supported through the SSTP project)42 or received from another farmer (whether a direct beneficiary or indirect beneficiary). Indirect farmer beneficiaries may have either tested the improved seeds and technologies and/or adopted them. For the purpose of the quantitative farmer survey, indirect farmer beneficiaries will be identified as being those who are aware of SSTP-supported seed/technologies but have not taken part in SSTP activities. 3. Non-beneficiary farmers: those smallholder farmers who work and reside either within the target areas of SSTP-supported activities (e.g., demonstration plots, trainings, input/service provision provides by the grantees) or within the target areas of agro-dealers selling SSTP￾supported seed and/or technologies. Non-beneficiary farmers have not had direct contact with SSTP-supported activities and have not purchased, tested, or adopted SSTP-supported technologies (whether improved seed varieties and/or management practices). For the purpose of the survey screening process, non-beneficiary farmers will be those who have not taken part in SSTP activities and are not aware of SSTP-promoted varieties. It is possible that some smallholder farmers may have received the same technologies through another project, so the ET will need to avoid sampling geographical areas where there are other projects 42 In general, SSTP does not support many agro-input dealers. In cases where SSTP does support agro-input dealers, it is unlikely that a farmer would know about this, so it would be very difficult, if not impossible, to identify SSTP-supported agro-input dealers. 68 promoting the same technologies. If this information is not known by the SSTP Country Coordinator, then inquiries will be made through the Regional or District Agriculture Office. Text Box 1. Beneficiary Definitions43 An individual is a direct beneficiary if s/he comes into direct contact with the set of interventions (goods or services) provided by the activity. Individuals who receive training or benefit from activity-supported technical assistance or service provision are considered direct beneficiaries, as are those who receive a ration or another type of good. The intervention needs to be significant, meaning that if the individual is merely contacted or touched by an activity through brief attendance at a meeting or gathering, s/he should not be counted as beneficiary. [Footnote 2] An indirect beneficiary does not necessarily have direct contact with the activity but still benefits, such as the population that uses a new road constructed by the activity, neighbors who see the results of the improved technologies applied by direct beneficiaries and decide to apply the technology themselves (spill-over), or the individuals who hear a radio message but don’t receive any other training or counseling from the activity. [Footnote 3] Primary direct beneficiaries: Enterprise-level data will focus on the seed companies with which the Partnership works directly. Data on the numbers of employees (or members), the gender of the employees and leadership/management, and information on the type of organization is being collected by the Partnership for each performance indicator, including the production of seeds, and the value/volume of seeds sold. [Page 8] Secondary direct beneficiaries: Smallholder farmer data is being collected by the sub-grantees, in order to assess impact on applying new technologies and varieties. These farmers are expected to participate in grantee activities such as extension services (e.g., demonstration plots), capacity training (e.g., short-term trainings), and other services and facilitation activities. The assumption is that these farmers would have adopted the improved seeds and technologies and demonstrated impact. [Page 8] Indirect beneficiaries: Effects on smallholder farmers who have bought the Partnership’s supported, improved seed varieties and/or associated technologies by non-grantee agro input dealers (where AGRA has pre-existing relationships built through other programs or who purchase/receive Partnership-supported improved seed varieties and/or improved technologies and/or improved management practices) will also be examined to look at what elements affect farmer demand, bring about uptake and have an impact on productivity and income. [Page 8] 43 Source: USAID/BFS. 2016. EOI – Performance Evaluation. Scaling Seeds and Technologies Partnership. Page numbers and footnotes refer to the EOI document (see Annex A). 69 ANNEX E. DATA COLLECTION METHODS 70 ANNEX E.1. LIST OF INDIVIDUALS INTERVIEWED BY COUNTRY Ghana Name Position Institution/Company Date of Interview Forster Boateng Country Coordinator SSTP Jan. 23 & Jan. 26 Evelyn Anfu Deputy Country Coordinator SSTP Jan. 23 & Jan. 27 Seth Osei Akoto Ag. Director of Crop Services Ministry of Food and Agriculture (DCS) Jan. 23 Dr. Solomon Gyan Ansah Deputy Director, DCS (also WASP Seed Specialist) Ministry of Food and Agriculture (DCS) Jan. 23 Rowland Addo, Agricultural Officer, DCS (Seed Unit) Ministry of Food and Agriculture (DCS) Jan. 23 Macbeth Appiah￾Dankwali, Seed Inspector (GSID) Ministry of Food and Agriculture (GSID) Jan. 23 Richmond Abeka Sey Acting Chief Executive Officer MAP TV/Viasat Broadcasting G Ltd. Jan. 23 Nancy Boakye MAP TV/Viasat Broadcasting G Ltd. Jan. 23 Hon. Issifu Mohammed Pangabu CEO PEE Farms Ltd. Jan. 24 Asare Kusaku, Accountant Accountant PEE Farms Ltd. Jan. 24 Owusuaa Alimata, Secretary PEE Farms Ltd. Jan. 24 Henry Azot Agricultural Extension Agent Ministry of Food and Agriculture (MOFA) Jan. 24 Abdulai Tiani Agricultural Extension Agent MOFA Jan. 24 Peter Gyinah Store Manager Jehovah is Faithful Jan. 24 Samual Adomako Facilitator AGS & More Jan. 24 Attah Abebrese Facilitator AGS & Moe Jan. 24 Richmond Adjei Agronomist AGS & More Jan. 24 Stephen Opuko Store Manager WAAF Agro Limited Jan. 24 9 Out-growers PEE Farms Ltd. Jan. 24 Gabriel Dake M&E Officer AGS & More Ltd. Jan. 25 Mr. Addo Manager B. Kaakyire Agro Chemicals Ltd Jan. 25 Francis Appiah KNUST Jan. 25 Eli Gaveh KNUST Jan. 25 K. Obeng-Antwi CRI Jan. 25 Francis Asempah Asempah Seed Co. Jan. 25 Simeon Mankalo Agri-Impact Jan. 25 Daniel Acquaye Agri-Imact Jan. 25 Evelyn Denchern Agri-Impact Jan. 25 Kingsley Sectey Agri-Impact Jan. 25 Edward Boateng Grain & Legume Development Board Jan. 25 Irene Annor-Frempong Director for Agricultural Research and Innovation FARA Jan. 26 Cecil Osei Country Coordinator Grameen Foundation Jan. 26 71 Name Position Institution/Company Date of Interview Amos Rutherford Azinu CEO and Founder Legacy Crop Improvement Center (LCIC) Jan. 26 Augusta Clottey Executive Secretary NASTAG Jan. 26 Juliana Asante-Dartey Country Director ASNAPP Jan. 26 Monica Awuku Co-founder & Marketing Manager M&B Seeds and Agric. Services GH Ltd. Jan. 27 Joshua Kuatsikor Assistant Marketing Manager M&B Seeds and Agric. Services GH Ltd. Jan. 27 Mawuli Amenyoame Out-grower M&B Seed Jan. 27 Gabriel Badasu Out-grower M&B Seed Jan. 27 Alen Owusu Out-grower M&B Seed Jan. 27 Dam Patience Employee Mavid Seeds Jan. 27 Benjamin Kemetse CEO M&B Seed and Ag. Services (Ghana) Ltd. Jan. 27 Afua Ansre Managing Director IVSC Jan. 27 Processing Manger M&B Seed Jan. 27 2 MOFA Extension Agents Extension Agents MOFA Jan. 27 1 AFRICARE staff AFRICARE Jan. 27 Michael Kumi Store Manager Greenshield Agrochemicals Jan. 27 Hellen Vitashie Store Manager Green Acres Ventures Jan. 27 3 women farmers Farmers Jan. 28 Roland Nii Ayi Quaye Head of Business RMG Ghana Ltd. Jan. 30 Jenna Tajchman-Trofim Ag Development Officer USAID/Accra Jan. 30 Kenya Name Position Institution/Company Date of Interview Itai Makanda Deputy Chief of Party SSTP Feb. 12 Richard Jones Chief of Party SSTP Feb. 12 Joe DeVries Chief of Agricultural Transformation AGRA Feb. 13 Mainza Mugoya Regional Coordinator Market Matters Inc. Feb. 13 Caroline Adala Oremo Business Development Support AGRA Feb. 13 Diana Odero SSTP Grants Officer AGRA Feb. 13 Ignatius Mutula AGRA Grants Manager AGRA Feb. 13 Simiyu Gaitano M&E Officer SSTP Feb. 13 Mozambique Name Position Institution/Company Date of Interview Tiana Campos Program Leader CLUSA Feb. 14 Name not recorded Technician from the SEEDS program CLUSA Feb. 14 Constantino Nobre Cuambe Project Coordinator IIAM, Nampula Feb. 14 72 Name Position Institution/Company Date of Interview Amilcar Lucas Benate CEO Oruwera Feb. 14 Mauro Gatabeta Agronomist Oruwera Feb. 14 Benvindo Verde Deputy Country Coordinator SSTP/AGRA Feb. 15 Managing Director Managing Director Rapid Agrodealer Landscaping Feb. 15 Francisco Macoua Technical IFDC responsible for Prequejo village Feb. 15 Carols Cobre Extension Agent IFDC Feb. 15 No name recorded Store Manger DADTCO Feb. 15 No name recorded Procurement Officer DADTCO Feb. 15 Elsa Timana Head of Seed Department National Directorate of Agriculture and Silviculture, Ministry of Agriculture and Food Security Feb. 16 Isabel Mazive Cassava Team Leader IFDC Feb. 16 Estevo Fraqueza Agronomist IFDC Feb. 16 Maria Estrela Alberto APROSE Executive Secretary and Co￾author of EGS study IIAM Feb. 16 Dr. Anabela M. Zacarias Director Ministry of Agriculture – Seed Department, Cassava Platform Feb. 17 Tanzania Name Position Institution/Company Date of Interview Baldwin Eghert Shuma Executive Director TASTA Feb. 20 Vianey Rweyendela Country Coordinator AGRA-SSTP Feb. 20 & Feb. 27 Ipyana Mwakasaka Deputy Country Coordinator AGRA-SSTP Feb. 20 & Feb. 25 Geoffrey Kirenga CEO Southern Agricultural Growth Corridor of Tanzania (SAGCOT) Feb. 20 Donald D. Temba Farm Manager African Plantation Kilimanjaro Ltd. Feb. 21 Patrick Paul Mwasapi Seed Production Manager Meru Agro-Tours & Consulting Co. Ltd. Feb. 21 Marco Martin Mwendo Agric. Officer Agricultural Seed Agency Feb. 21 Karen Hampson Regional Coordinator Farm Radio International Feb. 22 Linda Temba Project Coordinator Farm Radio International Feb. 22 Japhet Emmanuel Country Manager Farm Radio International Feb. 22 Rashid Kuma Chairman Uwasdu Farmer Group Feb. 22 Eunice Mmary Area Representative Meru Agro – Tours Feb. 22 Solomon Representative ERSF Farmer Survey Feb. 22 Rashid Kuma Chairman Uwasdu Farmer Group Feb. 22 Eunice Mmary Area Representative Meru Agro – Tours Feb. 22 Solomon Representative ERSF Farmer Survey Feb. 22 Jean Claude Rubyogo PM CIAT Feb. 22 Sylvia Monica Kalemera M&E GIS CIAT Feb. 22 Fadhili Kasubiri Asst. PM CIAT Feb. 22 73 Name Position Institution/Company Date of Interview Marco Mwendo Agric. Officer ASA Feb. 22 Muhammad A. Bajwa Managing Director Bajwa Farmers & Traders Ltd. Feb. 22 Mr. Akai Shop Owner Akai Agrovet Feb. 22 Ayubu Private Extension Officer Khebhandza Mark Co. Feb. 24 Paul Mchana Principal Farmer Training Center Feb. 24 Ayubu Private Extension Officer Khebhandza Mark Co. Feb. 24 Emmanuel Innko Marketing Officer Highland Seed Growers Feb. 24 Dr. Arnold Mushongi Project Manager, SSTP, ARI Uyole ARI Uyole Feb. 24 Leonard Sabula Co-Project Manager, SSTP ARI Uyole ARI Uyole Feb. 24 Elisha Kameta Accountant Highland Seed Growers Feb. 24 Bonaver Mlewa Human Resource Highland Seed Growers Feb. 24 Tulole Lugendo Bucheyeki Zonal Director ARI Uyole (Southern Higllands Zone) 24 Feb John Kibwinya Research Officer ARI Uyole Feb. 24 Estos Ronado ICT Officer ARI Uyole Feb. 24 Arnold Mushongi Maize Breeder, SSTP Project Manager ARI Uyole Feb. 24 Benjamin Kiwobele ARI Uyole Feb. 24 Nasibura ARI Uyole Feb. 24 Stephen Matotanje General Agronomist Sura International Agro Feb. 24 Merina Nyando Farmer (woman) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Wilson Ndali Farmer (man) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Stella Farmer (woman) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Saloum Marikano Farmer (man) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Annette Muhanga Farmer (woman) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Obede Kadodo Farmer (man) Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Ezekiel VBA Wanging’ombe District, Njombe Region Feb. 25 Browness VBA Wanging’ombe District, Njombe Region Feb. 25 Salome VBA Wanging’ombe District, Njombe Region Feb. 25 Gift VBA Wanging’ombe District, Njombe Region Feb. 25 74 Name Position Institution/Company Date of Interview Latifa VBA Wanging’ombe District, Njombe Region Feb. 25 Dani VBA Utiga Village, Wanging’ombe District, Njombe Region Feb. 25 Johanes Musau Coordinator FIPS – Tanzania Feb. 25 Malimi Leonard Master Trainer Centum Feb. 25 Melanda Scmid Human Resources Officer FIPS – Nairobi Feb. 25 Edward Kanju Breeder IITA Feb. 25 James Legg Entomologist/Virologist IITA Feb. 25 Owekisha Herman Kwigizile Potato Project Coordinator SAGCOT Feb. 25 Noel Lugenye Field Officer Mtanga Farms Ltd. Feb. 25 Anna Kanyangemu Project Assistant SAGCOT Feb. 25 Mhola Sybuko DCO Makambako FIPS Feb. 25 Richard Msikwa DAICO DAICO Feb. 25 Geoffrey Wiltioni DCO Songea FIPS Feb. 25 Lusungu Kitambole DCO Njombe FIPS Feb. 25 Johannes Musau FIPS Country Manager FIPS Feb. 25 Deo Msemwa FIPS Operations Manager FIPS Feb. 25 Leonard Malimi Sitta CENTUM Trainer Centum Feb. 25 Melanda Ochieng FIPS SSTP PM FIPS (Nairobi) Feb. 25 Betty Maida Ag. Research Specialist & Economic Growth USAID Feb. 25 Dave Charles Feed the Future Team Lead USAID Feb. 25 Edward Charles Technical Director Kilimo Trust Mar. 2 (Skype) Additional Interviews Name Position Institution/Company Date of interview Paul van Mele Co-Founder Agro-Insight Mar. 2 (Skype) Lesley Perlman Monitoring and Evaluation Specialist USAID-BFS Mar. 8 (Skype) Mark Huisenga Senior Program Manager USAID-BFS Mar. 8 (Skype) Eva Weltzin Millet/Sorghum Breeder (retired) Formerly ICRISAT Mar. 17 (in person) Yonas Sahlu Country Coordinator, Ethiopia AGRA-SSTP Mar. 23 (Skype) Gezahegn Ayele Deputy Country Coordinator, Ethiopia AGRA-SSTP Mar. 23 (Skype) Tom Osborn Seed Specialist (retired) FAO Mar. 24 (email) Kent Mphepo Project Coordinator Story Workshop Educational Trust Mar. 27 (Skype) Blessings Mtuwa Nkhata Head of Programs Story Workshop Educational Trust Mar. 27 (Skype) Noel Sangole Country Coordinator, Malawi AGRA-SSTP Mar. 28 (Skype) 75 Name Position Institution/Company Date of interview Geoffrey Kananji Deputy Country Coordinator, Malawi AGRA-SSTP Mar. 28 (Skype) Abdou Ndiaye Country Coordinator, Senegal AGRA-SSTP Mar. 28 email 76 ANNEX E.2. FOCUS GROUP DISCUSSIONS AND ROUND TABLE MEETINGS Location # Male Participants # Female Participants Date of FGD Female out-grower group, PEE Farms, Ashanti Region 10 Jan. 24 Male out-grower group, PEE Farms, Ashanti Region 9 Jan. 24 Aframso, Ashanti Region, GH 15 (including about 5 youths) 3 Jan. 24 Amfoe & Waya Communities, Adako District, Volta Region, GH 9 8 Jan. 28 Olima Ovilela Village, Ribaue District, MZ 9 11 Feb. 15 Prequejo Village, Murupula District, MZ 21 8 Feb. 15 Olkolili Village, TZ 7 3 Feb. 21 Bonga Village, TZ 31 1 Feb. 22 Nkowarisambo Village, Meru District, TZ 12 10 Feb. 22 Mashoma Village, near Mbeya, TZ 17 3 Feb. 24 Kilimo Chetu, Farmer Training Center, Inyala, TZ 4 6 Feb. 24 Lunguya Village, Njombe District, TZ 16 9 Feb. 24 77 ANNEX F. DATA COLLECTION INSTRUMENTS 78 INTERVIEW GUIDES Interview Guide for SSTP Country Staff 1. SSTP Country Priority Setting a. How have the baseline survey and the roadmap helped orient SSTP grant making? b. Are the grants aligned with the roadmap and are there indications that constraints are being addressed? 2. SSTP Grant Approval Process How was the SSTP grant advertised/publicized, and how were grantee applicants shortlisted and selected? What role did you play in these processes? What were the challenges? 3. SSTP Grantee Support a. Technical b. Financial c. Data management & reporting 4. SSTP Data Management a. Given the critical role between monitoring the progress of the grantees and the SSTP program-wide M&E system, what are the strengths and weaknesses in the data management system? b. What challenges have you encountered and how have you addressed these? 5. SSTP Communication and Learning a. Between grantees within country b. Between SSTP countries 6. SSTP Seed Sector Support Communication & Coordination a. Describe how SSTP has been able to bring coherence to the formal seed sector and especially enhance the role of the private sector while supporting the public sector? 7. SSTP Seed Policy Support a. To what extent is SSTP supporting the country seed strategy? b. To what extent is SSTP able to promote a pluralistic seed system with clear roles for the public and private sector? c. What country-level and regional-level policy and regulatory processes have you been engaged with? Describe any policy-related activities that SSTP has supported. 8. Looking Ahead a. Which grant investments do you feel have been most successful and why? b. Which have been disappointing and again why? c. Where on the seed chain do you feel future investment should focus? 79 Interview Guide for Seed Company Grantees 1. Could you please provide an overview of your SSTP grant? a. What types of improved variety seeds/planting materials does your company produce and sell? b. In what ways has SSTP assisted your company in developing your seed business? c. With the assistance provided by SSTP have you achieved your business targets? d. Are there other ways in which it could have helped more? 2. Based on your involvement with SSTP, in what ways has your company’s capacity to produce and distribute quality improved seeds/varieties changed? a. What improved seed, other technologies, and farming practices are your company promoting in your area? b. What methods/approaches are you using to promote the seed varieties and other technologies among smallholder farmers? Do you provide samples to agro-input dealers and/or farmers? How? c. How do you gather feedback from agro-input dealers/farmers? What have you learned from the feedback to date? What are your distribution channels? How effective are these? (quantities sold through different mechanisms) d. What challenges still remain in increasing availability of improved seeds/varieties and other technologies and what is your company or SSTP doing to address them? e. Do smallholder farmers understand the benefits of using improved varieties/technologies in terms of their “Return on Investment?” 3. Based on your experience with the SSTP project, what has been your experience in accessing breeder’s and foundation seed for multiplication and certified seed production? a. How is breeders’ seed/foundation seed allocated to different seed companies? b. Who is responsible for the quality of foundation seed used in certified seed production? c. Is foundation seed a class of certified seed? d. What are the major barriers in accessing foundation seed and what would make the process more efficient and profitable for your company? e. In your assessment, which improved varieties have been easier/more difficult to produce/distribute and why? 4. Does the practice of making foundation seed of the same improved variety available to all seed companies that want it help or hinder the availability of improved certified seed by smallholder farmers? a. In what ways does it help/hinder availability? b. What types of crops are suited/not suited to this approach and why? c. What changes are required in order to improve availability of improved varieties by smallholder farmers? 5. Based on your experience with the SSTP project, are there aspects of producing improved seed that can be best performed by other entities rather than by the seed company and why? a. What has been the most difficult aspect of producing improved seed for your company and why? b. How would you change it to improve the efficiency and profitability of your seed company? c. Does your company develop its own improved varieties? If yes, give examples. If not, explain why not. 80 6. Based on your experience with the SSTP project, what have been your major challenges in scaling up seed production? a. Are government policies favorable to private seed sector in seed production, certification, inspection, distribution, marketing, etc.? b. What policies/regulations are/are not working and why? c. What changes would improve scaling up seed and technologies through the private sector? 7. Could you please describe your seed production planning (forecasting) process and what inputs go into the planning process? a. Are planting, production, sale, etc. of foundation and certified seed reported and/or tracked by seed certification/seed growers, licensing authorities, etc.? b. In your experience with the SSTP project, what activities have helped you create demand by smallholder farmers of the improved varieties that you produce? c. In your assessment, what factors influence farmers’ demand for quality-improved varieties? d. Do smallholder farmers ask for these parameters when making buying decisions regarding improved varieties? e. What markets do your farmers sell their grain/produce to and how does this impact demand for improved varieties? 8. Has SSTP assisted your customers in accessing markets for the grain/produce that they produced? a. In your assessment, how can the private sector improve demand for improved varieties? b. In which ways can SSTP help in demand creation? 9. Please explain your seed/planting materials distribution, marketing, and sales. Is this the model used by other seed companies in the country? a. How are your improved varieties branded and how do farmers distinguish brands when making buying decisions? b. How does your company differentiate its brand of OPV maize from the same OPV maize sold by other companies? c. How would you improve the current distribution system to make it more efficient and profitable? 10. There are many ways of creating awareness about improved varieties/technologies among smallholder farmers. In your assessment, what method(s) have been most effective and why? a. For those conducting demonstration trials, please describe these (e.g., local, regional, or countrywide; farmer-managed or company-managed). How effective have they been in demonstrating the benefits of improved varieties/technologies to smallholder farmers? b. What has been the value of small seed packs given to smallholder farmers? c. How effective are radio programs in educating smallholder farmers about improved varieties and other technologies? Explain. d. What level of collaboration did you receive from government extension in promoting improved varieties and technologies? e. Based on your observations, what changes would you make to create awareness among smallholder farmers and why? 11. As a result of the SSTP grant, have you expanded the network of agro-dealers in the country? Explain how. Based on your involvement with the SSTP project, do agro-dealers and smallholder farmers have the knowledge required to demand improved varieties and technologies? Did the project activities contribute to capacity building of agro-dealers? Explain. 81 Interview Guide for Individual Agro-dealers 1. Please explain the processes for registering an agro-dealer business. a. For the product that you sell, do you have any exclusive selling rights or is your business a commission-based business? Explain. b. How many different companies are represented in the products that you sell? c. Is this your only selling point or do you sell through other outlets, e.g., weekly markets, etc.? d. How many other seed/agro-input selling points (agents) do you have to service your customers? e. How do farmers pay for the seed/products that they buy? 2. Could you explain the types of products that you sell to smallholder farmers in your area? a. From whom do your buy your products – seed companies, wholesalers, or other agro￾dealers? b. In what ways have your suppliers assisted you with your marketing program? c. What are your major challenges in obtaining improved seeds or other products to sell to smallholder farmers? d. In your opinion, what makes your customers repeat buyers of improved seeds and other technologies? Include questions on how the dealer markets/promotes/advertizes his/her products among farmers…. Include questions on distribution channels – how does dealer sell products? (e.g., from shops, through weekly markets, through distributors, etc.) Does the dealer provide inputs on loan/barter basis? What are repayment terms? 3. Based on your observations of your customers, what improved varieties and technologies are they buying? a. Have they increased the use of improved seed and technologies in the last two or three years? b. What motivates them to buy improved varieties and technologies? c. What prevents them from buying and using improved varieties and technologies? d. How do your customers differentiate the various brands of improved seeds that you sell? 4. Do your farmer customers look up to you for advice on what products to buy and how to use them correctly? If yes, what kinds of information/advice are they asking for? a. Describe how and where you get the information and whether you pay for the information and training that you receive. b. Do the farmers apply the information/products according to the instructions? c. What more could be done to increase their ability to use the information correctly? 5. In your view, do smallholder farmers have the knowledge required to demand and use improved varieties and technologies? 6. Have there been unforeseen effects from increased seeds/inputs/technologies provided by providers? 82 Interview Guide for Other Stakeholders, e.g., public sector, policy level, MOFA officials, Partners, etc. Note: Only those questions that are relevant to the specific individual will be asked. 1. Describe your/your organization’s role in developing country-level and regional policy and regulatory mechanisms in the seed sector. 2. How has SSTP supported the development of national and regional policies and regulatory mechanisms? a. Describe any (ongoing) policy development processes to date and what role SSTP has played. b. Has SSTP supported any workshops or events that brought together different stakeholders to develop policies and/or regulatory mechanisms? 3. How has SSTP contributed to the objectives/activities described in the New Alliance Country Framework? 4. Breeder’s seed availability and foundation seed multiplication are often a major challenge to the availability of improved varieties/seed to smallholder farmers. a. In your interactions with SSTP, what have you seen as major steps taken to address these constraints? b. In your assessment, has the situation improved/remained the same/worsened? c. What national/regional initiatives has SSTP taken to address these constraints? 5. Do mechanisms exist that provide limited exclusivity to private seed companies of new varieties released by the public sector? a. Would such exclusivity be a good or bad arrangement and why? b. Would it increase the availability of improved seed to smallholder farmers? c. What seed sector/system/crop would benefit from this type of arrangement? 6. Could you explain the seed certification and inspection process in the country and what is working or not working well? a. In what ways is SSTP supporting the seed certification and inspection processes in the country? b. Would relevant stakeholders view a seed company’s certification of their own seed favorably? c. In your assessment, who is best suited to provide seed inspection service to the seed sector? d. Are there other ways of improving these processes? 7. Are data on certified seed production and sale captured by seed certification agency or other entities? a. Would this help the seed sector’s seed demand planning process? b. What role should organizations such as Seed Producer Associations, Seed Traders Associations, etc. play in demand planning? 83 Interview Guide for Individual Out-Growers 1. Please explain the certification process to be a registered seed grower. 2. How many different seed crops did you grow during the last growing season? 3. Did you plant both certified seed/planting materials and food crops? 4. Do you produce certified seed/planting materials for more than one customer? If yes, how are the different materials handled to maintain identity preservation? 5. Do you receive a forward contract to produce certified seed/planting materials? If yes, what are some of the terms of the contract? 6. Do you produce certified seed for yourself? 7. What is the source of the foundation seed that you use to produce certified seed? a. Do you pay for the foundation seed and, if so, what is the price per unit? b. How do you ensure that the foundation seed is of good quality and that the certified seed produced is also of good quality? c. Who processes the certified seed that you produce? 8. What are the major challenges in producing certified seed/planting materials and what recommendations would you make to improve the process? 9. What is the role of a seed company in your certified seed/planting materials production? 10. Based on your experience, do you have the necessary knowledge and skills to produce good quality certified seed/plant materials? 84 Interview Guide for Individual Farmers Responsible for Farmer-Managed Demonstrations 1. Please explain the purpose of the demonstration plot that you managed. 2. How many crops/varieties were grown, and what are their names? Were any other improved technologies used on the demo? 3. How were you selected to be responsible for the demo? 4. How many years have you been doing demos? 5. What kind of training did you receive? 6. What other forms of support/inputs were provided? What did you have to provide? 7. What are the benefits of managing a demo? 8. How do other farmers learn from your demo? 9. How effective do you consider the demo to be in promoting awareness about the varieties/technologies? 10. What other promotional activities motivate farmers to try the varieties/technologies? 85 Format Used in Writing Up Interview Notes Note-taker: Date: City/Town: Start time: End time: Name Position Institution/Company Contact details (if necessary) Type of organization (delete as appropriate): SSTP Grantee; Public Sector; Private Sector; Summary of interview (bullet points): Additional direct observations: Delete as necessary: Implications for ERQ 1.1: Implications for ERQ 1.2: Implications for ERQ 1.3: Implications for ERQ 1.4: Implications for ERQ 2.1: Implications for ERQ 2.2: Implications for ERQ 2.3: Implications for ERQ 3.1: Implications for ERQ 3.2: Implications for ERQ 3.3: Implications for ERQ 4.1: Implications for ERQ 4.2: Follow up required (e.g., changes to interview questions; documents to refer to, etc.): Additional inputs from: 86 FOCUS GROUP DISCUSSION CHECKLISTS FGD Checklist for an Out-Grower Group 1. Please provide some background information about your group, e.g., when it was formed, how many members, how the members were selected, and how the group operates (i.e., by multiplying seed on a group farm or on the individual farms of the members). 2. Please describe the types of training you have received, and who provided the training. 3. Please explain the certification process to be a registered seed grower. 4. How many different seed crops did you grow during the last growing season? 5. Do you produce certified seed/planting materials for more than one customer? If yes, how are the different materials handled to maintain identity preservation? 6. Describe the contractual arrangements with your customer(s). 7. What is the source of the foundation seed that you use to produce certified seed? a. Do you pay for the foundation seed, and, if so, what is the price per unit? b. How do you ensure that the foundation seed is of good quality and that the certified seed produced is also of good quality? c. Who processes the certified seed that you produce? 8. What are the major challenges in producing certified seed/planting materials and what recommendations would you make to improve the process? 9. What is the role of a seed company in your certified seed/planting materials production? 10. Based on your experience, do you have the necessary knowledge and skills to produce good quality certified seed/plant materials? 87 FGD Checklist for Discussions with Direct Beneficiary Farmers 1. Introduction of the team and purpose of the discussion. 2. Introduction of the participants. 3. For Farmer Group: When was the group formed? How many members are there? Where do the members come from? Are there links with other groups? 4. For Farmer Group: What is the purpose of the group? What do you expect to gain through being a member? 5. For Farmer Group: What activities have been undertaken by the group and group members since the group was first established? Use follow-up questions in relation to the activities to determine the different technologies/varieties promoted by SSTP partners and ensure that you have a complete list. 6. For each technology/variety promoted through the group activities: a. What is the purpose of the technology, why is it being promoted? (to determine level of knowledge/understanding of farmers) b. How was the technology promoted? (include group activities, as well as more general radio ads, posters, etc.) c. And how did individual group members acquire it? (acquired free or through purchase? Where? Distance of agro-dealer? Quantity?) d. What are the local name(s) for technology? Be sure to record all local names for the improved varieties. e. Show of hands for how many farmers used the technology on their own farms in the last growing season and plan to use it again in the next season. What do you like about this technology/why are you using it? f. Show of hands for how many farmers tried the technology but have decided not to use it again. Why have you decided to stop using the technology? What don’t you like about it? 7. Has the group or any individual members tried to promote any of the technologies among other (non-group member) farmers? How? (provide examples where possible and record name and location of farmers for follow-up) 8. For the selected crop: What are the different types/varieties of this crop grown locally? Ask individual farmers to tell us the names of the varieties that they cultivated in the past season/year and whether they bought the seed or saved it from the previous year. This list doesn’t have to be exhaustive, but we need a rough indication of the number of different varieties that are commonly grown, how many different varieties a single farmer might grow in any one year, and where the seed comes from. 9. End of our questions – does anyone else have any questions, or additional points that they would like to raise? 10. Closing of the discussion – thank everyone for participation, and encourage them in their future group/farming activities. 88 Checklist for Discussions with Mixed Group of Direct Beneficiary Farmers, Indirect Beneficiary Farmers, and Non-Beneficiaries The discussion will focus on one selected crop. All farmers growing the selected crop will be invited to take part in the discussion. In Tanzania, it might be necessary to focus a single discussion on two different crops (to discuss all three crops in a single session would be too time-consuming); in this case all questions relating to one crop must be completed before proceeding to the second crop. The discussion is designed to be fast-paced, with lots of participation through raising hands and asking different individuals to provide answers to the same question to get a range of answers. 1. Introduction of the team and purpose of the discussion. 2. Introduction of the participants. 3. What are the main crops grown by farmers in this community? We would like to focus the discussion on [selected crop]. Is this grown for food or for cash, or both? 4. For the selected crop: What are the different types/varieties of this crop grown locally? Ask individual farmers to tell us the names of the varieties that they cultivated in the past season/year. This list doesn’t have to be exhaustive, but we need a rough indication of the number of different varieties that are commonly grown, and how many different varieties a single farmer might grow in any one year. 5. Has anyone heard about the [xxx] variety/ies (use local names to describe the varieties)? Focus on one variety only and make sure that everyone is clear about the variety in question (ask someone to describe it, and make a note of any other names that might be used to describe the same variety). Use a show of hands to determine how many have heard about the variety and how many have not. 6. Use a show of hands to determine if there anyone who has heard about the variety but is not growing it. Ask these farmers, individually, any one of the following questions: a. How did you hear about it, and why is it being promoted? b. What do you know about the variety, and how did you learn about it? c. If you want to grow it, do you know where you can get seed? How do you know that you can get seed there? (These questions aim to determine level of knowledge/understanding of farmers in relation to different promotional methods). 7. Did anyone grow the variety last season? Again, use a show of hands to count how many people grew the variety last season, and ask these people individually to determine: a. How many years different individuals have been growing it (so that you can establish roughly when it was first introduced to the village)? b. How they first heard about it. c. How they first acquired the seed (xx years ago). From whom? Free or purchased? In what quantities? d. How they acquired the seed for the last planting season (if they have been growing the variety for more than one season). From whom? Free or purchased? In what quantities? 8. Provide a recap on the different categories of farmers present in the discussion: a. Those who have not heard about the variety (if present). b. Those who have heard about it but are not planting it. c. Those who have heard about it and are planting it. Ask participants to describe the differences among these categories of farmers – are there differences in terms of wealth or social networks, or farm type or labor availability? Other differences? 9. For those who grew the variety last season: 89 a. Show of hands for how many farmers used the technology on their own farms in the last growing season and plan to use it again in the next season. What do you like about this technology/why are you using it? b. Show of hands for how many farmers tried the technology but have decided not to use it again. Why have you decided to stop using the technology? What don’t you like about it? 10. Are there any other agricultural technologies that have been promoted alongside the variety? What are these? (List all, including local names for the technologies). For each technology, ask for a show of hands as to how many farmers have used the technology on their farm last season. 11. End of our questions – does anyone else have any questions, or additional points that they would like to raise? 12. Closing of the discussion – thank everyone for participation, and encourage them in their future farming activities. 90 ONLINE QUESTIONNAIRE FOR GRANTEES Introduction: This online survey is administered by Mendez England and Associates (ME&A) who have been contracted by USAID to undertake a Mid-Term Performance Evaluation of the SSTP Program. All SSTP grantees in six countries (Ethiopia, Ghana, Malawi, Mozambique, Tanzania, and Senegal) have been invited to take part in the survey to gather information relating to the management of the Partnership, SSTP’s engagement with the Private Sector, and progress in policy and regulatory reform. The findings from this survey will feed into the Evaluation Report. The survey will take approximately 20 to 30 minutes to complete and can be filled out anonymously. We would, however, prefer if you provide your name, organizational affiliation, and email address in case of any follow-up. Your participation is entirely voluntary, and your feedback will only be shared with the Evaluation Team. Your name/organization will not appear in the Evaluation Report. Please contact Tom Remington (email: tom.remington.2014@gmail.com) if you have any questions. If you agree to take part in the survey, please click the button below to proceed. Q2. Indicate the country in which you are working (if regional check all relevant countries): ❑ Ethiopia ❑ Ghana ❑ Malawi ❑ Mozambique ❑ Senegal ❑ Tanzania Q3. Indicate your organizational type: ❑ Private Sector ❑ Public Sector ❑ International Organization ❑ Non-Governmental Organization (NGO) Q4. For this question, please indicate which activities your organization is undertaking as part of your SSTP grant. You may check more than one if it applies to your organization: ❑ Foundation Seed Production ❑ Certified Seed Production ❑ Awareness-Raising/Demand-Creation/Marketing of Seed of Improved Varieties ❑ Awareness-Raising/Demand-Creation/Marketing of Other Technologies/Management Practices ❑ Capacity Strengthening (of Farmers, Agro-Input Dealers, Seed Companies, Policy Makers, etc.) ❑ Activities to Promote Policy/Regulatory Development ❑ Other Activities 91 5a. Based on the Foundation Seed Production choice you checked, please check the crops that apply to your SSTP grant: ❑ Hybrid Maize ❑ OPV Maize ❑ Bean ❑ Cowpea ❑ Soybean ❑ Groundnut ❑ Cassava ❑ Yam ❑ Sweet Potato ❑ Pigeon Pea ❑ Chickpea ❑ Potato ❑ Rice ❑ Other (please write crop) ____________________ 5b. Based on the Certified Seed Production choice you checked, please check the crops that apply to your SSTP grant: ❑ Hybrid Maize ❑ OPV Maize ❑ Bean ❑ Cowpea ❑ Soybean ❑ Groundnut ❑ Cassava ❑ Yam ❑ Sweet Potato ❑ Pigeon Pea ❑ Chickpea ❑ Potato ❑ Rice ❑ Other (please write crop) ____________________ 92 5c. Based on the Awareness-Raising/Demand-Creation/Marketing of Other Technologies/Management Practices choice you checked, please check the boxes that apply to your SSTP grant: ❑ Land Preparation ❑ Seed Treatment/Inoculation ❑ Seeding ❑ Weed Control ❑ Soil Fertility ❑ Disease Control ❑ Insect Control ❑ Harvesting ❑ Storage ❑ Processing ❑ Other (please write crop) ____________________ 5d. Based on the Awareness-Raising/Demand-Creation/Marketing of Seed of Improved Varieties choice you checked, please check the ones that apply to your SSTP grant: ❑ Demonstrations Plots ❑ Farmer Training Sessions ❑ Television Programs ❑ Television Advertisements ❑ Community DVD Events ❑ Radio Programs With Call In ❑ Radio Advertisements ❑ Advertisements or Information Provided Over Loud Speaker ❑ Printed Publicity and Branding (for example, posters, stickers, etc.) ❑ Field Days ❑ Participation in Local Weekly Markets in Rural Areas ❑ Participation in or Facilitation of Seed Fairs or Agricultural Input Fairs ❑ Participation in or Facilitation of Agricultural Shows ❑ Free Seed Samples for Testing by Farmers ❑ Branding of Seed Packets ❑ Mobile Phone Service with Either SMS or Voice Information ❑ Information Available on Tablets ❑ Social Media Platforms such as Twitter, Facebook, etc. ❑ Newsletters ❑ Information Leaflets or Brochures 93 5d1. Based on the Awareness-Raising/Demand-Creation/Marketing of Seed of Improved Varieties choices you checked, please rank them in terms of perceived effectiveness in creating demand (return on investment): ______ Demonstrations Plots ______ Farmer Training Sessions ______ Television Programs ______ Television Advertisements ______ Community DVD Events ______ Radio Programs With Call In ______ Radio Advertisements ______ Advertisements or Information Provided Over Loud Speaker ______ Printed Publicity and Branding (for example, posters, stickers, etc.) ______ Field Days ______ Participation in Local Weekly Markets in Rural Areas ______ Participation in or Facilitation of Seed Fairs or Agricultural Input Fairs ______ Participation in or Facilitation of Agricultural Shows ______ Free Seed Samples for Testing by Farmers ______ Branding of Seed Packets ______ Mobile Phone Service with Either SMS or Voice Information ______ Information Available on Tablets ______ Social Media Platforms such as Twitter, Facebook, etc. ______ Newsletters ______ Information Leaflets or Brochures 5e. Based on the Capacity Strengthening choice you checked, please check to whom you provided support through your SSTP grant: ❑ Agro-Input Dealers/Retailers ❑ Seed Companies ❑ Out-growers/Seed Producers ❑ Policy Makers ❑ Smallholder Farmers ❑ Other (please specify) ____________________ 5f. Based on the Other Activities choice you checked, please specify activities your organization is undertaking as part of your SSTP grant: ____________________________________________________________________ 94 6. To what extent do you agree with the following statements: 6.1 The SSTP grant has increased my organization’s capacity to produce and deliver quality foundation seeds. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 6.2 The SSTP grant has increased my organization’s capacity to produce and deliver quality certified seeds. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 6.3 The SSTP grant has allowed my organization to increase the number and geographical spread of the agro-input dealers selling my certified seed. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 7. For those organizations producing certified seeds, please rank the crops that you produce in the order of profitability (for all crop seeds produced. Not only those supported by SSTP). Starting with 1 being most profitable (if producing seed of more than one kind). ______ Hybrid Maize ______ OPV Maize ______ Bean ______ Cowpea ______ Soybean ______ Groundnut ______ Cassava ______ Yam ______ Sweet Potato ______ Pigeon Pea ______ Chickpea ______ Potato ______ Rice ______ Other (please write crop) 95 8. In your experience, please rank the factors that most influence profitability of your seed enterprise: ______ Distribution network ______ Selling price ______ Cost of production ______ Quantity purchased by customers ______ Recurrent sales year after year ______ Bulkiness of seed or planting material ______ Perishability of seed or planting material ______ Storability/Viability ______ Other (please specify) 10. To what extent do you agree with the following statements: 10.1 The SSTP grant has strengthened my organization’s connection to seed policy actors. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 10.2 SSTP has supported coordination and harmonization of country-level seed production and delivery. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 10.3 SSTP has supported coordination and harmonization of regional-level seed production and delivery. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 10.4 The SSTP grant has allowed my organization to contribute towards sustainable development in this country/region. Please give examples. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 11. Please list any policy/regulatory events and/or processes supported by SSTP in which you/your organization has participated. 12. In your opinion, what are the most important barriers that have yet to be addressed in making quality seed of improved varieties available and accessible to smallholder farmers in your country? 96 13. To what extent do you agree with the following statements: 13.1 The SSTP grant has allowed my organization to work more closely with other private sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 13.2 The SSTP grant has allowed my organization to work more closely with public sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 14. To what extent do you agree with the following statements: 14.1 The SSTP grant has allowed my organization to work more closely with other public sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 14.2 The SSTP grant has allowed my organization to work more closely with private sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 97 15. To what extent do you agree with the following statements: 15.1 The SSTP grant has allowed my organization to work more closely with public sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 15.2 The SSTP grant has allowed my organization to work more closely with private sector organizations. ❑ ❑ ❑ ❑ ❑ ❑ Examples Strongly Agree Agree Neutral Disagree Strongly Disagree Don’t Know 16. Would it be alright for us to follow-up with you? ❑ Yes ❑ No 17. Please provide your contact details below for a follow-up. Details Name Email Country Phone Number 98 TEMPLATE USED TO GATHER GRANTEE INFORMATION FROM SSTP COUNTRY COORDINATORS Please list all grantees in your country and provide the details specified for each column. This information will be used by members of the Evaluation Team as background information, which can then be followed up by referring to the Grants folders on Dropbox. The email addresses will be used to contact the grantees in relation to the online grantee survey. Many thanks for your assistance. 1. Grantee name 2. Contact name and email address 3. Organizational type: State whether the grantee is a seed company, and/or doing outreach, and/or capacity development, etc. 4. Activity focus, i.e., crops focus (for seed companies) types of outreach (e.g., demonstrations, radio, TV, ICT, etc.) type of capacity development & for whom (e.g., technical and/or business support for seed companies, agro￾dealers, etc.) other activities. e.g., promoting fertilizer, mechanization, etc. 5. Means of Grant Selection: 1=Competitive, in response to EOI/RFP 2=Solicited because they were an AGRA grantee 3=Solicited because of AGRA linkage (but not an AGRA grantee) 4=Solicited through other linkages 5=Sole source 6=Other – please specify 6. Rating of overall performance: State whether very good, OK, underperforming, promising (for those grantees who are still relatively new), etc. 99 ANNEX G. ADDITIONAL DATA TABLES 100 Table G.1. Summary of Data Collection Methods and Actual Sample Sizes Per Country Approach Actors Totals Per Country Ghana Mozambique Tanzania TOTAL Other 44 Key Informant Interviews In-Person Interviews AGRA/SSTP Staff 2 1 2 6 11 USAID 1 1 2 Grantees (Seed production) 4 2 5 11 Grantees (Communications/Capacit y development/Policy) 3 1 2 1 7 Seed companies (non￾grantee) 1 1 Other partners 1 5 6 Government (Policy) 1 1 1 3 Government (Research) 1 1 2 EGS Consultant 1 1 1 3 Seed Association 1 1 2 Agro-Input dealers 5 2 2 9 Seed Out-growers 3 1 4 ICT Challenge Prog. Partner 1 1 1 3 Extension Agents 4 2 6 12 Farmer Beneficiaries 3 10 6 19 Total 31 23 34 7 95 Email/Skype/Telephone Interviews AGRA/SSTP Staff 3 3 USAID 2 2 Seed Experts 2 2 ICT/Communication Experts 1 1 In-country Grantees 1 1 1 3 Total 1 1 9 11 Round Table Discussions Sub-grantee and/or private/public partners 1 1 2 Total 2 Focus Group Discussions Out-growers 2 2 Direct, Indirect & Non￾Beneficiary Farmers 2 2 6 10 Total 12 Online Survey Monkey Grantees 58 Farmer Survey (Wave 1) Direct and Indirect Beneficiary Farmers 1205 1209 1325 3739 44 This includes Nairobi and elsewhere, also multi-country partners. 101 Table G.2. Types of Grantees Interviewed in Each Country Grant type Grantee type Seed & tech Cap Dev Outreach Policy Private Public NG O Intl Total Ghana All grantees working on focal crops 6 1 2 8 1 9 Grantees interviewed 4 1 2 7 7 Mozambiqu e All grantees working on focal crops 2 1 1 1 1 3 Grantees interviewed 2 1 1 1 1 3 Tanzania All grantees working on focal crops 4 2 1 5 1 1 7 Grantees interviewed 3 2 1 4 1 1 6 All 3 countries All grantees working on focal crops 12 1 5 1 14 2 1 2 19 Grantees interviewed 9 1 5 1 12 1 1 2 16 102 Table G.3. List of all SSTP Grantees Organization Country Purpose Status Ethiopian Agricultural Transformation Agency (ATA) Ethiopia Policy Ongoing Aybar Engineering PLC Ethiopia Technology Ongoing Yimam Tesema Seed Enterprise Ethiopia Seed production Ongoing Menagesha Biotech Industry Ethiopia Technology and seed Ongoing Becho Woliso Farmers Union Ethiopia Technology- blended fertilizer Ongoing Sasakawa and MoA Ethiopia Seed technologies - fertilizer Ongoing Amuari PLC Ethiopia Seed production Ongoing NONO PLC Ethiopia Seed production Ongoing Innovations Village Seed Co. Ghana Seed production Completed Josma Agro Industries Ltd. Ghana Seed production Completed Ohumpong investment Company Ltd. Ghana Seed production Ongoing M&B Seeds and Agricultural Services Ghana Limited Ghana seed production Ongoing PEE Farms Ltd. Ghana Seed production Ongoing Modern African Productions (MAP) Ghana Broadcasting -TV Ongoing University of Ghana (WACCI) Ghana Seed production Ongoing AGS & More Resources Ltd. Ghana Agro dealers development Ongoing FARA (ECoSIB) Ghana Seed business entrepreneurship Ongoing Legacy Crop Improvement Centre (LCIC) Ghana Seed production Ongoing Department of Agricultural Research Services, Malawi (DARS) Malawi Seed production Ongoing University of Malawi- Chancellor College: CASSAVA: ADDING VALUE FOR AFRICA PROJECT Malawi Seed production & value addition Ongoing The Story Workshop Educational Trust – phase 1 Malawi Awareness creation - Broadcast Completed The Story Workshop Educational Trust – phase 2 Malawi Awareness creation - Broadcast Ongoing Funwe Farms Malawi Seed production Ongoing Seed Trade Association of Malawi (STAM) Malawi Awareness creation Completed 103 Organization Country Purpose Status DARS (production of breeder of foundation seed) Malawi Seed production Ongoing Mgom'mera Sedds Co. Malawi Seed production Ongoing Global Seeds Malawi Seed production Ongoing Peacock Seeds Malawi Seed production Ongoing Multi Seeds CO Malawi Seed production Ongoing Premium Seeds Malawi Seed production Ongoing Instituto de Investigação Agrária de Moçambique (IIAM) Mozambique Seed production Ongoing Oruwera Limitada Mozambique Seed production Ongoing IFDC Mozambique Mozambique Awareness creation Completed Agrimerc Mozambique Awareness creation Ongoing Companhia De Zembe Lda Mozambique Seed production Ongoing African Fertilizer and Agribusiness Partnership (AFAP) Regional Policy Completed Columbia University in the City of New York Regional Seed related technologies - soil doc Ongoing GROUPE TOOL BAYE AGRO￾INDUSTRIE FARMING PROJECT Senegal Seed production Ongoing Concern Universal (CU) Senegal Awareness creation Ongoing Association Sénégalaise pour la Promotion du Développement à la Base (ASPRODEB) Senegal Seed production Ongoing Coumba Nor Thiam (SCNT) Senegal Seed production Ongoing SEMIS - ENSA Senegal Training Ongoing Syngenta Senegal Seed related technologies - seed treatment Ongoing Resopp Senegal Seed production Ongoing FCI Zanzibar Tanzania Seed production Ongoing SAGCOT NML Tanzania Policy Ongoing DRD-Southern Highlands Zone Agricultural Research Institute (ARI) Uyole Tanzania Seed production Ongoing Sugarcane Research Institute￾Kibaha Tanzania Seed production Ongoing AMINATA Quality Seeds and Consultancy Ltd. Tanzania Seed production Ongoing Crop Bioscience Solutions Limited Tanzania Seed production Ongoing SAGCOT-Mtanga Farms Tanzania Seed production Ongoing CIAT PABRA Tanzania Seed production Ongoing Africasia Tanzania Seed production Ongoing Meru Agro Tours Tanzania Seed production Ongoing 104 Organization Country Purpose Status FIPS - CENTUM Tanzania Capacity of small holder farmers to adopt quality seeds and technologies Ongoing Action for Enterprise Senegal Capacity building technologies Ongoing Market Matters Inc. Regional Awareness creation/seed systems Ongoing BAKO Seeds Co. Ethiopia Seed production Ongoing Morning Star Sesame Ethiopia Seed production Ongoing Avallo PLC Ethiopia Seed production Ongoing Bruckner Farms Ghana Seed production Ongoing Table G.4. Activities Undertaken by Organizational Type of Grantee # Activity Private Sector (N=31) Public Sector (N=10) International Organization (N=5) Non￾Governmental Organization (N=11) Total (N=57) 1 Foundation Seed Production 13 4 0 2 19 2 Certified Seed Production 21 3 1 1 26 4 Awareness￾Raising/Demand￾Creation/Marketing of Seed of Improved Varieties 18 5 3 8 34 3 Awareness￾Raising/Demand￾Creation/Marketing of Other Technologies/Managem ent Practices 12 3 4 7 26 5 Capacity Strengthening (of Farmers, Agro-Input Dealers, Seed Companies, Policy Makers, etc.) 17 6 5 7 35 6 Activities to Promote Policy/Regulatory Development 1 2 1 3 7 7 Other Activities 5 2 0 2 9 TOTAL ACTIVITIES 87 25 14 30 157 Average # activities per grantee 2.8 2.5 2.8 2.7 2.7 105 Table G.5. Examples of SSTP Partnerships 1. Linking input and output value chains: Cassava in Mozambique An example of a partnership promoting value chain linkages is the International Fertilizer Development Center (IFDC) grant for the promotion of quality planting material of higher yielding, disease resistant, cassava in Mozambique. With support from Mozambique's Institute of Agricultural Research, IIAM, (supported previously by AGRA PASS and currently by SSTP in a separate grant) and the International Institute of Tropical Agriculture (IITA), farmers are able to plant and harvest the improved varieties to sell to a Dutch-supported cassava processing company, DADTCO, for processing into cake for sale to CDM (Cervejas de Moçambique) for the brewing of Impala beer. IFDC and DADTCO were previously working together, and SSTP facilitated the partnership with IIAM to allow for the introduction of improved cassava varieties into the value chain. On the policy side, CDM has received a significant reduction in the excise tax for its cassava beer. 2. Supporting seed and other technologies in a pluralistic partnership: Beans in Tanzania Another example is the International Center for Tropical Agriculture (CIAT) grant in Tanzania. In this example, the findings of CIAT’s socio-economic research on bean markets and farmer preferences have been shared with agronomists from the national Agricultural Research Institute (ARI) who have screened and identified appropriate improved bean varieties for commercialization (with support from CIAT PABRA). Breeder seed from ARI is sent to and multiplied by the Agricultural Seed Agency (ASA). ASA and Meru Agro Tours then produce certified seed, which is promoted by CIAT through mother baby demonstrations (which also involve the use of DAP fertilizer and Apron Star seed treatment). CIAT also supports the sale of ASA and Meru Agro Tours certified seed through participating agro-dealers. And lastly, CIAT is partnering with FIPS in the Southern Highlands under their separate SSTP grant to extend mother and baby trials, support QDS production, and promote improved bean varieties. In this example, SSTP facilitated the links between CIAT and Syngenta (who produce Apron Star) and also FIPS. 3. Supporting SSTP grantees to promote certified seed through information technology: Partnering with the ICT Challenge Program in Tanzania A third example of support to varietal promotion is provided through SSTP’s linkage with the ICT Challenge Program in Tanzania, where Farm Radio International is working closely with SSTP grantees, Aminata Seed Company, and Kibaha Sugarcane Research Institute. 4. Supporting a pluralistic public-private sector partnership: ECoSIB in Ghana and FIPS in Tanzania a. Entrepreneurship for Commercial Seed Incubation Business (ECoSIB) brings together a seed sector service team to support nascent commercial seed companies in Ghana. The ECoSIB grant is managed by FARA (Forum for Agricultural Research in Africa) and implemented by Agri-Impact Consult. Key partners include: KNUST (Kwame Nkrumah University of Science and Technology), CRI (Crop Research Institute), and Ghana GLDB (Grain Legume Development Board). Organizational and staff changes within FARA meant that SSTP had to play a key role in ensuring the effective management of this grant, and Agri-Impact has successfully managed the relationships among the various public and private sector partners. b. The goal of the FIPS grant is to scale up a private, farmer-led, extension approach across three districts in Tanzania’s Southern Highlands. SSTP facilitated a partnership between FIPS and Centum Learning, a private company specializing in skills development and vocational training, in developing guidance and training material. FIPS collaborates with and supports the private sector technology companies (seed, fertilizer, crop protection products) and the Ministry of Agriculture Extension Service. As elaborated in Section VII, the approach includes village-based agents (VBAs) linked to and supported by MoA extension, high ratio mother and baby demonstrations (1:100) and QDS bean and cowpea. 106 Table G.6. Tanzania: Number of sample beneficiary farmers who have adopted or applied SSTP-promoted varieties, by crop. Farmer Status Adoption Status Total Pre-SSTP Adopters SSTP Adopters Non Adopters Direct Beneficiary Crop Maize Count 148 171 0 319 % 46.4 53.6 0.0 100.0 Beans Count 156 165 0 321 % 48.6 51.4 0.0 100.0 Irish Potatoes Count 135 175 11 321 % 42.1 54.5 3.4 100.0 Total Count 439 511 11 961 % 45.7 53.2 1.1 100.0 Indirect Beneficiary Crop Maize Count 50 58 11 119 % 42.0 48.7 9.2 100.0 Beans Count 49 59 10 118 % 41.5 50.0 8.5 100.0 Irish Potatoes Count 54 44 21 119 % 45.4 37.0 17.6 100.0 Total Count 153 161 42 356 % 43.0 45.2 11.8 100.0 Total Crop Maize Count 198 229 11 438 % 45.2 52.3% 2.5% 100.0 Beans Count 205 224 10 439 % 46.7 51.0 2.3 100.0 Irish Potatoes Count 189 219 32 440 % 43.0 49.8 7.3 100.0 Total Count 592 672 53 1317 % 45.0 51.0 4.0 100.0 107 Table G.7. Awareness and Use for the Seven Crops of Improved Seed Varieties and Associated Technologies Crop Awareness % (n) Use of % (n) Maize 99.5 (199) 98.5 (197) Pigeon peas 88 (176) 64 (128) Beans 92 (184) 62 (124) Cassava 85.5 (171) 55 (110) Sweet potatoes 97.5 (195) 80.5 (161) Rice 77.5 (155) 24.5 (49) Cowpeas 92.5(185) 56 (112) Source: Kabuli, 2017 108 ANNEX H. INCREASED AVAILABILITY OF SEED OF THE FOCAL CROPS 109 SSTP has supported its seed company grantees to increase the availability (supply) of promoted technologies and especially quality seed of improved varieties to markets in target areas by: 1. Scaling up production (quantities) of seed of existing varieties sold by the grantees for sale to smallholder farmers in target areas, 2. Commercializing developed and released varieties that are not yet in the market or available to farmers by producing quality seed that can be sold to farmers through the grantees and their distribution channel partners (agro-dealers and retailers). The number of improved varieties made available was variable depending on the crop, i.e., cassava, bean, maize, or potato. Most of the varieties being developed by SSTP grantees come either from International Research Centers or from National Agricultural Research Institutes (NARIs). There were none from the private sector, multinational, or local seed companies available for scaling up or commercialization. All had received the necessary release and registration in the country where they were being grown. It is important to keep in mind that while the SSTP program was initiated in 2014, actual seed production by the majority of the seed company grantees started in 2015 or 2016. Thus, available data covers only one or two years of production and consists of mixed varieties even in the same country. Therefore, there are gaps in the data and information provided but general information on quantities produced is summarized in the tables below by crop and class of seed produced. It is worth noting that most of the grants for seed production grants were scheduled to end in early 2017 and unless they are extended, further production of the varieties will be assumed by the seed companies themselves. MAIZE: FOUNDATION SEED PRODUCTION AND DISTRIBUTION The SSTP model for increasing the availability of foundation seed as the starting point for increasing the production and distribution of certified seed recognized that timely availability of foundation seed and quantities available were a major barrier facing local seed companies. Thus, it supported seed companies in Ghana, Malawi, Senegal, and Tanzania to gain access to inbred lines to produce increased quantities of foundation seed for use in further certified seed production. Data on foundation seed produced by SSTP grantees are as indicated in Table G.1. Table H.1. Maize Breeder/Foundation Seed Produced by SSTP Grantees Maize Breeder/Foundation Seed Production (MT) 2014 2015 2016 Ghana Foundation Seed (MT) 9 22 # of Grantees 1 2 Tanzania Foundation Seed (MT) 1.42 11.40 25.26 Sold (MT) 1.94 8.63 Carry over # of Grantees 1 2 2 Senegal Foundation Seed (MT) 8 65 Sold (MT) 43 Carry over 110 Maize Breeder/Foundation Seed Production (MT) 2014 2015 2016 # of Grantees 1 3 Malawi Foundation Seed (MT) 0.30 6.90 Sold (MT) 3.65 Carry over # of Grantees 1 2 Mozambique Foundation Seed (MT) 7.00 30.50 Sold (MT) 30.10 Carry over # of Grantees 1 1 Source: Data compiled by SSTP Country Coordinators and verified against data contained in Ki-Projects Ten grantees in Ghana (2), Senegal (3), Tanzania (2), Mozambique (1), and Malawi (2) were supported to increase the production of foundation seed. Foundation seed production involved the maintenance of parental inbred lines (breeder seed) and multiplication to make it available to foundation seed multipliers. Table G.1 shows that there was an increase in the quantities of foundation seed produced in each of the five countries by these grantees. In Senegal and Malawi, Tanzania, and Mozambique, 59 percent 60 percent, 28 percent, and 80 percent of the seed produced was sold following the second year of production to certified seed producers. Quantities produced in 2016 will be sold in 2017 depending on the timing of the growing season. Unsold quantities require proper storage to ensure quality and viability of the seed. Of the 31 MT produced by the two grantees in Ghana, none had been sold to other seed companies but was available for sale in 2017 growing season. The interviewees indicated that some of the foundation seed produced would be used in-house for the production of certified seed in 2017. All increases in foundation seed availability in the five countries represent the potential for further increased production of foundation and certified seed. It is important to keep in mind that the quantities of foundation seed produced depend on the parental lines being increased and their use either as male of female lines for seed production, and that it takes several seasons to bulk enough material for certified seed production. Foundation seed produced was reported either as pooled or single inbred line data. In either case, information on intended hybrids to be produced from the inbred lines was not provided. Therefore, it is not possible to specify the specific certified seed that will eventually be produced. The seed company producing the certified seed of the desired maize variety will make that determination. It can be argued that the use of public sector institutions to bulk up breeder seed for the production of foundation seed by private seed company grantees was successful in increasing the quantities of available foundation seed produced. However, as indicated above, the quantities of foundation seed produced must be tied to the target quantities of certified seed to be produced in the first year of production and beyond. For example, are the inbred lines being produced the ones that are required for the hybrid(s) to be produced? If the desired hybrid is a three-way hybrid that requires three different inbred lines and one of them is unavailable in sufficient quantities, then it is still not possible to produce certified seeds of that hybrid on time. 111 MAIZE CERTIFIED SEED PRODUCTION AND DISTRIBUTION The constraint (model) being addressed is the production of quality certified maize seed by SSTP seed company grantees for sale to agro-dealers, retailers and smallholder farmers in the countries of their operation. Data on the quantities of certified maize seed produced under SSTP support is presented in Table G.2 below. Scaling up refers to increasing the quantities of seed of maize varieties that are already available on the market; commercialization refers to increasing the quantities of seed of maize varieties that have been released but are not yet available on the market. The bulk of the production occurred in 2015 and 2016. There were 14 SSTP grantees engaged in certified maize seed production. Table H.2. Certified Maize Seed Produced by SSTP Grantees Scaling Up Program Commercialization Program 2014 2015 2016 2014 2015 2016 Ghana Produced (MT) 180 812 564 129 25 Sold (MT) 180 802 548 108 20 Carry over 0 10 16 21 5 # of Varieties 1 5 4 3 2 # of Grantees 1 3 3 2 1 Tanzania Produced (MT) 6 221 3 68 220 Sold (MT) 2 208 63 218 Carry over 4 13 3 5 2 # of Varieties 3 5 3 3 4 # of Grantees 1 2 1 1 2 Ethiopia Produced (MT) 388 757 130 Sold (MT) 328 170 Carry over 60 377 130 # of Varieties 3 3 1 # of Grantees 1 3 1 Senegal Produced (MT) 952 1377 Sold (MT) 952 Carry over 425 # of Varieties 1 4 # of Grantees 1 3 Malawi Produced (MT) 162 481 Sold (MT) 120 240 Carry over 42 241 # of Varieties 3 3 112 Scaling Up Program Commercialization Program 2014 2015 2016 2014 2015 2016 # of Grantees 3 3 Mozambique Produced (MT) 56 Sold (MT) 14 Carry over 42 # of Varieties 1 # of Grantees 1 Source: Data compiled by SSTP Country Coordinators and verified against data contained in Ki-Projects. Under the scaling up program, the production of certified maize seed increased from 2015 to 2016 in all countries except in Ghana, where one variety was dropped from production in 2016 for reasons discussed below under varieties. The quantities of certified maize produced in Tanzania by two SSTP grantees increased from 6 MT in 2015 to 221 MT in 2016. This increase was attributed to one variety (200 MT of Meru HB 513), which was in its first season of production. The increased production in both Ethiopia and Senegal was attributed to the increase in number of grantees producing certified seed from one to three in each country. The number of varieties in production remained the same in Ethiopia and Malawi and increased in Senegal and Tanzania. In Ghana, however, the number of varieties in production fell from five in 2015 to four in 2016 because one of the grantees (M&B) had discontinued the production of one variety because of quality issues with the inbred lines provided by the relevant Research Institute. Another grantee in Ghana, Innovations Village Seed Company (IVSC), reported similar issues with contaminated inbred lines from the National Research Institute, which affected the quantities of certified maize seed produced. These issues were under investigation, but had not been resolved by the time of the evaluation. Grantees in Ghana, Ethiopia, and Malawi reported that most of the certified maize seed produced, 98 percent, 84 percent, and 74 percent respectively, was sold in 2015, with some carry over into the next season. For most grantees, the quantities produced in 2016 will be sold in 2017, although some were sold for planting in 2016. The commercialization program was in effect in four countries – Ethiopia, Ghana, Tanzania, and Mozambique and involved five SSTP Grantees – and resulted in an increase in availability of seeds of improved maize varieties. Certified maize seed production in Ghana and Tanzania resulted in the production of 129 MT in 2015 and 68 MT respectively, of which 84 percent and 93 percent were sold. Two varieties were introduced into the commercialization program in Ghana in 2016 and of the 25 MT produced, 20 MT (80 percent) were sold. In Tanzania, 99 percent of certified seed produced was sold. Ethiopia and Mozambique produced certified maize seed under this program for the first time in 2016 and quantities produced will be sold for planting in 2017. It is important to keep in mind that maize seed produced in one growing season is for planting in the following growing season. In other words, seed produced in 2015 growing season will be planted in 2016 and so on unless the seasons are close together or the crop has a short growing cycle. Seed sales take place in anticipation of the start of the planting season. Thus, seed may be held in inventory by the seed company (carry over) or be moved into the distribution channel in anticipation of sale at the start of the 113 planting season. If a distributor or retailer has paid for the seed to be moved into their warehouse, then that seed may be classified as sold. Seed in inventory from a previous planting season may be sold for planting provided it meets certification quality standards such as percent germination standard. The type of certified maize seed produced by 14 grantees (out of 18) consisted of 27 hybrid maize varieties and 12 open-pollinated varieties (OPV) of maize. Ethiopia and Malawi produced no OPV varieties whereas Senegal produced only OPV maize varieties. Three recent (2016) grantees had not produced any maize seed yet. Certified seed production model through private seed companies has worked well in all the six SSTP countries in increasing the availability of certified maize seed. The challenges going forward will be the availability of improved varieties for an expanding market, the capacity of the seed companies staff and out-growers to manage expanded production, business development and financial services, etc. These are discussed below under barriers to increasing seed production section below. BEAN SEED PRODUCTION Bean production was evaluated in Tanzania and Malawi under the scaling up program and involved three SSTP grantees in Tanzania and three in Malawi (Table G.3). The constraints (models) being addresses were the production of foundation seed and scaling up the production of certified bean seed for sale to smallholder farmers. This required Public-Private Partnership collaboration in Tanzania and production by private seed companies in Malawi. The two models of increasing bean seed production and availability were successful in both Tanzania and Malawi. Table G.3 shows that bean seed production of five varieties increased more than three-fold in Tanzania, and a marginal increase in the production of the two varieties in Malawi. In both countries, sales figures were good for 2015, and the majority of the 2016 production is available for sale in 2017 based in the cropping cycle. As discussed under certified maize production above, bean seed produced in a growing season is for planting in the following growing season, which may be several months, a part. Once produced and processed, it may be held in inventory (carry over) or transferred into the distribution channels to be sold to farmers at the beginning of the next planting season. Table H.3. Bean Seed Produced by SSTP Grantees Scaling Up Bean Seed 2015 2016 Tanzania Produced (MT) 49 156 Sold (MT) 34 50 Carry over 15 106 # of Varieties 5 5 # of Grantees 3 3 Malawi Produced (MT) 95 103 Sold (MT) 95 81 # of Varieties 2 2 # of Grantees 3 3 Source: Data compiled by SSTP Country Coordinators and verified against data contained in Ki-Projects. 114 POTATO SEED PRODUCTION The ET covered two SSTP grantees in Tanzania. The applicable constraint (model) is lack of quality seed potato in adequate quantities for sale to potato farmers. The approach to addressing this constraint was scaling seed production was Certified Seed Production through the private sector. The scaling up of potato production in Tanzania involves two steps of producing potato seed. The first step involves the production of clean planting materials produced by tissue culture method. Tissue culture produced plantlets were imported from Kenya by SAGCOT/Mtanga Farms and grown into minitubers under screen houses. Minitubers were then harvested and planted in the field and multiplied over two seasons into certified seed potato for sale to farmers (Table G.4). Table H.4. Potato Seed Production in Tanzania Scaling Up Seed Potato Scaling Up Potato Minitubers Tanzania 2015 2016 2016 Produced (MT) 493 345 Produced (MT) 94,124 Sold (MT) 130 288 Sold (MT) 0 Used in Demonstration plots 365 Carry over 57 # of Varieties 3 3 # of Varieties 4 Source: Data compiled by SSTP Country Coordinators and verified against data contained in Ki-Projects. Under the SSTP scaling up seed production program, 493 MT of potato seed of three varieties were produced in 2015 and 345 MT in 2016. Of the quantities produced in 2015, 365 MT were provided to farmers along with other inputs for use in demonstration plots and training programs on good agricultural practices, etc. as part of the SSTP grant. The expectation was that farmers would experience the benefits of planting improved potato varieties and the uses of good potato production practices and buy future seed from Mtanga Farms, the grantee. The balance of 130 MT was sold (26 percent) to potato farmers. Of the 345 MT produced in 2016, 288 MT (83 percent) were sold to potato farmers in 2016. The balance of 57 MT is available for sale in 2017. The ET was able to observe some of the 2017 crop in the field. Thus, this program has increased the quantity of potato seed produced and made available to potato farmers in the target areas and farmers are willing to pay for the seed. A second SSTP grantee, Crop Bioscience Solutions Ltd. started a commercial tissue culture company and started to produce potato plantlets and minitubers of four varieties in 2016. A total of 94,124 potato minitubers were produced in 2016 and will be distributed to contracted decentralized potato seed multipliers for multiplication to potato seed in the coming seasons. The success of this multiplication approach could not be assessed because of its short duration. However, given the fact that plantlets can be sold after a short period of time (two months), several cycles can be produced in the course of a year to meet demand for minitubers production. Minitubers planted in the field will increase the quantities of seed potatoes produced over the course of one or two growing seasons thereby increasing the quantity of seed potatoes available for sale to smallholder farmers. A private seed company such as Crop Bioscience Solutions thus has two potential revenue streams, i.e., the sale of plantlets and minitubers. 115 CASSAVA SEED (CUTTINGS) PRODUCTION The production of cassava seed (cuttings or planting material) was evaluated in Mozambique under the scaling up program. Those production figures are provided by Table G.5. The applicable model/constraint being addressed is the production and dissemination of quality cassava planting materials to smallholder farmers. The approach used was an integrated Public-Private Partnership with linkage to the output market – value chain development. The public sector (IIAM) in provided suitable cassava varieties for multiplication by private sector companies (Corredor Agro) working as a contract Hub outgrower and Oruwera Limitada (an SSTP grantee), IFDC distributed cassava cuttings to farmers as well as providing training and other technical assistance as well as facilitating the development of value chain linkages. Smallholder farmers as growers produced cassava root for sale to a processor (DADTCO) of cassava into cassava cake, which was sold to the beer brewing company, CDM. Local cassava varieties are low yielding and susceptible to pest and diseases such as Cassava Mosaic Disease (CMD) and Cassava Brown Streak Disease (CBSD). The government research program aims to breed varieties for different ecological regions with such traits as high yield, CBSD and CMD tolerance/resistance, drought tolerance, quality flour and leaves for consumption, early maturity, etc. The government also wants to expand the use of cassava for industrial purposes such as breweries, and industries where dry matter content, etc. are desired characteristics. A range of varieties for all these applications have been developed by IIAM, and three were identified by IIAM, CDM, and DADTCO as being appropriate for the SSTP program. There were three cassava varieties in the 2015 and six in the 2016 programs. A total of 687,100 cassava cuttings were produced in 2015 of, which 659,840 were sold to contract out-growers for multiplication and/or distributed free of charge to smallholder farmers by the IFDC to promote the new varieties to smallholder farmers in 2016. The quantities of cassava cuttings produced in 2016 increased to 4,342,393 of, which 3,467,390 were sold out-growers for multiplication, farmers for planting, and NGOs for distribution as relief material in response to drought. Other cuttings were used in farmer training programs such as nursery maintenance, etc. Based on the 2015 and 2016 production numbers, the scaling up program had increased the availability of cassava planting materials in target areas. Interviews with farmer groups indicated that there was a shortage of cassava planting materials in 2016 largely due to drought conditions in 2015. Lead farmers or farmer groups were encouraged to multiply cassava-planting materials through rapid multiplication techniques from their own cassava plantings and to distribute them to farmers that had not received the freely distributed materials from IFDC. Needless to say, there were a number of issues associated with the arrangement. Some farmers refused to buy planting materials produced by farmers that had received free materials; others refused to contribute free labor in exchange for planting materials, and lead farmers had failed to sell planting materials to other farmers as a business transaction while waiting compensation from IFDC. This indicates that this type of PPP approach will require better implementation going forward. Furthermore, the closure of the IFDC grant (which ended in February 2017) was of concern to DADTCO who acknowledged their own lack of capacity to assume the role of training, sensitization, and the distribution of planting materials to smallholder farmers. There was lack of recognition that partners in the PPP must plan on self￾sustainability after a project comes to an end. Nonetheless, farmers interviewed were excited about the fact that DADTCO bought all the cassava roots produced and delivered, paid higher prices than market aggregators, and paid in less than four days after delivery. While there were issues of implementation of certain aspects of the PPP approach, such as developing a viable mechanism for farmer multiplication of cassava planting material which is seen as a business by farmers, and delivering planting material and extension services to farmers, linking seed companies and 116 farmers to viable output markets has the potential for increasing use of improved cassava varieties by farmers. The key issue to be addressed going forward is whether the sale of cassava roots, cassava cake, and beer will generate profits for each partner in the value chain to fund variety development, the production and distribution of planting materials, farmer services etc. to ensure the sustainability of the PPP. Cassava production under the commercialization program was initiated in 2016 and was in the multiplication phase to increase the number of planting materials for three new clones of cassava under development. This program will require at least another year of multiplication and selection before sufficient quantities are available for farmer sensitization activities. Table H.5. Production of Cassava Planting Materials in Mozambique Scaling Up Cassava Commercialization of Cassava 2015 2016 2016 Mozambique Produced 687,100 4,342,393 38,300 Sold 659,840 3,467,390 Carry over 27,260 875,003 Multiplication 2,024,500 4,356,050 # of Varieties 3 6 3 # of Grantees 3 3 1 Source: Data compiled by SSTP Country Coordinators and verified against data contained in Ki-Projects. 117 ANNEX I. OTHER FINDINGS 118 Table I.1. SSTP Progress at the End of Year 3 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment Obj. 1: Improved capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers 1-1: Quantity (1,000 MT) of targeted improved seeds produced by enterprises supported by SSTP The cumulative total of 19,321 MT at the end of Y3 is 44 percent of the overall SSTP program target of 43,698 MT. A total of 22,877 MT (52 percent of target) have been committed45 by seed grantees during the lifetime of their grants. The amount committed is just over half of the target figure. New or extended grants before the end of the program might increase the amount, but only slightly. It is likely that the target was set before the program was initiated, and that seed production in the first year was affected by late disbursement of grants and insufficient foundation seeds for bulking up. The target should have been revised at the end of the first year. 1-2: Value (USD) of targeted improved seeds sold by enterprises supported by SSTP No data are reported due to challenges in authenticating data to date. The M&E team is working to verify existing data and to enhance the integrity of future data and will report on this indicator in the next reporting period. Seed companies are reluctant to provide sales and financial information, which they consider confidential, competitive business information. There needs to be agreement on what target value of seeds sold is to be captured i.e., at the distributor/agro-dealer or retailer level, and how discard seed or seed in inventory should be reported. 1-3: Volume (1,000 MT) of targeted improved seeds sold by enterprises supported by SSTP Not included in Y3 report, presumably as for Indicator 1-2 above. As above. 45 “Committed” implies that the grantee has been contracted to reach a particular target and they have indicated that they will do so. 119 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment Outcome 1.1 Improved access to business development and financial services by agro enterprises 1.1-1: Value of new private sector investment in the agriculture sector or food chain leveraged by Feed the Future implementation (RiA) (FTF 4.5.2-38) (New Alliance Indicator) The committed total of $3,006,779 is just over 6 percent of the project target of $50,000,000. SSTP is collaborating with other initiatives to increase this figure. It is unlikely that the target will be reached within the lifetime of the project. The evaluators consider the target to be unrealistic. The target figure of $50 million appeared in the original program document46 which also contained much greater emphasis on working with donors, international and regional groups who were expected to contribute to the investment. 1.1-2: Number of new seed or other technology supply enterprises, seed production, and delivery businesses established, owned, and/or led by women supported through SSTP Project target = 12. At the end of Y3, 11 new women-led enterprises had been established, with a committed total of 12. Target will be met. A measure of how many survive beyond five years would be a good metric to capture. Outcome 1.2 Increased sustainable supply of quality seeds 1.2-1: Number of targeted seed varieties/technologies commercialized with SSTP support SSTP had supported 131 seed varieties and complementary technologies (65 through commercialization; 66 through scaling) by the end of Y3, exceeding the project target of 50. Target exceeded by over 250 percent. However, it cannot be implied that this represents a sustainable supply of seeds unless there is also farmer demand for these varieties. Outcome 1.3 Improved capacity of staff for quality seed production and delivery See Common Indicator C-1 below See Common Indicator C-1 below See Common Indicator C-1 below 46 The Program document refers to “$50m in investment in private sector seed supply.” not “$50m private sector investment” (as in Indicator 1.1-1); the two are not the same thing. 120 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment Outcome 1.4 Improved infrastructure to support quality seed production and delivery 1.4-1: Number of infrastructure and equipment sub-activities completed as a result of SSTP There is no project target, but grantees committed to a total of 80 infrastructure and equipment sub￾activities, of which 36 (45 percent) had been completed at the end of Y3. A budget increase was requested for this activity; if granted, it is likely that this target will be met. 1.4-2: Average distance (km) from farmers to input agro dealers Project target was a reduction in distance from an average of 20 km to 6 km. Survey data collected for the evaluation revealed a reduction in distance from an average of 11.4 km to 4.5 km in Ghana and from 3.0 km to 2.7 km in Tanzania. Given that the current averages are less than the 6 km target, it can be considered that the target has been met for these two countries for which data are available. Outcome 1.5 Increased public-private partnerships 1.5-1: Number of PPPs formed as a result of Feed the Future assistance (FTF 4.5.2-12) Project target = 12. Six PPPs had been formed at the end of Y3, and a total of 12 were committed. It is likely that this target will be met. Obj. 2: Increased use of quality seeds and other technologies by smallholder farmers 2-1: Number of hectares of land under improved technologies or management practices as a result of United States Government (USG) assistance (FTF 4.5.2-2) Project target was 3,129,745 ha; 199,488 ha have been committed by partners; and 667,410 ha had been reached by the end of Y3. Partners have exceeded their commitment by over 300 percent, but the hectares by Y3 end was only 20 percent of target. Target is unrealistically high47 whereas commitment from partners is low. Challenges in measuring this indicator mean that it will be difficult to know whether or not it is reached. 47 The target assumes an average of 4.6 ha per farmer planted to improved technologies or management practices. Even though the measurement system allows for the same land to be double counted if two technologies are used (e.g., improved seed and another associated technology), average farm size in the farmer survey was found to be 6.2 ha for Ghana, 3.8 ha for Mozambique, and 3.3. ha for Tanzania. It is highly unlikely that farmers would use improved technologies on all their land. 121 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment 2-2: Number of farmers and others who have applied improved technologies or management practices as a result of USG assistance (RiA) (WOG) (FTF 4.5.2-5) Project target = 684,000; Partners committed to 576,909 and had reached 489,00948 (85 percent of commitment and 72 percent of target) by Y3 end. It is likely that this target will be met. 2-3: Number of private enterprises, producers organizations, water users associations, women’s groups, trade and business associations, and community-based organizations (CBOs) that applied improved technologies or management practices as a result of USG assistance (FTF 4.5.2-42) Project target = 250; Partners committed to 9849 and had reached 465 by Y3 end. Target exceeded by over 185 percent by end of Y3. Outcome 2.1 Improved smallholder farmers’ access to knowledge and skills 2.1-1: Number of SSTP extension service events completed Project target = 90,000. Partners committed to 134,221, and had reached 246,837 by Y3 end. See Table H.2 below for breakdown of different extension products. Target exceeded by approximately 275 percent by end of Y3. Target was underestimated, especially for radio episodes and demo plots. 2.1-2: Number of farmers and others participating in SSTP supported extension services Project target = 66,000. Partners committed to 58,300 and had reached 762,297 by Y3 end. Target exceeded by over 1,000 percent by end of Y3. Target was vastly underestimated by project and partners alike, which is surprising, given that this is a scaling-up project.50 48 This figure excludes Ethiopia where the data were collected late. 49 Partner commitment was based on the partners at the time, and then new partners came on board. 50 It would appear that “scaling” with the SSTP program is used to refer only to the number of varieties promoted and the quantities of seed produced, not the numbers of farmers reached. 122 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment Outcome 2.2 Increased capacity of farmer organizations to support use of quality seeds and technologies See Common Indicator C-1 below See Common Indicator C-1 below See Common Indicator C-1 below Outcome 2.3 Increased use of ICT enabled extension services by smallholder farmers 2.3-1: Percent of targeted smallholder farmers using ICT-enabled services To be reported by ICT Challenge Fund, not SSTP 2.3-2: Percent of targeted smallholder farmers with access to ICT-enabled services To be reported by ICT Challenge Fund, not SSTP 2.3-3: Percent of smallholder farmers stating that their practices changed as a result of the ICT enabled extension services To be reported by ICT Challenge Fund, not SSTP Obj. 3: Improved regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers 3-1: Number of agricultural and nutritional enabling environment policies completing the following processes/steps of development as a result of USG assistance in each case: 1. Analysis 2. Stakeholder consultation/public debate 3. Drafting or revision 4. Approval (legislative or regulatory) Full and effective implementation (FTF 4.5.1-24) No target had been set. By end of Y3, a total of 73 policies were in the process of development – see Table H.3 below for a breakdown of the stages reached. This is considered to be an impressive amount of policy implementation, though the measurement indicators do not adequately reflect the amount of policy work achieved because they assume that there is a policy vacuum. In fact, policies exist in most countries, but have yet to be implemented. SSTP is heavily involved in supporting policy implementation. Outcome 3.1 Increased consultation by private and public sector groups 3.1-1: Number of stakeholder policy/regulatory events supported through SSTP efforts Project target = 14. Partners have committed to 61, and 4 had been supported by Y3 end. It is likely that this target will be met. 123 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment on key seed production and delivery policy and regulatory issues 3.1-2: Number of stakeholders participating in policy and regulatory events supported through SSTP efforts Project target = 280. Partners have committed to 300, and 87 had participated by Y3 end. It is likely that this target will be met. Outcome 3.2 Increased coordination and harmonization of country and regional-level seed production and delivery 3.2-1: Number of national governments and regional entities supported to improve policies and regulations Not reported in Y3 Annual Report. PIRS document states that targets will be set through Grantee work plans. Not reported in Y3 Annual Report. 3.2-2: Number of New Alliance Cooperation Framework policy commitments implemented with support from SSTP (New Alliance) Not reported in Y3 Annual Report. PIRS document states that targets will be established by January 2015. Not reported in Y3 Annual Report. Outcome 3.3 Increased capacity of producer and consumer associations to engage in seed sector policy formulation and implementation See Common Indicator C-1 below See Common Indicator C-1 below See Common Indicator C-1 below Outcome 3.4 Increased capacity of the public and private sector to monitor and regulate seed production and delivery 3.4-1: Number of public and private sector seed institutions accredited through support by SSTP Not reported in Y3 Annual Report. PIRS document states that targets will be established by January 2015. Not reported in Y3 Annual Report. 3.4-2: Number of public and private sector seed/crop inspectors licensed through support by SSTP Not reported in Y3 Annual Report. PIRS document states that targets will be established by January 2015. Not reported in Y3 Annual Report. Outcome 3.5 Increased self￾monitoring, measurement, TBD Not reported in Y3 Annual Report. Not reported in Y3 Annual Report. 124 Objective/Outcome Performance Indicator Target and Achievement at the End of Year 3 (as reported in Y3 Annual Report) Evaluators’ Comments and Assessment documentation, and communication of progress by SSTP focus country governments Various: Outcomes 1.3, 2.2, 3.3, 3.4 (as indicated above): C-1: Number of individuals who have received USG supported agricultural sector productivity or food security training (FTF 4.5.2-7) Project target = 40,644. Partners committed to 26,551, and 98,366 individuals had been supported by Y3 end. Target has been exceeded by almost 250 percent. Many grantees are using demonstration plots to train farmers, which was not originally envisioned. Various: Objectives 1, 2, 3 C-2: Number of members of producer organizations and community based organizations receiving USG assistance (FTF 4.5.2-27) Not reported in Y3 Annual Report. PIRS document states that targets will be set through Grantee work plans. Not reported in Y3 Annual Report. Various: Objectives 1, 2, 3 C-3: Number of individuals in the agriculture value chain directly impacted by SSTP Project target = 7,600,000. Partners committed to 4,683,435, and 7,173,910 individuals were directly impacted by Y3 end. Target is likely to be met by the end of the project. Color-coding for evaluator’s assessment: Likely that target will be exceeded, met, or met within 10 percent by program end. Unlikely the target will be met by program end; in most cases, target unrealistically high. 125 Table I.2. Extension Related Events (2.1-1) Indicator Cumulative Total Committed Total Project Target Number of SSTP extension service events completed Demonstration plots 59,706 49,693 6,000 Field days/Seed fairs 350 922 120 Radio episodes aired 543 1,456 30 TV episodes aired 40 93 20 Video programs developed Information materials distributed - 85 12 134,221 246,837 90,000 Table I.3. Policy Efforts Supported with USG Assistance (3-1) Indicator Cumulative total Committed total Project target Number of agricultural and nutritional enabling environment policies completing the following processes/steps of development as a result of USG assistance 73 - - By Process/Step Analysis 68 - - Stakeholder consultation/public debate 2 - - Drafting or revision 3 - - Approval (legislative or regulatory) 0 - - Full and Effective implementation 0 - - 126 Table I.4. Assessment of Progress Towards Anticipated Results, as Summarized in SSTP Program Document Anticipated Results for SSTP Activities (taken from the SSTP Program Document) Evaluators’ Assessment of Implementation and Results Activity 1: Develop “Road Maps” of specific public and private sector actions needed Results for Activity 1: Detailed, country-specific plans that map how Partnership objectives will be met within the context of CIPs leading to a greater awareness by government and donors of the way third-parties view their efforts. AGRA will also complete assessments of potential scalability of key technologies. Road Maps were completed for each country and provided a clear plan for subsequent SSTP activities. Key stakeholders met by the ET were all familiar with the Road Maps, which proved to be effective in guiding SSTP’s activities in each country. Activity 2: Coordinate and align efforts among public and private sector actors and donors Anticipate results for Activity 2: Outputs for Activity 2 will include: convening regular meetings on key input sector issues; holding broader events; planning meetings; and workshops with key stakeholders to ensure strong coordination. The impact of Activity 2 will be a much better coordinated input sector system and stronger relationships (and trust) among the stakeholders. Coordination activities took different forms in the different countries. In Ghana, regular meetings for SSTP partners and stakeholders took place to encourage coordination. In Tanzania, some of the different SSTP grantees were linked into the same value chain, thus promoting alignment and coordination through the value chain. In all countries visited, the SSTP Country Team was seen to be effective in building a sense of trust and collective effort among the stakeholders. Activity 3: Provide technical support locally for building capacity of public and private actors Anticipated results for Activity 3: As a result of technical assistance and procurement, this Activity will lead to: significant improvement in gender equitable distribution and use of improved inputs within AGRA’s target countries; greater capacity of input suppliers and government agencies to lead the evolving input market systems; and at least six new, successful private sector led initiatives in input systems in each country. Improvement in input use among female and male farmers is presented in Section 6. With the exception of one grant in Ghana, it is not clear how SSTP has increased the capacity of agro input dealers; this was not a major focus of the SSTP Program. It is not known whether six new, successful private sector led initiatives are in place in each country; this is not an indicator that is being tracked. A total of 12 new women-led enterprises are expected to be established by the end of the program. Activity 4: Model and pilot priority activities through grant-making Anticipated results for Activity 4: Twenty new technologies and methods identified and models/pilots implemented each year of the Partnership. At least 50 percent of these considered successful and scaling initiatives initiated. At least 50 percent of model/pilot beneficiaries are women. By the end of Year 3, 131 seed varieties/ technologies had been commercialized. The total number of farmers participating in extension services was 762,297 by end Y3. Although this figure is not broken down by gender, it is very likely that at least 50 percent of these are women. 127 Anticipated Results for SSTP Activities (taken from the SSTP Program Document) Evaluators’ Assessment of Implementation and Results Activity 5: Provide regional technical support for finding solutions to cross-cutting issues Anticipated results for Activity 5: Regional technical support and regional dialogue will: accelerate learning and action on key bottlenecks to scaling seed; leverage investment from additional donors and governments; and help to achieve consensus on effective solutions. Regional facilitation by AGRA on scaling seed supply will lead to stronger capacity of regionally important organizations and partners, contributing to a more harmonized regional input platform. Regional support has focused on Eastern and Southern Africa because there is another USAID grant to support the West African seed program. Prior to SSTP, AGRA had limited activities at the regional level so this represents an expanding space for AGRA. Work with COMESA has focused on the seed sector and fertilizer trade harmonization, leading to additional support from IFDC. Activity 6: Monitor and benchmark progress toward goals Anticipated results for Activity 6: Clear data, analysis, and understanding of progress in each country toward achieving CIPs; Stronger capacity to collect and analyze data; and contributions toward a coordinated industry system for input supply and demand data collection. SSTP’s M&E system includes over 20 different indicators which must be measured and reported. It has been a challenge to get private sector grantees to provide the necessary data. Within the target countries, challenges still remain in determining national seed demand, making it difficult for seed companies to plan their production. The capacity of the government and National Seed Associations to compile and analyze data varies across the countries visited. Activity 7: Address constraints to regional harmonization Anticipated results for Activity 7: Identification and publication of obstacles to regional harmonization and their insertion into the national debate of recalcitrant countries; Alleviation of constraints that negatively impact cross-border movement of seed and other agricultural inputs in New Alliance countries; and Improved regional relationships and coordination to address issues in the future. The ET is not aware of the publication of obstacles to regional harmonization, though SSTP is actively setting the agenda for national debate in each country. Different countries have different attitudes to the cross-border movement of seed and other inputs. Improved regional relationships have been supported in Eastern and Southern Africa. 128 ANNEX J. FARMER SURVEY REPORT – WAVE 1 08 Fall 129 SSTP SUPPLEMENT REPORT SCALING SEEDS AND TECHNOLOGIES PARTNERSHIP (SSTP) PROGRAM: MID-TERM EVALUATION FARMER SURVEY REPORT WAVE 1 September 22, 2017 This publication was produced at the request of the United States Agency for International Development under USAID Contract Number AID-OAA-A-13-00040. It was prepared independently by: Prepared by: Florence Nyangara, Ph.D. Senior Evaluation Specialist Program Evaluation for Effectiveness & Learning (PEEL) Project Contractor: Program Evaluation for Effectiveness and Learning (PEEL) ME&A 1020 19th Street NW, Suite 875 Washington, DC 20036 Tel: 240-762-6296 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 130 BACKGROUND The Scaling Seeds and Technologies Partnership (SSTP) is a five-year program implemented by the Alliance for a Green Revolution in Africa (AGRA) in Ethiopia, Ghana, Malawi, Mozambique, Senegal, and Tanzania between July 2013 and July 2018). Originally, SSTP was designed to contribute to commitments agreed upon within the New Alliance for Food Security and Nutrition, which was established in 2012. The SSTP aims to achieve the following three main objectives: 1. To improve capacity of public and private sector groups to deliver quality seeds and other technologies to smallholder farmers; 2. To increase the use of quality seeds and other technologies by smallholder farmers; and 3. To improve regional and country level policy and regulation mechanisms for the production and delivery of quality seeds and technologies to smallholder farmers. To achieve these objectives, the SSTP employs a number of key strategies including: 1. Developing “Road Maps” of specific public and private sector actions needed; 2. Coordinating and aligning efforts among public and private sector actors and donors; 3. Providing technical support locally for building the capacity of public and private actors; 4. Modeling and piloting priority activities through grant-making; 5. Providing regional technical support for finding solutions to cross-cutting issues; 6. Monitoring and benchmarking progress toward goals; and 7. Addressing constraints to regional harmonization. Overall, the program is to create a better-aligned and coordinated set of donor and government actions, which will leverage greater private sector investment in marketing of improved seeds and other technologies. In conjunction, AGRA convenes international and regional groups involved in developing seed systems to discuss specific challenges and solutions, and to gather regional-level momentum. SSTP PROGRAM EFFORTS TO INCREASE USE OF IMPROVED SEEDS AND OTHER TECHNOLOGIES To encourage and increase the use of improved seeds and other technologies, SSTP employed a number of innovative approaches including: 1. Foundation Seed Production 2. Certified Seed Production 3. Awareness-Raising/Demand-Creation/Marketing of Seed of Improved Varieties 4. Awareness-Raising/Demand-Creation/Marketing of Other Technologies/Management Practices 5. Strengthening Capacity of Farmers, Agro-Input Dealers, Seed Companies, and Policy Makers, etc. 6. Activities to Promote Policy/Regulatory Development 7. Other Specific Activities in each Country such as: a) Involving highly specialized and experienced organization that apply or adapt tried-and￾tested approaches developed and honed over many years of practical implementation. b) Combining key elements of traditional outreach and private sector marketing strategies so they operate at scale. c) Using media communications such as Farm Radio International (FRI) in Tanzania. FRI is supported by the ICT Extension Challenge Fund, and works with two SSTP grantees and agricultural experts. Together, they deliver technically sophisticated, integrated 131 radio, short message service (SMS), and interactive voice response (IVR) services based on accurate agricultural information fine-tuned to farmers’ interests and real-time seasonal changes. OVERALL EVALUATION In October of 2016, an evaluation team—with support from the USAID/Bureau of Food Security (BFS)—was tasked to work through the PEEL project to design and implement a mid-term performance evaluation of the SSTP project. The aim of the mid-term evaluation was to help inform programming and policy decisions by assessing different aspects of the SSTP program, including management, strategies, and achievements to-date. The evaluation used a mixed methods approach to collect data in three countries, Ghana, Mozambique, and Tanzania. Methods included documentation review, interviews and focus group discussions, compiling seed production and sales data for focal crops, an online questionnaire for grantees, and a farmer panel survey. Triangulation analyses of the data from different sources were conducted to address the evaluation questions (Final SSTP Performance Evaluation Report, September 2017). THE FARMER SURVEY Part of the overall mid-term performance evaluation of the SSTP program involved a survey of smallholder farmers in the program areas in three countries. The farmer survey aimed to provide information helpful in assessing the Program’s progress and achievements made to date towards increasing use of improved seeds and other technologies by smallholder farmers. Data from the survey data were supplemental to data from other sources, including qualitative data conducted for the mid￾term evaluation of the SSTP. SURVEY OBJECTIVES The key objective of the SSTP farmers’ survey was assessing the rates at which smallholder farmers in Ghana, Mozambique, and Tanzania adopted improved seed technologies for four focal crops—beans, cassava, Irish potatoes, and maize. The findings from this survey will inform future programming decisions on how to increase adoption of improved seeds. Specifically, the farmer survey aims to address three key evaluation questions: 1. Use of quality seed and improved technologies by farmers. To what extent have AGRA and its sub￾grantees increased the use of quality seeds and improved technologies by smallholder beneficiary farmers? By indirect beneficiary farmers? 2. Encouraging adoption vs. application51 by different farmer types. What types of strategies, innovations, or support have been most effective in encouraging use of improved seeds and technologies for just one season? Over multiple seasons (adoption vs. application)? What important barriers remain by: a) Direct project beneficiaries? b) Indirect beneficiaries (defined as, those in “catchment areas” of activities who aren’t directly targeted by the activity)? c) Early adopters? Majority adopters? Late adopters? d) The poor, women, and socially marginalized groups? 51 The term “application” as used in this report, refers to farmers’ testing of or “trying out” an improved seed variety or technology for just one season before deciding whether or not to plant it again the following season. 132 e) What other types of approaches and methodologies could the Partnership incorporate in its projects to encourage indirect beneficiaries to both try out and continue to use new agricultural technologies and practices? 3. Take-off points: At what level of market penetration of potential market demand for targeted technologies is there likely to be spontaneous adoption (over 2+ growing seasons) by indirect beneficiaries (e.g., what is the take-off point at which technologies could “go viral”)? Between March and May of 2017, the PEEL project worked with two survey organizations to implement this round of the farmer survey (Wave 1). In Tanzania, the organization was Economic and Social Research Foundation (ESRF). In Ghana and Mozambique PEEL worked with Kantar Public. The follow-on survey (Wave 2) will be conducted around the same period one year later—2018. This report presents the findings from Wave 1 of the survey. SURVEY METHODOLOGY The SSTP farmer survey is designed as a panel survey and involved two rounds of data collected one year apart. Wave 1 data was collected in 2017 and Wave 2 data will be collected in 2018. To maintain accuracy, data will be collected from the same set of sampled smallholder farmers for two groups – direct and indirect beneficiaries in the program areas. STUDY SITES AND TARGET POPULATION The study sites were defined by location. SSTP grantees implement their program activities for the focal crops in Ghana, Mozambique, and Tanzania, and are given funds to implement their program in specific regions, districts, and communities based on the local of their agro-dealers and network of partners for each crop. Table J.1. SSTP Program Areas by Focal Crop and Improved Seed Varieties in Ghana, Mozambique, and Tanzania Country (Crop) SSTP Grantees Working to Increase Use of Improved Seeds and Technologies SSTP Program Regions by Focal Crops (as reported by grantees) SSTP* Improved Seed Varieties by Focal Crop Ghana (Maize) Bruckner, IVSC, LCIC, M&B, and PEE FARMS LTD Ashanti, Brong Ahafo, Central, Eastern, and Volta Abontem (Aburo kokoo), Aseda, Mamaba, Obatampa (40 days), Omankwa, Opeaburoo, and Tintim. Mozambique (Cassava) Instituto de Investigacao Agraria de Mocambique (IIAM), Oruwera limitada, and International Fertilizer Development Center (IFDC) Limpopo and Nacala corridors Orera, Mokhalana, Eyope, clone 170, Chinhembwe, Colicanana, Liconde, and Varuiya Tanzania (Beans) AMINATA, ARI – Uyole, and International Centre for Tropical Agriculture (CIAT) Arusha, Coast, Iringa, Njombe, Manyara, Mbeya, Morogoro, Rukwa, Katavi, Kilimanjaro, Ruvuma, and Tanga Lyamungo/Rosekoko/Nyayo/Kite nge (Lyamungo 90), Njano/Njano ndefu/Njano dume (Uyole Njano), Pundamilia, Kombati, Iringa ndefu (Jesca), Kalima (Calima Uyole), and Njano Uyole (Uyole 96) 133 Country (Crop) SSTP Grantees Working to Increase Use of Improved Seeds and Technologies SSTP Program Regions by Focal Crops (as reported by grantees) SSTP* Improved Seed Varieties by Focal Crop Tanzania (Irish Potatoes) SAGCOT – MTANGA and Crop Bioscience Solutions (CBS) Arusha, Njombe, Manyara, and Kilimanjaro. Asante, Meru, Sherekea, Tengeru, and Obama (Cyangi) Tanzania (Maize) AMINATA, ARI – Uyole, and Meru Agro-Tours and Consultants Co. Ltd (MATCC) Arusha, Coast, Iringa, Njombe, Manyara, Mbeya, Morogoro, Rukwa, Katavi, Ruvuma, and Tanga NATA/Lulu (NATA H104, NATA H105, NATA K6Q), Nyati (Meru HB 515), Ngamia (Meru HB513), Boko (Meru HB 623), Uyole 615 (UH615), and Uyole 03 (UH6303). * Source: Information on SSTP Improved Varieties was provided by SSTP in-country teams and was validated with each grantee and sub-grantee and during fieldwork. The target population for this survey is smallholder farmers who are direct or indirect SSTP program beneficiaries for each focal crop. Direct beneficiaries in this survey are smallholder farmers who have come into direct contact with SSTP grantee activities promoting improved seed or planting materials adoption, or adoption of other related technologies for beans, cassava, Irish potatoes, and maize. Indirect beneficiaries are smallholder farmers who live in the program area. Indirect beneficiaries have not had direct contact with SSTP grantee activities, but, through sources like other farmers, agro-input dealers, or suppliers not associated with the SSTP program, have learned about, used, or adopted improved seed, planting materials or other related technologies for the focal crop. See Annex D for additional information on farmer beneficiary definitions and how they were identified. SAMPLE CALCULATION AND SIZE The sample size used in this survey was calculated to detect a 10-percentage point difference in farmer adoption of improved seeds of each focal crop between the two groups—direct and indirect beneficiaries—with 80 percent power and alpha of 0.05. Though, this is a beneficiary survey, it has similar components and complexity to most surveys, and sample size calculations account for a design effect of about 1.40 based on estimates from other population-based surveys conducted in Ghana. Also, because this survey will be repeated within a year, the sample calculation for beneficiaries considered the non-response rate of five percent for the first-round survey, and a dropout rate of about 10 percent a year later. The sample survey will be used to estimate statistical differences between different groups of farmers (beneficiary status, sex, district, and socio-economic status) in use and adoption of improved seed technologies. The procedures used in the survey have been applied before in other farmer household surveys to calculate and determine the total sample size of farmer beneficiaries (Feed the Future Guide, 2016 https://agrilinks.org/sites/default/files/resource/files/Sampling-Guide-Beneficiary￾Based-Surveys-Feb122016.pdf). The sample size calculation is based on two key indicators—percent of farmers using improved seed technologies, and the number of hectares under SSTP grantee-supported improved seed technologies. This sampling methodology in the first-survey was expected to yield the required total sample size of 600 direct beneficiaries and 600 indirect beneficiaries for a total of 1,200 per country (Ghana, Mozambique, and Tanzania). The sample size estimation for Tanzania was calculated in a slightly different way because there were three focal crops (beans, Irish potatoes, and maize). To avoid double counting 134 farmers growing more than one of these crops in the sample, the three crops were each allotted to a region where SSTP varieties for a specific focal crop are commonly grown. For each crop in each region, a minimum of 300 direct and 100 indirect beneficiary farmer interviews were expected. SAMPLING STRATEGY The survey sample is designed to provide estimates of smallholder farmer indicators measuring the extent of use and adoption of improved seeds for the selected focal crops (beans, cassava, Irish potatoes, and maize) in the program areas within Ghana, Mozambique, and Tanzania. This section describes the sampling design, the procedures used at each stage of the sample selection, and determination. The beneficiary sample selection followed a multistage cluster sampling strategy described below: Stage 1: Selection of regions or provinces The regions were selected using a non-probabilistic purposive sampling strategy. Initially, the research team worked with SSTP country teams in Ghana, Mozambique, and Tanzania to obtain a complete list of the SSTP grantees for each focal crop. To increase the likelihood of finding a large enough sample of eligible farmers exposed to program activities, regions with the highest concentration of SSTP activities were selected from the list. Also, to minimize field costs and other logistical challenges (including the possibility of not finding SSTP direct beneficiaries) the research team purposely selected and focused on 2-3 regions or provinces per country. In Ghana, the focus was on Ashanti, Brong Ahafo, and Central regions. In Mozambique, Inhambane and Nampula provinces were the focus. In Tanzania, the selected regions were Arusha, Manyara, and Njombe. In Tanzania, the three regions were selected based on where each of the three focal crops (beans, Irish potatoes, and maize) are supported by SSTP programs. Arusha (beans), Njombe (Irish potatoes), and Manyara (maize). Stage 2: Selection of communities/villages The communities or villages selected were ones where the SSTP grantees are implementing activities relating to the focal crops within the selected regions. For each focal crop, the grantees provided a list of all the communities where they implement SSTP activities. The research team validated and revised the lists with grantees. The lists then provided the sampling frames for selecting communities for each crop, region, and country. Before the survey, the research team randomly ordered all the communities in the sampling frame by region, and then randomly selected 10 or 15 communities per region to total 30 communities per country. Four or five communities were selected to serve as back-ups, or replacements. A few communities were replaced when, on further investigation and consultation with the agro-dealers, they were found to be non-SSTP, and therefore ineligible. A total of 42 communities were included in this survey in Ghana. In Mozambique, 30 communities, evenly distributed between the two provinces of Nampula and Inhambane, were chosen. In Tanzania, the 30 selected communities were evenly distributed across three regions—Arusha had bean farmers, Njombe had Irish potatoes farmers, and Manyara had maize farmers. 135 Table J.2. Selected Communities for the Farmer Survey by Region and Country Country (Crop) Selected Regions Selected Communities Ghana (Maize) Ashanti Hiawoanwu, Frante, Yabraso-Aframso, Ejura Nkwanta, Teacher krom, Kobriti, Mmesuo-Drumakumah, Sekyeredumase, Akomadan (Nkramomu), and Atobiase Brong Ahafo, Swanshi-Jato Zongo, Amantin, Dobidi Nkwanta-Fante Newtown, Asanti boa-Busunya, Bamiri-Oforikrom, Kokroko￾Aworowa, Kuntunso-Tanoso, Awisa, Nsoko, and Nkrankrom Central regions Dominase-Ojobi, Akwakwaa-Mankrong, Osae Krodua-Krodua, Mfafo-Obrachire, Denkyera Oboasi, Assin Sienchem, Assin Gangan, Assin Bediadua (Bungalo), Assin Nsuta, and Agona Nkum Mozambique (Cassava) Limpopo corridor (Inhambane) and Sizal, Conjo, Benzala, Munguambe, Mwengue, Nharrumbo, Nhambudoz, Cambine, Mindu, Nhambele, Helena, Dongane, Sengane, Inhacoongo, and Bule Nacala corridor (Nampula) provinces Carrupeia, Namalili, Bueto, Mucuili-unidade Mocambique, Tiponha Nachaca, Nihessiue Muihia, Nacuca, Chilapane, Nathuko, Namatoro, 1 de Maio, Mariri, Nhamiconha, Caiaia, and Nacuatane Tanzania (Maize) Arusha Karatu District Council: Slahamo, Kainam, Rhotia Kati, K/Tembo, G/Arusha, and Endamarariek Arumeru DC District: Lemanyata, Imbibia, Patandi, and Nasholi Tanzania (Beans) Manyara Hanang District Council: Endasaki and Maeskron Babati District Council: Riroda Mbulu District Council: Labay, Dongobeshi, Bashay, Kainamu, Basonyangwe, Harar, and Ngwandaqw Tanzania (Irish Potatoes) Njombe Njombe Town Council: Magoda, Kisilo, Lugenge, and Utengule Njombe District Council: Matembwe and Ikuna Wanging'ombe District Council: Ujindile, Utelewe, Usalule, and Igima Stage 3: Selection of direct and indirect farmers During fieldwork, data collectors conducted a listing exercise for all eligible farmers in each selected community. The listing exercise involved administering a pre-survey questionnaire (page 207) aimed at comprehensively inventorying all eligible farmers within the sampled community before randomly selecting qualifying farmers. Tablets were used to capture Global Positioning System (GPS) coordinates of the selected villages and of the respective households with eligible farmers. The listing of eligible farmers was conducted a day before the interviews. During the listing, the data collectors visited households and asked to speak to a farmer who makes decisions related to the focal crops. After he or she consented, the farmer was asked whether they grow the focal crop. If their answer was yes, they were asked follow-up questions about whether they had heard of improved seeds and other technologies. If the answer was yes, they were asked whether they had attended or participated in any activities promoting any of the improved seed technologies. If the farmer said yes again, that farmer was eligible to be included as a “direct or indirect beneficiary” for the study. The qualified farmers were informed that they were within the targeted group and were asked if they were available and willing to answer more questions about improved seeds. If they were, they were, their name and contact information was added to list for possible random selection. If the farmer was 136 selected, the data collector contacted the farmer to schedule an interview. Farmers who didn’t reply after three attempts at contact were dropped from the list and replaced. The lists of all eligible farmers in each community, both direct and indirect beneficiaries, were submitted electronically (synchronization) to a central server. Once the listing data were submitted, the survey manager centrally analyzed and developed two-parallel lists based on the farmers’ responses to the screening questions. This determined whether the farmer was included as a direct or indirect beneficiary. The two lists served as sampling frames for each community. A total of 1,200 randomly selected farmers per country were expected. A total of 40 farmers per community was necessary to achieve the required sample size of 1,200 from the 30 randomly selected communities. Therefore, from each developed sampling frame, 20-22 direct and 20-22 indirect beneficiaries were randomly selected from each corresponding list. This made a total of 40-44 farmers for interviews. Up to four extra farmers were selected in each community to serve as replacements. The final sample selection and distribution of farmers for each country and crop is summarized below: • Ghana: In total, 3,201 farmers were listed within the 42 selected communities across the regions of Ashanti, Brong Ahafo, and Central. The survey sample was equally divided among the three regions (200 direct and 200 indirect), for a total of 600 direct and 600 indirect beneficiaries. A final sample size of 1,205 farmers were interviewed for the main survey. • Mozambique: In total, 3,047 farmers were listed in the 30 randomly selected communities. There were 15 communities in Inhambane province, and 15 in Nampula province. A total of 600 direct and 600 indirect beneficiaries were expected. A final sample of 1,209 farmer interviews were completed. • Tanzania: In total, 3,600 farmers were listed in all of the 30 selected SSTP communities and villages. For each community, 44 farmers were randomly sampled for interviews. However, the distribution of the selected farmers differed from the other two countries because there were three crops. Of the expected 1,200 farmers, 900 were direct beneficiaries (300 for each crop) and 300 were indirect. Therefore, out of 44, 32 direct SSTP beneficiary farmers were chosen, and twelve 12 indirect SSTP farmer beneficiaries were selected for each community and for each crop. A total of 1,325 interviews were completed in the three Tanzanian regions. Table J.3 shows the overall sample of smallholder farmers included in the survey (N=3,839), of whom 2,172 were direct and 1,567 were indirect beneficiaries across the three countries. The survey sample included 2,091 males (54.5 percent) versus 1,748 females (45.5 percent). Table J.3. Distribution of Final Sample of Farmers Interviewed by Beneficiary Status, Sex, and Country Country (Crop) Direct Beneficiaries Indirect Beneficiaries Total Male Female Total Male Female Total Male Female Total Ghana Maize N 371 233 604 267 334 601 638 567 1205 % 61.4 38.6 100.0 44.4 55.6 100.0 53.0 47.1 100.0 Mozambique 137 Country (Crop) Direct Beneficiaries Indirect Beneficiaries Total Male Female Total Male Female Total Male Female Total Cassava N 279 328 607 213 389 602 492 717 1209 % 46.0 54.0 100.0 35.4 64.6 100.0 40.7 59.3 100.0 Tanzania Maize N 247 72 319 82 38 120 329 110 439 % 77.4 22.6 100.0 68.3 31.7 100.0 74.9 25.1 100.0 Beans N 211 110 321 79 40 119 290 150 440 % 65.7 34.3 100.0 66.4 33.6 100.0 65.9 34.1 100.0 Irish Potatoes N 188 133 321 54 71 125 242 204 446 % 58.6 41.4 100.0 43.2 56.8 100.0 54.3 45.7 100.0 Total N 646 315 961 215 149 364 861 464 1325 % 67.2 32.8 100.0 59.1 40.9 100.0 65.0 35.0 100.0 Grand Total N 1296 876 2172 695 872 1567 1991 1748 3739 % 59.7 40.3 100.0 44.4 55.6 100.0 53.2 46.8 100.0 Data Collection Procedures Questionnaire development and translation. The main instrument was a farmer survey questionnaire and a pre-screening questionnaire. The main survey questionnaire was developed in collaboration with the evaluation team using existing questions from adoption studies that were modified to be specific to the SSTP program activities, country, and targeted crops. The survey included: five subsections; background characteristics of respondent farmers; general exposure to ANY improved seed varieties and other technologies; application of SSTP promoted improved seed varieties of the focal crops; adoption of SSTP improved seed varieties; and any discontinuation of tried SSTP-promoted varieties (see Annex J.1). The questionnaire was developed in English and translated into the local languages for the selected areas of the survey and took respondents an average of 30–45 minutes to complete. In Ghana, the questionnaire was translated into Twi, in Mozambique into Shangana and Emakwa, and in Tanzania it was translated into Kiswahili. Data entry program design. Because of their expertise in using computer-assisted personal interviewing (CAPI) to collect, process, and manage data, the survey companies ESRF (in Tanzania) and Kantar Public (in Ghana and Mozambique) were selected to conduct the survey. The data managers initially programmed the questionnaire in Census and Survey Processing System (CSPro) software installed in hand held digital tablets for CAPI. The program and the questionnaire were adapted for each crop, pilot tested, and then loaded into the tablets. The data entry program included automated checks for quality control. Recruitment and training of data collection teams. Data collectors and supervisors with diverse backgrounds and, at minimum, a college degree were recruited and trained to work as data collectors in each country. The training was five days long, and provided trainees with comprehensive knowledge about the survey and how they should implement it in the field. The training covered how to obtain 138 consent of each respondent before administering the survey, and included two days to pilot-test and refine the survey tools, and familiarize the trainees with the survey tool using CAPI. The pilot tests were conducted in local languages and in communities where SSTP grantees work, however, they were not included in the study sample. Fieldwork. Data collection for the farmer survey took place between March 16 and April 6, 2017, in Tanzania, and between March 31 and May 5, 2017, in Ghana. In Mozambique, data collection was between April 10 and May 18, 2017. A quality control supervisor was responsible for overseeing the overall data collection process in each country and for communicating weekly with the principal investigator. Data quality procedures. The field team implemented and adhered to strict quality control procedures throughout preparation and implementation of the fieldwork. The field teams used tablets equipped with navigational GPS that captured the coordinates of selected households and communities. Furthermore, CAPI allowed for in-built quality checks and real-time monitoring of the fieldwork to track data collection progress. All captured interviews were reviewed daily by supervisors who checked for completeness and anomalies in the data. They also gave the interviewers feedback. The country survey manager produced weekly status reports to document the progress and to describe to the principal investigator any problems that had arisen. Data processing and cleaning. The field teams entered data for each community (synchronized) and submitted the data, which were automatically uploaded to an established central office server. Data processors accessed the data daily to ensure that all coding and cleaning was done to established data quality standards. Upon completion of fieldwork, the responses were translated from the local language in to English, crop variety names (both local and official) were validated, and they were checked for logical errors. Some of the responses, mainly from the open-ended questions and answers, were coded. Data analysis and triangulation. STATA software was used to analyze the farmer survey data collected from all three countries to address evaluation questions 3.1 and 3.2. Triangulation of the survey data and data collected by the evaluation team were conducted to help explain or interpret some of the findings. The survey analysis includes descriptive statistics of key variables and standard errors, confidence intervals, and cross-tabulations for key outcome indicators for the adoption and application of new seed varieties. As appropriate, simple regression analysis determined key contributors to the application and adoption of improved varieties. In the second round of survey in 2018, the two surveys will be matched, and, to determine the change of adoption rate over a year’s time, difference-in￾differences group analysis will be undertaken. Ethical Considerations Before the start of the survey, the evaluation team submitted a comprehensive SSTP Evaluation Protocol package with all survey instruments and interview guides for ethical review and approval by ICF’s Institutional Review Board (IRB). The full study protocol and instruments were also submitted to local ethics boards in each of the study countries, and reviews were waived. Permission letters to conduct the evaluation study in each country were obtained from local authorities. Survey Limitations The survey methodology used in this study has a number of limitations that should be considered when interpreting the data, including the following: 139 1. Lack of comprehensive lists providing complete sampling frames for selecting beneficiaries. Some of the grantees and their sub-grantees do not keep updated lists, or are unwilling to share their beneficiary lists. To address this limitation, only communities in regions with program activities were included in the sampling frame. 2. The survey focused on direct and indirect beneficiaries of the smallholder farmers in limited sites within the SSTP program communities, which may be systematically different than other farmers in the general population. Thus, the findings from these sites cannot be generalized across the entire population of smallholder farmers. 3. During fieldwork, some of the selected program communities were found to be too small in population to yield enough sample size of farmers required. As a result, neighboring or adjacent communities were annexed, which may have introduced some selection bias. 4. The SSTP program supports about 17 crops across six countries. Hence, the four selected crops and three countries are not necessarily representative of the 17 crops and associated technologies included under the SSTP program. 5. Given the different approaches and expected outcomes by each program grantee, results may be hard to compare across grantees. 6. Wave 1 utilized a post-test study design, hence, round 1 does not generally allow for analysis of change over time. In a year’s time the changes over two time periods will be analyzed. Findings In this section, farmer survey findings are presented by the evaluation question addressed by the survey as part of the SSTP mid-term program performance evaluation. It should be noted that the SSTP program implementation began in 2014, and this study provides a snapshot of its current status after three years of program introduction and its advances to full implementation. In one year, the findings will be used as the benchmark against which SSTP program progress and achievements will be assessed to show change over time. Evaluation Question 3.1. Use of Quality Seed and Improved Technologies by Farmers Use of quality seed and improved technologies by farmers: To what extent have AGRA and its sub-grantees been able to increase the use of quality seeds and improved technologies by smallholder beneficiary farmers? By indirect beneficiary farmers? Overall Levels of Using ANY Improved Seeds and Other Improved Technologies The SSTP program is a scale-up program that aims to support grantees already producing and selling improved seed varieties so their production can increase (in terms of volume, quantity, and quality). The Program also helps grantees scale up distribution to reach more farmers than they could with their own resources. It is worth noting that some of the SSTP grantees may have also received support from other funders, hence, it was not possible to determine the direct effects of the SSTP activities. As such, the researchers have tried to systematically sort through the adoption process and determine what changes the SSTP supported activities contributed to the adoption of four focal crops in Ghana, Mozambique, and Tanzania. Overall, a majority of smallholder farmers in both Ghana (90.0 percent) and Tanzania (94.8 percent) have tried at least one or more of any improved seed or other technologies in their plots. In Mozambique, only 42.1 percent of farmer respondents said that they have tried improved seeds or other technologies. The results were not surprising since, compared to cassava in Mozambique, both Ghana and Tanzania have a long history of promoting improved agricultural technologies, especially for maize and beans. 140 Figure J.1. Percentage of Farmers Who Have Used ANY Improved Seeds and Other Technologies by Beneficiary Status by Country 0 20 40 60 80 100 Ghana Mozambique Tanzania Direct Indirect Total Table J.4 shows the specific types of improved agricultural technologies farmers have tried in their plots, including improved seeds, improved farm management practices, fertilizer applications, other agro￾chemicals, and post-harvest processing and storage technologies. The results show that there is extensive use of improved technologies in Ghana and Tanzania. In Ghana, a higher proportion of farmers have tried other improved technologies like improved farm management practices (74.2 percent), agro￾chemicals (69.3 percent), and fertilizer application (61.5 percent) compared to those that have used improved seed varieties on their plots (53.7 percent). In Tanzania, a higher proportion of farmers have tried improved seeds (95.1 percent), followed by fertilizer users (65.1 percent), agro-chemicals (52.2 percent), and management practices (40.0 percent). In Mozambique, about 57.2 percent of farmers have tried improved seeds, 34.4 percent tried farm management practices, 33.9 percent tried fertilizer, and 18.3 percent tried agro-chemicals. Across all countries very few farmers tried post-harvest processing and storage technologies. As expected, compared to indirect beneficiaries, a higher proportion of direct beneficiaries in all countries tried improved seeds and other technologies in their plots. Table J.4. Percentage of Farmers who have tried ANY Improved Seeds or other Technologies by Beneficiary Status and by Country Type of Improved Technology Ghana Mozambique Tanzania Direct (N=604) Indirect (N=601) Total (N=1205) Direct (N=607) Indirect (N=602) Total (N=1209) Direct (N=961) Indirect (N=364) Total (N=1325) Improved seed varieties N 365 218 583 216 75 291 909 286 1195 % 62.9 43.2 53.7 57.8 55.6 57.2 96 92.6 95.1 Farm management practices N 442 363 805 149 26 175 449 54 503 % 76.2 71.9 74.2 39.8 19.3 34.4 47.4 17.5 40.0 Fertilizer applications N 387 280 667 137 31 168 628 190 818 % 66.7 55.4 61.5 36.6 23 33 66.3 61.5 65.1 Other agro￾chemical use N 414 338 752 67 26 93 487 169 656 % 71.4 66.9 69.3 17.9 19.3 18.3 51.4 54.7 52.2 N 94 40 134 47 13 60 171 26 197 141 Type of Improved Technology Ghana Mozambique Tanzania Direct (N=604) Indirect (N=601) Total (N=1205) Direct (N=607) Indirect (N=602) Total (N=1209) Direct (N=961) Indirect (N=364) Total (N=1325) Post-harvest processing/ storage % 16.2 7.9 12.4 12.6 9.6 11.8 18.1 8.4 15.7 Farmer Adoption of Improved Seed Varieties To estimate the extent to which SSTP grantee activities were able to increase farmers’ motivation to use improved seeds, we used four measures as proxies including: 1. Change in the proportion of farmers that started using improved seeds of the focal crop within the past three years (SSTP program period). It is expected that SSTP supported grantees will intensify their promotion and distribution activities and, as a result, more farmers will become aware and start using improved seed varieties on their plots. 2. Increased amount of improved seeds/planting materials planted (kilograms) from the first-time it was tried and last season (2016). Usually farmers use small amounts of new seed varieties as a trial package for the first time, and as they get motivated to adopt, they start acquiring and using more for themselves (adoption) 3. Reduced distance between farmer's home and where the farmers obtain the improved seed (Kilometres) for first-time and last season planted. This measure assumes that SSTP grantees establish distribution outlets of improved seeds in the target communities and make them accessible to the farmer; and 4. Increased land size (acres) where improved seed varieties were planted for the first-time and last season the farmers used them. Typically, before farmers adopt a new variety, they first try the improved seed in a small plot before expanding to more acreage in later seasons. In addition, direct and indirect beneficiaries will be used as two comparison groups in all these measures. These measures will also be tracked in Wave 2 to assess progress overtime. Usage Levels of Improved Seed Varieties for the Focal Crops Over Time Figure J.2 shows the proportion of farmers in all countries combined using improved seed varieties by season (year) when first planted for the four focal crops. Figure J.2. Percentage of Farmers Using Improved Seeds by Beneficiary Status by First Season Used in all Three Countries 142 The highest proportion of farmers reported starting to use improved seeds more than three years ago (pre-SSTP period). Then, the proportion declined before it started increasing again within the last season (SSTP implementation period). A higher proportion of SSTP direct beneficiaries than indirect beneficiaries tried improved seed varieties in the past two seasons. Progress in the Use of Improved Seed Varieties Table J.5, below, shows the average amount of SSTP seed varieties planted per farmer in the first and last season (2016) they were planted across countries. On average, farmers used a larger amount of SSTP-supported improved seeds in the last season compared to the first. Specifically, the average amount increased in Ghana and Mozambique from 12.8 to 18.7 and 263.6 to 561.7 kilograms respectively. Between the first season and last season planted in Tanzania, the amount range increased from 17.5 to 21.1 for maize, from 32 to 33.8 for beans, and 119.9 to 179 kilograms for Irish potatoes. Direct beneficiaries in all countries also had a larger increase compared to indirect beneficiaries. These findings are in line with the adoption process, where farmers test small packages of seed the first time they plant before committing larger plot sizes and more seeds in later seasons. Table J.5. Average Amount of SSTP Improved Seed Varieties Planted (Kilograms) by Country, Crop, and Farmer Beneficiary Status Country Crop Direct Beneficiaries Indirect Beneficiaries All Beneficiaries Season First Planted Last Season Planted (2016) Season First Planted Last Season Planted (2016) Season First Planted Last Season Planted (2016) Ghana Maize 13.2 19.4 12.2 17.6 12.8 18.7 Mozambique Cassava 300.9 487.3 55.8 976.3 263.6 561.7 Tanzania Maize 18.1 21.9 15.9 18.9 17.5 21.1 Beans 33.3 35.7 28.2 27.9 32 33.8 Irish Potatoes 125.2 183.5 103.1 164.8 119.9 179 Table J.6, below, shows the average size of land where improved seed varieties were planted last season. Table J.6. Average Plot Sizes (in Acres) Planted to SSTP Varieties in the Last Season (2016) Country Crop Direct Beneficiaries Indirect Beneficiaries All Beneficiaries Ghana Maize 2.35 1.92 2.30 Mozambique Cassava 9.48 1.67 8.47 Tanzania Maize 2.54 3.60 2.64 Beans 1.66 1.09 1.52 Irish Potatoes 0.93 0.61 0.91 143 It’s a snapshot of the variations between direct and indirect beneficiary groups. The results show that compared to indirect beneficiaries, direct beneficiaries have slightly larger land size under SSTP￾supported seed varieties across the different crops. The largest difference is with cassava in Mozambique. It is not clear what accounts for the large differences, but it may be that cassava was recently introduced in Mozambique compared to other crops like maize and beans, which have been promoted for a long time. Table J.7, below, shows, by beneficiary status and crop, the average kilometer distance between the farmers’ home and where they obtained the seed they planted the first-time and the last season. The results show that, on average, the distance the farmers travel to acquire improved seeds has decreased significantly between the first-time used and last season (2016). The results also indicate that, in Mozambique, the improved cassava planting materials were obtained within or around the community. This finding is in line with the strategy used in Mozambique to produce and distribute cassava planting materials through farmer associations. Table J.7. Average Distance (in kilometers) Between Farmer's Home and Where They Obtained Seed Planted by Season, Crop, and Program Beneficiary Status by Country Conclusion Overall the results show that SSTP program interventions are having an effect on the application and adoption of improved seed varieties in the three surveyed countries. Consistently, a higher proportion of direct beneficiaries of the SSTP program have tried (and some have adopted) improved seeds compared to indirect beneficiaries. The results also show that, on average, the amount of improved seeds planted and the size of plots where improved seeds were planted have increased. Also, the average distance to where farmers obtained improved seeds for all focal crops has been reduced within the past three years. The observed differences can partly be explained by the efforts currently promoted by SSTP-grantees through farmer associations, agro-input dealers, demonstration plots, and agricultural extension agents to promote and distribute improved seed varieties and make them available in the communities for easy accessibility to the farmers. Recommendation Although, the program is making progress, overall use of improved seeds is still low. To reach more farmers, we recommend increasing support by grantees on farmer-managed demonstrations and agricultural extension services. Country Crop Direct Beneficiaries Indirect Beneficiaries All Beneficiaries Season First Planted Last Season Planted (2016) Season First Planted Last Season Planted (2016)) Season First Planted Last Season Planted (2016) Ghana Maize 6.70 3.50 10.00 4.20 8.00 3.80 Mozambique Cassava 0.50 0.11 0.82 0.98 0.55 0.24 Tanzania Maize 2.76 2.37 3.57 3.06 2.96 2.54 Beans 2.41 2.12 1.58 0.94 2.21 1.82 Irish Potatoes 1.24 0.95 1.11 0.14 1.21 0.75 144 Evaluation Question 3.2: Encouraging Adoption Versus Application by Different Farmer Types Encouraging adoption vs. application52 by different farmer types: What types of strategies, innovations, or support have been the most effective in encouraging use of improved seeds and technologies for just one season? Over multiple seasons (adoption vs. application)? What other types of approaches and methodologies could the Partnership incorporate in its projects to encourage indirect beneficiaries to both try out and continue to use new agricultural technologies and practices? What important barriers remain for: Direct project beneficiaries? Indirect beneficiaries (defined as, those in “catchment areas” of activities who aren’t directly targeted by the activity)? Early adopters? Majority adopters? Late adopters? The poor, women, and socially marginalized groups? Encouraging Adoption Versus Application by Different Farmer Types The farmers were categorized into three groups based on whether, and when, regardless of their beneficiary status, they tried any of the improved seed varieties for the first time. The groups are: 1) Pre￾SSTP Adopters: all those farmers (direct or indirect beneficiaries) that started using improved seeds more than three years ago; 2) SSTP Adopters: all farmers who started using improved seeds two or three years ago or last season (during SSTP implementation period – from 2014 to 2016); and 3) Non-Adopters: all farmers who have never used improved seeds. Figure J.3 shows the proportion of farmers who have used improved seeds in all countries by crop, maize, beans, cassava, and Irish Potatoes, and by adoption group as a proxy to estimate the effects of SSTP on adoption. Figure J.3. Percentage of Farmers by Adoption Status by Crop 0 20 40 60 80 Pre-SSTP Adopters SSTP Adopters Non Adopters Ghana (Maize) Mozambique (Cassava) Tanzania (Maize) Tanzania (Beans) Tanzania (Potatoes) The results indicate that the proportion of farmers who adopted during the SSTP period was high for all three crops in Tanzania and for cassava in Mozambique compared to pre-SSTP type. However, for Ghana there was very little difference between the two groups. The percentage of non-adopters in Mozambique was very high. Most of the increase observed for the SSTP adopters can be attributed to SSTP program efforts. Based on this finding, we can argue that the SSTP program is making deep in￾roads in Mozambique and Tanzania. 52 The term “application” as used in this report, refers to farmers’ testing of or “trying out” an improved seed variety or technology for just one season before deciding whether or not to plant it again the following season. 145 Factors Influencing the Application and Adoption of Improved Seeds and Technologies The SSTP program started implementing activities in 2014. Direct beneficiaries are farmers who have had contact with activities implemented by SSTP. Indirect beneficiaries are those living in the same program communities who have learned about improved seeds and technologies from other sources. It is expected that more direct beneficiaries will apply and finally adopt the improved seeds than indirect beneficiary farmers. It is widely acknowledged that a number of socioeconomic factors play an important part in influencing farmers’ decisions on application and adoption of new technologies. The factors include, among others: gender; level of education; social and cultural status (tribe, caste, community associations; poverty; access to information; access to land; and the number of household members (dependents). Poverty is known to be negatively associated with adoption of improved seed varieties because cost for the farmers is prohibitive. In this survey, poverty was defined in terms of the average household land size. Farmers were categorized into the following three “levels:” ● ● ● “Poor:” those with two acres or less; “Medium:” those with greater than two and less than or equal to five acres; and “Rich:” those with more than five acres of household land size. Ownership of a radio and/or a television set was also used as a proxy for measuring farmers’ access to information related to agricultural technologies. The respondents were asked if they had someone in their household with a radio (yes/no), television (yes/no), and mobile phone (yes/no). Ownership of mobile phones was dropped from further analysis because almost all the households and people interviewed across the three countries had a mobile phone. For this evaluation, the socially marginalized farmers were defined as those in households where no family member belongs to an organized group like a farmer based organization, association, or other community based group. The outcome measures for this evaluation were the use of improved seeds of any crops and the use of SSTP varieties. The main strategies used by SSTP-grantees to promote, distribute, and motivate farmers to acquire and use the improved seed varieties were also examined. The SSTP grantee strategies include demonstration plots, government extension agents, agro-dealers, community events, radio and TV programs, ICT messages, and, as in Mozambique, work through farmer associations. Table J.8, below, shows the important factors motivating farmers to use the improved seed varieties for maize, beans, cassava, and Irish potatoes. The findings indicate that some of the strategies used by SSTP grantees have increased farmer motivation to start using improved seeds on their plots. Across the three countries, the highest motivating factor was other farmers—neighbors or relatives who used improved seeds. This broke down to 52.2 percent in Ghana, 33.1percent in Mozambique, and 60.8 percent in Tanzania. The next three important motivating factors by country include learning though government agricultural extension agents. In Ghana, the break down was 21.2 percent through extension agents, 6.3 percent through radio programs, and 3.5 percent though agro-dealers. In Mozambique, the next three most important motivating factors were demonstration plots at 28.8 percent, government extension agents at 12.3 percent, and community events promoting agricultural technologies at 9.3 percent. 146 Table J.8. Main Motivation Factor That Persuaded the Farmer to Plant the Improved Variety for the First Time by Country Source of Information Ghana Mozambique Tanzania Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Farmer-managed demonstration N 13 3 16 56 1 57 117 0 117 % 3.0 1.1 2.2 21.9 2.2 18.9 12.3 0.0 9.3 Seed company/Project￾managed demo/Agro￾dealer N 26 10 36 25 3 28 73 14 87 % 6.0 3.6 5.1 9.8 6.5 9.3 7.7 4.5 6.9 Neighbor/Relative N 187 185 372 68 32 100 495 273 768 % 43.4 65.8 52.2 26.6 69.6 33.1 52.1 86.9 60.8 Field Days/Agricultural show N 13 2 15 6 1 7 9 2 11 % 3.0 0.7 2.1 2.3 2.2 2.3 0.9 0.6 0.9 Community events N 9 1 10 26 2 28 89 11 100 % 2.1 0.4 1.4 10.2 4.3 9.3 9.4 3.5 7.9 Government extension agent/VBA N 122 32 154 41 1 42 86 8 94 % 28.3 11.4 21.6 16.0 2.2 13.9 9.1 2.5 7.4 Farmer training course N 1 1 23 2 25 36 0 36 % 0.2 0.0 0.1 9.0 4.3 8.3 3.8 0.0 2.8 Radio program N 22 23 45 1 0 1 6 1 7 % 5.1 8.2 6.3 0.4 0.0 0.3 0.6 0.3 0.6 Agriculture Organization/TV/Messag e on Farm N 14 4 18 0 0 0 1 0 1 % 3.2 1.4 2.5 0.0 0.0 0.0 0.1 0.0 0.1 Others N 24 21 45 10 4 14 38 5 43 % 5.6 7.5 6.3 3.9 8.7 4.6 4.0 1.6 3.4 Total N 431 281 712 256 46 302 950 314 1264 % 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 147 In Tanzania, farmer demonstration plots were the next most important factor at 11.1percent. This was followed by community events promoting agricultural technologies at 7.9 percent, and government extension agents at 6.3 percent. Although important factors vary by country, the influence of demonstration plots and government extension agents are important across all countries. The results show that, particularly in Mozambique, the farmer-managed demonstration strategy, the farmer training courses, and other strategies engaging farmers as agents of change have motivated more farmers to use improved cassava planting materials. However, the remaining strategies, including use of agro-dealers, community events, and agricultural extension agents, only have moderate effects. This could be due to the strategy used in Mozambique— distributing cassava-planting sticks through farmer associations, which requires farmers to belong to an association to access the planting materials from other farmer members of the association. Whereas in other countries and crops, farmers can choose where to obtain the improved seeds they plant mainly through formal outlets. In comparing direct and indirect beneficiaries, the results show that other farmers or relatives had the most influence on encouraging indirect beneficiary farmers to use improved seeds—more so than for direct beneficiaries across all three countries. In Ghana, the radio programs were an important factor encouraging indirect beneficiaries to use improved maize seed varieties. The findings show that farmer￾based organizations, agro-dealers, and government extension agent services were important factors for direct beneficiaries, where their influence on indirect beneficiaries was less. These findings are a reflection of the efforts SSTP has made using these strategies to promote and distribute the improved seeds. Gender Variation in Adoption Status of Improved Seed Varieties Figure J.4 shows that in Tanzania, regardless of their program beneficiary status, more male farmers than female farmers were adopting improved seed varieties. Figure J.4. Percentage of Farmers Using SSTP Improved Seeds by Sex, Beneficiary Status, and Country 148 However, for Ghana, there was only a small difference between males and females in the use of improved maize seeds. While in Mozambique, compared to males, more female farmers (59.3 percent) were adopting the improved planting material for cassava. However, when all other factors were constant, there were no significant differences between females and males in the use of improved seeds in Ghana and Tanzania. In Mozambique, though, there was a marginally significant difference between females and males (P=0.05). Overall, female farmers, regardless of their beneficiary status, were significantly less likely to use improved seeds compared to male farmers. Application and Adoption of Improved Seed Varieties by Social Economic Status It is widely acknowledged that poverty and social marginalization of farmers have negative effects on adoption of improved seed varieties. In this survey, poverty was defined in terms of the average household land size as explained earlier (poor, medium, and rich), while, social marginalization was defined as those farmers in households without membership in an organized social group (i.e., religious, farmer-based, associations, or community-based group). Figure J.5 shows that across the three countries, poverty was an important factor in adoption. When everything else is held constant, poverty was an important factor in Ghana and Tanzania. In Tanzania, controlling for all other factors, only land size was an important key predictor of using improved seeds. In Ghana, when everything else is held constant, educational level was a marginal predictor of using improved maize seeds. In Mozambique, additional factors like farmer literacy level, household size, and radio and television ownership were significantly associated with improved cassava seed use. Social marginalization was not an important predictor of using improved seeds in any of the three countries. Figure J.5. Household Land Size by Use of Improved Seeds by Beneficiary Status by Country 0 10 20 30 40 50 60 Direct Indirect Total Direct Indirect Total Direct Indirect Total Ghana N= 1,205 Mozambique N= 1,209 Tanzania N= 1,325 <=2 acres >2<=5 acres > 5 acres Further analysis on the key predictors of farmer’s use of improved seed varieties for each country show that, across all three countries, the beneficiary status of farmers continued to be an important predictor of improved seeds use—even when all other factors are constant. In Tanzania, direct beneficiaries of the SSTP program interventions were about 12 times (11.6 times) more likely to use improved seeds 149 compared to indirect beneficiaries. Compared to indirect beneficiaries in Ghana and Mozambique, direct beneficiaries were, respectively, 4.3 and 5.6 times more likely to be appliers or adopters. Barriers to Access of Improved Seed Varieties SSTP program efforts are aimed at reducing and eliminating barriers to improved seed variety access. Efforts are made through supporting grantees to increase production and distribution of seeds they are producing. The findings from this study show that the ways farmers acquire the SSTP seed varieties varies across the three countries and by crop (Tables J.9 and J.10). Table J.9 shows that in both Ghana and Tanzania a majority of farmers purchased the seeds at full cost. In Mozambique, almost all of the farmers interviewed had obtained their improved cassava planting materials for free. In general, seed acquired for free is usually a gift from another farmer or is a trial package—such as the potato seed supplied through the SSTP Program. The SSTP grantees provided potato seed to farmers for free in the first year of the grant. In subsequent years farmers were expected to purchase the seed. The success of this approach will be explored with the next wave survey. Table J.9. Mode Used by Farmers to Acquire the Improved Seeds When They First Planted Them in Their Plots by Beneficiary Status by Country Source of Seeds (Q402) #/% Ghana Mozambique Tanzania Purchased at full cost N 441 5 1087 % 61.9 1.7 86.0 Purchased at reduced cost N 29 9 22 % 4.1 3.0 1.7 Got it for free N 229 287 139 % 32.2 95.0 11.0 In-kind or any exchange N 13 1 16 % 1.8 0.3 1.3 Total N 712 302 1264 % 100.0 100.0 100.0 Table J.10, below, shows the various sources where farmers obtained seeds for the first time. In Ghana and Tanzania, many obtained their seeds from agro-dealers (30.8 percent and 49.8 percent respectively), while market traders were a source for about 15 percent of farmers in both countries. In Mozambique, the largest source of seeds for farmers was farmer-based organizations (43.7 percent). Table J.10. Sources of Improved Seed That Farmers Used in the First-Time Planted by Country and Crop Source of Seeds (q403) #/% Ghana Mozambique Tanzania Direct Indirect Total Direct Indirect Total Direct Indirect Total FBO/NGO N 22 2 24 123 9 132 42 2 44 % 5.1 0.7 3.4 48 19.6 43.7 4.4 0.6 3.5 Seed company/ Agro-input N 128 91 219 20 4 24 492 137 629 % 29.7 32.4 30.8 7.8 8.7 7.9 51.8 43.6 49.8 150 Source of Seeds (q403) #/% Ghana Mozambique Tanzania Direct Indirect Total Direct Indirect Total Direct Indirect Total Govt. extension agent/VBA N 115 36 151 70 2 72 64 6 70 % 26.7 12.8 21.2 27.3 4.3 23.8 6.7 1.9 5.5 Other farmer or relative N 140 102 242 71 32 103 299 136 435 % 32.5 36.3 34 27.7 69.6 34.1 31.5 43.3 34.4 Market trader N 46 63 109 0 1 1 141 53 194 % 10.7 22.4 15.3 0 2.2 0.3 14.8 16.9 15.3 Total N 431 281 712 256 46 302 950 314 1264 % 100 100 100 100 100 100 100 100 100 Table J.11, below, indicates the sources of improved seeds that farmers planted last season. Results show that most farmers used self-saved seeds for planting last season in all countries (Ghana = 51.1 percent; Mozambique = 36.4 percent, and Tanzania= 36.0 percent). Other key sources for improved seeds in Tanzania were agro-dealers (44.5 percent) and other farmers or relatives (17.1 percent). In Mozambique, most farmers continue to get their seed from farmer based organizations and NGOs (28.8 percent), government extension agent (20.2 percent), and other farmers or relatives (25.2 percent). Table J.11. Sources of the Improved Seeds That Farmers Planted Last Season by Country and Crop Source of Improved Seeds (q410) #/% Ghana Mozambique Tanzania Direct Indirect Total Direct Indirect Total Direct Indirect Total Self-saved seed N 209 155 364 89 21 110 327 132 459 % 48.5 55.2 51.1 34.8 45.7 36.4 34.4 42.0 36.3 Farmer Based Organization/NGOs N 13 0 13 84 3 87 34 0 34 % 3.0 0.0 1.8 32.8 6.5 28.8 3.6 0.0 2.7 Agro-input dealer N 81 48 129 6 2 8 441 122 563 % 18.8 17.1 18.1 2.3 4.3 2.6 46.4 38.9 44.5 Gov’t extension agent/Village based agent N 52 17 69 59 2 61 35 3 38 % 12.1 6.0 9.7 23.0 4.3 20.2 3.7 1.0 3.0 Other farmer or relative N 57 32 89 52 24 76 152 64 216 % 13.2 11.4 12.5 20.3 52.2 25.2 16.0 20.4 17.1 Purchased in Market N 37 48 85 0 1 1 109 33 142 % 8.6 17.1 11.9 0.0 2.2 0.3 11.5 10.5 11.2 Others N 2 0 2 2 0 2 15 3 18 % 0.5 0.0 0.3 0.8 0.0 0.7 1.6 1.0 1.4 Total N 431 281 712 256 46 302 950 314 1264 % 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 151 Table J.12, below, shows that only a small number of farmers have tried and abandoned any SSTP improved seed variety (N=332) across the three countries. Most of these are bean appliers in Tanzania and maize appliers in Ghana. Further investigation shows that in Ghana, most of the farmers abandoned seeds after planting them for one or two season/s (61 percent). Also, there was no difference by how they were acquired (46.4 percent of farmers got them free and 45.6 percent purchased the seeds at full cost). In addition, the abandoned seed varieties in Ghana were acquired within the community. Table J.12. Number of Farmers That Have Tried and Abandoned SSTP Improved Seed Varieties in their farm by Beneficiary Status and Country Crops Ghana Mozambique Tanzania Total Crop Maize Cassava Beans Irish Potatoes Maize Direct beneficiaries 67 38 103 26 19 253 Indirect beneficiaries 41 5 27 3 3 79 Total 108 43 130 29 22 332 Table J.13, below, shows the main reasons for abandonment by country. In Ghana, the topmost include storage, marketability, yield, and cost. In Mozambique, it was marketability, labor, taste, and cost. In Tanzania, cost and marketability were the top reasons for farmers to abandon the seed varieties they had used. All of these reasons were followed by weather, and availability of alternative varieties. Table J.13. Main Reasons for Abandoning Improved Seed Varieties by Country Country Top Four Reasons for Abandonment of improved Seeds Ghana (N=237) 1. Harvested grain has short storage duration (N=37) 2. Harvested grain was not marketable (N=35) 3. Variety has low yield (N=30) 4. Seeds are expensive to purchase (N=28) Mozambique (N=43) 1. Crop not marketable (N=8) 2. Crop requires a lot of labour (N=7) 3. Farmer doesn’t taste (N=6) 4. Planting material are expensive to purchase (N=4) Tanzania (N=181) 1. Seeds are expensive to purchase (N=62) 2. Harvested crop are not marketable (N=51) 3. Weather variability (N=40) 4. Availability of alternative variety (N=19) Tanzania – Maize(N=22) 1. Weather vulnerability (N=7) 2. Availability of alternatives (N=5) 3. Others (N=5) 4. Expensive to purchase (N=4) Tanzania – Beans(N=130) 1. Expensive to purchase (N=46) 2. Others (N=38) 3. Not marketable (N=33) 4. Weather vulnerability (N=30) Tanzania – Irish Potatoes (N=29) 1. Not marketable (N=17) 2. Expensive to purchase (N=12) 3. Availability of alternatives (N=3) 4. Weather vulnerability (N=3) 152 Conclusion In sum, the survey shows varying reasons for abandoning improved seeds. The most common reasons across the three countries are the prohibitive costs of purchasing improved seeds and the lack of markets for the produce. The results also show that most farmers are influenced and motivated to adopt by neighbors and relative farmers across the three countries. Investments in commercial advertisement strategies such as ICT messages, and media or radio/TV programs showed little effect in motivating farmers, perhaps because seeing other farmers’ practices was more accessibly practical. Recommendations Based on the survey results, in order to increase affordability and accessibility of improved seeds for sustainable agriculture, the partnership and its grantees should find creative ways of working directly with farmers’ social networks to influence use and adoption of improved seeds. This could involve working to promote and demonstrate the improved varieties by working with cooperatives, farmer based organizations, through community events, or peer-groups of other farmers. 153 ANNEX J.1. SSTP FARMERS SURVEY QUESTIONNAIRE Consent form read and accepted: Yes SECTION 0: FARMER IDENTIFICATION FARMER NAME: ________________________________________ Phone Number: ____________________________________ House No./Address: _________________________________________ FARMER ID (Unique Study ID): ____________________________ Farmer Status: 1. Direct Beneficiary 2. Indirect Beneficiary INTERVIEW INFORMATION DATE: …..…/………/…..…. INTERVIEWER NAME: _______________________________________________ INTERVIEW START TIME: _______________________________________________ INTERVIEWER CODE: LANGUAGE OF QUESTIONNAIRE: _______________________________________________ LANGUAGE: DATA QUALITY CHECK (If Applicable) SUPERVISOR Accompanied by Supervisor QC present NAME: ____________________ DATE: ____________________ NAME: ____________________ DATE: ____________________ NAME: ___________________ DATE: ___________________ SECTION 1: FARMER’S BACKGROUND CHARACTERISTICS No. Questions and Filters Coding Categories Skip 101. Sex of farmer (Mark one without asking.) Male…………………………….. Female………………………….. 1 2 102. How old are you? Age in completed years…. 154 103. What is the highest level of formal education you have completed? (List Country Specific Educational Levels, e.g., grade.) None…………………………….. Primary Incomplete ……………… Primary Complete ……………….. Secondary Complete ……………. Higher…………………………….. 0 1 2 3 4 104. Can you read and understand a letter or newspaper with ease in English or local language? In English……………………….. In local language (Swahili)………………. Both English and local (Swahili)………... Not at all………………………… 1 2 3 4 105. What is your marital status? Never Married…………………. Informal/Living together………… Married………………………….. Widowed….…………………….. Separated……………………….. Divorced……….......................... 1 2 3 4 5 6 106. How many people live in your household (including you and children)? Total number of people……... 107. How many males and females live in your household? Number of Males............ Number of Females........ 108. In total, how many acres of agricultural land did your household cultivate last season? Farm size in acres.... 109. Do you/your household own the following? [items which have been functioning within the last 6 months.] Radio? Television? Mobile Phone? Yes No Yes No Yes No 110. What was the main source of your household income last year? Salaried employment/Job…………. Sale of crops…………..………… Sale of livestock…………..…….. Petty trade………………………… Sale of labor………………………. Remittances……………………… 1 2 3 4 5 6 155 Other: ___________________ (SPECIFY) 7 111. Which crops were the main sources of your household income last year? (Mark all that apply.) Maize……………………………… Beans………………… Irish potatoes………………….. Cassava….……………………… Other: ___________________ (SPECIFY) Other: ___________________ (SPECIFY) None……………………… 1 2 3 4 5 6 7 112. Do you or any member of your family belong to any type of organized groups? Yes…………………………….. No……………………………. 1 2 If 2 skip -> sec. 2 113. What types of groups do you or any member of your family belong to? (Mark all that apply.) Farmer-based organization…. Religious group………….………… Credit/Microfinance group (i.e., Savings/Merry-go-round)…….. Mutual help/Insurance group (i.e., burial societies)………….… Trade and business association… Other: ___________________ (SPECIFY) None……………………………. 1 2 3 4 5 6 7 SECTION 2: GENERAL EXPOSURE TO ANY IMPROVED SEEDS & OTHER TECHNOLOGIES Interviewer Notes: Now, I’d like to ask you about improved seeds and agricultural technologies you are aware about in your area. No. Questions and Filters Coding Categories Skip 201. What crops did you plant on your farm in the last long rainy season? Maize…………………………… Beans….…………………….. Irish potatoes…………………….. Cassava………….….…….. Other: ___________________ (SPECIFY) Other: ___________________ (SPECIFY) 1 2 3 4 5 6 156 202. Have you ever heard of improved seeds of any crop varieties and other technologies recommended for your area? Yes…………………………….. No……………………………. 1 2 203. What are the different improved technologies of any crop have you heard about? [Spontaneous responses only. Mark all that apply.] Improved seed varieties……… Farm management practice…. Fertilizer applications…………. Other agro-chemical use…….. Improved post-harvest processing……….. Improved post-harvest storage……………… Other: ___________________ (SPECIFY) 1 2 3 4 5 6 7 204. When did you first hear about this or any of these improved technologies? This season………………. Last season….…………. Two years ago………..……….. Three years ago……….….…….. More than 3 years ago……… 1 2 3 4 5 205. Within the past three years, how have you heard about these improved technologies? (This list must include the specific radio or TV shows or ICT channels that SSTP is supporting, e.g., KUAPA TV series in Ghana.) Demonstration…………………. Agro-dealer…………………… Neighbor/Relative………………. Field days……………………….. Community events to promote agricultural technologies………. Agricultural show……………… Government extension agent… Village-based agent (VBA)……. ____________ TV show……… ___________ Radio program… _____________ ICT details… Other: ___________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 10 11 12 206. Have you ever tried any of the improved technologies you have heard about for yourself on your farm? Yes…………………………….. No……………………………… 1 2 If No -> Q.209 157 207. Which improved technologies did you use last season? (Mark all that apply.) Improved seed varieties…… Farm management practice… Fertilizer applications…………. Other agro-chemical use………. Improved post-harvest processing…. Improved post-harvest storage… Other: ___________________ (SPECIFY) 1 2 3 4 5 6 7 208. Which of the following agro-chemicals did you use in the last season? Herbicide…………………………………………… Insecticide…………………………………………… Fungicide……………………………………………… Other:__________________________________ (SPECIFY) Yes No Yes No Yes No Yes No Skip --> Sect. 3 209. What are the main reasons for not having tried any improved technologies? [Spontaneous response only. Mark all that apply.] No interest in trying something….. Unable to afford the technology…. Don’t know where to get the technology……………………… Don’t have enough/type of land… Don’t have enough labor….…… Uncertain about market for outputs…………………… Don’t know how……………….. 1 2 3 4 5 6 7 END SECTION 3: CERTIFIED SEED OF IMPROVED VARIETIES TRIED OR TESTED Interviewer Notes: Now, I’d like to ask you about the Certified Seeds of improved (CROP X) varieties that you have used in your farm. No. Questions and Filters Coding Categories Skip 301. Have you ever tried and used any improved seed of (CROP X) (maize, cassava, beans, or Irish potatoes) varieties on your farm? (Some farmers may be growing improved seeds in a group/communal farm, e.g., in Tanzania.) Yes…………..………………….. No ……………….………………. 1 2 If, No- > Q.309 302. Which improved seed (CROP X) varieties have you ever tried or used in your farm? (Spontaneous response only. Probe and mark all that apply.) Variety 1………………. Variety 2….…………. Variety 3………..……….. Variety 4……….….…….. Variety 5……………….. Other: ___________________ (SPECIFY) 1 2 3 4 5 6 158 Interviewer Notes: Now, I am going to ask you about each of the improved (CROP XX) varieties you have ever tried or used in your farm. I will ask you to indicate which of these varieties of CROP X were planted last season, and which of these were tried but later abandoned (i.e., not planted last season). (Read each variety the farmer mentioned above and mark appropriately). Which of these varieties did you plant last season, and which of these have you tried but later abandoned (i.e., not planted last season)? Planted last season Abandoned (not planted last season) Skip --> Sect. 4 303. Variety 1 304. Variety 2 305. Variety 3 306. Variety 4 307. Variety 5 308. Other (Specify) 309 . Why have you not tried or used any improved seed of (CROP X) (maize, cassava, beans, or Irish potatoes) varieties on your farm? (Mark all that apply.) No interest in trying something….. Unable to afford the technology…. Don’t know where to get the improved seed…………………… Don’t have enough/type of land… Don’t have enough labor….…… Uncertain about market for outputs…………………… Don’t know how………………. Other: ___________________ (SPECIFY) 1 2 3 4 5 6 7 8 310. Do you intend to use improved seed of (CROP X) varieties on your farm in future? Yes……….……………………… No……………………………… … 1 2 If 1 or 2 skip - > Sect. 6. SECTION 4: TECHNOLOGY ADOPTION – USE OF IMPROVED CROP X VARIETIES IN LAST SEASON Interviewer Notes: Now, I am going to ask you about (VARIETY YY – choose name of the 1st mentioned and an SSTP Variety) that you have said you planted in last season in your farm. 401. When did you first use or try out (VARIETY YY)? Last season….…………. Two years ago………..……….. Three years ago……….….…….. More than 3 years ago……………….. 1 2 3 4 402. When you first used the (VARIETY YY) (N years ago), how did you acquire it? Purchased at full cost……….…….. Purchased at reduced cost……… Got it for free……………………….. In-kind or any exchange……… 1 2 3 4 159 403. When you first used (VARIETY YY) (N years ago), from whom did you acquire it? (multiple response) Farmer-based organization… NGOs/AID distribution………… Seed company…………………. Agro-input dealer….…………… Government extension agent………..…………………….. Village-based agent (VBA)…… Other farmers or relative.………. Market trader………….. …………. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 404. When you first used (VARIETY YY) (N years ago), what was the quantity of seed did you use? (Note: In most cases, the seed is supplied in packets of a known weight. Check seed packet sizes that are sold locally and convert their weight to kilograms.) Quantity in (Kilograms)… 405. When you first used (VARIETY YY) (N years ago), what was the distance (km) between your home to where you acquired it? (Indicate in zero kilometers (00 km) if the seed was acquired/received within the home village/town.) Distance from home in (Km).... 406. How did you first learn about (VARIETY YY)? (Mark all that apply.) Farmer-managed demonstration... Seed company/Project-managed demo…………………………… … Agro-dealer…………………… Neighbor/Relative ………………. Field days……………………….. Community events to promote agricultural technologies……… Agricultural show…………….. Government extension agent………….. Village-based agent……….. Farmer training course……….. Radio program………………….. TV program…………………….. ICT……………………………….. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 407. What was the main motivation that persuaded you to plant VARIETY YY for the first time? Farmer-managed demonstration... 1 160 [Spontaneous response, indicate one answer only – that which was most important in persuading farmer to plant this variety.] Seed company/Project-managed demo…………………………… … Agro-dealer…………………… Neighbor/Relative ………………. Field days……………………….. Community events to promote Agricultural technologies……… agricultural show…………….. Government extension agent………….. Village-based agent (VBA)……….. Farmer training course……….. Radio program………………….. TV program…………………….. ICT……………………………….. Other: ____________________ (SPECIFY) 2 3 4 5 6 7 8 9 10 11 12 13 14 408. What is the size of the farm plot (in acres) where (VARIETY YY) was planted last season? Plot size in acres.... 409. What quantity of seeds of (VARIETY YY) did you plant last season? Quantity in (Kilograms)... 410. What was the source of the seed of variety YY that you planted last season? (Probe: Mark all that apply.) Self-saved seed………………… Farmer-based organization … NGOs/AID distribution…………. Agro-input dealer….……………. Government extension officer…… Village-based agent (VBA)……. Other farmers or relative.………. Purchased in market………….. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 411. When you acquired the seed of (Variety YY) last season, what was the distance (km) between your home to where you acquired it? (Indicate 00km if the seed was acquired/received within the home village/town.) Distance from home in (Km).... 161 412. What benefits have you experienced in your household from using this (Variety YY)? [Spontaneous response only. Do not read out loud the list of responses. Mark all that apply.] Increased yield with this variety… More income from crop sales of this variety………………………… Reduced labor demands from this variety…………………………….. . Better tasting food products from this variety………………………… Reduced fuel used in cooking this variety…………………. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 413. Do you intend to use this (Variety YY) again next season? (If No, make sure you complete Q415.) Yes…………………………….. No…………………………… 1 2 414. Since you started using (Variety YY), have you changed any of the farm management practices because of the technology’s characteristics or requirements? Yes…………………………….. No…………………………....... 1 2 415. Why have you decided not to continue using VARIETY YY next season? (Mark all that apply.) Expensive to purchase the inputs... Requires a lot of labor....…. Don’t like taste………………... Not marketable…………………. Other: ____________________ (SPECIFY) 1 2 3 4 5 Interviewer text: I am now going to read a few statements about your perceptions of the (Variety YY). I am interested in the extent to which you agree or disagree with each statement. After I have read each statement, please indicate whether you strongly disagree with the statement, you disagree with the statement, you are neutral about the statement, you agree with the statement or you strongly agree with the statement. Statement: Strongl y disagre e Disagre e Neutral Agree Strongl y agree 416. I like using (Variety YY) in my farm. 1 2 3 4 5 417. I would recommend (Variety YY) to other farmers or relatives. 1 2 3 4 5 SECTION 5: TRIED AND DISCONTINUED SEED VARIETY Interviewer Notes: Now, I’d like to ask you about (Variety ZZ) that you have tried but then abandoned or discontinued. 162 501. When did you first use or try out (VARIETY ZZ)? Last season……………... Two years ago……………………… Three years ago……………………. More than 3 years ago………………... 1 2 3 4 502. What was the main motivation that persuaded you to plant this variety for the first time? [Spontaneous response. Indicate one answer only – that which was most important in persuading farmer to plant variety.] Farmer-managed demonstration... Seed company/Project-managed demo……………………………… Agro-dealer…………………… Neighbor/Relative ………………. Field days……………………….. Community events to promote agricultural technologies……… Agricultural show…………….. Government extension agent………….. Village-based agent……….. Farmer training course……….. Radio program………………….. TV program…………………….. ICT……………………………….. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 503. For how many seasons or years did you use VARIETY ZZ? Number of seasons.... 504. When you first used VARIETY ZZ (N years ago), how did you acquire it? Purchased at full cost……….…….. Purchased at reduced cost/subsidy……….. Got it for free……………………….. In-kind or exchange………………. 1 2 3 4 505. When you first used VARIETY ZZ (N years ago), what was the quantity of seed that you acquired? Quantity in (Kilograms)... 163 506. When you first used VARIETY ZZ (N years ago), from whom did you acquire it? Farmer-based organization…………. NGOs/AID distribution……….. Seed company………………… Agro-input dealer….…………... Government extension officer………..…………………… Village-based agent…………… Other farmers or relative.………. Market trader………………….. Other: ____________________ (SPECIFY) 1 2 3 4 5 6 7 8 9 507. When you first used VARIETY ZZ (N years ago), what was the distance (km) between your home to where you acquired it? (Indicate 0 km if the seed was acquired/received within the home village/town.) Distance from home in (Km).... 508. Why did you decide not to continue using VARIETY ZZ? (Mark all that apply.) Expensive to purchase the inputs... Requires a lot of labor....…. Don’t like taste………………... Not marketable…………………. Other: ____________________ (SPECIFY) 1 2 3 4 5 509. Do you plan to use VARIETY ZZ again in future? Yes……….………………………….. No………………………………….. 1 2 SECTION 6: FARMER RE-CONTACT CONSENT FOR SECOND-ROUND OF SURVEY Interviewer Read: Thank you for talking to me today. This survey will also be conducted again next year as a follow-up on the same study topic. We will like to contact you again next year around the same time. Your participation for the follow-up survey is voluntary as well. If you agree to participate, a representative from our team will contact you again next year. 601. Do you have any questions about the survey next year or what I have said? (Interviewer: Clarify any question asked). Yes……….………………………….. No………………………………….. 1 2 602. Do you agree to be contacted again next year for the follow-up survey? Yes……….………………………….. No………………………………… …. 1 2 If 2 -- → END 603. Are the phone number(s) you provided earlier the best contact information to reach you next year? Yes……….………………………….. No………………………………….. 1 2 If 1 -- → END 604. What is the best contact information to reach you next year? Phone:___________________ END 164 END OF INTERVIEW - THANK YOU! 165 ANNEX J.2. SSTP FARMERS LISTING QUESTIONNAIRE PRE-SURVEY: SCREENING QUESTIONS OF FARMER HOUSEHOLDS AND IDENTIFICATION COUNTRY: __________________________________________________ PROVINCE: _________________________________________ DISTRICT: __________________________________________ VILLAGE/COMMUNITY NAME: __________________ PROJECT NAME/AREA _________________________________ A. BENEFICIARY: ________________________________ B. INDIRECT BENEFICIARY: _______________________ PROVINCE: DISTRICT: CSA/SEA: __/__/___ | __/__/__ Yes (Mark Only ONE). Yes Hello, My name is __________ and my colleague’s name is ________________we are from a ______(ESRF or SURVEY FIRM). We are visiting here in (NAME VILLAGE) to talk to farmers who grow [Maize/Beans/Irish Potatoes] in their farms. We will like to talk to the person/farmer that makes day-to-day farming decisions including what to grow and how to grow it on the farm plots. Interviewer Note: ● If the farmer took part in an activity and are aware of the SSTP varieties then they are a direct beneficiary. ● If they did not take part in an activity but are aware of the varieties then they are an indirect beneficiary. ● If they did not take part in an activity and are not aware of the varieties then they are a non-beneficiary. 1. Do you or anyone living in this household grow maize/beans/Irish Potatoes? (If they say NO do not list, move on to the next household.) (If YES, ask to speak to the person responsible for growing maize.) Yes No 2. In the past 3 years, have you participated in any activities about improved (maize/beans/Irish Potatoes) varieties such as demonstration plots, field-day, attended a training, or listened to radio or TV program and then made inquiries to learn more? Yes No 3. Have you ever heard of (Variety 1, Variety 2, or Variety 3) xx or xx or xx varieties of maize? (Where Variety 1, 2, 3, etc. are the maize/bean/Irish potato varieties promoted by SSTP in the area such as region.) Yes No 4. Based on what you have told me about improved maize/bean/potato seed varieties, I will like to come back at (XX TIME/DAY) and ask you some more questions about them. If your name is selected to represent other farmers in your village [NAME OF VILLAGE], may I contact you again at that time to interview you? Yes No (If No, END) 5. Is there a phone number where I can reach you if necessary before I come? Yes No 6. What is the number? Farmer’s Contact Phone Number: 7. In-case we don’t find you by phone, who else could we contact to know where you are? Yes No 8. What is their number? Alternative Person’s Phone Number: Farmer Name: Alternative Person Name: Household Address and Features: If GPS is used write the geocodes here. Otherwise, provide enough information on how this household can be traced. 166 ANNEX J.3. ADDITIONAL SURVEY TABLES 167 Table J.3.1. Main Reasons For Not Having Tried Improved Seed & Other Technologies Main Reason Ghana Mozambique Tanzania Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total Direct Beneficiaries Indirect Beneficiaries Total No interest in trying something N 4.0 30.0 34.0 28.0 44.0 72.0 14.0 55.0 69 % 16.7 31.3 28.3 12.0 9.4 10.3 100.0 100.0 100.0 Unable to afford the technology N 12.0 41.0 53.0 43.0 86.0 129.0 10.0 20.0 30 % 50.0 42.7 44.2 18.5 18.4 18.4 71.4 36.4 43.5 Don’t know where to get the technology N 3.0 22.0 25.0 99.0 271.0 370.0 8.0 24.0 32 % 12.5 22.9 20.8 42.5 58.0 52.9 57.1 43.6 46.4 Don’t have enough/type of land N 2.0 10.0 12.0 20.0 24.0 44.0 1.0 4.0 5 % 8.3 10.4 10.0 8.6 5.1 6.3 7.1 7.3 7.2 Don’t have enough labor N 1.0 19.0 20.0 24.0 32.0 56.0 0.0 1.0 1 % 4.2 19.8 16.7 10.3 6.9 8.0 0.0 1.8 1.4 Uncertain about market for outputs N 0.0 8.0 8.0 6.0 10.0 16.0 5.0 3.0 8 % 0.0 8.3 6.7 2.6 2.1 2.3 35.7 5.5 11.6 Don’t know how N 0.0 32.0 51.0 83.0 3.0 28.0 31 % 0 0 0 13.7 10.9 11.9 21.4 50.9 44.9 Others N 7 4 11 21 18 39 0 % 29.2 4.2 9.2 9.0 3.9 5.6 0.0 0.0 0.0 Total N 24 96 120 233 467 700 14 55 69 % 100 100 100 100 100 100 100 100 100 168 Table J.3.2. Background Characteristics by Adoption and Country Ghana Mozambique Tanzania Adoption Adoption Adoption Background Pre- Pre- Pre- SSTP Non- Tota SSTP Non- Tota SSTP Non- Tota Characteristics SSTP SSTP SSTP Adopter Adopter l Adopter Adopter l Adopter Adopter l Adopter Adopter Adopter s s s s s s s s s Sex Male N 215 174 182 571 22 127 329 478 398 426 35 859 % 55.3 53.9 47.5 52.1 50 49.2 37.5 40.5 67.2 63.4 66 65.2 Female N 174 149 201 524 22 131 549 702 194 246 18 458 % 44.7 46.1 52.5 47.9 50 50.8 62.5 59.5 32.8 36.6 34 34.8 Age group 16 - 29 yrs N 33 42 70 145 9 83 274 366 50 102 7 159 % 8.5 13 18.3 13.2 20.5 32.2 31.2 31 8.4 15.2 13.2 12.1 30 - 59 yrs N 285 241 257 783 25 143 463 631 454 486 34 974 % 73.3 74.6 67.1 71.5 56.8 55.4 52.7 53.5 76.7 72.3 64.2 74 60+ yrs N 71 40 56 167 10 32 141 183 88 84 12 184 % 18.3 12.4 14.6 15.3 22.7 12.4 16.1 15.5 14.9 12.5 22.6 14 Grade complete d None N 101 94 177 372 12 58 271 341 0 0 0 0 % 26 29.1 46.2 34 27.3 22.5 30.9 28.9 0 0 0 0 Primary incomplet e N 113 99 102 314 24 132 403 559 29 32 4 65 % 29 30.7 26.6 28.7 54.5 51.2 45.9 47.4 4.9 4.8 7.5 4.9 169 Background Characteristics Ghana Mozambique Tanzania Adoption Tota l Adoption Tota l Adoption Tota l Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Primary complete or higher N 175 130 104 409 8 68 204 280 507 590 40 1137 % 45 40.2 27.2 37.4 18.2 26.4 23.2 23.7 85.6 87.8 75.5 86.3 Read and understan d Can't read and understan d N 0 0 0 0 17 110 421 548 51 51 9 111 % 0 0 0 0 38.6 42.6 47.9 46.4 8.6 7.6 17 8.4 Can read and understan d N 389 323 383 1095 27 148 457 632 541 621 44 1206 % 100 100 100 100 61.4 57.4 52.1 53.6 91.4 92.4 83 91.6 Average Househol d Size Average N 6.9 6.6 6.2 6.6 6.5 5.5 5 5.1 5.8 5.7 5.2 5.7 Average Farm Size Average N 6.4 5.6 6.4 6.2 4.4 3.7 3.8 3.8 3.3 3.3 3.1 3.3 Land size Less than or equal to 2 acres N 123 91 95 309 12 79 238 329 262 289 22 573 % 31.6 28.2 24.8 28.2 27.3 30.6 27.1 27.9 44.3 43 41.5 43.5 Greater than 2 and less N 122 101 129 352 20 132 477 629 247 298 27 572 170 Background Characteristics Ghana Mozambique Tanzania Adoption Tota l Adoption Tota l Adoption Tota l Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s than or equal to 5 acres % 31.4 31.3 33.7 32.1 45.5 51.2 54.3 53.3 41.7 44.3 50.9 43.4 More than 5 acres N 144 131 159 434 12 47 163 222 83 85 4 172 % 37 40.6 41.5 39.6 27.3 18.2 18.6 18.8 14 12.6 7.5 13.1 Own radio? YES N 308 249 276 833 18 104 389 511 484 508 41 1033 % 79.2 77.1 72.1 76.1 40.9 40.3 44.3 43.3 81.8 75.6 77.4 78.4 NO N 81 74 107 262 26 154 489 669 108 164 12 284 % 20.8 22.9 27.9 23.9 59.1 59.7 55.7 56.7 18.2 24.4 22.6 21.6 Own TV? YES N 243 175 160 578 8 31 107 146 141 160 5 306 % 62.5 54.2 41.8 52.8 18.2 12 12.2 12.4 23.8 23.8 9.4 23.2 NO N 146 148 223 517 36 227 771 1034 451 512 48 1011 % 37.5 45.8 58.2 47.2 81.8 88 87.8 87.6 76.2 76.2 90.6 76.8 Belong to a group of any kind? YES N 337 289 331 957 28 218 625 871 333 388 43 764 % 86.6 89.5 86.4 87.4 63.6 84.5 71.2 73.8 56.3 57.7 81.1 58 NO N 52 34 52 138 16 40 253 309 259 284 10 553 % 13.4 10.5 13.6 12.6 36.4 15.5 28.8 26.2 43.8 42.3 18.9 42 171 Background Characteristics Ghana Mozambique Tanzania Adoption Tota l Adoption Tota l Adoption Tota l Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s Pre￾SSTP Adopter s SSTP Adopter s Non￾Adopter s N 389 323 383 1095 44 258 878 1180 592 672 53 1317 Total % 100 100 100 100 100 100 100 100 100 100 100 100