PERFORMANCE EVALUATION OF THE PRIVATE SECTOR-DRIVEN AGRICULTURAL DEVELOPMENT ACTIVITY IN RWANDA August 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. PERFORMANCE EVALUATION OF THE PRIVATE SECTOR-DRIVEN AGRICULTURAL DEVELOPMENT ACTIVITY IN RWANDA This publication was produced at the request of the United States Agency for International Development under Contract/Task Order Number: AID-OAA-I-15-00024/72069618F00003. It was prepared independently by: David Rinck – Team Leader Didier Nkubito – Local Evaluator Elie Nsabimana – Agribusiness/ Value Chains Specialist Raphael Rurangwa – Policy Analysis and Development Specialist Mel Schnapper – Organizational Management Specialist Contractor: ME&A Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 240-762-6296 http://www.meandahq.com/ Cover photo: Enterprise Nkubiri & Sons (ENAS) staff surveying a silo site in the Kirehe Grain Storage Facility in Kirehe District partially funded by a PSDAG grant – By David Rinck for USAID. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT This performance evaluation of the Private Sector-Driven Agricultural Growth (PSDAG) activity, implemented August 2014 - August 2019 by the Research Triangle Institute (RTI), aims to assess the activity’s progress until April 2018 on its two main objectives, and the effectiveness of its technical approach and its responsiveness to supporting the Government of Rwanda’s goal of increasing private investment in agribusiness. The evaluation, implemented May-July 2018, employed a mixed methodology, including document review, 66 key informant interviews, 16 focus group discussions with 104 farmers, and a beneficiary-based survey of 902 farmers to answer seven evaluation questions. The evaluation found that PSDAG’s assistance to the Rwanda Development Board (RDB) has been effective in attracting private sector investment in agriculture, but less so in public-private dialogue. The Agriculture Land Information System (ALIS) is a valuable tool for providing timely information for new investment projects. Contract farming and improved post-harvest handling are among the innovations that have the most significant impact on improving farmers’ incomes. The evaluation also concluded that PSDAG’s grant facility has been effective in promoting investment in agriculture; the activity has taken reasonable steps to prevent double-dipping from grant recipients, and it has pursued a vigorous approach to social inclusion, with mixed results. Key recommendations to PSDAG include: continue the systematic approach to building RDB’s capacity; provide support to the Ministry of Agriculture and Animal Resources (MINAGRI) to improve and continue funding ALIS; and continue working with MINAGRI on developing the legal framework for contract farming and expanding grants to contract farming scheme implementers. TABLE OF CONTENTS EXECUTIVE SUMMARY................................................................................................................................................... i 1.0 EVALUATION PURPOSE AND QUESTIONS ................................................................................................... 1 1.1 Evaluation Purpose ........................................................................................................................................ 1 1.2 Evaluation Questions.................................................................................................................................... 1 2.0 ACTIVITY BACKGROUND.................................................................................................................................... 2 2.1 Activity Description ...................................................................................................................................... 2 3.0 EVALUATION METHODS AND LIMITATIONS .............................................................................................. 4 3.1 Data Analysis................................................................................................................................................... 7 3.2 Potential Methodological Limitations........................................................................................................ 8 4.0 FINDINGS .................................................................................................................................................................... 9 4.1 EQ 1: Most Effective Interventions to Assist GOR............................................................................... 9 4.2 EQ 2: Interventions Most Likely to Lead to Lasting Change............................................................. 16 4.3 EQ 3: Risks to Sustainability...................................................................................................................... 18 4.4 EQ 4: Top Three Technologies and Innovations................................................................................. 19 4.5 EQ 5: Effectiveness of the PSDAG Grant Facility................................................................................ 27 4.6 EQ 6: Preventing Double Dipping ........................................................................................................... 29 4.7 EQ 7: Social Inclusion ................................................................................................................................. 30 5.0 CONCLUSIONS....................................................................................................................................................... 32 5.1 EQ 1: Most Effective Interventions to Assist GOR............................................................................. 32 5.2 EQ 2: Interventions Most Likely to Lead to Lasting Change............................................................. 33 5.3 EQ 3: Risks to Sustainability...................................................................................................................... 34 5.4 EQ 4: Top Three Technologies and Innovations................................................................................. 34 5.5 EQ 5: Effectiveness of the PSDAG Grant Facility................................................................................ 34 5.6 EQ 6: Preventing Double Dipping ........................................................................................................... 35 5.7 EQ 7: Social Inclusion ................................................................................................................................. 35 6.0 RECOMMENDATIONS.......................................................................................................................................... 35 7.0 LESSONS LEARNED................................................................................................................................................ 37 ANNEXES ......................................................................................................................................................................... 38 Annex A: Statement of Work............................................................................................................................... 39 Annex B: Evaluation Team..................................................................................................................................... 48 Annex C: Citations and References Reviewed ................................................................................................. 51 Annex D: Contact List............................................................................................................................................ 55 Annex E: Evaluation Tools..................................................................................................................................... 59 List of Figures Figure 1: PSDAG Results Framework........................................................................................................................... 3 Figure 2: Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models................................................................................................................................................................................ 11 Figure 3: Case Study – Les Minoteries Industrielles Import-Export (MINIMEX)............................................. 23 Figure 4: Case Study – Tuzamurane Cooperative ................................................................................................... 24 List of Tables Table 1: PSDAG Provinces and Districts..................................................................................................................... 4 Table 2: Key Informant Groups by Stakeholder Group........................................................................................... 5 Table 3: Focus Groups Discussions by Province/District and Sector................................................................... 5 Table 4: Calculation of Sample Size for Farmer Survey............................................................................................ 6 Table 5: Beneficiary-Based Surveys by Province/District and Sector.................................................................... 7 Table 6: VCCF “Investments” Summary .................................................................................................................... 20 Table 7: Change in Agricultural Productivity at PSDAG-Assisted Cooperatives............................................. 21 Table 8: Sources of Market Information for PSDAG-Assisted Cooperative Members.................................. 25 Table 9: Overall Change in Crop Production........................................................................................................... 25 Table 10: Reasons for Increased Production ............................................................................................................ 26 Table 11: Types of Technical Assistance Received by PSDAG............................................................................. 26 Table 13: Sources of Loans at PSDAG-Supported Cooperatives Disaggregated............................................. 31 Table 14: Changes in Income at PSDAG-Supported Cooperatives.................................................................... 32 ACRONYMS Acronym Description AFR Access to Finance Rwanda Agri-PPD Agriculture Public-Private Dialogue ALIS Agriculture Land Information System AOR Agreement Officer’s Representative APS Annual Program Statement ASWG Agriculture Sector Working Group BBS Beneficiary-Based Survey BDF Business Development Fund BDS Business Development Services BDU Business Development Unit BEE Business Enabling Environment BRD Rwanda Development Bank (Banque Rwandaise de Développement) CAPI Computer-Assisted Personal Interview CDCS Country Development and Cooperation Strategy CESB Capacity Development and Employment Center CIP Crop Intensification Program COP Chief of Party COR Contracting Officer’s Representative CPP Cooperative Professionalization Program DEC Development Experience Clearinghouse DFID United Kingdom Department for International Development DO Development Objective DRC Democratic Republic of Congo EAGC East Africa Grains Council EDC Education Development Center EDPRS Economic Development and Poverty Reduction Strategy EIA Embedded Investment Advisor ENAS Enterprise Nkubiri & Sons EQ Evaluation Question ET Evaluation Team EUCORD European Cooperative for Rural Development FARA Forum for Agriculture Research in Africa FDI Foreign Direct Investment FGD Focus Group Discussion FY Fiscal Year GDP Gross Domestic Product GOR Government of Rwanda GSI Gender and Social Inclusion HG Huguka Dukore (project) ICT Information and Communications Technology ID Identifier Acronym Description IDEC Industrial Development Export Council IFAD PASP International Fund for Agricultural Development Post-Harvest and Agribusiness Support Project IFC International Finance Corporation INES Institute of Applied Sciences IP Implementing Partner IRG International Resource Group ISU Investment Support Unit kg Kilogram KII Key Informant Interview KCB Kenya Commercial Bank KNU KNU Agro Mechanized Services Ltd. LSP Local Service Provider M&E Monitoring and Evaluation MINECOFIN Ministry of Finance and Economic Planning MINAGRI Ministry of Agriculture and Animal Resources MINICOM Ministry of Trade and Industry MINIMEX Les Minoteries Industrielles Import-Export MOU Memorandum of Understanding MT Metric Ton NAEB National Agricultural Export Development Board NAIPS National Agribusiness Investment Promotion Strategy PCC Potato Collection Center PPD Public-Private Dialogue PE Performance Evaluation PEBEC Producing and Aggregating Export-Quality Birds-Eye Chili PPP Public-Private Partnership PSDAG Private Sector-Driven Agricultural Growth PSF Private Sector Federation PSTA-IV Fourth Strategic Plan for the Transformation of Agriculture PWD Persons with Disabilities PY Project Year QMS Quality Management Standards RALIS Rwanda Agricultural and Livestock Inspection and Certification Services RCAL Rwanda Chamber of Agriculture and Livestock RDB Rwanda Development Board RDO Rwanda Development Organization RFA Request for Applications ROI Return on Investment RPPD Rwanda Private-Public Dialogue Secretariat RSB Rwanda Standards Board RTI Research Triangle International RWF Rwandan Franc RYAF Rwanda Youth in Agribusiness Forum Acronym Description SACCO Savings and Credit Cooperative Organization SAIS Smart Agriculture Information System SID Strategic Investment Department SME Small and Medium-Sized Enterprise SOW Statement of Work SPF Seed Potato Fund Ikigega Ltd. STTA Short-Term Technical Assistance TA Technical Assistance TAF Technical Assistance Fund UNCTAD United Nations Conference on Trade and Development US$ United States Dollars USAID United States Agency for International Development VAT Value Added Tax VCCF Value Chain Competitiveness Fund i EXECUTIVE SUMMARY EVALUATION PURPOSE The Private Sector-Driven Agricultural Growth (PSDAG) activity is funded by the United States Agency for International Development (USAID) and implemented by the Research Triangle Institute (RTI) from August 2014 to August 2019.1 The purpose of the PSDAG performance evaluation was to assess: 1. Progress made to date in achieving the activity’s two main objectives; and 2. The effectiveness of the technical approach and the activity’s responsiveness to support the Government of Rwanda’s (GOR) goal of increasing private investments in agribusiness in Rwanda. The USAID/Rwanda Mission and its implementing partner (IP) will use the evaluation findings to prioritize and strengthen activity implementation and help determine what steps to take to sustain its achievements. The evaluation’s results will also help inform the implementation of other related programs designed to advance private sector engagement. PROJECT BACKGROUND The overall goal of PSDAG is to “increase smallholder incomes by promoting private sector investment (internationally and domestically) that contribute to the Government of Rwanda Vision 2020 aim of transforming agriculture into a market-oriented, competitive, and high-value sector.”2 PSDAG’s primary objectives are to assist the GOR to increase private sector investment, and to facilitate private sector investment in upgrading value chains. In addition, PSDAG has two cross-cutting objectives: promoting the use of information and communications technology (ICT) and the social inclusion of women, youth (defined as ages 18-35), and persons with disabilities (PWDs). EVALUATION METHODOLOGY To conduct the evaluation, the evaluation team (ET) utilized a mixed-methods approach that included qualitative and quantitative data collection. Qualitative data was collected through an extensive document and secondary data review, 66 key informant interviews (KIIs) with four key PSDAG stakeholder groups, and 16 focus group discussions (FGDs) with 106 respondents. The ET implemented data collection in all PSDAG provinces with the exception of Kigali, and in five randomly chosen districts (one each in South, West, North, and two in East to reflect the distribution of PSDAG beneficiaries). Quantitative data was collected through a beneficiary-based survey (BBS) with a representative sample of 902 beneficiary farmers who are members of farmer cooperatives in areas that have received activity support. As with the FGDs, the BBS was implemented in all PSDAG provinces with the exception of Kigali, and in five randomly selected districts reflecting the distribution of PSDAG beneficiaries. Data collected was analyzed using STATA, SPSS, and Excel. Qualitative data obtained from the document review and the KIIS and FGDs was compiled on a real-time basis to discern emerging trends and develop probing questions, and then aggregated around common themes related to the seven evaluation questions (EQs) (see table below) to identify potential emergent findings. The evaluation design includes several potential methodological limitations: selection bias, recall bias, halo bias, and the absence of a counterfactual. The ET addressed these potential limitations through a variety of methods, including a two-stage sampling approach for the FGDs and farmer BBS, and triangulation of evidence from different methods. 1 USAID initially awarded PSDAG to International Resource Group (IRG). However, in January 2017, when RTI bought IRG, the PSDAG contract was transferred to RTI. 2 “Contract No.: AID-696-C-14-00002/International Resources Group Private Sector-Driven Agricultural Growth (PSDAG), SECTION C – Statement of Work,” USAID. ii FINDINGS AND CONCLUSIONS The evaluation findings and conclusions are summarized in the table below: Findings Conclusions EQ 1: Within the objective of assisting the GOR to increase private sector engagement, what types of interventions supported by PSDAG (public-private dialogue [PPD]; policy and regulatory reform, including restructuring Rwanda Development Board [RDB] investment targeting and promotion strategies; institutional strengthening; facilitating public￾private partnerships [PPPs] etc.) have been the most effective in attracting private sector investment in agriculture • According to PSDAG and USAID staff, activity interventions have been delayed for a number of reasons. Consequently, interventions have yet to realize their potential to generate specific investments. • Multiple GOR and PSDAG informants, as well as project documents, describe PSDAG supporting the GOR’s shift to a facilitative approach to investment promotion through “systematic” assistance as opposed to specific projects. This includes capacity development and support through Embedded Investment Advisers (EIAs) at the Ministry of Agriculture and Animal Resources (MINAGRI) and RDB, which staff describe as effective due to the “flexible” nature of assistance. • Reflecting this approach, in 2016 PSDAG supported the “Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models,” which resulted in expanded exports by five companies, increasing exports by US$1.2 million and expanding private sector use of a National Agricultural Export Development Board (NAEB)-owned cold storage facility. • Project documents and GOR and PSDAG informants described review of agriculture strategies, including the National Agriculture Investment Promotion Strategy (NAIPS) and the Fourth Strategic Plan for the Transformation of Agriculture (PSTA-IV), as well as the legal framework for contract farming as useful activities. • Private Sector Federation (PSF) is described as the primary forum for PPD but stakeholders cite a multiplicity of others, including the Agriculture Sector Working Group (ASWG), Agro Small and Medium-Sized Enterprise (SME) Cluster Forums, AgriProFocus, as well as forums supported by Value Chain Competitiveness Fund (VCCF) grants. • In KIIs, GOR and PSDAG respondents stated that the PPD process is “not (effectively) covering agriculture due to convoluted institutional support,” including staffing, and described challenges in addressing issues identified at sub-national levels. PSDAG documents indicate a plan to analyze the factors behind poor agriculture PPD. PSDAG staff states that the activity has already identified and trained district-level “champions” and provided a capacity development grant to PSF. • PSDAG staff and activity documents show that in 2016 PSDAG implemented a “Poultry Competitiveness Study for Day￾Old Chick,” which informed a policy decision to exempt locally produced feeds from the value-added tax (VAT). This increased investment in feed sector and facilitated the US$2 million PPP privatization of the National Hatchery. Given the relative recentness of PSDAG interventions, it is difficult to attribute specific investments to specific interventions. Interventions supporting the GOR’s shift in targeting and investment promotion to a more facilitative approach will have an increasingly significant impact in attracting private sector investment as capacity to implement improves. Outcomes related to the “Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models” and “Poultry Competitiveness Study for Day-Old Chick” illustrate this potential impact. EIAs support to MINAGRI and RDB will have significant impact due to the flexible nature of embedded assistance and ability to provide long-term continuity of support. To date, the PPD process has not been effective in the agricultural sector due to convoluted institutional support and challenges in addressing issues identified by the private sector at the sub-national level. iii Findings Conclusions EQ 2: To what extent are the PSDAG interventions supporting Objective 1 (which includes the Agricultural Land Information System [ALIS], support to MINIAGRI’s budget process, restructuring RDB investment targeting and promotion strategies, support to the PPD structure and process, and aggregation service centers) likely to result in long-lasting impact in attracting private sector investment in agriculture beyond the conclusion of PSDAG? • Multiple GOR informants described the ALIS land-mapping system as a “very useful tool to guide agricultural investors” by providing timely information on available sites for new projects. Informants state that the multi-year embedded budget support for hiring staff and planning has been secured. • Officials at the RDB positively assessed PSDAG’s approach to restructuring investment targeting and promotion strategies through a more “systematic” approach, and focusing on institutional capacity building, including through EIAs. • GOR officials at the district level assessed the PPD process more positively than officials at the national level where informants stressed the need to address the functionality constraints noted above related to coordination and issue resolution. With its obligated seven-year budget to facilitate staffing and planning, ALIS is highly likely to produce positive impact through timely provision of available data to investors on potential new project news. PSDAG’s support to restructuring RDB investment targeting and promotion strategies through a more facilitative approach, including sector-wide analyses and intensive institutional capacity building, is very likely to produce a lasting impact on the capacity of the board to implement investment promotion through better capacity for provision of required investor data and increased capacity to generate future data. Coordination challenges, due to multiple forums, weak PSF leadership, and unclear processes for addressing issues identified (especially between sub-national and national forums) currently undermine the potential of PPD to generate long-lasting positive impact. EQ 3: What are the risks to sustainability and the actions needed to address them within the remaining project terms? (Note: In the context of this question, sustainability is defined as the ability of the GOR to continue the key interventions or gains achieved under Objective 1 after the conclusion of the PSDAG activity.) • GOR and PSDAG KII respondents noted sustainability risks to ALIS and PPD, indicating: - ALIS needs a dynamic updating process to provide real time data on land availability, which is planned through Esri Rwanda Ltd. support; and a legal framework for leasing needs to be developed between the Rwanda Land Management and Use Authority and ALIS as planned under the Land Use Management Guidelines project is currently being implemented. Respondents suggested that ALIS be consolidated under the Land Use Authority’s management to expand land data. - Multiple stakeholders stated that PPD requires greater coordination between stakeholders, greater staff consistency, improved issue resolution process, and improved leadership by PSF. Sustainability risks to ALIS and PPD include: • The ALIS system requires development of a dynamic updating process and the establishment of a clear and transparent process for reviewing and processing applications for land and implementing leases in a timely manner. • PPD requires greater coordination between stakeholders, greater staff consistency, improved issue resolution process, and improved leadership by PSF. iv Findings Conclusions EQ 4: Which top three technologies and business model innovations promoted by PSDAG have resulted in the highest return on investment for farmers and for agricultural businesses? What were the reasons for any success or failure? • Multiple informants and activity documents described significant linkages between PSDAG-assisted farmer cooperatives and SMEs through contract farming schemes that have improved access to finance for production. In the BBS, 10 out of 16 cooperatives surveyed reported applying for loans from financial institutions, of which four were successful. • Bank staff cited contract farming as having increased its agriculture portfolio from 14 percent to 24 percent within the last two years. • Bank staff also stated that an effective mechanism for contract dispute resolution and enforcement is crucial. • Through PSDAG support to post-harvest handling companies, 126 farmer cooperatives accessed post-harvest handling upgrades, resulting in 60,000 farmers accessing new markets. • Activity documentation shows that partnership with commercial crop producers increased incomes for up to 142,875 smallholders, resulting in US$20 million in gross farm incomes. • Seven PSDAG grantees are involved in potato seed multiplication and signed grant agreements with PSDAG, with a total value of US$590,895, to increase the supply of quality seeds. Contract farming has produced the most significant positive impacts from among the technologies and business model innovations promoted by PSDAG. This is evidenced by cooperative members’ improved access to finance, which has in turn expanded members’ access to new inputs and markets, as evidenced by increased production and incomes found in the BBS. Loans collateralized through cooperative contracts have provided funding for smallholder production and post￾harvest handling, which, in turn, has led to increased production and quality of products, as well as new market linkages. Improved post-harvest handling and other increased vertical supply chain linkages have also produced a significant increase in farmers’ incomes for widely produced basic food crops. Finally, commercial (high value) crops and potato seed multiplication have had a similar effect on smallholder incomes. However, the more limited number of farmers impacted in both cases renders the income effect for commercial crops and potato seed multiplication lower than for contract farming and improved post-harvest handling. EQ 5: Has PSDAG’s grant facility been an effective investment tool in promoting agriculture? • Activity documents show that PSDAG grantees leveraged over US$12.3 million in co-investment via its grant mechanism, surpassing its objective. • SME informants reported that, in some cases, PSDAG facilitation of loan negotiations between grantees and financial institutions, as well as business development services (BDS) support, were useful in securing co-investment. • SME staff described PSDAG-supported technical assistance (TA) to SMEs—such as profitability analysis studies to assess business models and loan repayment capacity—as key to leveraging co-investment. • VCCF grants also improved the performance of cooperatives, and 65.8 percent of BBS respondent farmers reported improved access to markets. The PSDAG grant facility has been an effective tool in promoting investment, with VCCF grantees leveraging over US$12.3 million in co-investment from self-funding, banks, and equity investors. Key factors in leveraging co￾investments have been PSDAG-supported TA to SMEs, as well as PSDAG’s BDS support and TA in implementing business projects, which helped mitigate activity risk and provided a critical guarantee of activity success to banks and investors. EQ 6: How effective has PSDAG been in preventing double dipping from its grant recipients (i.e., is PSDAG providing grants to support proposals being funded by other donors)? • PSDAG approached the issue of double dipping through a self-reporting procedure, followed up by due diligence through a review process with other donors to ensure that VCCF grants did not duplicate efforts. • No SMEs interviewed acknowledged double dipping, although several received funding from other donors for prior stages of their projects or for assets not funded by PSDAG grants. PSDAG took reasonable steps to ensure that systematic problems with double dipping have not occurred. As a result, the activity appears to have been effective to date in avoiding the provision of grants to support proposals already funded by other donors. v Findings Conclusions EQ 7: Did the PSDAG approach to social inclusion ensure the involvement of women, youth, and PWD? Are there innovations or other approaches that could enhance the meaningful engagement of these targeted beneficiary groups? • Of the 26 SMEs interviewed, four had a social inclusion policy in place prior to PSDAG support, while three reported developing a policy with activity support. • Changes to social inclusion reported by SMEs were mostly related to “increase in awareness.” • Several SME staff described existing practices as favoring gender inclusion based on existing gender roles, but limited changes in staffing. • Compared to SMEs, PSDAG-supported cooperative staff reported more robust efforts in social inclusion. For example, through outreach to women farmers, the proportion of women cooperative members increased to the current 42.9 percent. • FGDs suggest that considerable efforts have been made to increase the active participation of women in cooperative activities. Two cooperatives described efforts to recruit young people through a dedicated small start-up loan targeting youth. • In contrast, SMEs and cooperatives did not report efforts to expand membership or launch new activities to expand roles for PWD. Likewise, respondents at SMEs and cooperatives stated that PWD are under-represented in the Rwandan workforce and. PSDAG pursued a vigorous approach to social inclusion. However, only a limited number of PSDAG partner SMEs have made substantive changes to staffing as a result of participation in the activity (although the impact of PSDAG social inclusion strategies may not yet have fully accrued). In most cases, gender barriers to employment remain intact, and women continue to be employed in areas associated with existing gender roles. In contrast, approaches to social inclusion have resulted in improvements at PSDAG-assisted cooperatives, including increased women’s membership and specific outreach and training activities targeting women. Some cooperatives have also developed initiatives to recruit more youth, including start-up loans and membership fee waivers. On the other hand, initiatives to engage PWD have not been developed at SMEs or cooperatives at a significant scale. RECOMMENDATIONS 1. As planned under the existing TA grant to PSF, PSDAG should support PSF leadership to develop a more effective coordination approach for PPD and generate consensus on methodologies for addressing issues identified by investors. 2. PSDAG, USAID/Rwanda, and other donors should continue the current “systematic” approach to GOR capacity development, including providing support for sector-wide analysis and institutional capacity building. 3. PSDAG should continue to support MINAGRI and the Rwanda Land Management and Use Authority to develop the process and legal framework for leasing needs, as planned under the Land Use Management Guidelines project currently being implemented. MINAGRI should consider consolidating ALIS under the Land Authority, possibly placing it under their management. 4. PSDAG, USAID/Rwanda, and other donors should continue to work with MINAGRI to leverage the legal framework in place for contract farming to engage potential investors, including expanding contract farming for selected high-value crops, as well as developing a contract dispute resolution mechanism. 5. Blending finance and TA has been effective in improving project bankability. PSDAG, USAID/Rwanda and other donors should focus on institutional capacity development with GOR entities and Rwandan finance institutions to strengthen linkages between SMEs seeking investments for projects and BDS and TA providers to better leverage financing. 6. USAID/Rwanda and other donors should consider innovations suggested by respondents to enhance the social inclusion of targeted beneficiary groups, which include: introducing agricultural mechanization, promoting digital technologies, and promoting access to existing financial services vi opportunities. Several cooperatives have also successfully implemented “youth start-up” grants in order to attract new youth members. LESSONS LEARNED 1. The flexible and ongoing nature of embedded technical assistance makes this type of support an effective strategy for the adoption of a facilitative approach to investment promotion. 2. The enabling environment in Rwanda makes the country well placed to realize improvements in access to inputs and markets through contract farming schemes that can expand farmer incomes. 3. The limited scale of agricultural sub-sectors challenges the development of investment funds in Rwanda. Nonetheless, an innovative blended finance approach, marrying TA and finance, can be effective in leveraging investment capital and expanding agricultural loan portfolios. 1 1.0 EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE The Private Sector-Driven Agricultural Growth (PSDAG) Activity is a United States Agency for International Development (USAID)-funded activity implemented by the Research Triangle Institute (RTI) from August 2014 to August 2019.3 The purpose of the PSDAG performance evaluation (PE) is to assess: 1. Progress made to date in achieving the activity’s two main objectives; and 2. The effectiveness of the technical approach and the activity’s responsiveness to support the Government of Rwanda’s (GOR) goal of increasing private investments in agribusiness in Rwanda. USAID/Rwanda and its implementing partner (IP) will use the findings of the evaluation to prioritize and strengthen activity implementation and help determine what steps to take to sustain its achievements. The evaluation’s results will also help inform the implementation of other related programs designed to advance engagement of the private sector in driving growth in Rwanda’s agriculture sector (see Annex A: Statement of Work). The evaluation was implemented by a five-person evaluation team (ET), assembled by ME&A, between May-July 2018. The ET included the Team Leader, a Policy Analysis and Development Specialist, an Agribusiness Market Systems/Value Chain Specialist, an Organizational Management Specialist, and a Local Evaluator, who supervised a survey team of 16 enumerators and focus group discussion (FGD) moderators (see Annex B: Evaluation Team). Home office staff at ME&A supported the ET in the field throughout the evaluation process. 1.2 EVALUATION QUESTIONS This evaluation answers the following evaluation questions (EQs) related to PSDAG’s objectives and crosscutting objectives: 1. Within the objective of assisting the GOR to increase private sector engagement, what types of interventions supported by PSDAG (public-private dialogue [PPD]; policy and regulatory reform, including restructuring Rwanda Development Board [RDB] investment targeting and promotion strategies; institutional strengthening; facilitating public-private partnerships; etc.) have been the most successful or effective in attracting private sector investment in agriculture? 2. To what extent are the PSDAG interventions supporting Objective 1 (which includes the Agriculture Land Information System [ALIS], support to the Ministry of Agriculture and Animal Resources’ [MINAGRI] budget process, restructuring RDB investment targeting and promotion strategies, support to the public-private dialogue structure and process, and aggregation services centers) likely to result in long lasting impact in attracting private sector investment in agriculture that will continue beyond the conclusion of PSDAG? 3. What are the risks to sustainability and the actions needed to address them within the remaining project term? (Note: In the context of this question, sustainability is defined as the ability of the GOR to continue the key interventions or gains achieved under Objective 1 after the conclusion of the PSDAG activity.) 3 USAID initially awarded PSDAG to International Resource Group (IRG). However, in January 2017, when RTI bought IRG, the PSDAG contract was transferred to RTI. 2 4. Which top three technologies and business model innovations promoted by PSDAG have resulted in the highest return on investments for farmers and for agricultural businesses? What were the reasons for any success or failure? 5. Has PSDAG’s grant facility been an effective tool in promoting investment? (Note that “effective” is defined as improvements to the grantees and review of the grants process including the activity hypothesis.) 6. How effective has PSDAG been in preventing double dipping from its grant recipients (i.e., is PSDAG providing grants to support proposals being funded by other donors)? 7. Did the PSDAG approach to social inclusion ensure the involvement of women, youth, and persons with disabilities (PWD?) Are there innovations or other approaches that could enhance the meaningful engagement of these targeted beneficiary groups? 2.0 ACTIVITY BACKGROUND 2.1 ACTIVITY DESCRIPTION Agriculture has been a major source of national income and growth for Rwanda and will continue to be a longer-term engine of growth. The sector accounts for 68 percent of employment directly and contributes up to 31 percent of gross domestic product (GDP), more than 50 percent of all exports of goods, and more than 25 percent of annual labor productivity growth.4 However, despite efforts by the GOR, the Rwandan agricultural sector still faces major challenges for growth. Some of these challenges include: low quality produce; lack of business skills and entrepreneurship; poor and/or inadequate agricultural infrastructure to enable farmers to access input and outputs markets; lack of sustainable markets and post-harvest facilities; limited capital to invest in farm development due to lack of access to funding from commercial banks; poor access to market information; lack of knowledge about modern farming methods including irrigation; lack of necessary equipment to use on the farm during planting, harvesting, and value addition; etc.5 The original contract value for the PSDAG was US$24,582,958. This was subsequently increased to US$25,425,958. PSDAG develops strategic partnerships with the private sector to increase investment in agriculture and expand economic opportunities for smallholder farmers through more competitive value chain market systems. The activity has provided support to small and medium-sized enterprises (SMEs) and farmer cooperatives to implement/upgrade new technologies and business models to address the issues mentioned above in order to respond to market opportunities. It has promoted new technologies and business models that include contract farming, cooperative professionalization, potato seed multiplication, irrigation mechanization, provision of agro-inputs, information communication drone technologies, and access to finance for farmers. PSDAG supports the GOR’s Rwanda Vision 2020 development pillar “Private Sector-Led Development”6 and the USAID/Rwanda Country Development and Cooperation Strategy (CDCS) goal of “Accelerating Rwanda’s progress to middle income status and better quality of life through sustained growth and reduction of poverty.” The overall goal of PSDAG is to “increase smallholder incomes by promoting private sector investment (internationally and domestically) that contribute to the GOR Vision 2020 aim of transforming agriculture into a market-oriented, competitive, and high-value sector.”7 In addition, PSDAG has two primary objectives, each with four results (see Figure 1). 4 “Second Economic Development and Poverty Reduction Strategy (EDPRS II) for 2013-2018,” GOR, 2013. 5 “Strategic Plan for the Transformation of Agriculture in Rwanda - Phase III 2013-2017,” MINAGRI, 2013. 6 “Rwanda Vision 2020 (revised 2012),” Republic of Rwanda, 2012. 7 “Contract No.: AID-696-C-14-00002/International Resources Group Private Sector-Driven Agricultural Growth (PSDAG), SECTION C – Statement of Work,” USAID. 3 Figure 1: PSDAG Results Framework Goal: Increase smallholder incomes by promoting private sector investment Objective 1: Assist the GOR to increase private sector investment Objective 2: Facilitate increased private sector investment in upgrading value chains Result 1.1: Support establishment and implementation of a comprehensive investment framework for private sector investment Result 2.1: Identify market opportunities and link value chain actors to identified opportunities Result 1.2: Identify and define the roles and responsibilities of public entities Result 2.2: Strengthen vertical and horizontal value chain linkages and build capacity in selected value chains Result 1.3: Strengthen capacity of relevant public entities Result 2.3: Strengthen input supply and other supporting services and improve smallholder access to these interconnected markets Result 1.4: Enhance the ability of government and business to productively engage each other Result 2.4: Improve the enabling environment for local and regional private sector investment Objective 1 includes the development and implementation of a roadmap for attracting and supporting private investment, including definition of roles and responsibilities within the GOR to improve the business enabling environment and work with potential investors from their expression of interest to the implementation of the investment. Objective 2 includes interventions to assist the private sector in identifying market opportunities and facilitating investments to take advantage of those opportunities.8 In addition, PSDAG has two crosscutting objectives: 1) Promoting the use of information and communications technology (ICT) and 2) Social inclusion of gender, youth (defined as ages 18-35), and PWD. In terms of implementation, PSDAG emphasizes three approaches: 1. Participation and ownership by the GOR and private sector partners and corollary institutional capacity building around investment promotion; 2. Facilitation by deploying a US$5 million grant fund, or “Value Chain Competitiveness Fund” (VCCF), to catalyze investments in technology and technical assistance, as well as access to finance facilitation and business development services (BDS); and 3. A portfolio approach around core value chains (maize, beans, Irish potatoes, and horticulture), although the activity also supports ICT, agro-input technology, and access to finance in addition to other commercially viable sectors (except tea and coffee). 8 “Contract No.: AID-696-C-14-00002/International Resources Group Private Sector-Driven Agricultural Growth (PSDAG), SECTION C – Statement of Work,” USAID. 4 Geographically, PSDAG interventions have been implemented in all five provinces of Rwanda, in the 23 districts shown in Table 1. Table 1: PSDAG Provinces and Districts Kigali Eastern Northern Western Southern Gasabo Kayonza Rulindo Rubavu Kamonyi Nyarugenge Kirehe Burera Nyabihu Muhanga Kicukiro Ngoma Musanze Rusizi Ruhango - Rwamagana - - Nyanza - Gatsibo - - Huye - Nyagatare - - Nyamagabe - Bugesera - - Gisagara Source: “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF)” 3.0 EVALUATION METHODS AND LIMITATIONS To conduct the PSDAG evaluation, the ET utilized a mixed-methods approach, which included both qualitative and quantitative data collection methods to answer the seven EQs. These data collection methods included the following: Document and Secondary Data Review: Prior to departure for fieldwork, the ET conducted a comprehensive review of activity documents related to PSDAG. In addition, the ET reviewed additional documents as they became available over the course of fieldwork. These documents included: • PSDAG activity documents and presentations • PSDAG analyses and studies • PSDAG monitoring and evaluation (M&E) plans • PSDAG annual reports • PSDAG quarterly reports • Secondary background references These documents are cited as footnotes throughout this Evaluation Report. They also appear as a list of references annexed to the report (see Annex C: References). Key Informant Interviews: For the purposes of this PE, KIIs were in-depth, semi-structured interviews employing an evolving subject-driven approach, which refers to an iterative process of interviews focused on broad thematic subjects, aggregating forward issues as they arise into subsequent interview topic lists. The KIIs were conducted with individuals or small groups of two to three key informants drawn from key stakeholder groups, face-to-face, by telephone, or over Skype. The Team Leader and Technical Specialists on the ET conducted all KIIs, taking detailed notes immediately or soon after completing each interview. Based on information available at the outset of this PE, the ET identified four key stakeholder groups for participation in the KIIs - 1) USAID/Rwanda and PSDAG staff; 2) banks and finance institutions; 3) GOR and development partners; and 4) PSDAG-assisted SMEs and cooperatives - and selected participants purposively from among these groups. Ultimately, the ET carried out a total of 66 KII (see Table 2), including 10 with USAID/Rwanda and PSDAG staff, seven with staff of banks and finance institutions, 23 with GOR and development partner staff, and 26 with staff at PSDAG-assisted SMEs and cooperatives (see Annex D: Contact List). 5 Table 2: Key Informant Groups by Stakeholder Group Stakeholder Group No. USAID/Rwanda and PSDAG Staff 10 Banks and finance institutions 7 GOR and development partners 23 PSDAG-assisted SMEs and cooperatives 26 Total 66 Over the course of conducting the KIIs, the ET also conducted site visits to processing and farming sites in order to provide interviewees with an opportunity to demonstrate firsthand the factors that facilitated or hindered outcomes of PSDAG activities, such as packaging and storage units funded through the PSDAG VCCF and stands of crops grown by farmers working with grantees. Site visits were selected to include both male and female respondents. Focus Group Discussions: For the purposes of this PE, FGDs were in-depth moderated discussions with groups of 6-10 participants. FGDs carried out over the course of this PE utilized a set of standard discussion guides, while employing a semi-structured, evolving subject-driven format that relied heavily on moderator probing, combined in this case with group discussion dynamics that allowed discussion moderators to explore different topics in depth (see Annex E: Evaluation Tools). To ensure meaningful discussion, the FGDs were constructed to be as homogenous as possible (where appropriate and feasible) in terms of status, sex, and age, and took place in venues where participants would be comfortable. Between May 22, 2018 and June 4, 2018, the ET conducted a total of 16 FGDs with 106 respondents out of a planned 120, or 88.4 percent. These were comprised of smallholder farmers belonging to activity￾supported local farmer cooperatives, including representatives of other groups, such as women, youth, and PWD. The FGDs were implemented in all PSDAG provinces with the exception of Kigali, and in five districts (one each in South, West, North, and two in East to reflect the distribution of PSDAG beneficiaries). Districts were chosen through a random identifier (ID) generated by MS Excel (see Table 3). Table 3: Focus Groups Discussions by Province/District and Sector Date Province/District Sector FGDs (#) Respondents May 22 South/Kamonyi Gacurabwenge, Rukoma 2 13 May 23 South/Kamonyi Gacurabwenge, Rukoma 2 13 May 24 West/Nyabihu Jenda 2 14 May 25 West/Nyabihu Jenda 1 10 May 28 North/Burera Gahunga, Nemba 1 7 May 29 North/Burera Gahunga, Nemba 1 7 May 30 East/Ngoma Mugesera 2 12 May 31 East/Ngoma Jarabana 1 6 June 1 East/Nyagatare Karama, Tabagwe, Mukama 4 24 June 4 East/Nyagatare Karangazi, Kagitumba 0 0 Total 16 106 Beneficiary-Based Survey (BBS) of Farmers: The ET implemented a BBS with a representative sample of beneficiary farmers who are members of farmer cooperatives in areas that have received activity support. The survey consisted of a set of largely closed-ended questions taking approximately 30-40 minutes to administer. As with the FGDs, the BBS was implemented in all PSDAG provinces with the exception of Kigali, and in five districts reflecting the distribution of PSDAG beneficiaries. Districts were chosen through a random ID generated by MS Excel. To select the sample of farmers, the ET used a two￾stage cluster sampling approach with a modified “take all” selection of beneficiaries. In this approach, the ET randomly selected one or more cooperatives in each of the selected clusters and then focused all data collection efforts within those cooperatives on a random sample of cooperative members. Under the modified take all approach, the ET used the established cooperative meeting place as a locus to which cooperative members were invited to participate in the survey. This approach was selected for its cost￾efficiency benefits (i.e., enumerators need not visit beneficiary farmers at their homes or farming plots). In selecting cooperatives, the ET distributed selection across cooperatives engaged in the bean, horticulture, Irish potato, and maize value chains and also ensured a selection of both direct and indirect cooperatives, e.g., direct cooperatives that received PSDAG in-kind grants and technical assistance (TA) and indirect cooperatives that have a business relationship with SMEs supported by PSDAG through a grants mechanism. Using the USAID Feed the Future sample size calculator for two-stage cluster designs with a modified take all selection of beneficiaries, the ET calculated a required sample size of 891 respondents for the farmer survey, as outlined in Table 4 below.9 Table 4: Calculation of Sample Size for Farmer Survey DESIGN OPTION 4 – Two-stage cluster design of farmer groups with modified take all selection of beneficiary farmers INDICATOR 2 – Value of incremental sales (collected at farm level) attributed to United States Government (USG) implementation - N Population of beneficiaries Estimate of standard deviation available? 83,388 NO - - If YES, write estimate here (in units of indicator): - - - If NO, provide estimates of minimum and maximum: - - max Estimate of maximum (per beneficiary) 210.00 - min Estimate of minimum (per beneficiary) 0.00 - s Standard deviation 35.00 - p Acceptable percentage error (Margin of error parameter) 10.0% - - Target value of indicator (Margin of error parameter) 83,388 - MOE Margin of error 8,338.8 - - Confidence level 95% - z Critical value from Normal Probability Distribution 1.96 - n initial Initial sample size 423.801 ADJUSTMENT 1 - Use adjustment 1? (YES for all survey design options but only if ninitial /N is greater than 5%) NO Finite population correction ninitial / N Ratio of initial sample size to population size (%) 0.994 nadj1 Adjusted sample size (1) 423.801 ADJUSTMENT 2 - Use adjustment 2? (YES for survey design options 1, 2, and 4) YES Design effect D Design effect 2.00 - - - - - nadj2 Adjusted sample size (2) 846 ADJUSTMENT 3 - Use adjustment 3? (YES for all survey design options) YES Non-response - Non-response rate 1.05 - nadj3 Adjusted sample size (3) 891 SAMPLE SIZE n final Final sample size 891 The ET calculated the sample size using the Feed the Future incremental sales indicator as the relevant sampling variable sales are higher-level outcomes that sit at (or nearer) the top of the activity causal chain 9 The calculations utilized the Feed the Future Sampling Guide for Beneficiary Based Surveys and its accompanying Sample Size Calculator (see: https://www.agrilinks.org/sites/default/files/resource/files/Sampling-Guide-Beneficiary-Based-Surveys-Feb2016.pdf). 6 7 and are thus of greater relative importance. Information on incremental sales and number of beneficiary farmers (83,388) was derived from PSDAG performance monitoring data as reported in its Fiscal Year (FY) 2017 Annual Report.10 For the purposes of this PE, and in the interest of striking a reasonable balance between statistical efficiency and cost-effectiveness, the ET assumed an average of 178 survey respondents per selected cluster, for a total of five clusters which, depending on the structure of cooperative membership in each cluster, translated into one or two cooperatives per cluster. (Feed the Future sampling guidelines recommend 15-35 respondents per cluster.) In the modified take all selection approach, survey respondents in each relevant cluster were selected using simple random sampling methods without attempting to stratify (or disaggregate) the sample by beneficiary (farmer) characteristics. In each cooperative, moreover, the ET over-sampled by randomly selecting 40 farmers to ensure reaching the target number of surveys per cooperative. With this method, if a selected farmer could not participate in the survey, the ET moved on to select the next available replacement on the list). Based on this sampling methodology, the ET surveyed 902 farmers in the five randomly selected districts, reflecting the distribution of the farmers across all provinces of Rwanda (excluding Kigali) (see Table 5). Table 5: Beneficiary-Based Surveys by Province/District and Sector Date Province/District Sector # of BBS Respondents May 22 South/Kamonyi Gacurabwenge, Rukoma 73 May 23 South/Kamonyi Gacurabwenge, Rukoma 48 May 24 West/Nyabihu Jenda 104 May 25 West/Nyabihu Jenda 100 May 28 North/Burera Gahunga, Nemba 112 May 29 North/Burera Gahunga, Nemba 92 May 30 East/Ngoma Mugesera 97 May 31 East/Ngoma Jarabana 86 June 1 East/Nyagatare Karama, Tabagwe, Mukama 116 June 4 East/Nyagatare Karangazi, Kagitumba 74 Total 902 3.1 DATA ANALYSIS Qualitative data obtained from document review and KIIs was compiled on a real-time basis to discern emerging trends and develop probing questions. Following the conclusion of data collection, the ET aggregated data obtained from these sources around common themes related to the seven EQs to identify potential emergent findings. To the extent possible, the ET also compared relevant PSDAG indicators and targets derived from M&E Plans. Data analysis methods employed by the ET to generate findings included: • Content Analysis: Content analysis entailed the ET’s intensive review of KII and FGD notes to identify and highlight notable examples of factors related to the EQs. • Trend Analysis: The ET used trend analysis to identify patterns of convergence (or divergence) that affected PSDAG outputs and outcomes. • Social Inclusion Analysis: The ET worked across all EQs to capture and compare the results of the activity as it specifically benefited (or did not benefit) women, youth, and PWD. The ET disaggregated all data collected through KIIs, FGDs, and the BBS by target inclusion group in order to analyze effects on these sub-groups of beneficiaries. 10 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 8 The ET also conducted systematic analysis of the quantitative data from the FGDs and BBS. For the BBS, the ET collected data using computer-assisted personal interview (CAPI) software to allow for quality control and analysis in near real-time, enabling fast decision-making by the ET for quality control and quick course corrections on fieldwork plans and data collection instruments. Furthermore, the ET edited this data progressively during and after the field survey. Raw data obtained via CAPI outputs was cleaned using Stata and imported to SPSS for analysis and to Excel for development of tables. The ET focused on the most robust findings—i.e., those that appeared with relatively more frequency across multiple stakeholders and were triangulated by several data sources—for qualitative data analysis. 3.2 POTENTIAL METHODOLOGICAL LIMITATIONS The design of this PE includes several potential methodological limitations: Activity Implementation Delays: The implementation of activity interventions has, in some cases, been delayed by events beyond PSDAG control; thus, the results timeframe may not have fully accrued at the time of this evaluation. In interviews with PSDAG staff, interviewees indicated that activity interventions under both Objective 1 and Objective 2 were delayed by turnover of senior activity staff, including the termination of the PSDAG Chief of Party (COP) in March 2015.11 In addition, PSDAG Annual Progress Reports describe delays in implementation of specific activities outlined in Memoranda of Understanding (MOUs) attributable to partner GOR agencies. For example: “The MOU with RDB, which was planned to have been signed in Quarter 1, was signed in Quarter 4 due to restructuring and staff changes. Consequently, there were delays in eight planned activities. In addition, during MOU negotiations with RDB, three activities originally requested by RDB and included in the FY 2017 Work Plan were dropped.”12 Return on Investment Calculation: Return on investment (ROI) is traditionally calculated as a financial equation generating a percentage as follows: R / I where R = Return to Investor and I = Total Investment Value Comparative evaluation of this ROI equation for technologies and business model innovations promoted by PSDAG is challenged by several factors. First, PSDAG investments rarely present discreet typologies of technologies and business model innovations, i.e., in numerous cases, investments were with companies that employed multiple technologies and innovations simultaneously, such as high-value crops, contract farming, and new marketing and processing technologies. Second, in many cases, return has not yet accrued, i.e., in many cases, SMEs visited by the ET were still in the start-up phase, or early development, and had yet to generate positive cash flow from investments. Furthermore, in many cases, the full value of “return” does not accrue to the investor, e.g., PSDAG. For example, a processor realizes return on processing equipment invested by the activity but does not realize return through increased income earned by supplier farmers, which would likewise need to be derived from baseline farmer income data across a wide variety of distinct farming systems. Additionally, in most cases, investment (grant) value is exogenous to total activity value, i.e., the investment (grant) amount represented specific components of a project (especially capital assets and TA), while the grantee self-funded construction and other costs. Finally, a number of PSDAG investments are “demand bound,” i.e., potato seed multiplication supplied an unmet demand for quality seed. However, further investment may not generate similar return now that seed producers supported by the activity are meeting this demand. 11 KII PSDAG Staff, Kigali, June 4, 2018. 12 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 9 Given these challenges, and as a proxy for ROI, the ET reports the BBS value of additional income derived from a specific investment associated with increased on-farm production, reduced post-harvest losses, and improved access to the markets, specifically assessing the technologies and innovations where KII and FGD respondents in KII suggested these effects are the greatest. Selection Bias: As some key informants may have either declined to be interviewed or did not speak the relevant languages, there was a possibility of selection, or sampling, bias during KIIs and FGDs. Respondents who agreed to be interviewed may have differed from those who did not in terms of their attitudes and perceptions, affiliation with government/non-government structures, socio-demographic characteristics, experience, etc. Recall Bias: Key informants and FGD participants may have systematically had difficulties remembering details about the past relative to the present. This presented potential issues of bias in their recall of information. In particular, recall biases may have made it difficult to accurately compare situations before and after activity interventions. Halo Bias: Key informants may have underreported socially undesirable answers or altered their responses in accordance with what they perceive as prevailing social norms. The extent to which key informants were prepared to reveal their true opinions may have also varied for some questions that called upon the respondents to assess the performance of their colleagues or people on whom they depend on for the provision of services. Absence of Counterfactual: The evaluation design was not able to generate a counterfactual, i.e., what the situation would look like without the PSDAG interventions, to compare to the situation with PSDAG interventions. Without a counterfactual, and without clear baseline measures of conditions at the inception of the activity, the ET had a limited ability to attribute observed results to PSDAG interventions in a statistically valid manner. Mitigation of Potential Limitations: The evaluation design addressed these potential limitations through a variety of methods. To minimize sampling bias, the ET sought to achieve a purposive sampling of key stakeholder groups and key informants within those groups for KIIs and FGDs that was as comprehensive as possible. In addition, the evaluation used a two-stage sampling approach for the FGDs, as well as the farmer BBS. These data collection activities, moreover, focused on activity-supported farmer cooperatives as a mechanism for achieving cost-effective administration. In addition, the ET guaranteed all key informants’ and beneficiaries’ anonymity to facilitate more open and honest discussion and more forthright responses. The ET also triangulated evidence from different qualitative and quantitative data sources, which served to increase the credibility of findings via validation from multiple data sources. Finally, the ET collaborated closely with PSDAG staff and sub-awardees to facilitate introductions and mobilize key informants and beneficiaries. This mobilization strategy helped ensure that key informants and beneficiaries were willing to participate in the evaluation, and that they were well informed of its purposes. 4.0 FINDINGS 4.1 EQ 1: MOST EFFECTIVE INTERVENTIONS TO ASSIST GOR Evaluation Question 1: Within the objective of assisting the GOR to increase private sector engagement, what types of interventions supported by PSDAG (public-private dialogue; policy and regulatory reform, including restructuring RDB investment targeting and promotion strategies; institutional strengthening; facilitating public-private partnerships; etc.) have been the most successful or effective in attracting private sector investment in agriculture? 10 Since its launch in August 2014, PSDAG has engaged with GOR entities, including MINAGRI, RDB, and the National Agricultural Export Development Board (NAEB), to “increase private sector engagement” in order to “attract private sector investment in agriculture.” Specifically, PSDAG works through technical specialists and dedicated embedded investment advisers (EIAs) to provide ongoing support to strengthen the overall GOR approach to investment and implement a range of discrete activities related to public￾private dialogue (PPD) policy and regulatory reform, institutional strengthening, and facilitating public￾private partnerships (PPPs). One of the PSDAG EIAs began work in 2014 at MINAGRI, and a second began work in 2016 at RDB. The MINAGRI EIA has provided ongoing TA to the Investment Support Unit (ISU) for privatization of MINAGRI flagship projects and programs. This EIA also serves as the focal point for the Industrial Development Export Council (IDEC) under the Ministry of Trade and Industry (MINICOM), and serves on several organizing committees for investment forums, such as the Grow Africa 2016 World Economic Forum, the 2016 Forum for Agriculture Research in Africa (FARA), and the ApiExpo Africa 2016. The EIA at RDB has provided ongoing TA to staff at the RDB investment unit for restructuring the board’s investment targeting and promotion strategies. PSDAG had initially intended that this EIA would also provide TA to the RDB’s Strategic Investment Department (SID), but it dropped this because the United Kingdom (UK) Department for International Development (DFID) was already providing this support.13 GOR Investment Approach: To promote private sector investment, “the GOR had been using several tools in the past, such as business plans and value chain one-pagers, but systemic improvements in investment lagged.”14 In 2012, the GOR collaborated with the United Nations Conference on Trade and Development (UNCTAD) to develop an investment guide to Rwanda, which highlighted agriculture as among the key industries that offer good domestic and international investment opportunities. However, “internal capacity to develop and market agriculture opportunities to potential investors proved to be a constraint.”15 Likewise, most of the GOR investment strategy focused on creating strategic PPPs for subsequent privatization.16 To address these constraints, PSDAG implemented a set of integrated activities related to PPPs, including restructuring RDB investment targeting and promotion strategies, institutional strengthening through a capacity building program, and adopting a more facilitative approach to PPPs. In line with its objectives, and as described by activity staff: “PSDAG works with [these] counterpart entities in strategy development and capacity building to position GOR investment promotion units and officers within these institutions to support the GOR goal of moving from a direct actor and investor in value chains to a facilitator of investment by providing the information and resources to investors that they need and by improving an enabling environment so that it is more conducive to private sector investment in agriculture.” – KII, PSDAG Staff As such, through the EIAs and other inputs, PSDAG has promoted a shift in the GOR investment promotion approach away from a focus on privatizing specific PPPs toward providing benchmarking, sector analyses, and sector competitiveness studies as well as capacity building. “Competitiveness studies conducted provided clear insights of specific value chains regarding their business viability to help potential investors making decision.” – KII, GOR Staff In addition, according to RDB staff, PSDAG EIAs work with their RDB and MINAGRI counterparts to evaluate the effectiveness of existing tools and methodologies.17 13 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 14 Ibid. 15 “Investment Guide to Rwanda - Opportunities and Conditions,” UNCTAD, 2012. 16 KII GOR Staff, Kigali, June 6, 2018. 17 Ibid. 11 This “systematic” approach is demonstrated in the way PSDAG approached sector analysis, as opposed to discreet support on “one off” privatization, in support to the GOR on the privatization of Uzima Chicken through a poultry sector study, as well as the Airfreight Competitiveness Study implemented by PSDAG in 2015. Figure 2: Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models One example of the more “systematic” PSDAG approach is the “Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models.” In 2015, PSDAG funded the Airfreight Competitiveness Study, which summarized the status of airfreight options from Rwanda to international markets and described the current logistics and infrastructure environment within the country to support airfreight services. Following up on this study, PSDAG supported an additional study in 2016 titled “Proposed Business Model for Airfreight Export Aggregation Services.” The rationale of this study was to design a model that would enable exporters to negotiate lower airfreight costs through the design of an aggregation services model for goods moving to the same destinations. This study described airfreight export commodity aggregation practices in neighboring countries and recommended the design of a practical and probable business model for export commodity aggregation services in Rwanda.18 One of the study’s main findings was that low volumes of ready-for-export agricultural products were causing airfreight prices for exports from Rwanda to be significantly higher than those from neighboring countries. To address this challenge, the study recommended increasing agricultural exports and creating an export aggregation center. Although the NAEB had anticipated this need and completed a facility that will house the aggregation center, the facility lacked the business model, and the operating plan needed to be fully operational. In this context, MINAGRI/NAEB requested that PSDAG develop the model for the aggregation center and provide suggestions on how it will be implemented. Based on this study, NAEB negotiated with RwandAir for a reduction in airfreight by 80 percent from (about US$2 to US$1 per kilogram [kg]). As result, five companies (including Garden Fresh) increased exports by US$1.2 million, and these companies are now using a NAEB-owned cold storage facility located near the airport to preposition products for export.19 Despite these outcomes, RDB staff state that results are yet to be fully realized: “The GOR is holding consultations with airfreight companies on prices and the business deal was concluded between RDB and RwandAir too recently. RDB and NAEB are just starting the implementation of the aggregation business model, so there are few tangible results yet.” – KII GOR Staff Specific PSDAG interventions implemented under activity Objective 1 that have made important incremental contributions to improving the enabling environment for attracting increased private investment in agriculture are described below. While the ET in most cases was unable to link specific investment flows to any particular intervention (beyond the illustrative examples provided), or to the interventions as a whole, the findings indicate that a number of activity interventions have been effective at addressing structural constraints to increased private agricultural investment and that, in the absence of activity interventions, these constraints either would have remained unaddressed or would have been addressed on a slower timetable and/or with a potentially lower level of technical expertise. Public-Private Dialogue: Since at least 2013, the GOR has had a legal framework and implemented PPD in order to “strengthen the legal and regulatory environment of business in the country, and thus, boost the investment climate in order to attract foreign direct investment (FDI) in key development sectors namely, agriculture, industry, and services.”20 In line with this objective, a key PSDAG intervention to support PPD was support to the Pilot National Agricultural Public-Private Dialogue (Agri-PPD) Framework for which the activity provided analysis, consultation, and revision. The objective of the Agri-PPD Framework was to “improve the 18 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 19 KII PSDAG Staff, Kigali, June 6, 2018. 20 “Second Economic Development and Poverty Reduction Strategy (EDPRS II) for 2013-2018,” GOR, 2013. 12 conditions for investment mobilization and to create more economic opportunity for smallholder farmers through increased value chain market integration.”21 According to activity documents, PSDAG interventions are primarily focused on “assessment of the current public/private dialogue process at multiple levels to support the redesign and revival of the PPD forums with the Private Sector Federation (PSF).”22 These documents note that PSF is supposed to be the primary forum for PPD; however, respondents reported that currently there are other forums. 23 As described by a District Official in Rubavu, “PPD is happening at all levels in many forums.”24 Also, PSDAG staff described creating additional PPD forums through the VCCF grant process: “[The] PSDAG grant contribution is a valuable support to extend the PPD Process at district levels. The objective is to establish effective and efficient mechanisms to solve issues pertaining the agriculture sector and agribusinesses through agriculture Public-Private Dialogue processes at district level and support farmers and agribusinesses to access more remunerative markets.” – KII, PSF Staff However, both GOR key informants and PSDAG staff described the effect of simultaneous PPD forums as “complicating” the process of raising issues identified at sub-national to national PPD forums due to “unclear information flow” and “multiple points of contact”. 25 In addition, one PSDAG staff stated that the PPD process is particularly complicated by multiple forums and is not effectively covering agriculture due to “convoluted institutional support, especially from RDB and PSF, and [therefore] does not work for agriculture.”26 Other respondents went on to say: “[There is a] need to dedicate staff to this process to avoid ‘confusion,’ and RDB needs a clearer process for dealing with issues raised in the PPD process at various forums.” – KII, PSDAG Staff According to KIIs with PSDAG staff, however, the activity has “so far only signed an MOU for PPD with PSF during 2017,” and hence activities under the MOU have only been underway since that date. 27 As a result, informants believe this restructured process has yet to fully realize its potential to address private sector investment issues related to agriculture through improved identification of issues. PSDAG staff report that, going forward, they foresee implementing a ranking process for issues identified in order to improve issue resolution. 28 “The MOU for PPD with PSF was signed in March 2017, and the grant agreement [was] signed in November 2017. The implementation started in March 2018. The main milestones include identification of issues and ranking, both at districts and national level, and conducting dialogue in five districts. About the progress, PPD processes in four districts were conducted and concrete resolutions made. But the remaining PPD in the remaining district[s] is planned with one at the national level.” – KII, PSF Kigali As such, the activity plans to “analyze the causal factors behind poor PPD performance on national agriculture issues and pilot the implementation of new, more effective mechanisms for 700 private and civil-sector stakeholders 21 Ibid. 22 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 23 Although the following list may not be comprehensive, additional PPDs described in KIIs include: 1. Agriculture Sector Working Group (ASWG). ASWG is a technical working forum through which the GOR and stakeholders meet to discuss agriculture sector and cross-sector planning and prioritization according to strategic plans and development programs. 2. Agro SME Cluster Forums: This forum engages SME forums selected by MINICOM to identify key challenges identified by SMEs and define action points and responsible agents to address these challenges. 3. AgriProFocus: AgriProFocus is a Dutch-based international network operating in Rwanda. This network brings together entrepreneurs, private sector companies, civil society organizations, “knowledge institutes,” and government entities together around specific questions, issues, and opportunities. 24 KII GOR Staff, Rubavu, June 1, 2018. 25 KIIs, GOR Staff, Kigali, May 22, 2018 and PSDAG Staff, Kigali, June 4, 2018. 26 KII PSDAG Staff, Kigali, June 4, 2018. 27 Ibid. 28 Ibid. 13 in order to realize further improvements.”29 In addition, PSDAG has supported the identification and training of seven “champions” (opinion leaders) in five pilot districts (Rubavu, Musanze, Gasabo, Nyagatare, and Nyaruguru). These include the Director of Agriculture and Natural Resources, the Director of the Business Development Unit (BDU), the district-level coordinators at the Business Development Fund (BDF), representatives for female agricultural sector stakeholders and staff at the Rwanda Youth in Agribusiness Forum (RYAF), district representatives, model farmers, and the chairs of district level PSF. Policy and Regulatory Reform: Several PSDAG activities addressed policy and regulatory reform through analysis, facilitating consultation/validation and drafting/revision of specific GOR strategic frameworks. While these frameworks could not be reasonably expected to attract specific attributable “private sector investment in agriculture” in and of themselves, they represent key strategic guidelines for improving investment and, as such, are fundamental to multiple PSDAG Objective 1 interventions. These include: • National Agribusiness Investment Promotion Strategy (NAIPS): The GOR has made the attraction of investment and its role in the Rwandan economy a key policy priority. To this end, it created the RDB to integrate all of the agency’s investment dealings. Since 2002, the GOR developed and implemented the National Investment Strategy with the aim to improve the prioritization of investment, both public and private, and to increase its impact on beneficiaries. NAIPS provides guidelines for all public-sector stakeholders, including MINAGRI and RDB, on promoting increased private sector agribusiness investment in Rwanda. It also provides measures to improve their performance in attracting private investors in the agricultural sector through an Investment Framework Management System, which is an “evidence-based iterative process” that will “build GOR capacity to plan, execute, measure, learn, and adjust implementation of the Agribusiness Investment Framework.”30 • Fourth Strategic Plan for the Transformation of Agriculture (PSTA-IV): The PSTA-IV, of which NAIPS forms a part, is a continuation of the previous PSTAs developed by MINAGRI since 2004. MINAGRI expects PSTA-IV to provide a policy framework for agricultural transformation covering the period 2018-2024, aligning national agriculture sector development activities to national, continental, and international objectives. At the national level, PSTA-IV will implement an updated National Agriculture Policy aimed at creating “a productive, green, and market and private sector-led agricultural sector,” as described in Rwanda Vision 2020.31 At the continental level, the framework will contribute to achieving the commitments under the Malabo Declaration and the Sustainable Development Goals (SDGs). Strategic areas of PSTA-IV include strengthened land management, market-led and profitable agriculture, strengthened private sector service delivery and investment (through PPD), promotion of high-value exports in value chains, enhanced focus on diversified animal resources (e.g., fisheries, poultry, pork), and investment in research and skills development. NAIPS was ultimately incorporated into PSTA-IV, which has become the primary agricultural program in Rwanda. Specific areas adopted from NAIPS in the current draft of PSTA￾IV include: information systems, Agri-PPD, investment promotion and aftercare, and PPPs.32 PSTA￾IV was subsequently approved by the GOR during the week of July 9, 2018 following a review by stakeholders. • PPD Workshop on Contract Farming: The GOR has pursued promotion of contract farming as a strategy to address limited access to inputs and output markets, as well as agricultural extension and financial services. Support activities have included: developing contract farming schemes; 29 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 30 KII, PSDAG Staff, Kigali, June 4, 2018. 31 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 32 Ibid. 14 facilitating contract-farming training to individuals from private companies in the maize, beans, potato, and horticulture value chains; and developing standardized contract farming templates to be shared with the private sector. Restructuring RDB Investment Targeting and Promotion Strategies: As noted above, PSDAG initially provided support for the privatization of specific PPPs slated for privatization. Subsequently, however, PSDAG took a more “systematic” approach to assist the GOR to enhance tools and methodologies for analyzing marketable opportunities and targeting investors in the agriculture sector, including supporting sub-sector selection, benchmarking competitiveness activities, and developing investment promotion materials with support from the EIAs. 33 In KIIs, RDB respondents specifically noted the effectiveness of the EIAs, due to the flexible nature of their assistance: “[The] EIA has been particularly helpful in addressing needs. The flexible nature of EIA support is very good. There is room for flexibility in the topics to address specific needs as they arise.” – KII, GOR Staff “The EIA is very important and has been useful in supporting a range of ‘tasks.’ The EIA works at the investment support unit and supported negotiations with investors.” – KII, GOR Staff In addition, PSDAG activities will support capacity building and institutional resources of GOR counterparts with regards to market opportunities, which may include, but not be limited to, speaking to investors, understanding different types of investors and their needs, and building effective networks to engage with new investors.34 At the same time, PSDAG will support benchmarking and selection of specific sub-sectors, as well as “guidance on how to select and maintain a local pipeline of investment-ready businesses and opportunities to present to interested investors.”35 Institutional Strengthening: In line with the PSDAG strategy of shifting to a more systematic approach to building GOR capacity, the activity has implemented a capacity building program with GOR partner entities and provided an institutional strengthening grant to PSF. PSDAG has also implemented several reviews of GOR agriculture support programs in order to make them more suitable to investment support. For example, PSDAG implemented a review of the National Input Subsidy Distribution Program.36 The program addresses one of the most significant challenges Rwandan smallholder farmers face, which is the lack of working capital for purchasing inputs (e.g., improved seeds, fertilizers, and pesticides). This consequently prevents them from potentially maximizing the agricultural production needed to increase household income and achieve self-sufficiency in food. As a response to this challenge, in 2009, the GOR expanded the use of agricultural subsidies, including a subsidy voucher system designed to expand the role of the private sector in supporting the supply of farm inputs. As part of this effort, PSDAG plans to use the findings of this report to support the GOR to strengthen input supply and other support services and improve smallholder access to inputs, equipment, and market information, as well as improve formal and informal linkages with buyers with the aim of facilitating partnerships and private investment across agro-dealers, equipment companies, financial institutions, and private buyers.37 In addition, PSDAG plans to continue supporting the fertilizer input and distribution system by assisting MINAGRI in providing capacity building and mentorship in the use of the mFarms ICT 33 KII PSDAG Staff, Kigali, June 4 and July 16, 2018. 34 KII GOR Staff, Kigali, June 6, 2018. 35 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 36 Ibid. 37 Ibid. 15 inventory management system, which will also generate feedback on changes in the input subsidy program.38 Likewise, PSDAG implemented a review of the “Guidelines on Marketing Irish Potato through Potato Collection Centres (PCCs).” Irish potato is one of the key priority crops targeted under the MINAGRI Crop Intensification Program (CIP).39 Since CIP’s introduction in 2006, Irish potato productivity has steadily increased as a result of enhanced advisory services and efficient use of agricultural inputs, mainly fertilizers and improved seeds, as farmers get these inputs through the government subsidy scheme. Following this increase, the potato trading system has faced a large disparity between the price dealers pay farmers and the final price consumers pay for potatoes, especially in Kigali. Consequently, farmers complain that they sell their products below the cost of production. They also expressed concern that middlemen exploit price fluctuations.40 To address these issues, the GOR developed guidelines on marketing Irish potato through potato collection centers (PCCs) as a market regulation measure to ensure farmers receive fair prices. PSDAG identified Irish potato as a priority value chain. As such, it has supported the GOR to improve the cost of marketing Irish potato through PCCs. Subsequently, in Project Year (PY) 4, PSDAG activities focused on assessing market opportunities and constraints within the Irish potato value chain, including identifying and mapping major stakeholders working in the value chain and developing the potato seed production capacity with private sector investors.41 Additionally, PSDAG has helped build MINAGRI’s capacity to implement a land mapping system in order to provide potential investors with accurate and timely information on available land for agricultural activities, including through the “Budget Allocation for the Smart Agriculture Information System (SAIS) and ALIS for GOR FY2017/2018 to 2023/2024.” The GOR recognizes that one of the key constraints to private sector investment in the agricultural sector is access to land ownership usage and other land-related information. To address this constraint, and in collaboration with MINAGRI, PSDAG has provided technical support for the development of ALIS, an ICT-based platform that provides accurate and up-to￾date information on available land for new projects. The system provides a land information database business tool for both MINAGRI and prospective investors in the sector. It incorporates information on all land parcels owned by MINAGRI throughout the country, and through it prospective investors are now able to locate land that meets the selection criteria for their prospective investments. Key information available within the system includes: plot size; agro-ecological conditions; and proximity to water sources, feeder, or main roads. In addition, the system serves as a tool for MINAGRI to monitor and track usage of available arable land and to provide up-to-date information on available infrastructure. However, the use of ALIS has been constrained by the absence of a regulatory framework and legal process for implementing land leasing, which PSDAG is currently addressing by developing Land Use Management Guidelines for MINAGRI in PY 4.42 Additional capacity building activities have been scheduled, which GOR staff consider essential for implementing their mandates: “A training on how to communicate with investors has been organized by PSDAG during the week of May 7th to 11th 2018. Other modules on contract negotiation and in marketing will follow in the future. These trainings are really important. The most important thing is to keep increasing our knowledge of how to respond to the potential investor data needs for [a] well-informed decision-making process.” – KII, GOR Staff 38 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 39 “Strategic Plan for the Transformation of Agriculture in Rwanda - Phase III 2013-2017,” MINAGRI, 2013. 40 Ibid. 41 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 42 KII PSDAG Staff, Kigali, June 6, 2018. 16 “These (the trainings) were very useful, because our officials are dealing with investors every day. They helped us to identify which sector to focus on, how to improve the investment environment, and how promote investment. We are always dealing with investors, guiding them to which sector to invest in, competitive advantage.” – KII, GOR Staff Facilitating PPPs: As noted previously, PSDAG support to the GOR has shifted from a focus on business plans and analysis of specific PPPs created by MINAGRI and slated for privatization to a more “sector￾oriented” approach, which provides analysis of entire sectors in order to attract investments throughout the value chain.43 One example of this shift to a more sector-oriented approach is the implementation of a “Poultry Competitiveness Day-Old Chick Study.” In 2016, PSDAG supported MINAGRI to conduct a poultry competitiveness analysis.44 This study identified the determinants of the high cost of day-old chicks in Rwanda.45 The study found that exempting feed producers from the value-added tax (VAT) is an effective measure to incentivize and facilitate private sector investment, thereby making the poultry industry more competitive domestically and regionally. MINAGRI used the study’s findings to inform a policy decision to exempt locally produced feeds from the VAT. “They produced a white paper on removing [the] VAT on animal feed, which MINAGRI used to convince MINCOM to remove [the] VAT and subsequently increased competitiveness of [the] domestic animal feed sector.”- KII, GOR Staff Subsequent to the decision to exempt the VAT on feeds, there are an increasing number of investors in the poultry feed industry, and the National Hatchery PPP was privatized through a US$3 million investment by Uzima Chicken (Ethio Chick).46 4.2 EQ 2: INTERVENTIONS MOST LIKELY TO LEAD TO LASTING CHANGE Evaluation Question 2: To what extent are the PSDAG interventions supporting Objective 1 (which includes the ALIS, support to MINAGRI’s budget process, restructuring RDB investment targeting and promotion strategies, support to the PPD structure and process, and the aggregation services centers) likely to result in long-lasting impact in attracting private sector investment in agriculture that will continue beyond the conclusion of PSDAG? A number of PSDAG interventions offer reasonable potential to exert a long-lasting impact on attracting private sector investment in agriculture beyond the life of the activity. These are discussed below. MINAGRI Budgeting Process and ALIS: The ALIS land-mapping system has received significant enthusiasm from MINAGRI officials as “a very useful tool to guide investors in agricultural businesses” by providing timely information on available sites for new investment projects.47 Going forward, MINAGRI officials have stated that, in order to more fully realize the potential of the system in facilitating investments, they intend to “focus on the (corollary) development of high-value crops, capacity building of farmers, and improvement of infrastructure facilities of the commodities collection centers (to complement ALIS’ potential as a marketing tool).”48 On the other hand, district-level GOR officials stated that they are aware of tool but noted that it is not yet used at the district level, although they have high expectations for this to occur soon. “We expect it to be rolled out for use in the near future. We need this tool.” – KII, GOR District Official 43 KII GOR Staff, Kigali, June 6, 2018. 44 “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2016 Quarterly Progress Report: April 1, 2016 – June 30, 2016,” IRG for USAID, July 2016. 45 Jenkins, Dr. Glenn P., et. al. “Poultry competitiveness Study,” IRG with MINAGRI, September 1, 2016. 46 KII PSDAG Staff, Kigali, June 6, 2018. 47 KII GOR Staff, Kigali, June 6, 2018. 48 Ibid. 17 In KIIs, respondents provided a mixed assessment about the long-term impact of PSDAG TA on MINAGRI’s budget process around ALIS. PSDAG support specifically focused on securing Ministry of Finance and Economic Planning (MINECOFIN) embedded funding for SAIS in the GOR budget cycle, and ultimately US$18 million over seven years was obligated. While MINAGRI officials in Kigali noted this outcome, they also described the need for “improvements,” highlighting coordination at the regional level as requiring further development, although this is beyond the current scope of PSDAG objectives. 49 “The project has not been involved in the district budgeting process, and no specific projects have been funded with the knowledge of the planning department of the district.” – KII, GOR District Official Restructuring RDB Investment Targeting and Promotion Strategies: PSDAG interventions to support restructuring of RDB investment targeting and promotion strategies have included a benchmarking analysis of Rwanda’s investment attractiveness relative to its regional competitors for FDI, selection of seven priority sub-sectors for targeted investment promotion, and ongoing support through the EIA. Officials at RDB positively assessed the PSDAG approach to restructuring investment targeting and promotion strategies through a more systematic approach and focusing on institutional capacity building. “PSDAG support has been more about training on processes (as opposed to one-off privatization of specific PPPs), which is more helpful in building our capacity.” – KII, GOR Staff In addition, GOR informants cited interventions related to sector-wide improvements as most likely to lead to lasting change. For example, RDB staff described an initiative to improve poultry competitiveness as “likely to lead to lasting impact because of high interest of investors in the industry and high demand of poultry products resulting from the growth of tourism industry and high rate of urbanization in the country.”50 Along these lines, contract farming was also cited as a business model that will also result in lasting impact because “it supports the government to address key challenges that smallholder farmers are facing, namely limited access to finance, advisory services as well as markets.”51 In addition, agro-processing industries will also increase their operational capacities because farmers will be able to supply sufficient raw materials through contract farming.52 Public-Private Dialogue: As noted, PSDAG has taken several measures to address the performance of PPD, including review and revisions to the PPD framework, identification of district champions, VCCF grants for PPD forums, and capacity building with PSF (see EQ 1: Findings). Nevertheless, one USAID respondent indicated that GOR discussion of issues facing private sector business remains limited and that the GOR needs to be encouraged to engage more substantively. Likewise, this respondent suggested that PSF needs to be “empowered to exercise its advocacy role more effectively.”53 In a KII with PSF staff, the Director of the Agriculture Chamber of Commerce stated that he has only been in his role for two months, which may account for some degree of inactivity at this unit.54 Although not yet fully realized, the potential long-lasting impact of the PPD process was more highly assessed by GOR officials at the district level than in Kigali, albeit with caveats at this level as well: “Agri-PPD meetings have been successful. We are getting feedback on policies and procedures. [However] there hasn’t been consistency on staff.” – KII, GOR Staff 49 KII GOR Staff, Kigali, May 23, 2018. 50 KII GOR Staff, Kigali, June 6, 2018. 51 KII GOR Staff, Kigali, May 22, 2018. 52 Ibid. 53 KII USAID Staff, Kigali, July 11, 2018. 54 KII GOR Staff, Kigali, May 22, 2018. 18 “PPD is generating results. One example is taxes on rice (VAT). Millers were paying farmers a lower price because they had to pay VAT on milled rice.” – KII, GOR Staff Notably, a key issue raised by informants in KIIs for inclusion in the Agri-PPD process was aflatoxin, which African Improved Foods raised as a significant raw materials problem requiring sector-wide coordination, although PSDAG partners in the maize value chain are currently investing in measures to control this threat. 55 Aggregation Services Centers: In KIIs, respondents were unable to provide an assessment of the potential for long-lasting impact of the export aggregation services model. However, they described this as “a new concept, likely to lead to long-lasting impact (given the high political support to increase exports to reduce trade deficit).”56 4.3 EQ 3: RISKS TO SUSTAINABILITY Evaluation Question 3: What are the risks to sustainability and the actions needed to address them within the remaining project term? (Note: In the context of this question, sustainability is defined as the ability of the GOR to continue the key interventions or gains achieved under Objective 1 after the conclusion of the PSDAG activity.) In KIIs, respondents enthusiastically assessed PSDAG VCCF grant activities as sustainable, largely due to the business nature of the grantees: “The project and the company exit strategies are clear, and the project sustainability will depend on the strong links between stakeholders and the role and responsibilities of every stakeholder. This will be entrenched in the training and coaching of the partners.” – KII, SME The investment sustainability of PSDAG interventions is ensured since most of the project outcomes are led by the cooperatives and the market for their produce both outside and inside Rwanda.” – KII, SME “The VCCF will obviously not continue, but SMEs are very likely to continue.” – GOR Staff On the other hand, risks to sustainability cited by respondents were limited to two PSDAG activities: 1) MINAGRI budgeting process and ALIS and 2) PPD. MINAGRI Budgeting Process and ALIS: In a KII, one GOR respondent said that ALIS needs “improvement and MINAGRI and Land Management Office of [the Ministry of Lands and Forestry] MINILAF should work together to make the tool user friendly and dynamic because land regularization and usage are a dynamic process and businesses should work based on an updated agriculture land database.”57 PSDAG staff agreed with this assessment, stating: “ALIS needs a dynamic updating process, and there needs to be a process for leasing land between Rwanda Land Management and Use Authority and ALIS. Rwanda Land Management and Use Authority issues land titles.” – PSDAG Staff Specifically, the sustainability of ALIS required MINAGRI’s engagement in prioritizing and advocating for MINECOFIN to allocate resources to the land system through the planning and budget process. It is in that context that PSDAG and the MINAGRI Planning Directorate have worked together to secure a budget for the implementation of SAIS and ALIS and the elaboration of a new Project Profile Document that became effective in July 2018 in the budget law for FY 2018/2019. To ensure their sustainability beyond the PSDAG activity life cycle, PSDAG has supported MINAGRI in developing proposals and 55 Ibid. 56 KII GOR Staff, Kigali, May 23, 2018. 57 Ibid. 19 justifications for increased budget allocations for implementation of these programs and new staffing resulting in the seven-year budget allocation of US$18 million. PSDAG also seeks to add related tasks and outcomes to official job descriptions and performance contracts.58 However, one respondent in KIIs suggested that the new Rwanda Land Management and Use Authority would be better positioned to manage ALIS, as well as be able to provide information on lands not under MINAGRI control, and recommended the system be transferred to the authority’s mandate.59 For ALIS to be sustainable, there is a need to complete and finalize further steps. These include the development of SAIS, which is an integrated agriculture land information system that includes ALIS 2 (a simplification of the investor application of ALIS 1); the development of a more complex management application that can manage the large quantities of data requested; and the inclusion of improved weather information for agriculture from the International Center for Tropical Agriculture (CIAT)-sponsored data sources.60 This is currently planned through technical support from Esri Rwanda Ltd. 61 Other steps include the establishment of a clear and transparent process for reviewing and processing applications in a timely manner; review of the current Land Lease Client Charter, the Land Use Consolidation Ministerial Orders, and any other processes proposed to manage agriculture land; and development of the criteria, guidelines, and processes for managing public land concessions and private land demarcated for agricultural use according to Rwandan law.62 Project documents indicate that development of Land Use Management Guidelines has been underway since November 2017 and is set to conclude in Q4 of FY2018. Public-Private Dialogue: As noted, despite measures taken by PSDA – including review and revisions to the PPD framework, identification of district champions, VCCF grants for PPD forums, and capacity building with PSF – numerous respondents, including GOR and PSDAG staff, described challenges to the performance of PPD. These include an inadequate resolution process for issues identified in sub-national and alternative forums, GOR engagement with enterprises, and poor ownership by PSF. “Agro PPD is not taking off. Quarterly meetings should be strengthened and should be taken seriously for success in the partnership dialogue and to get feedback on policy implementation. [Nonetheless] discussions around tax exemption, postharvest handling, and mechanization are good examples that showed how the PPD can potentially be a good forum to make progress.” – KII, GOR Staff In KIIs, respondents suggested that elements required for PPD sustainability include strong coordination, ownership, and strong political support to materialize the PPD framework and ensure it yields impactful results. “PPD is poorly coordinated between stakeholders, and there are problems in dealing with issues at various levels and between various forums. These need to be addressed.” – KII, GOR Staff 4.4 EQ 4: TOP THREE TECHNOLOGIES AND INNOVATIONS Evaluation Question 4: Which top three technologies and business model innovations promoted by PSDAG have resulted in the highest return on investments for farmers and for agricultural businesses? What were the reasons for any success or failure? PSDAG has used two mechanisms for partnering with private sector companies that promote the adoption of improved technologies and business model innovations: 1) TA under an MOU and 2) grant funding support through the VCCF. The activity identified partners and VCCF grantees through Annual Program Statements (APS), which included a series of Requests for Applications (RFAs) seeking 58 KII GOR Staff, Kigali, June 6, 2018. 59 KII, USAID Staff, Kigali, July 7, 2018. 60 PSDAG FY 2017 Annual Progress Report, October 2017. 61 Esri Rwanda Ltd. is a developer of geographic information systems (GIS) software: https://www.esri.rw/. 62 PSDAG FY 2017 Annual Progress Report, October 2017. 20 cooperatives, SMEs, and “other types of applicants” that could “increase investment in agriculture and expand economic opportunities for smallholder farmers through more competitive value chain market systems.”63 Specific RFAs prioritized applications under the following areas: • Cooperative Professionalization; • Women’s Leadership & Entrepreneurship; • Asset Financing; • Innovation Challenge Fund (subsequently cancelled); and • Institutional Capacity Building. PSDAG provided grants to private businesses, cooperatives, and private sector associations in the areas of agro-processing and value addition, agro-input technologies and services, post-harvest handling and marketing, and institutional capacity building. In most cases, grants provided funding for TA and capital assets. In all cases, applicants were required to provide a significant co-investment. As of May 18, 2018, PSDAG had supported 181 partners through the VCCF leveraging a total of approximately US$12.3 million in co-investment (Table 6).64 Table 6: VCCF “Investments” Summary Investment by Type of Beneficiary and Primary Value Chain Number of Grantees Estimated Grant Value ($) Leverage Commitment ($) Agro-Processing/Value Addition Horticulture 11 $1,004,497 20% $2,179,987 Maize 16 $929,473 18% $3,564,918 Other 5 $418,513 8% $1,408,579 Potato 2 $144,486 3% $186,849 Beans 1 $49,087 1% $56,653 Subtotal 35 $2,546,056 51% $7,396,985 Agro-Input Technologies and Services Cross-cutting 7 $850,352 17% $2,378,096 Potato seed 7 $590,895 12% $1,099,206 Horticulture 2 $288,147 6% $781,564 Subtotal 16 $1,729,394 34% $4,258,866 Post-Harvest Handling & Marketing Support to Cooperatives Maize and beans 98 $227,348 5% $239,500 Potato 13 $186,390 2% $88,270 Horticulture 15 $7,447 0.30% $16,324 Subtotal 126 $421,184 7% $344,094 Institutional Capacity Building for Private Sector Associations Cross-cutting 3 $380,091 8% $231,764 Other 1 $41,303 1% $28,879 Subtotal 4 $421,394 8% $260,643 Grand Total 181 $5,040,636 $12,311,950 Source: “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF)” This section discusses the technologies and business model innovations promoted by PSDAG that have produced the most significant change in incomes for farmers and for agricultural businesses, as suggested by the BBS, KIIs with private business representatives, and farmer FGDs. While the ET was unable to 63 “PSDAG Annual Program Statement (APS) No. IRG- APS-17-01.” 64 Pfeiffer, Kirsten, Chantal Umuhire, and Liliane Gasana, “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF),” PowerPoint presentation, IRG for USAID, May 18, 2018. 21 calculate an ROI for these technologies and business models (see Section 2.5 above), five emerged from the BBS, KIIs, and FGDs as being particularly effective: contract farming; improved post-harvest handling; commercial crops, including pineapple and macadamia; potato seed multiplication; and linking farmers to markets. Rather than exclude one of these, the report discusses all five in no particular order of importance. Contract Farming: PSDAG VCCF grants have linked numerous farmer cooperatives to SMEs and to new inputs and markets through contract farming schemes using a facilitation approach. This, in turn, has linked cooperative members to new sources of finance for production using the contracts to collateralize loans. “The contract farming model has been an effective tool for smallholder farmers to access finance. Many farmers’ cooperatives approached [micro-finance institutions] MFIs and successfully managed to get small loans for seeds and agriculture equipment using contract farming as a collateral.” – KII, SME Staff “Contract farming has proven successful for both the company and farmers. Farmers are well connected to the company, have a guaranteed and sustainable market. Annual income for farmers increased/doubled compared with before the partnerships with PSDAG.” – KII, SME Staff Out of 16 cooperatives surveyed, 10 reported applying for loans from financial institutions, of which four were successful. The institutions from which they received loans include Duterimbere, Clecam, and Savings and Credit Cooperative Organizations (SACCOs), with the amounts obtained ranging from RWF 4 million (Rwandan Francs) to RWF 10 million and contract farming agreements serving as collateral for the loans. In the BBS, 25 percent of respondents who obtained loans indicated that the funding was used for the construction of maize-drying hangars, while 75 percent used the funds to purchase agriculture inputs, including improved seeds, fertilizers, and pesticides. According to the BBS, 80 percent of the farmers receiving loans reported an increase in production. Specifically, they reported that maize production increased, on average, from 15 kg per acre in 2014 to 34.6 kg per acre in 2018; beans from 9.44 kg per acre in 2014 to 13.5 kg per acre in 2018; and Irish potato from 65.6 kg per acre in 2014 to 82.3 kg per acre in 2018 (Table 7). The BBS further found that through contract farming, farmers have increased access to both markets and improved income as a result of increased crop production and sales. Table 7: Change in Agricultural Productivity at PSDAG-Assisted Cooperatives Value Chain Unchanged Increased Decreased Maize 7.4% 86.1% 6.5% Beans 21.4% 64.0% 14.6% Irish potato 6.5% 89.2% 4.3% Horticulture 9.5% 82.3% 8.2% Average 11.2% 80.4% 8.4% In KIIs, banks also described contract farming schemes as ideal for facilitating access to finance for smallholder farmers who would otherwise not qualify for financing, noting that this mechanism has facilitated an increase in their agricultural lending portfolios: “Contract farming is among the key topics that we discussed during the training as a means of supply chain governance. After the training we piloted the model, and it was successful. The contract farming model increased our agriculture portfolio from 14 percent to 24 percent within the last two years as a result of contract farming, which contributed to the decrease of default rates. It is a great tool overcoming factors that constrain smallholder commercialization, including finance deficiencies.” – KII, Bank Staff 22 “No new products per se, but with increased on-lending through contract farming cooperatives, loan amounts have been between 4,000,000 to 10,000,000 [Rwandan] Francs.” – KII, Bank Staff The CEO of another finance institution, to which PSDAG provided TA to increase its agricultural loan portfolio, agreed with this positive assessment of contract farming schemes in expanding smallholders’ access to finance: “By financing (through) this model, the company successfully expanded the agriculture loan portfolio with an increase of 100 million Francs over one year. No repayment default reported up to date. The business model was proven successful as a result of capacity building of farmers.”65 – KII, Bank Staff Furthermore, in KIIs, several respondents described contract farming mechanisms as resulting in smallholder access to new commercial markets. For example, one “innovative technology (introduced through contract farming) contributed to a consistent market access for smallholder farmers”66, while another stated that a contract farming scheme supported by PSDAG led to “increased volumes of high-value products.”67 One SME respondent described how a contract farming scheme had provided his company access to new regional and international export markets for smallholder products: “We got the grant for certification and automatization system for the processing plant, so we could increase sales. We are selling now in East Africa, Ghana, and in Europe, so the contract farming linked the farmers to these markets.” – KII, SME Staff Notwithstanding, several KII informants also suggested that an effective mechanism for contract dispute resolution and enforcement is “crucial in order to preserve the rights of the parties” and, ultimately, to ensure the success of contract farming schemes in providing finance.68 In KIIs, bank staff indicated that contract enforcement remains a significant major challenge (this is a component of PY 4 district-level PPD activities): “Contract farming is a means of supply chain governance, and sometimes it is affected by external or internal factors, which impact on contractual obligations. The party’s obligations are sometimes interrelated, and the ability of one party to perform depends on the prior compliance by the other party. In this scenario, the financial services provider, as the third party in the agreement, suffers from the conflicts which impact on the repayment of the loan by farmers.” – KII, Bank Staff Improved Post-Harvest Handling: PSDAG entered into partnerships with several private sector companies that implemented strategies for building vertical supply chain linkages. In addition, the activity worked with 60 cooperatives under its Cooperative Professionalization program that included post￾harvest crop quality management. According to one SME staff member, “PSDAG was able to identify those buyers and aggregators that have a strong interest in investing in the farmer supply chain, in particular to improve quality, aggregation, and reliability.”69 As a result, 126 farmer cooperatives accessed post-harvest handling upgrades, resulting in 60,000 farmers accessing new markets for their on-farm production.70 (For an example of how this worked in practice, see the case study in Figure 3). “Improved [post-harvest handling] improves consistency of product quality and improves market linkages. This can in turn improve the capacity of farmers to attract investment capital for improved production.” – KII, GOR Staff 65 KII, Bank, Kigali, May 24. 66 KII, Development Partner, Kigali, May 22, 2018. 67 KII, SME, Kigali, May 25, 2018. 68 KII, Bank, Kigali, June 5. 69 KII, SME, Kigali, May 24. 70 Pfeiffer, Kirsten, Chantal Umuhire, Liliane Gasana “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF),” PowerPoint presentation, RTI for USAID, May 18, 2018. 23 “The purchase of new processing and drying machinery, with the potential of upgrading the cassava leaves processing technologies, was for an electric drying machine to manage growing demand for the product, a blanching machine to replace existing equipment, and six electric sealing machines to increase packaging capacity. This led to high volumes of production. We raised prices for [the] purchase of fresh cassava leaves from farmers as an incentive to continue encouraging production and supply of fresh cassava leaves.” – KII, SME Staff Figure 3: Case Study – Les Minoteries Industrielles Import-Export (MINIMEX) Case study: Les Minoteries Industrielles Import-Export (MINIMEX) milling facility based in Kigali signed a grant facility agreement with PSDAG in February 2017 to promote post-harvest handling technologies for high-quality maize producers. With PSDAG support, the company introduced a number of improved technologies and quality management practices into the smallholder maize supply chain with the objective of reducing average post-harvest maize losses. The company has a contract with 8,685 smallholder farmers (3,147 women) in districts in the Eastern Province. It also entered into joint ventures with two other companies, one providing extension services and another one in charge of post-harvest handling. Farmers were organized into cooperatives, acquired proper drying and storages equipment with PSDAG support, and engaged in supply of quality maize to address issues of inconsistent and poorly tracked volumes. Twenty-one cooperatives were supplied with assorted post-harvest handling equipment that improved the ability of farmers to manage quality after harvest, which is particularly important in Rwanda where the humidity is high and rains are possible during harvest. Farmers received: 1) extension services, which are crucial to achieving high yields and meeting quality standards; 2) timely coaching on post-harvest handling techniques; and 3) a higher price (for high-quality maize) resulting in higher incomes. According the milling plant CEO, “By developing the maize sector through linking thousands of maize farmers to a guaranteed market, the project helped in creating a sustainable source of income for the rural poor, as well as increasing access to affordable food for the urban poor.”71 Commercial Crops, Including Pineapple and Macadamia: PSDAG selected four core value chains: Irish potato, beans, maize, and horticulture (fruits, vegetables, and nuts). However, the activity also worked with commercial, or high-value, crops, which refers to non-traditional crops, such as intensive vegetable, fruit, and nut production; chilies; and flowers that are mainly grown for their cash value in domestic and export markets. Under grant agreement partnerships, PSDAG engaged several private partners and cooperatives that are investing in their supply chains with high potential market opportunities. Activity documents suggest that through the VCCF, these companies increased agricultural production and value addition of these selected high-value crops, which had a positive impact on their business growth, in addition to increasing farmers’ incomes. Farmers also benefited from high premium prices. Four-hundred farmers are actively involved in the chili production and supply. The total value in cash is 70 million Rwandan francs over last 12 months. – KII, SME Staff “There is high demand for organic products related to macadamia over the world. Farmers benefited from high premium prices. We export to Vietnam; the market is not yet satisfied. Increased production due to PSDAG financial support to upgrade farming practices for farmers, raised the volume from 20 to 80 tons. Also, export[s] shifted from one to three times a year due to [the] increased volume of production. Two thousand farmers are involved in production and benefit from high-value markets, and there is an increase of farm gate price from 750 to 2,000 RWF per kg.” – KII, SME Staff Aggregate results from PSDAG’s partnerships indicate that the number of smallholder farmers who increased incomes by applying improved technologies or management practices in these high-value crops was about 142,875, including 63,236 women. Income increases were due to stronger relationships with 71 KII, SME, Kigali, May 24. 24 buyers, resulting in $20 million in gross farm incomes for smallholder farmers (US$8 million for women).72 (See Figure 4 for a case study of the Tuzamurane pineapple farmer cooperative.) Figure 4: Case Study – Tuzamurane Cooperative Case Study: Tuzamurane is a pineapple farmer cooperative based in Kirehe District in Eastern Province and is among 46 PSDAG partners that benefited from the PSDAG grant facility. The cooperative was formed by farmers with the intention to look for a favorable market for their produce after realizing that there is high potential production in the area. Subsequently, the cooperative set up a pineapple processing unit based on an idea originally conceived in 2009. Currently, 141 farmers are involved in organic farming and benefit from market exports. The cooperative board members reported revenues amounting to over US$360,000 per year, or RWF 300 million, as result of PSDAG support in upgrading their pineapple products for export. Exports volumes are estimated at 2.5 metric tons (MT) per year. This contributed significantly to household conditions of farmers as a result of increased income from pineapple farming and access to high-value markets. Export earning revenues contributed to the economic impact of the cooperative on household conditions for cooperative members. Cooperative members state that this performance is a result of cooperative professionalization with PSDAG and other development partner support and linkage to high-value markets.73 Potato Seed Multiplication: Irish potato is one of the key priority crops under the GOR CIP. Since the introduction of this program in 2006, Irish potato productivity has steadily increased as a result of enhanced advisory services and efficient use of agricultural inputs, mainly fertilizers and improved seeds, as farmers obtain these inputs through the government subsidy input scheme.74 In KIIs, district officials in the potato growing districts estimated the current demand for potato seed at over 40 MT.75 However, 75 percent of currently available seeds are distributed informally and are of poor quality. Current production of Irish potatoes stands at between 25 and 29 MT per hectare but with improved seeds and use of new farming techniques, the produce could increase to 40 MT per hectare. The implementation of a formal seed system is expected to increase food production to satisfy the growing market demand, help farmers to access adequate quality seeds at planting, and increase business linkages among potato seed value chain actors.76 PSDAG identified Irish potato as a priority value chain and devised a strategy of developing potato seed production capacity with private sector investors. Currently, seven grantees are actively involved in potato seed multiplication and signed grant agreements with PSDAG with a total value of US$590,895 to increase the supply of quality seeds.77 One company reported already receiving a PSDAG VCCF grant for farmer capacity building and acquisition of appropriate machinery for seed multiplication. Over 3,000 farmers organized into 50 farmer groups and are set to benefit from PSDAG support through training, on-the-job coaching, and easy access to high-quality seeds at affordable prices (RWF 150 per plant for in vitro plants, RWF 50 for mini-tubers, and RWF 400 per kg for pre-basic seeds). In KIIs, the company reported that it generated increased incomes from these three core seed products due to “coaching of farmers and consistent supply of seeds.”78 However, KII respondents indicated that further progress in this value chain is threatened by policy changes related to potato pricing. In KIIs, cooperative staff specifically indicated that these new government regulations have disrupted production, aggregation, and marketing of Irish potato over the 72 Pfeiffer, Kirsten, Chantal Umuhire, Liliane Gasana “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF),” PowerPoint presentation, RTI for USAID, May 18, 2018. 73 KII, SME, Kirehe, May 30, 2018. 74 “Strategic Plan for the Transformation of Agriculture in Rwanda - Phase III 2013-2017,” MINAGRI, 2013. 75 KII, GOR, Rubavu, June 1, 2018 and KII, SME, Musanze, May 31, 2018. 76 Ibid. 77 Pfeiffer, Kirsten, Chantal Umuhire, Liliane Gasana, “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF),” PowerPoint presentation, RTI for USAID, May 18, 2018. 78 KII, SME, Musanze, May 31, 2018. 25 last two seasons. They also state that changes in marketing regulations have affected not only farmer cooperatives, but also the overall seed potato market as the main clients are farmers who are prohibited from marketing through their PCCs and “do not have any other businesses now.”79 Improved Market Linkages: Linking the farmers to the market was another area in which PSDAG has been effective. The evaluation assessed how the farmers were gaining the market information, where the information came from, and whether the farmers had a reliable market where they could sell their produce at reasonable price. According to the BBS, cooperatives were a primary source of market information for 95.7 percent of respondent farmers (Table 8). This finding confirmed the effort of PSDAG to strengthen the farmers’ access to the market through cooperatives. Other important sources of information included mass media (28.5 percent), sector and districts agronomists/staff (23.6 percent), neighbors (18.6 percent), input suppliers (17.5 percent), other development projects (16.0 percent), and the agricultural promoters (15.5 percent). It was also important to note that 53 percent of survey respondents affirmed that the market information received was new to them, while 75.3 percent of respondents used the marketing information to decide on the timing of crops sales, 74.9 percent used the information to decide the location for crops sales and which types of crops to sell, and 70.7 percent used marketing information to decide on the sales price. Table 8: Sources of Market Information for PSDAG-Assisted Cooperative Members Source Male Female Total Cooperatives 95.2% 96.6% 95.7% Mass media (radio, TV, newspapers) 29.6% 26.6% 28.5% Sector/district agronomists or staff 23.5% 23.9% 23.6% Neighbors 19.1% 17.7% 18.6% Input suppliers 18.3% 16.0% 17.5% Customers 18.3% 15.7% 17.3% Other development projects 15.1% 17.4% 16.0% Agricultural promoters 14.5% 17.1% 15.5% Government projects 12.9% 13.0% 12.9% Distributors 10.7% 9.9% 10.4% Informal saving and lending groups 8.2% 8.9% 8.4% Financial institutions (SACCO, banks) 0.8% 2.4% 1.4% Beyond the five technologies/business models discussed above, the BBS, KIIs, and FGDs produced other notable findings of a general nature related to changes in on-farm production and outcomes resulting from the adoption of improved farming techniques and household income that merit discussion. These are discussed briefly below. Improved Production Through New Farming Techniques: According to the BBS, 80.4 percent of respondent farmers reported an increase in crop production over the past four years (Table 9). Kamonyi led the way with 90.2 percent of respondent farmers reporting an increase in crop production, followed by Ngoma (85.3 percent), Burera (81.0 percent), Nyagatare (75.6 percent), and Nyabihu (69.9 percent). Table 9: Overall Change in Crop Production District Unchanged Increased Decreased Nyagatare 9.8% 75.6% 14.5% Ngoma 3.3% 85.3% 11.4% Kamonyi 1.5% 90.2% 8.3% Burera 6.9% 81.0% 12.2% Nyabihu 3.4% 69.9% 26.7% Overall 5.0% 80.4% 14.6% 79 KII, SME, Musanze, May 31. 26 In the FGDs, farmers attributed much of the increased production to the adoption of improved farming techniques as a result of PSDAG support. The BBS confirms this finding in that it reveals a widespread use of fertilizers and pesticides (94.9 percent of farmers), extension services (93.0 percent), proper farm management (90.7 percent), and the use of improved agricultural techniques (82.9 percent) (see Table 10). In probing further, FGDs respondents, moreover, indicated that each of the above four improved farming techniques were adopted specifically as a result of the PSDAG interventions, as opposed to other projects that may also have supported the cooperatives. Table 10: Reasons for Increased Production Reasons for Increased Production % Proper use of improved seeds, fertilizers, and pesticides 94.9% Extension services 93.0% Proper farm management 90.7% Use of improved agricultural techniques 82.9% Use of equipment and tools 62.9% Climate condition 55.1% Increase in land 48.1% Promising market 47.1% Extension services for production (89.6 percent of respondent farmers) and post-harvest handling (85.6 percent) were the most prevalent types of TA received by respondent farmers (see Table 11), followed by assistance with inputs (76.0 percent), marketing (71.9 percent), production tools and equipment (52.7 percent), and post-harvest tools and equipment (55.1 percent). For the most part, male and female respondents received similar types of TA with the exceptions that fewer female respondents reported receiving assistance with tools and equipment than male respondents. The BBS also assessed the providers of the technical assistance. Respondents indicated that the cooperatives were the primary source of assistance, extension services, and capacity building for the farmers across all four value chains, with an average of 70.3 percent of respondents reporting receiving extension services from their cooperative. In addition, farmers in FGDs reported that these extension services were obtained from cooperative representatives trained through a training of trainers approach, either by private sector service providers or directly by PSDAG. Table 11: Types of Technical Assistance Received by PSDAG TA Received Male Female Total Extension services for production 89.5% 89.8% 89.6% Extension services for post-harvest handling 84.6% 87.4% 85.6% Assistance with inputs (fertilizers, pesticides, seeds) 76.5% 76.9% 76.0% Assistance with marketing 72.9% 70.3% 71.9% Assistance with production tools and equipment 49.0% 58.9% 52.7% Assistance with post-harvest tools and equipment 52.9% 58.9% 55.1% Effects on Household Income: With regard to household income, the BBS results indicate that over the past four years, 79.9 percent of respondent farmers experienced an increase in household income as the result of increased on-farm production and access to reliable markets. Notably, 80.4 percent of women and 85.8 percent of female youth reported an increase in household income compared to 77.4 percent of men who reported this increase. This suggests that the activity not only contributed to increased income through increased on-farm production and sales but also contributed to social inclusion by empowering women and youth via engagement in market-oriented rather than subsistence agriculture. 27 4.5 EQ 5: Effectiveness of the PSDAG Grant Facility Evaluation Question 5: Has PSDAG’s grant facility been an effective tool in promoting investment? PSDAG launched the first RFA for VCCF grants in 2015.80 One of the key criteria to access the VCCF is a co-financing commitment by the grantee at the time of signing, which can include both capital and non￾capital investment. A minimum co-investment of 50 percent is required by PSDAG with exceptions for groups targeted for social inclusion (women, youth, and PWD). In KIIs, grantees described several challenges in obtaining co-financing from equity investors and banks: • Equity investors are interested in Rwandan agriculture, but are looking at investments over US$1 million (social impact investors prefer a minimum of US$2-5 million), while potential PSDAG partners require investments well below this scale; • Some grantees were not ready for loans, which made it difficult to link them to financial institutions due to their limited capacity to develop bankable projects;81 • Some grantees seeking bank loans already had outstanding loans or lacked physical collateral, which prevented them from accessing finance from formal financial institutions; and • Most Rwandan banks do not have staff with specialized skills required to appraise a broad range of agricultural investment projects.82 In response, several bank and SME staff suggested increased access to equity investment as a strategy for improving access to SME finance: “PSDAG should [put] emphasis on the mobilization of more impact investment, such as access to equity for SMEs, which remains a major challenge. Interest rates are very prohibitive, and there is a lack of collateral. The scale of the business is still a problem as the financial environment is not quite attractive for SMEs” – KII, SME Staff “[PSDAG should be] promoting other investment strategies, such as equity, mix of equity and loans, and guarantee funds for smallholder farmers to access to finance.” – KIIs, SME Staff Bank and SME staff also noted that equity investments can bring in much needed technical expertise required by the enterprises, even suggesting that PSDAG should “shift from a grant approach [to] supporting SMEs entering into the [equity] financial market to source capital.”83 Furthermore, these respondents stated that TA, such as that provided by PSDAG, is effective in de-risking finance. However, bank staff also noted that the vast majority of equity funds target specific sectors, and that the scale of these sectors in Rwanda is limited to opportunities significantly lower than what these investors would accept. 84 “There are really limited opportunities for equity investment funding given the limited value of potential investments. Equity fund managers are looking for sectors that can absorb over US$20 million in investment.” – KII, Bank Staff “We work on sourcing capital from private institutions, like structuring equity funds, but this is challenged by the limited scale of investments.” – KII, Bank Staff Despite these challenges, PSDAG grantees leveraged over US$12.3 million in co-investment, enabling grantees to expand to new markets, invest in international quality certifications, and/or attract new investors. For example, one company accessed an export loan facility for acquisition of additional 80 “PSDAG Competitiveness Grant - Request for Application # IRG-RFA-15-01 USAID Contract No. AID-696-C-14-00002.” 81 “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2017 Quarterly Progress Report: October 1, 2016 – December 31, 2016,” IRG for USAID, January 2017. 82 KII, Bank, Kigali, May 24. 83 Ibid. 84 Ibid. 28 equipment to process chili for export, while another company secured a loan to finance equipment for agricultural mechanization. A third company secured an investment and working capital loan to finance maize processing activities.85 In KIIs with SME staff, numerous PSDAG grantees described the VCCF grant as “helpful” in building their creditworthiness with banks. For example: “We managed to get a loan from the [International Finance Corporation] IFC using the PSDAG grant as a reference. PSDAG provided 50 percent of the total investment, and the remaining 50 percent came from [European Cooperative for Rural Development] EUCORD and our own equity as well as the loan from IFC.” – KII, SME Staff “PSDAG was not directly involved in leveraging investment for the company. Therefore, [the] PSDAG partnership was a reference for the company to get [a] loan from the bank. The project played a catalytic role in the process, so it was successful in that way.” – KII, SME Staff “We applied for a loan, which is not yet granted due to the long process from the bank. But the PSDAG grant agreement was presented as a reference to get access to loan, so we [as a] company expect to get approval very soon.” – KII, SME Staff More importantly, PSDAG provided TA to SMEs through profitability analysis studies to assess their business models and loan repayment capacity. In some cases, PSDAG facilitated loan negotiations between grantees and financial institutions as well as BDS support. Through this support, the SMEs developed tools and documentation to present to investors and banks for financing. Finally, SMEs stated that PSDAG TA implementing their business projects provided a critical guarantee of success to banks. Likewise, in the case of equity shareholders, a potato seed company working in Northern Province stated that PSDAG TA was helpful in building investor confidence in his business project.86 In some cases, the VCCF facility also interested the GOR in investing in corollary infrastructure facilities. For example, in one case, the Managing Director of a grantee firm operating in Kirehe District described how the GOR invested in irrigation infrastructure in the area where his SME is operating.87 In its work with financial institutions, PSDAG conducted a feasibility study among various agriculture value chains, which helped build the capacity of finance institutions and used the grant facility to develop their agriculture loan portfolio. PSDAG offered two technical training sessions on agriculture credit risk and management as well as cash flow lending and financial analysis. A total of 43 people (21 women) from seven financial institutions were trained.88 The training enabled them to better understand the dynamics of agriculture value chains and how important it is to understand the bottlenecks, which business models are relevant for the agriculture financing, and how business relationships can be used to mitigate some risks. In addition, VCCF grants also improved the performance of cooperatives. In the BBS, 65.8 percent of respondent farmers reported improving market access through their PSDAG-assisted cooperatives, while 86.9 percent of the farmers said that they sell their produce to the cooperatives “very often” (see Table 12). In contrast, however, 65.6 percent reported continuing to sell some portion of their production at local village markets with the same frequency. 85 “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2018 Quarterly Progress Report: October 1, 2017 – December 31, 2017 (redacted),” RTI for USAID, January 2018. 86 KII, SME, Musanze, May 31. 87 KII, SME, Kirehe, May 28. 88 Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2017 Quarterly Progress Report: October 1, 2016 – December 31, 2016,” IRG for USAID, January 2017. 29 Table 12: Market Access by PSDAG-Assisted Cooperatives Markets Not at All Rarely Often Very Often Neighbors 57.9% 13.4% 23.2% 5.5% Cooperatives 6.9% 2.7% 3.5% 86.9% Local small village market 3.9% 21.9% 8.6% 65.6% Peri-urban market 82.4% 3.3% 10.6% 3.7% Local large village market 75.3% 5.3% 14.4% 5.0% SMEs/Private sector with agreements 83.5% 2.9% 10.5% 3.1% Analysis of BBS data also suggests that VCCF support to cooperatives increased commercialization, with farmers reporting the quantity of crops marketed increasing from 17.5 percent of household production in 2014 to 36.8 percent in May 2018, while the cooperatives reported an average increase in sales of 35 percent during this same period. In line with the increased production and access to the market, farmers also reported increased household income, with average annual income per household growing from RWF 207,456 to RWF 438,347, realizing an increment of RWF 150,891, equivalent to 72.73 percent of the total. Cooperative member responses in the FGDs also suggest that the PSDAG grant facility has been effective in promoting agricultural production through the increased availability and accessibility of infrastructure for post-harvest handling and market access as well as the creation of linkages between the farmers and the buyers. In FGDs, cooperative members in all districts described how new post-harvest facilities and infrastructure increased on-farm production as they gained confidence that their crops will not spoil due to lack of drying and storage facilities. 4.6 EQ 6: PREVENTING DOUBLE DIPPING Evaluation Question 6: How effective has PSDAG been in preventing double dipping from its grant recipients (i.e., is PSDAG providing grants to support proposals being funded by other donors)? PSDAG has approached the issue of double dipping through a self-reporting procedure followed up by due diligence that includes a review process with other donors operating in Rwanda prior to grant approval in order to avoid duplication of funding. “PSDAG asks grantees to self-report, and we send out the information on a particular grantee to other donors to review for previous funding. This process has improved, as we’re able to reach more potential grantees.” – KII, PSDAG Staff During KIIs with SME staff, none acknowledged double dipping from other donors to co-invest in activities also being funded by USAID under PSDAG. That said, respondent SMEs did acknowledge receiving co￾investment funding from other sources; however, they described these co-investment sources as primarily: 1) self-funding (16 respondents), 2) debt finance through banks (six respondents), and 3) equity investments (four respondents). When asked whether other donor sources were secured, five staff answered affirmatively but described the funding as either financing prior stages of their projects or as assets not funded by PSDAG grants, such as the construction of facilities. For example, Seed Potato Fund Ikigega Ltd. (SPF) reported receiving support from the United Nations International Fund for Agricultural Development Post-Harvest and Agribusiness Support Project (IFAD PASP) for the construction of its laboratory and seed storage facilities in Musanze, while a PSDAG grant funded equipment assets. 89 MINIMAX received funding from an IFC facility, but post-harvest handling equipment, which was not included in this funding, was provided through a PSDAG grant.90 PSDAG staff explained that Kigali Farms was originally funded by the Dutch embassy, but after restructuring, PSDAG 89 KII, SME, Musanze, June 2, 2018. 90 KII, SME, Kigali, May 24, 2018. 30 provided funding for the purchase and installation of substrate production facilities, construction of mushroom growing houses, and installation of mushroom production equipment, including a baler, tractor, generator, cold room, and pre-cooler.91 4.7 EQ 7: SOCIAL INCLUSION Evaluation Question 7: Did the PSDAG approach to social inclusion ensure the involvement of women, youth, and PWD? Are there innovations or other approaches that could enhance the meaningful engagement of these targeted beneficiary groups? PSDAG interventions emphasize the social inclusion of women, youth, and PWD. Social inclusion activities have been guided by a Gender and Social Inclusion Integration Plan, which was developed in FY 2016.92 This plan includes five strategies: Strategy 1: Mainstream gender and social inclusion in PSDAG’s policies, procedures, communications, and employment recruitment methods. Strategy 2: Mainstream gender and social inclusion in the grantee selection process. Strategy 3: Support grantees/value chain actors to develop and implement gender and social inclusion strategies. Strategy 4: Advocate with national institutions, financial institutions, and investors to include the specific needs of youth, women, and PWD in policy implementation and investment frameworks. Strategy 5: Support specific innovative agribusiness initiatives for youth, women, and PWD groups through the grant mechanism in order to showcase their capacity and potential. In addition, by investing in socially inclusive enterprises and companies that actively work with women, youth, and PWD, VCCF investments benefited these groups. One example of this is Masaka Creamery, which employs hearing impaired persons in its activities. Of the 26 SMEs interviewed in the KIIs, eight reported that they had attended PSDAG social inclusion trainings. Four of the 26 SMEs had a social inclusion policy in place prior to PSDAG support, while three reported developing a policy with the support of the project. (Masaka Creamery had a specific mission to employ PWD prior to receiving PSDAG support in line with its stated Christian mission).93 Of the SMEs that reported receiving training in social inclusion, only two implemented any changes in staffing or policies as a result; one hired a PWD as a Finance Officer, and one appointed a female PWA to its Board of Directors.94 Rather than changes implemented within the business, the changes SMEs reported related to social inclusion were much more related to an increase in awareness. “More awareness to think about women and to promote them.” – KII, SME Kigali “[The training] created an awareness and feasibility for the future. [We] have done an analysis and survey of current jobs, for future activity. [We] will think about impact on ordinary employee and family impact mostly for women.” – KII, SME Kigali On the other hand, several SME staff described their existing practices as favoring gender inclusion based on existing gender roles. For example, one SME stated that women are involved in sorting and grading 91 KII GOR Staff, Kigali, June 6, 2018 and “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. 92 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Activity - Project Fiscal Year (FY) 2016 Annual Progress Report: October 1, 2015 – September 30, 2016,” IRG for USAID, October 2016. 93 KII, SME, Kigali, May 24, 2018. 94 KII, SME, Kigali, May 23, 2018 and KII, SME, Kigali, May 30, 2018. 31 maize after harvest, a task they “are good at.”95 At the same time, SMEs cited traditional gender roles as persistent barriers to employing women, for example, “females are not good at mechanization.”96 Several SMEs described focusing on youth due to their “openness to new ideas.”97 They also described hiring youth because, as university graduates, they possessed the required skills. “The average age of our staff is 25 years old because we need skilled staff. We take only university graduates.” – KII, SME Kigali Compared to SMEs, approaches to social inclusion seem to have resulted in more robust results at PSDAG-assisted cooperatives. For example, two cooperatives described explicit efforts to recruit youth through a dedicated small start-up loan targeting youth,98 while another project-assisted cooperative in Koduibi (Ngoma District) is attempting to recruit youth members by offering to waive their cooperative membership fees. Analysis of BBS responses suggests that all cooperatives surveyed reported receiving training on social inclusion, including gender and youth mainstreaming. Subsequently, activity documents show the proportion of women cooperative members increased to the current level of 42.9 percent in 2018 up from 25 percent in 2014.99 FGDs reflected this increase. For example, following the social inclusion training at Dukorekarama Cooperative, the number of women members increased from three to 12. At a pineapple producing cooperative in Kirehe District, women form the majority of cooperative members. Feedback from FGDs also suggests that the increased participation of women in cooperatives is producing positive changes, such as increasing women’s confidence to request loans, either as individuals or as a group, from formal financial institutions with cooperative contract farming deals serving as loan collateral. Findings from the BBS indicate that prior to PSDAG interventions, women accessed loans at lower rates than men from all sources of credit with the exception of Savings and Credit Unions (see Table 13). Following PSDAG interventions, the level at which women accessed credit from all sources increased. Notably, while still lower than men for most sources of credit, women’s level of accessing credit from the cooperatives surpassed that of men, and the BBS suggests that the level of access for loans from savings and credit unions more than doubled. Table 13: Sources of Loans at PSDAG-Supported Cooperatives Disaggregated Sources of Loans Pre-PSDAG Current Men Women Average Men Women Average Microfinance Institutions 14.9% 12.0% 13.8% 24.3% 12.4% 19.1% Savings & Credit Union 21.5% 27.0% 23.6% 47.1% 56.2% 51.0% Banks 17.2% 8.7% 14.0% 29.4% 14.3% 22.8% Cooperatives 13.7% 11.7% 12.9% 54.3% 60.3% 57.7% Agro dealers 2.3% 0.3% 1.5% 8.1% 1.0% 5.0% Others 12.5% 11.0% 11.9% 19.9% 14.3% 17.4% Likewise, in cooperative FGDs, respondents reported that considerable efforts have been made to increase the active participation of women in cooperative activities across all districts. For example, in Kamonyi District, the Kabiaki cooperative reported that, as a result of PSDAG gender training, it introduced a literacy class exclusively targeting women. In the same district, the Impabaruta and Kabiaki cooperatives reported mobilizing women to form savings and credit groups and empowering them in the 95 KII, SME, Kigali, May 23, 2018. 96 KII, SME, Rwamagana, May 28, 2018. 97 KII, SME, Kigali, May 22, 2018. 98 KII, Cooperative Staff, Musanze, May 29, 2018 and KII, Cooperative Staff, Kirehe, May 27, 2018. 99 “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2018 Quarterly Progress Report: October 1, 2017 – December 31, 2017 (redacted),” RTI for USAID, January 2018. 32 area of property rights. Members of the Impabaruta Cooperative in Kamonyi District also reported that the PSDAG gender training inspired them to launch a youth training in computer literacy. Analysis of the BBS data further indicates that 76.3 percent of respondents at PSDAG-assisted cooperatives reported an increase in income over the last four years as the result of increased production and improved access to markets, with a higher proportion of women and youth reporting higher on-farm income than men (see Table 14). As noted above (see EQ 4: Findings), the average percentage of respondents reporting an increase in income was 79.9 percent, while this number was 85.8 percent for adult females, 77.4 percent for adult males, 80.4 percent for female youth, and 73.2 percent for male youth. Table 14: Changes in Income at PSDAG-Supported Cooperatives Category Measure Unchanged Increased Decreased Adult Men Frequency 76 387 37 Percent 15.2% 77.4% 7.4% Adult Women Frequency 23 242 17 Percent 8.2% 85.8% 6.0% Youth Men Frequency 12 52 7 Percent 16.9% 73.2% 9.9% Youth Women Frequency 5 41 5 Percent 9.8% 80.4% 9.8% Total Frequency 116 722 66 Percent 12.8% 79.9% 7.3% However, the BBS failed to find similar positive results in terms of expanded incomes related to PWD. Furthermore, in FGDs, respondents stated that their cooperatives had not expanded opportunities for PWD or launched new activities focusing on this group, while in KIIs numerous respondents from various stakeholder groups stated that more needed to be done at the national level to address the needs of this group. 5.0 CONCLUSIONS 5.1 EQ 1: MOST EFFECTIVE INTERVENTIONS TO ASSIST THE GOR Evaluation Question 1: Within the objective of assisting the GOR to increase private sector engagement, what types of interventions supported by PSDAG (PPD; policy and regulatory reform, including restructuring, RDB investment targeting, and promotion strategies; institutional strengthening; facilitating PPPs; etc.) have been the most successful or effective in attracting private sector investment in agriculture? Previously, GOR investment promotion approaches were focused on a limited set of tools along with the creation and subsequent privatization of discreet PPPs. However, PSDAG has supported an overall shift across GOR institutions to a more systematic approach to investment promotion through a focus on capacity building, benchmarking and competitiveness assessments, supporting sub-sector selection, and developing investment promotion materials, including through “flexible” support from EIAs. Although, given the PSDAG activity timeframe, it is too soon to realize the full impact of this shift in approach, this is likely to be the most effective intervention in attracting private sector investment in agriculture. Already, this approach has generated positive outcomes, as illustrated by the implementation of an “Airfreight Competitiveness Study and Review of Export Commodity Aggregation Services Models,” which resulted in expanded exports by five companies of US$1.2 million and increased private sector use of an NAEB-owned cold storage facility located near the airport. 33 The GOR, with PSDAG support, has developed a Pilot National Agri-PPD Framework in order to revitalize and re-launch this process under the PSF and has identified district-level champions to lead PPD forums. In addition, PSDAG has supported the establishment of several PPD forums. However, to date, this process has not been effective in attracting private sector investment in the agricultural sector due to convoluted institutional support and processes, which undermine action on issues raised by the private sector. As such, the PPD process has yet to fully realize its potential to address policy bottlenecks related to private sector investment in agriculture. In the area of restructuring RDB investment targeting and promotion strategies, institutional strengthening, and facilitating PPPs, PSDAG initially provided support for the privatization of specific PPPs. Subsequently, the activity transitioned to a more systematic approach to assistance, including through EIA assistance. Although it is too early to fully assess the effectiveness of this shift, there are promising signs of success. For example, the implementation of lessons learned from a “Poultry Competitiveness Day-Old Chick Study” led to a policy decision to exempt locally produced feeds from the VAT, stimulating investment in the feed sector, while the same study simultaneously facilitated the privatization of the National Hatchery PPP through a US$3 million investment by Uzima Chicken. 5.2 EQ 2: INTERVENTIONS MOST LIKELY TO LEAD TO LASTING CHANGE Evaluation Question 2: To what extent are the PSDAG interventions supporting Objective 1 (which includes the ALIS, support to MINAGRI’s budget process, restructuring RDB investment targeting and promotion strategies, support to the PPD structure and process, and the aggregation services centers) likely to result in long lasting impact in attracting private sector investment in agriculture that will continue beyond the conclusion of PSDAG? PSDAG supports an overall shift in GOR investment promotion approach away from discreet projects toward to a more facilitative role through broader sector competitiveness analyses, which can provide data to a range of investors. As such, this shift is most likely to result in significant long-lasting impact in attracting investment in agriculture. With PSDAG support to the MINAGRI budgeting process, the GOR has an obligated seven-year budget support for SAIS that will facilitate staffing and long-term planning for the ALIS system. However, the system still requires the development of a dynamic updating process, and the GOR needs to develop a clear process for land leasing (see EQ 3: Risks to Sustainability). Once these constraints are addressed, ALIS is likely to be an effective tool for providing timely information on land availability, which is difficult to obtain in other regional markets for a broad range of new investment projects. PSDAG interventions supporting restructuring RDB investment targeting and promotion – including benchmarking and priority sub-sector identification, sector-wide analyses, and intensive institutional capacity building, including through the EIA – exemplify the more systematic approach to investment promotion. In this regard, these interventions are resulting in the transformation from a “static” approach toward more dynamic agency capacity. As such, these are likely to have a significant long-lasting impact on the capacity of RDB to implement investment promotion through the embedded capacity and analytics approaches it has fostered. In contrast, PPD forums are currently not effective in delivering long lasting impact due to challenges in information flow between multiple sub-national and natation forums and “convoluted” institutional support, as well as due to weak leadership by PSF. The potential of this intervention to deliver lasting change may increase in the future as a result of recent and planned improvements in the PPD process, including the identification of regional “champions” and capacity strengthening at PSF. Likewise, the potential for aggregation service centers to deliver lasting change is unclear as recent interventions have yet to generate impact. 34 5.3 EQ 3: RISKS TO SUSTAINABILITY Evaluation Question 3: What are the risks to sustainability and the actions needed to address them within the remaining project term? (Note: In the context of this question, sustainability is defined as the ability of the GOR to continue the key interventions or gains achieved under Objective 1 after the conclusion of the PSDAG activity.) Stakeholders, including GOR and PSDAG staff, perceive risks to sustainability related to two interventions: 1) ALIS and 2) PPD: ALIS: Ensuring the sustainability of the ALIS system entails improving functionality through the development of a dynamic updating process to produce real time data on available land. MINAGRI plans to address this through support to be provided by Esri Rwanda in 2018. The establishment of a clear and transparent process for reviewing and processing applications for land in a timely manner, moreover, is key to ensuring the sustainability of this system through the review of the current Land Lease Client Charter, the Land Use Consolidation Ministerial Orders, and other elements of the legal framework for leasing land, which are currently being addressed through a Land Use Management Guidelines project started in November 2017. Some stakeholders believe that the new Rwanda Land Management and Use Authority would be better positioned to manage ALIS due to its greater influence on issues related to land policies and access to additional land managed by the authority. PPD: The sustainability of the PPD process related to agriculture is threatened by poor coordination between multiple forums and challenges in effective processes to deal with the policy bottlenecks and issues identified in forums, especially between the sub-national and national levels. These challenges risk losing private sector interest in participation. Further complicating these challenges are confusion and turnover in staffing roles. Going forward, clearer processes for addressing identified issues need to be developed, and the leadership role of PSF needs to be clarified and strengthened. 5.4 EQ 4: TOP THREE TECHNOLOGIES AND INNOVATIONS Evaluation Question 4: Which top three technologies and business model innovations promoted by PSDAG have resulted in the highest ROIs for farmers and for agricultural businesses? What were the reasons for any success or failure? Contract farming, driven by investments in agri-SMEs, has produced the most significant positive impacts from the technologies and business model innovations promoted by PSDAG. This is evidenced by increased incomes among cooperative members due to improved access to finance through loans collateralized by cooperative contracts, which in turn is financing improved access to inputs, production techniques, and markets. Likewise, the expansion of contract farming has expanded agriculture portfolios at several finance institutions. Furthermore, the expanded adoption of commercial inputs and services generates increased income to agri-service providers, such as inputs suppliers and transporters. Improved post-harvest handling and other increased vertical supply chain linkages have also produced increased farmer incomes for widely-produced basic food crops. Finally, expanded production of commercial (high value) crops – such as intensive vegetable and fruit production, nuts, and flowers and potato seed multiplication – have had a similar positive effect on smallholder incomes. However, the more limited number of farmers impacted in both cases renders the income effect for commercial crops and potato seed multiplication lower than for contract farming or improved post-harvest handling. 5.5 EQ 5: EFFECTIVENESS OF THE PSDAG GRANT FACILITY Evaluation Question 5: Has PSDAG’s grant facility been an effective tool in promoting investment? The US$5 million PSDAG VCCF grant facility has been an effective tool in promoting investment with VCCF grantees leveraging over US$12.3 million in co-investment from self-funding, banks, and equity 35 investors over the life of the activity to date against a target of US$10,674,000 by the end of FY 2017. After grants funding, several PSDAG-assisted SMEs have continued to access private funding, including Sarura, FarmFresh, and Tuzemurane Cooperative. Key factors in leveraging co-investments have been PSDAG TA to SMEs, including profitability analysis studies to assess business models and loan repayment capacity, as well as PSDAG-provided BDS support and TA in implementing business projects, which improved the bankability of projects and provided a critical guarantee of project success to banks and investors. 5.6 EQ 6: PREVENTING DOUBLE DIPPING Evaluation Question 6: How effective has PSDAG been in preventing double dipping from its grant recipients (i.e., is PSDAG providing grants to support proposals being funded by other donors)? PSDAG has been taken reasonable steps to ensure that systemic problems with double dipping have not occurred. These include due diligence through a review process with other donors operating in Rwanda, and self-reporting by grantees. Although some grantees did receive funding from other donors, this was limited to either financing prior stages of their projects or assets not funded by PSDAG grants. 5.7 EQ 7: SOCIAL INCLUSION Evaluation Question 7: Did the PSDAG approach to social inclusion ensure the involvement of women, youth, and PWD? Are there innovations or other approaches that could enhance the meaningful engagement of these targeted beneficiary groups? PSDAG has pursued a vigorous approach to social inclusion of women, youth, and PWD and has also supported SMEs that employ target groups and cooperatives that include target groups as members. However, only a limited number of PSDAG partner SMEs have made substantive changes to their staffing as a result of participation in the activity (although the impact of PSDAG social inclusion strategies may not have yet fully accrued). In most cases, gender barriers to employment remain intact, and women continue to be employed in areas associated with existing gender roles. In contrast, approaches to social inclusion have resulted in more robust results at PSDAG-assisted cooperatives, including increased women’s membership and specific outreach and training activities targeting women. Some cooperatives have also developed initiatives to recruit more youth, including start￾up loans and membership fee waivers. On the other hand, initiatives to engage PWD have not been developed at SMEs or cooperatives at a significant scale. 6.0 RECOMMENDATIONS Public-Private Dialogue: In KIIs, respondents described a “convoluted” institutional support structure for PPD related to agriculture, especially at RDB, and related to the leadership of PSF. Respondents also described challenges in the process to address issues raised in PPD forums, especially between sub￾national and national forums. Presumably, the PSDAG-supported “assessment of the current public/private dialogue process at multiple levels to support the redesign and revival of the PPD forums with the Private Sector Federation (PSF)” and review of the “Pilot National Agricultural Public-Private Dialogue (Agri-PPD) Framework” has improved institutional support for Agri-PPD. However, as planned in the existing capacity building grant to PSF, PSDAG should continue to engage MINAGRI, RDB, and the leadership of PSF to convene stakeholders in the Agri-PPD process to develop a more effective coordination approach and generate consensus on methodologies for addressing issues identified by agricultural investors. Particular attention should be paid to developing a coordination process between sub-national levels where issues are identified by investors. Investment Strategy: In restructuring the GOR investment targeting and promotion approach and provision of institutional capacity building, PSDAG has transitioned from providing discreet support for 36 specific projects to a more systematic approach that includes a strong focus on institutional capacity building. PSDAG, USAID/Rwanda and other donors should continue its current systematic approach, including RDB capacity development, providing support for sector-wide analysis, and institutional capacity building. Given the limited opportunities for exposure to “state-of-the-art” investment promotion methodologies in Rwanda, PSDAG, USAID/Rwanda, and other donors should consider engaging with initiatives that support inbound and outbound exchange visits between RDB staff and other investment promotion agencies in other countries where selected strategies are currently being implemented. ALIS: Stakeholders note that the currently static nature of land holding data presents a risk to the sustainability of the ALIS tool. PSDAG should provide MINAGRI with technical support to develop a dynamic updating process for ALIS data, potentially entailing an integrated registration process of transfers of currently mapped land as leases and other transfers are transacted so that data is systematically updated. In PY 4, MINAGRI plans to obtain support from Esri Rwanda Ltd. to address this. PSDAG should also engage sub-national governmental entities, which are already aware of the system but do not yet have access, in the use of the system, possibly promoting it as a potential regional and district-level investment promotion tool. PSDAG should support MINAGRI and the Rwanda Land Management and Use Authority to develop a process and legal framework for land leasing, as planned under the Land Use Management Guidelines project launched in November 2017. These stakeholders should consider transferring management of the system to the Rwanda Land Use Authority in order to expand information to wider selection of lands and apply the authority’s budget to ALIS as well. Contract Farming and Investment Promotion: Contract farming schemes have facilitated improved access to finance for inputs and market access for smallholders, as well as expanded technology transfer. PSDAG plans to address conflict dispute resolution in PY 4. PSDAG, USAID/Rwanda, and other donors should continue to work with MINAGRI on promoting the legal framework already in place for contract farming to promote this mechanism to potential investors, including as a strategy to expand production of high-value and other crops identified in the sub-sector prioritization process. Given the comparatively well-developed infrastructure for contract farming in Rwanda, USAID/Rwanda and other donors should support MINAGRI and the Rwanda Land Use Authority on the integration of ALIS data to promote opportunities for investment in strategic contract farming projects. Blended Finance: Blending finance and TA has been effective in improving project bankability, even when banks themselves lack the capacity to evaluate the financial feasibility of projects. PSDAG, USAID/Rwanda, and other donors should focus on institutional capacity development with GOR entities and Rwandan finance institutions to strengthen linkages between SMEs seeking investments for new projects and BDS and TA providers in order to better leverage financing. This effort should include work with finance institutions seeking to expand their agricultural portfolios – such as Duterimbere, Clecam, and SACCOs – to understand analytical products developed by TA and BDS providers in order to leverage increased financing. PSDAG should continue to assess investment opportunities in seed production of other crops beside Irish potato to continue improvements in seed supply quality. Social Inclusion: In KIIs and FGDs, innovations or other approaches suggested by respondents to enhance the meaningful engagement of targeted beneficiary groups included: • Introducing agricultural mechanization; • Promoting digital technologies, especially to improve engagement of youth; and • Promoting access to existing financial services opportunities. USAID/Rwanda, and other donors should consider promoting pilot projects deploying these approaches with cooperatives seeking to increase the engagement of women, youth, and PWD. 37 7.0 LESSONS LEARNED Embedded Technical Assistance: The impact of the PSDAG EIAs at MINAGRI and RDB were assessed highly by their GOR counterparts. As previously stated, RDB respondents specifically noted that the flexible nature of their support and their ability to engage in a broad range of tasks contributed to the effectiveness of the EIAs (see EQ 1: Findings). Likewise, the EIAs are presumably able to provide long￾term engagement in specific areas, such as support to follow-on activities identified in discreet studies and analyses in a consistent manner. Finally, the EIAs are presumably able to develop long-term understanding of staff capacities and needs, enabling them to contribute effectively to capacity development in an informed way. The flexible and ongoing nature of embedded TA, in contrast to short-term technical assistance (STTA), makes this type of support an effective strategy for adoption of a facilitative approach to investment promotion, as envisioned by the GOR. Given the long-term nature of most investment promotion activities, the presence of such assistance facilitates an effective strategy for consistent and sustained support. Contract Farming: Contract farming has been identified in several regional countries as a mechanism for improving finance for access to inputs and markets. However, “side selling,” i.e., the practice of contract default due to finding buyers willing to pay higher prices than contracted, has undermined the effectiveness of these schemes in countries such as Kenya, Malawi, Tanzania, and Zambia. In contrast, several respondents indicated that, although side-selling is a problem, better contract enforcement mechanisms in Rwanda makes this less of a problem. Likewise, as noted, PSDAG has planned to implement activities around developing contract dispute resolution mechanisms in PY 4 of the activity that can further strengthen contract enforcement. Finally, in one KII, a respondent indicated that credit bureaus, which can record data on past contract performance, are an option in Rwanda, in contrast to other regional countries such as Kenya.100 Hence, the enabling environment in Rwanda makes the country well placed to realize improvements in access to inputs and markets through contract farming schemes that can expand farmer incomes, and the activities planned in PSDAG PY 4 may lead to improvements in this area. Blended Finance: De-risking credit by marrying TA with equity and quasi-equity capital through blended finance structures, such as in the case of a technical assistance fund (TAF), has proven to be an effective approach to expanding available investment capital in many frontier markets. However, as noted, given that most investment fund managers seek markets in the range of US$20 million, the limited scale of specific agricultural sub-sectors in Rwanda challenges the development of investment funds in the country. Nonetheless, as described above, the combination of PSDAG-supported TA and BDS has been effective in facilitating required co-investments to VCCF grantees in several cases (see EQ 5: Findings). This suggests that blended finance approaches that marry TA and finance can still be effective in leveraging investment capital and expanding agricultural loan portfolios in Rwanda, even in cases where banks may lack the capacity to appraise specific projects. This suggests that TA and capital sources can be linked in innovative ways to generate successful outcomes, such as loosely affiliating TA and BDS providers with finance institutions. In addition, it may be possible to leverage social impact investment through focused TA support to potential social impact sources. 100 KII, Bank Staff, Kigali, May 22, 2018. 38 ANNEXES 39 ANNEX A: STATEMENT OF WORK 40 I. OBJECTIVE The purpose of this evaluation is to assess the progress made to date in achieving the activity’s two main objectives, and the effectiveness of the technical approach and the activity’s responsiveness to support the GOR goal of increasing private investments in agribusiness in Rwanda. The findings will be used by USAID/Rwanda and the PSDAG implementer to prioritize and strengthen the project’s implementation in order to improve performance over the remaining life of the project, and help USAID determine what steps to take in order to sustain the achievements of the project. In addition, the evaluation will help to inform the implementation of other related programs in order to advance engagement of the private sector in driving growth of Rwanda’s agriculture sector. II. BACKGROUND A. Context 1. Problem or Opportunity Addressed by the Project/Activity being evaluated The Government of Rwanda’s (GOR’s) commitment to agriculture as the driver of economic growth is well established. Three of Rwanda’s key socio-economic development strategies identify private sector￾led agricultural intensification and commercialization as the next phase of economic growth and job creation. The Crop Intensification Program (CIP) is the main implementing mechanism of the GOR’s Strategic Plan for the Transformation of Agriculture in Rwanda (PSTA). The CIP has been responsible for significant improvements in the sector over the past nine years. The bulk of CIP has been in supply-focused public investments (including subsidized fertilizers) designed to increase production and achieve food security. The CIP program has contributed to increased GDP and a 45 percent reduction in poverty from 2001 to 2010. While agriculture productivity has dramatically increased, the government has shifted from not only increasing productivity but also ensuring a market for surpluses. However, the private sector has not had the capacity to take over the functions in the market that it would normally perform in a developed economy. The perceived weakness in the private sector has driven the government, for example, to take over transactions and investment in processing facilities, storage, and the state-owned Rwanda Grains and Cereals Corporation (RGCC) to manage bulk commodity purchases. A feedback loop has developed where the government performs more functions in the market to ensure that farmers have a steady market, and the private sector remains weak because it is unable to manage more functions in the market. Consultations among local and regional private sector actors show an interest in such purchasing, productivity, and quality investments, but also highlight policy or capacity constraints. With a central budget consisting of approximately 51 percent of donor funds, the GOR has limited resources with which to expand new agricultural investments. Nonetheless, the pursuit of Rwanda’s economic strategy of modernizing agriculture requires significant new investments if it is to be successful. The GOR strategy envisions a broader role for market-led agriculture in Rwanda as expressed in the Economic Development and Poverty Reduction Strategy (EDPRS) 2013 – 2018. In this document, the GOR has expressed an interest in shifting resources into greater infrastructure and energy investments but needs a robust private sector in the agricultural sector in order to move money from direct agriculture investments. The Agriculture Sector Investment Plan recommends at least 30.9 percent of private sector investment in agriculture to help achieve the target agriculture GDP growth rate of 8.5 percent. It is in this context that USAID designed and awarded the Private Sector-Driven Agricultural Growth (PSDAG) activity to assist the GOR in increasing private sector investments in agribusiness, including upgrading agricultural value chains. 2. Target Areas and Groups PSDAG supports GOR agencies involved in investment promotion, mainly the Ministry of Agriculture and Animal Resources (MINAGRI) and the Rwanda Development Board (RDB), by facilitating the transition 41 of the GOR’s role from that of a direct investor to a regulator and facilitator for growth of the agriculture sector. PSDAG also strengthens private sector capacity and facilitates expanded investments for existing and new private sector entities (local and international businesses located in Rwanda). PSDAG supports private entities which promote inclusive growth which demonstrate potential to benefit smallholder farmers, women’s economic empowerment, women’s leadership, and engaging youth and persons with disabilities (PWDs). While the zone of influence of the Rwanda Feed the Future strategy is the entire country except the three districts of Kigali, PSDAG has actively been involved in 15 districts. B. Intended Results of the Project/Activity being evaluated PSDAG has two primary objectives: 1. Assist the GOR to increase private sector investment 2. Facilitate increased private sector investment in upgrading agricultural value chains These two objectives are seen as interrelated and interventions are designed to support both. Objective 1 includes the development and implementation of a clear roadmap for attracting and supporting private investment, including clear definition of roles and responsibilities within the GOR to improve the enabling environment and work with potential investors from their expression of interest to the implementation of the investment. Objective 2 includes interventions to assist the private sector in identifying market opportunities and facilitating investments to take advantage of those opportunities. There are also a number of systemic and/or cross-cutting issues that are critical to achieving these objectives, including gender-equitable solutions to improved productivity and competitiveness, integration of youth (defined as ages 18-35), and sustainable natural resource management that supports productivity improvements. PSDAG expects to achieve multiple results in addressing these objectives and crosscutting issues. 42 PSDAG Results Framework Key results achieved to date Objective 1 milestones: Assist the GOR to increase private sector investment PSDAG supported completion of at least one stage of the following 11 Agricultural Business Enabling Environment policies, strategies, frameworks, and or guidelines as shown below: Objective 2 milestones: Facilitate increased private sector investment in upgrading agricultural value chains Under Component 2, the PSDAG implementer is using a Value Chain Competitiveness Fund (VCCF) to catalyze relationships between investors, financial institutions, and investment facilitators and to demonstrate value-enhancing innovations and reduce risk. Aggregate results from all of PSDAG’s partnerships with the private sector include: • $5,484,770 gross income for 85,234 farmers (40,910 women); • $1,328,535 in new private sector investment in the agriculture sector or food chain leveraged by Feed the Future implementation; • $5,845,500 in exports of agricultural products; • $2,683,944 in incremental sales (collected at farm level); • $11,076,001 in value-added domestic sales; • 101 new jobs (43 women) created by PSDAG private sector partners; • $2,923,745 in loans and credit to upgrade agricultural business models; and • 47,354 farmers and others who have applied improved technologies or management practices (20,918 women). Detailed data on achievements to date can be accessed through reports uploaded to the USAID Development Experience Clearinghouse (DEC). 43 Approach and Implementation PSDAG implementation has focused on addressing underlying systems issues and technical gaps that prevent increased investment in the agriculture sector. For example, while RDB and other government agencies with investment promotion mandates can improve their performance, the impact of such improvements is dependent on other parts of the system—from agricultural policy to tax collection to the cost of electricity. Once the system problems have been identified, technical interventions can be very valuable. Thus, the overall approach for this objective is to facilitate interactions and discussions among relevant public and private sector organizations in order to define the investment promotion system, and from that definition, identify additional reforms, training, incentives, or other needs that can be addressed through technical assistance. Reform priorities must therefore be based on GOR priorities, and support should be flexible enough to change course based on GOR decisions. Also, while PSDAG focuses on economic growth, the enabling environment for agriculture and agro￾business depends on the system of governance. In this way, PSDAG encourages participation, inclusion, transparency, and accountability through support to public-private dialogue structure and processes which help improve the enabling environment for growth of targeted beneficiaries. The second component of PSDAG implementation includes two approaches: the portfolio approach and facilitation. The portfolio approach allows PSDAG to manage the risks of working with the limited number of participants currently or potentially involved in Rwanda’s agricultural markets. By applying the principles of facilitation, PSDAG ensures that market functions are not overtaken by the project and that changes created throughout implementation are sustainable. PSDAG organizational management structure is composed of Research Triangle Institute (RTI), which is the prime contractor, and three subcontractors: Crown Agents, which supports the activity Component 1, and both Connexus and Catholic Relief Services which support Component 2. II. EVALUATION RATIONALE Evaluation Purpose The purpose of this evaluation is to assess the progress made to date in achieving the activity’s two main objectives, and the effectiveness of the technical approach and the activity’s responsiveness to support the GOR goal of increasing private investments in agribusiness in Rwanda. The findings will be used by USAID/Rwanda and the PSDAG implementer to prioritize and strengthen the project’s implementation in order to improve performance over the remaining life of the project, and help USAID determine what steps to take in order to sustain the achievements of the project. In addition, the evaluation will help to inform the implementation of other related programs in order to advance engagement of the private sector in driving growth of Rwanda’s agriculture sector. Audience and Intended Uses The audience of this evaluation will primarily consist of USAID/Rwanda, the implementer of PSDAG, and the GOR. Given that the final report of the evaluation will be posted on the USAID Development Experience Clearinghouse, a publicly available repository of technical and programmatic documents related to USAID programming, its audience will also include the broader development community. Findings from this performance evaluation will help identify constraints to PSDAG’s ability to achieve its objectives, and opportunities to scale up interventions for greater results. These findings and recommendations will be used to inform necessary modifications/adaptation to the Year 5 work plan of the PSDAG activity and any other Mission programming. The results of the evaluation also will inform broader audiences relevant to related activities, including the GOR. 44 Questions to be Addressed in the Evaluation The evaluation will respond to the following questions and provide recommendations, where appropriate. The questions are based on the project objectives described above and in project documents. The evaluation will assess implementation from project initiation to present time. The evaluation will make conclusions based on findings, identify opportunities and challenges, and formulate recommendations for improvements for the remaining period of performance. The key evaluation questions are: Objective 1: 1. Within the objective of assisting the GOR to increase private sector engagement, what types of interventions supported by PSDAG (policy and regulatory reform; institutional strengthening; facilitating public-private partnerships, etc.) have been the most successful or effective in attracting private sector investment in agriculture? 2. To what extent are the PSDAG interventions supporting Component 1 [which includes the Agriculture Land Information System (ALIS), the support to MINAGRI’s budget process, and the support to the public-private dialogue structure and process, the aggregation center of services] are likely to result in long lasting impact in attracting private sector investment in agriculture that will continue beyond the conclusion of PSDAG? 3. What are the risks to sustainability and the actions needed to address them within the remaining project term? [Note: in the context of this question, sustainability is defined as the ability of the GOR to continue the key interventions or gains achieved under Objective 1 after the conclusion of the PSDAG activity.] Objective 2 1. Which top three technologies and business model innovations promoted by PSDAG have resulted in the highest return on investments for farmers and for agricultural businesses? What were the reasons for any success or failure? 2. Has PSDAG’s Grant facility been an effective tool in promoting investment? 3. How effective has PSDAG been in preventing double dipping from its grant recipients (i.e., is PSDAG providing grants to support proposals being funded by other donors)? 4. Did the PSDAG approach to social inclusion ensure the involvement of women, youth, and people with disabilities? Are there innovations or other approaches that could enhance the meaningful engagement of these targeted beneficiary groups? Methodology: Data collection and analysis The Evaluation Contractor must propose its own methodology, which must be approved by USAID/Rwanda. At a minimum, it will include: • Thorough review of key project documents. USAID/Rwanda will provide a detailed document review matrix that lists relevant documents, including contract, annual work plan, Monitoring, Evaluation, and Learning (MEL) plan, and progress reports, as well as other information available regarding previous similar projects and other initiatives, to provide a foundation for the evaluation work to be undertaken in the field. A notional document review matrix is provided in Attachment 1. • Key informant interviews with project staff, sub-partner organizations, government officials, private sector, and other stakeholders. An initial list of key stakeholders for key informant interviews that should be considered is provided in Attachment 2. • Focus group discussions with GOR officials and program beneficiaries, which should be representative of both men and women. 45 • Analysis of PSDAG data generated by its M&E team. • Analysis of relevant indicators and data from USAID’s annual Performance Plan and Report (PPR) and Performance Management Plan (PMP). • Integration of USAID’s Collaboration, Learning, and Adapting (CLA) approach in order to enhance the engagement of stakeholders and usefulness of the recommendations for current and future programming. III. SCHEDULE AND MANAGEMENT Timeframe The Period of Performance for this Task Order is defined in Section F.1 of this Task Order (see schedule for deliverables described in Section F.4). Logistics/Management The Evaluation Team implementing this activity (comprised of the personnel described in Section F.3) is required to travel outside of Kigali to the four provinces (Eastern, Northern, Western, and Southern provinces) to carry out focus group discussions, key informant interviews, and any needed survey. The evaluation Contractor is responsible for the administrative support and logistics required to fulfill this task; this includes all travel arrangements, appointment scheduling, secretarial services, report preparations services, printing, duplicating, and translation services. USAID will assist the Contractor in obtaining any additional program documents and contacts necessary to fulfill the task. USAID will also update indicator data to reflect FY 2017 results. IV. DELIVERABLES The following deliverables are required. All written deliverables must be in English and submitted in electronic format (Microsoft Word or PowerPoint). The Evaluation Contractor must submit all draft documents to the Contracting Officer’s Representative (COR) at USAID/Rwanda. The COR will provide comments as noted below. The Evaluation Contractor must incorporate these comments and submit the final deliverables to USAID. The Evaluation Contractor must submit an electronic copy to the Development Experience Clearinghouse at http://dec.usaid.gov. Deliverable 1: Work plan Within 10 calendar days of award, the Evaluation Contractor must submit the work plan to USAID. The work plan must include the timeline, key questions, evaluation design, evaluation methodology, data collection instruments, and data analysis plan. The evaluation design must include: A detailed evaluation design matrix that links the evaluation questions in the Statement of Work (SOW) to data sources, methods, and the data analysis plan; • Draft questionnaires and other data collection instruments or their main features; • List of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan; • Known limitations to the evaluation design; and • Dissemination plan. The COR will provide comments within five working days of receipt of the work plan. Deliverable 2: In-brief with USAID Upon arrival in-country, the Evaluation Contractor will have an oral briefing with USAID/Rwanda for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the evaluation design, if necessary. 46 Deliverable 3: Oral presentation on preliminary results After the completion of the field work, the Evaluation Contractor will deliver a presentation (verbal and with PowerPoint) on preliminary results, key findings, and recommendations. The audience will include USAID/Rwanda and activity stakeholders. The presentation will: • Review and assess PSDAG performance based on the questions listed above. The core of the presentation should be aligned with the main topics of this evaluation under Objective 1 and Objective 2; • Provide the methodology and approach used to evaluate the project; • Provide preliminary analysis, findings, and conclusions; and • Provide sources of information. Deliverable 4: Draft narrative report (in MS Word) Within 50 calendar days of award, the Evaluation Contractor will submit the draft performance evaluation report. The draft report will include details of the methodology followed, key findings to evaluation questions, initial conclusions, and recommendations. USAID will provide comments no later than eight (8) working days after receiving the draft report. The draft report must not exceed 40 pages, excluding the cover page, table of contents, and annexes; it must follow the structure outlined below. Deliverable 5: Within 80 calendar days of award, the Evaluation Contractor will deliver a final performance evaluation report meeting USAID evaluation quality standards as defined in ADS 201maa and 201mah. This performance evaluation will comply with USAID evaluation policy requirements for evaluations, which is available in Annex A of the Evaluation Policy and at the following link: http://www.usaid.gov/sites/default/files/documents/2151/USAIDEvaluationPolicy.pdf The format of the Performance Evaluation Report must include: • Cover page using the standard USAID Evaluation Branding Template • Table of Contents • Acronyms • List of tables or charts (if any) • Executive summary (not to exceed two pages) • Background/Introduction (not to exceed five pages) o Description of the project including goals and objectives o Evaluation rationale/purpose/objectives, including a list of the main evaluation questions o Description of the evaluation design, methodology, and limitations • Findings – Describe the findings related to each of the evaluation questions • Conclusions • Recommendations – Based on the evaluation purpose and the findings, describe what remains to be done; what changes can be made in program design or implementation to result in more effective and/or efficient execution and improved results; identify potential new solutions to problems the project has faced; identify adjustments/corrections that need to be made; and recommend actions and/or decisions to be taken by management. • Lessons Learned – in terms of program implementation, coordination, and beneficiary satisfaction • Annexes o Statement of Work o List of places visited; types of people and groups interviewed (e.g., “female beneficiaries in Village X,” “Senior PSDAG Staff Member,” etc.), including contact information o Copies of all tools, survey instruments, and questionnaires used 47 o Electronic copy of data sets o List of background documents reviewed o Copies of background documents which were used by Contractor, but which were not provided by USAID-Rwanda o Any unaddressed comments submitted by the PSDAG project team (if applicable) Note: The final report must not exceed 40 pages, excluding the cover page, table of contents and annexes. Deliverable 6: Upon acceptance of the report by USAID, the Evaluation Contractor must submit the final report to the Development Experience Clearinghouse (DEC), and relevant data sets and supporting documents must be submitted to the USAID Development Data Library. As per the USAID Evaluation policy approved in 2011, the Evaluation Contractor must submit completed or approved evaluations to the agency’s Development Experience Clearinghouse (DEC) and a cover sheet attached indicating the type of evaluation conducted and design. Each completed evaluation must include a 5-to-10-page summary of the purpose, background of the project, main evaluation questions, methods, findings, conclusions, recommendations, and lessons learned (if applicable) of the evaluation. 48 ANNEX B: EVALUATION TEAM 49 David Rinck, Team Leader. Mr. Rinck is an Agricultural Economist with over 20 years of leadership experience in development programming and monitoring and evaluation, with technical specializations in agriculture and agro-enterprise development, value chain assessment, market systems, economic policy analysis, food security, and advocacy. He has led over 10 evaluations for USAID, including: the Final Evaluation of the USAID Aquafish Innovation Lab funded by the USAID Bureau for Food Security, which he recently completed for ME&A; the Performance Evaluation of the Solutions for African Food Enterprises; the Final Evaluation of the Supporting Opportunities for Livelihoods Development Activity in Sri Lanka; etc. He has worked extensively in Africa, including in Rwanda, Djibouti, Ethiopia, Côte d’Ivoire, Kenya, Malawi, Zambia, Senegal, Ghana, etc. Mr. Rinck’s experience includes five years at the USAID East Africa Regional Mission, where he was responsible for providing technical support to U.S. Government-funded agricultural development and assistance programs across Eastern and Southern Africa, In this role he also developed and managed new initiatives, including the Alliance for Commodity Trade in Eastern and Southern Africa (ACTESA), a specialized agency for agricultural market systems development under the Common Market for Eastern and Southern Africa (COMESA) regional trade zone, and the Market Linkages Initiative (MLI) to improve systems for staple crops in Central and Southern Africa. Between 2001-2006, he served as Regional Technical Adviser for Agro-Enterprise Development in Eastern and Southern Africa for CRS, where he represented the agency in an Agro-Enterprise Learning Alliance with the Centro International de Agricultura Tropical (CIAT) to develop smallholder market systems. He was also the Market Systems Recovery Expert at the Business Development (BDS) Seminars implemented by the International Labour Organization (ILO) between 2003 and 2008. His publications include “Implementing Sustainable Private Sector Development: Relief to Development in Crisis￾Affected Economies” ILO BDS Reader, September 2006 and “Market Development in Crisis-Affected Environments: Emerging Lessons for Achieving Pro-Poor Economic Reconstruction” Small Enterprise Education and Promotion Network (SEEP), 2007. David holds a Master's Degree in Social Sciences from the University of Chicago, and a degree in Agricultural Economics from the University of California at Davis. He has also studied economics, languages and sociology at many locations worldwide, including Beijing University, as well as in the Middle East at Damascus, Beirut, and Sana’a. He speaks English, French, Portuguese, and Spanish. Didier Nkubito, Local Evaluator. Mr. Nkubito is a Rwandan Evaluation Specialist with more than 15 years of experience managing and evaluating agribusiness, food security, and nutrition projects. He has led and conducted a number of performance evaluations of projects focused on rural economic development, such as a recent Midterm Evaluation of the European Commission-funded Inclusive Engagement for Change Project, which promotes sustainable agriculture and food security in Rwanda. He was a member of the ME&A team that conducted the evaluation of the USAID Integrated Improved Livelihood Program. Mr. Nkubito led the: Baseline Survey for the USAID/Improved Services for Vulnerable People (ISVP) Program; Survey of Knowledge, Attitudes, and Practices for the USAID/IILP; and Baseline Survey for the Solar Milk Chilling Innovation Project, etc. He also was a lead consultant for the evaluation of the Rwanda Education Alternatives for Children in Tea-Growing Areas Project and served as the Gender and Nutrition Expert for the Baseline Survey for the Promoting Financial Inclusion for Small Holder Farmers. Mr. Nkubito holds a Master’s degree in Nutrition and Rural Development from the University of Ghent, and a Bachelor’s degree in Food Science and Technology from Kigali Institute of Science and Technology of Rwanda. He speaks Kinyarwanda, English, Swahili, and French. Raphael Rurangwa, Policy Analysis and Development Specialist. Mr. Ruranwga is a Rwandan consultant with more than 20 years of experience in the agriculture sector in Rwanda. A former Director of Policy and Planning at the Rwandan Ministry of Agriculture and Animal Resources, he oversaw the formulation of policies, strategies, and investment plans for the agriculture sector. He has worked with 50 donors, NGOs, farmer organizations, the private sector, and other stakeholders in the agriculture sector to build consensus around policy. Since leaving the GOR, he has consulted as an agriculture specialist for a number of international organizations, including UNDP and the International Fund for Agricultural Development, as well as for USAID projects. Recently, he was a consultant for an internal evaluation of the DFID-funded Program of Support to Agriculture in Rwanda. Mr. Rurangwa holds Bachelor’s and agricultural engineering degrees from the University of Burundi. He speaks Kinyarwanda, English, French, and Swahili. Elie Nsabimana, Agribusiness Market Systems/Value Chains Specialist. Mr. Nsabimana is a Rwandan Agribusiness Specialist with 15 years of experience working with value chains in Rwanda. In particular, he has supported agricultural cooperatives in business planning, management skills development, market linkages, and access to finance. He has worked with a number of international donors, including USAID, the SNV NDO, CARE International, and the International Labor Organization in Rwanda. For SNV NDO, Mr. Nsabimana worked on agribusiness value chains, especially those targeting women and small agriculture producers, provided technical assistance in developing strategic plans and new programs and scaling up current programs, and supported access to credit for underserved groups (women, youth, and PWD). In this role, he led the formation and capacity building of district and provincial agribusiness clusters. He also advised small producer organizations and private businesses in the development and implementation of pro-poor business models in the agricultural sector and facilitated the enhancement of the agricultural value chain by developing financing business models that specifically facilitate access for small businesses. Mr. Nsabimana holds a Master’s in Business Administration from Mount Kenya University in Kigali and a Bachelor’s Degree in Sociology from the National University of Rwanda. He speaks Kinyarwanda, English, French, and Swahili. Mel Schnapper, Organization Management Specialist. Dr. Mel Schnapper is an organizational development (OD) and human and institutional capacity development (HICD) expert with over 25 years of experience in performance assessment, gap analysis, human resources assessment, management development and training, and evaluation. He has worked in many countries, including Rwanda, Ethiopia, Ghana, Côte d’Ivoire, Kenya, Lesotho, Liberia, Malawi, Namibia, Nigeria, Somalia, Tanzania, and Uganda. Throughout his career, he has been able to develop various HICD and training strategies to increase organizational and individual productivity and teamwork among various stakeholders, sectors, and communities in ways that are culturally appropriate. Dr. Schnapper has conducted numerous evaluations and assessments of projects related to HICD and OD. He was an Organizational and Institutional Review Specialist in Kenya, a Senior Evaluator for the evaluation of the Support to Performance Management System in Namibia, a Change Management Specialist in Liberia and, also in Namibia, an Evaluation Expert for the Country Ownership and Organizational Capacity Building Assessment, among other positions he has held. He holds a PhD in Organizational Behavior from the University of Pittsburgh, a Master’s degree in Linguistics and Anthropology from Northwestern University, and a Bachelor’s degree in English Literature from Howard University. 51 ANNEX C: CITATIONS AND REFERENCES REVIEWED 52 PSDAG Project Documents and Presentations “Competitive Benchmarking of Rwanda – Comparison of Rwanda’s Investment Attractiveness,” PSDAG, July 18, 2018. “Contract No.: AID-696-C-14-00002/International Resources Group Private Sector Driven Agricultural Growth (PSD-AG), SECTION C – Statement of Work,” USAID “Investment Promotion Coordinating Committee - Short-Listing Survey, Results and Recommendations,” RTI International, February 12, 2018. Pfeiffer, Kirsten, Chantal Umuhire, Liliane Gasana “USAID/Rwanda Private Sector-Driven Agricultural Growth Project – Value Chain Competitiveness Fund (VCCF),” PowerPoint presentation, RTI for USAID, May 18, 2018. “PSDAG Annual Program Statement (APS) No. IRG- APS-17-01.” “PSDAG Annual Program Statement (APS) No. RTI- APS-18-01.” “PSDAG Competitiveness Grant – Request for Application # IRG-RFA-15-01 USAID Contract No. AID-696-C-14- 00002.” “PSDAG Competitiveness Grant – Request for Applications for Potato Farm Book and Aggregation Center Upgrading # IRG-RFA-15-02 USAID Contract No. AID-696-C-14-00002.” “PSDAG Private Partners + Cooperatives by Location,” Excel database, RTI, May 22, 2018. PSDAG Analyses and Studies Hastings, Mary Beth “Gender and Social Inclusion in Rwanda’s Agricultural Value Chain Private Sector Training Curriculum to Build GESI Capacity and Economic Growth - Facilitator Guide,” IRIS Group for RTI, March 10, 2017. Jenkins, Dr. Glenn P., Mikhail Miklyaev, Noel Ujeneza, Shahryar Afra, François-Xavier Ndungutse, and Majid Hashemi, Melanie Bitter, Joseph Nzomoi “Poultry Competitiveness Study,” IRG with MINAGRI, September 1, 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) – Beans Value Chain Analysis,” IRG for USAID, September 2015. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Project – Horticulture Chain Analysis,” IRG for USAID, September 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Project – Irish Potato Chain Analysis,” IRG for USAID, September 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) – Maize Value Chain Analysis,” IRG for USAID, September 2015. “Smart Agricultural Information System (SAIS) - Concept and Cost-Benefit Assessment,” MINAGRI, January 20, 2017. PSDAG M&E Plans “Rwanda Private Sector Driven Agricultural Growth (PS-DAG) Monitoring & Evaluation Plan FINAL,” IRG for USAID, December 2014. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Monitoring and Evaluation Plan – Fiscal Year (FY) 2016: October 1, 2015 – September 30, 2016 FINAL,” IRG for USAID, January 2016. 53 “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Monitoring and Evaluation Plan – Fiscal Year (FY) 2017: October 1, 2016 – September 30, 2017,” IRG for USAID, November 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Fiscal Year (FY) 2018 Monitoring and Evaluation Plan –: October 1, 2017 – September 30, 2018,” RTI for USAID, November 2017. PSDAG Annual Reports “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Fiscal Year (FY) 2015 Annual Progress Report: October 1, 2014 – September 30, 2015,” IRG for USAID, October 2015. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Activity - Project Fiscal Year (FY) 2016 Annual Progress Report: October 1, 2015 – September 30, 2016,” IRG for USAID, October 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) - Project Fiscal Year (FY) 2017 Annual Progress Report: October 1, 2016 – September 30, 2017,” IRG for USAID, October 2017. PSDAG Quarterly Reports “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2015 Quarterly Progress Report: September 1, 2014 – December 31, 2014,” IRG for USAID, January 2015. “Rwanda Private Sector Driven Agricultural Growth (PSDAG) Project – Fiscal Year (FY) 2015 Quarterly Progress Report: January 1, 2015 – March 31, 2015,” IRG for USAID, April 2015. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2015 Quarterly Progress Report: April 1, 2015 – June 30, 2015,” IRG for USAID, July 2015. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2016 Quarterly Progress Report: October 1, 2015 – December 31, 2015,” IRG for USAID, January 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2016 Quarterly Progress Report: January 1, 2016 – March 31, 2016,” IRG for USAID, April 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2016 Quarterly Progress Report: April 1, 2016 – June 30, 2016,” IRG for USAID, July 2016. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2017 Quarterly Progress Report: October 1, 2016 – December 31, 2016,” IRG for USAID, January 2017. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2017 Quarterly Progress Report: January 1, 2017 – March 31, 2017 (redacted),” IRG for USAID, April 2017. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2017 Quarterly Progress Report: April 1, 2017 – June 30, 2017 (redacted),” IRG for USAID, July 2017. “Rwanda Private Sector Driven Agricultural Growth (PSDAG)– Fiscal Year (FY) 2018 Quarterly Progress Report: October 1, 2017 – December 31, 2017 (redacted),” RTI for USAID, January 2018. Background References Boudreaux, Karol C. “A Better Brew for Success: Economic Liberalization in Rwanda’s Coffee Sector,” World Bank, 2010. “Concept Paper on the Organization of the Irish Potato Value Chain,” Ministry of Trade and Industry, June 2015. Doss, Cheryl “The Role of Women in Agriculture,” Agricultural and Development Economics Division (ESA) Working Paper no. 11-02, Food and Agriculture Organization (FAO), March 2011. “Invest in Rwanda – Investment Opportunities,” RDB, November 2017. 54 “Investment Guide to Rwanda - Opportunities and Conditions,” UNCTAD, 2012. King, Gary, Robert Keohane, and Sydney Verba “Designing Social Inquiry: Scientific Inference in Qualitative Research,” Princeton University Press, 2016. Midmore, David “Principles of Tropical Horticulture,” CABI, 2015. “Rwanda Vision 2020 (revised 2012),” Republic of Rwanda, 2012. “Second Economic Development and Poverty Reduction Strategy (EDPRS II) for 2013-2018,” GOR, 2013. “Strategic Plan for the Transformation of Agriculture in Rwanda - Phase III 2013-2017,” MINAGRI, 2013. 55 ANNEX D: CONTACT LIST 56 Key Informant Interviews Date Name Organization Title USAID/Rwanda and PSDAG Staff May 14 Celestin Mutumayi USAID M&E Specialist May 14 Emmanuel Gasana USAID Senior M&E Specialist May 14 Malik Haidara USAID Econ. Growth Director May 14 Patrice Hakizimana USAID Ag. Dev. Specialist May 14 Janvier Mwitirehe USAID Financial Analyst May 14 Frank Ntaganzwa USAID Financial Analyst May 14 Melanie Bittle PSDAG Chief of Party May 14 Kirsten Pfeiffer PSDAG Deputy Chief of Party May 14 Wilson Kaikai PSDAG M&E Specialist May 24 Eve Gossiaux PSDAG Deputy M&E Specialist June 4 Melanie Bittle PSDAG Chief of Party June 4 Wilson Kaikai PSDAG M&E Specialist June 6 Kirsten Pfeiffer PSDAG Deputy Chief of Party Banks and Finance Institutions May 24 Dative Nzasingizimana Duterimbere CEO May 24 Marie Claire Umugwaneze Duterimbere Head of Marketing May 24 Esperance Mukarugwiza IFC Agribusiness Specialist May 25 Benjamin Manzi Rwanda Development Bank (BRD) Senior Exports Advisor May 25 Jackson Ndaruhutse BRD Investment Officer May 25 Malaika Rousseau BRD Fund Mobilization June 5 Jules Ndahayo Umutanghula Finance Managing Director Government of Rwanda and Development Partners May 22 Eric Mbonigaba PSF Director, Chamber of Agriculture May 22 Dr. Charles Murekezi MINAGRI DG Agricultural Development May 23 Dr. Octave Semwaga MINAGRI DG Planning May 23 Eugene Rwibasira East Africa Grains Council (EAGC) Country Director May 23 Beatrice Uwumukiza Rwanda Agricultural and Livestock Inspection and Certification Services (RALIS) Director General May 23 Isidore Nsengimana RALIS Professional May 23 Eugene Rwibasira Rwanda Development Organization (RDO) Country Director May 24 Sajji Ryakunze AgriProFocus Coordinator May 24 Fabien Ngoga EUCORD Project Coordinator May 24 Providence Mavubi MINAGRI (PSDAG) EIA May 25 Jean Baptiste Hategekimana RYAF Chairman May 25 Rosine Mwiseneza RYAF Treasurer May 25 Patrice Habinshuti RYAF Programs Director May 25 Esperance Nyiramucyo RYAF Finance Director May 31 Peter Bagirishya Musanze District Gov. Executive Secretary June 1 Blaise Harerimana Rubavu District Gov. Director of Planning June 4 Fred Mugabe MINICOM Industrial Dev. Policy June 5 Sandrine Urujeni NAEB Deputy CEO June 5 Epimaque Nsanzabaganwa NAEB Horticulture Division Manager June 6 Birasa Nyamulinda RDB (PSDAG) EIA June 6 Winifred Kabega RDB Head of Investment Promotion 57 Date Name Organization Title June 6 Livingstone Nshemereirwa Access to Finance Rwanda (AFR) Head of Agriculture and Rural Finance Jun6 6 Raymond Murenzi Rwanda Standards Board (RSB) Director General PSDAG-Assisted SMEs and Cooperatives May 22 Leon Nduwayezu AgroPlast Ltd. CEO May 22 Leonidas Mushimiyinama Deyi Ltd. Manager and Owner May 22 Mbrute Suraith Green Harvest Ltd. Managing Director May 23 Dennis Mutawazi Norlega Macadamia Operations Manager May 23 Augustin Mutijima Sarura Commodities Managing Director May 23 Roman Kasema Shekina Deputy MD May 24 John Porter Makasa Creamery Managing Director May 24 Trevor Augustine Minimex General Manager May 25 Emmanuel Harelimana E & Food Direct Owner May 28 Michael Kalisa KNU Agro Mechanized Services Ltd. (KNU) Managing Director May 28 Eugene Gisagara Agri-Construction Owner May 28 Alfred Nkubili Enterprise Nkubiri & Sons (ENAS) CEO May 28 Justin Havugimana ENAS Finance and Admin Manager May 28 Madeleine Niyonagze ENAS Managing Director May 29 Jean Hakuzimana Tuazamuarane President May 29 Sylvestre Barajiginywa Tuazamuarane Manager May 29 Medaro Ndashyikirwa Tuazamuarane Member May 30 David Bugingo Agriseed Africa Shareholder May 30 Ntizo Senkesha Agriseed Africa Agronomist May 30 Thijs Boer Winnaz (Holanda Fair Foods) Managing Director May 30 Foutine Hagenimana Cooperative Intego Secretary May 30 Josephine Nyiramariza Cooperative Intego Auditor May 31 Dr. Fabien Hagenimana Institute of Applied Sciences (INES) – Ruhengeri Rector May 31 Filbert Icyishaka INES – Ruhengeri Lab Analyst June 2 Salomon Mbarushimna SPF Managing Director June 7 Matt Gantz Agri-Lift Senior Manager 58 Focus Group Discussions and Beneficiary-Based Surveys FGD BBS Date Province/District Sector # Male Fem Male Fem May 22 South/Kamonyi Gacurabwenge, Rukoma 2 7 6 45 29 May 23 South/Kamonyi Gacurabwenge, Rukoma 2 7 6 25 33 May 24 West/Nyabihu Jenda 2 9 5 49 55 May 25 West/Nyabihu Jenda 1 5 5 61 41 May 28 North/Burera Gahunga, Nemba 1 4 3 44 53 May 29 North/Burera Gahunga, Nemba 1 4 3 56 36 May 30 East/Ngoma Mugesera 2 6 6 45 53 May 31 East/Ngoma Jarabana 1 3 3 52 34 June 1 East/Nyagatare Karama, Tabagwe, Mukama 4 10 14 69 48 June 4 East/Nyagatare Karangazi, Kagitumba 0 0 0 36 40 Total 16 55 51 482 422 59 ANNEX E: EVALUATION TOOLS 60 Key Informant Interview – Bank/Finance Institution Date of interview: MM-DD-YYYY Interviewer name: - Respondent name: - Organization and position: - Contact: - Questions Does your bank have any direct or indirect relationships with PSDAG or any of their partners (SMEs or cooperatives)? If yes, how and when did your Bank become involved with PSDAG and what was your role? How would you assess the current “access to finance” situation for agriculture? To what extent has PSDAG intervention influenced/impacted rural finance for agriculture? To the best of your knowledge, has there been an increase in the number of loans made in agriculture due to your partnership with PSDAG? To what extent has PSDAG’s intervention influenced/impacted the financial services sector in Rwanda? What new products have been developed for agribusinesses that were not previously available (i.e., prior to PSDAG involvement)? What progress has been made in recent years in the numbers of women able to take out loans for their agribusinesses? To your knowledge, have PSDAG’s interventions influenced “access to finance” for women individually or as a group? In your view, how can PSDAG best improve access to finance for agribusinesses, including SMEs and farmer cooperatives, in the future? 61 Key Informant Interview – GOR/Development Partner Date of interview: MM-DD-YYYY Interviewer name: - Respondent name: - Organization and position: - Contact: - Questions 1. How successful has the PSDAG activity been in assisting the GOR to increase private investment in agriculture through the following different types of interventions? a. Facilitating public-private partnerships? b. Institutional strengthening? c. Improving the enabling environment for local and regional private sector investment? 2. What are the reasons for the success of these types of interventions in attracting private sector investment in agriculture, or lack thereof? 3. How likely to continue are the following interventions in support of agricultural investment supported by the PSDAG activity once the activity ends? a. Agriculture Land Information System (ALIS)? b. Support to Ministry of Agriculture and Animal Resources’ (MINAGRI) budget process? c. Support to the public-private dialogue structure and process? d. Services aggregation center? 4. What are the reasons that these interventions are likely to be sustained in the future, or not? 5. What measures could PSDAG take to increase the chances that these interventions will be sustained? 62 Key Informant Interview – SME / COOPERATIVE Date of interview: MM-DD-YYYY Interviewer name: - Respondent name: - Organization and position: - Contact: - Questions 1. How successful was the partnership between PSDAG and your company in increasing private investment in agriculture through the PSDAG grant facility? 2. What are the reasons for the success of this type of intervention? 3. What were the technologies and business model innovations promoted by PSDAG through the private sector that have resulted in the highest return on investments for farmers and for agricultural businesses (please list specific examples)? 4. What do you think are the reasons the technologies or business models you have listed were successful? 5. What are some examples of less successful technologies and business model innovations supported by PSDAG, and why were they less successful? 6. Has PSDAG’s grant facility been an effective tool in promoting investment? Why or why not? 7. Has PSDAG been providing grants to support activities that are also being funded by other donors? a. If so, is the funding for separable tasks within those activities, or for the same work? If it is for the same work, how widespread is the double dipping, in your opinion? 8. What was the PSDAG approach to social inclusion to involve women, youth, and persons with disabilities? a. How successful was that approach? Why? b. What possible innovations or other approaches could enhance the meaningful engagement of these targeted beneficiary groups? 9. How likely is agricultural investment supported by PSDAG activity likely to continue once the activity ends? 10. What measures could PSDAG take to increase the chances that these interventions will be sustained? 63 Focus Group Discussion (FGD) Guide Date of Interview - (DD/MM /YY) /___/___/2018 Facilitator Name: - Note Taker Name: - Respondents’ Full Names Cooperative Sex Age Addresses (District, Sector, and Cell) - - - - - Questions 1. How has your cooperative helped you in the production of (maize, beans, Irish potatoes, horticultural products)? (please list) 2. Has its support changed during the past four years? How? 3. Has your cooperative changed its approach to working with women, youth, and persons with disabilities during the past four years? How? 4. Have you been able to obtain adequate and timely seed and fertilizer for each of these crops? From where? 5. Have you utilized any form of agro-financing (bank, savings and credit union, intermediaries, etc.) during the past season for growing (maize, beans, Irish potatoes, horticultural crops listed)? 6. If so, for what purpose, and how does this compare with your use of agro-financing four years ago? 7. What are the opportunities for women, youth, and persons with disabilities to access agro-financing? 8. Have those opportunities changed during the past four years? How? 9. Where have you sold your (maize, beans, Irish potatoes, horticultural crops listed) crop after your most recent harvest? 10. Are there more places where you have the opportunity to sell your (maize, beans, Irish potatoes, horticultural crops listed) than there were four years ago? If so, what kinds? 11. What are the opportunities for women, youth, and persons with disabilities to access those opportunities? Have they changed in the past four years? 12. Is agro-infrastructure (roads, storage facilities, etc.) adequate to meet your needs for marketing of (maize, beans, Irish potatoes, horticultural crops listed)? 13. How does this compare with agro-infrastructure four years ago? 64 Informed Consent to Participate in an Interview/FGD/BBS Name of the project Private Sector Driven Agricultural Growth (PSDAG) Contractor ME&A Contractor Telephone +1 (301) 652-4334 Contractor Email meainfo@engl.com Investigator Name - Investigator phone number - Investigator email - Address (District, Sector, Cell) - Date - My name is and I am here on behalf of USAID. We are conducting a performance evaluation of USAID Private Sector-Driven Agricultural Growth (PSDAG) program to assess the progress made it has made to date in achieving its objectives. For that purpose, we would like to gather some information from the beneficiaries, program staff, and other stakeholders who may be knowledgeable about the PSDAG’s activities. PSDAG is a project funded by USAID that is designed to support the Government of Rwanda in promoting private sector investment in agribusiness and agricultural value chains. PSDAG also strengthens private sector capacity and facilitates expanded investments for existing and new private sector entities (local and international businesses located in Rwanda). We have selected you because we think you may have worked with PSDAG, benefited from its activities, or because you may have useful information about the program. The interview/focus group/survey should only take about 45 minutes to complete. Your answers will be added to those of other stakeholders and analyzed to help understand PSDAG’s progress. All of your answers are completely confidential; if any question makes you uncomfortable, you do not need to answer it. And if you decide you do not want to continue with the discussion, you can also ask the interviewer to stop at any time. The information you provide us with will in no way affect your partnership with PSDAG or the services you are currently receiving from it. Do you agree to participate in this interview? Yes ◻ Proceed with signing the consent form No ◻ Thank him/her and stop the interview Participant’s Agreement: If you have understood the information and you voluntarily agree to participate in the research please sign your name or make your mark below. Participant Name Participant Signature Date Impartial Witness Name (If participant is illiterate) Impartial Witness Signature Date 65 Beneficiary-Based Survey Questionnaire Date of Interview - (DD/MM /YY) /___/___/2018 Respondent Name: - Enumerator Name: - Supervisor Name: - 1. Section 0-Respondent Identification Questions Answers Province [code 0.1] - District [code 0.2] - Sector [write the name of the sector] - Sex: 1 = Male, 2 = Female (observe don’t ask) - Age [year of Birth] - Marital status (1=Single, 2=Married, 3=Divorced, 4=Widow/er, 5=Other [specify]) - Position of the respondent in the household [code 0.3] - Household type (1=Male headed, 2=Female headed, 3=Child headed) - Do you belong to an Agricultural Cooperative? =Yes, 0=No - If yes, name of your cooperatives - What is your primary occupation [code 0.4] - Any other Primary occupation? 1=Yes, 0=No - Household size (Number of members in the households including yourself) - [code 0.1] 1=Southern 2=Northern 3=Eastern 4=Western [code 0.2] 1=Kamonyi 2=Burera 3=Nyagatare 4=Ngoma 5=Nyabihu [code 0.4] 1=Agricultural production 2=Non-ag labor (daily wage) 3=Salaried worker 4=Self-employed technical activities, e.g., carpentry, mason, 5=Petty trading 6=Remittance from relatives 7=Students 8=Other [Specify]________ [code 0.3] 1=Head 2=Spouse 3=Son 4=Daughter 5=Hired worker 6=Family relative 7=Non-family relative Questions Sub Question/Codes Answers 1. Do you grow any of the following crops? 1=Maize, (If no, go to the question 2) 2=Beans, (If no, go to the question 3) 3=Irish potatoes, (If no, go to the question 4) 4=Horticultural crops? (If no, go to the question 4) 2. If not growing maize what are the reasons? 3. If not growing beans what are the reasons? 4. If not growing Irish potatoes what are the reasons? 5. If not growing horticultural crops what are the reasons? 6. What is the size of your farm? 7. What was the quantity of each of the inputs you used to grow each of those crops four years ago? 66 Questions Sub Question/Codes Answers a. What was the quantity of each of the inputs you used to grow maize four years ago? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) b. What was the quantity of each of the inputs you used to grow beans four years ago? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) c. What was the quantity of each of the inputs you used to grow Irish potatoes four years ago? ∉ Land, (Area) ∉ Labor (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other (Specify types and quantity) d. What was the quantity of each of the inputs you used to grow horticulture four years ago? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) 8. What is the quantity of each of the inputs you used in your last harvest (Season)? a. What is the quantity of each of the inputs you used in your last harvest to grow maize? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) b. What is the quantity of each of the inputs you used in your last harvest to grow beans? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) c. What is the quantity of each of the inputs you used in your last harvest to grow Irish potatoes? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) d. What is the quantity of each of the inputs you used in your last harvest to grow horticulture? ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) 67 Questions Sub Question/Codes Answers 9. What is the cost of the following inputs? ∉ Cost of lending a unit land, (Area) per season ∉ Cost of one labor person/day, (Person/day) ∉ Cost of fertilizer per type, (Kg) ∉ Cost of pesticides, (Specify types per unit) ∉ Cost of Tools and equipment, (Specify types) ∉ Other, (Specify types) 10. What was your gross production (in Kg) of per harvest four years ago? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 11. What was your gross production (in Kg) of each of those crops in your last harvest? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 12. What was your production per unit (hectare, m2 of greenhouse, etc.) per harvest four years ago? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 13. What was your production per acre in your last harvest? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 14. How did the agricultural production change over the last three years? ∉ 2=unchanged ∉ 1=increased ∉ 0=decreased 15. What do you think was the main reason for this change in production (increase)? ∉ Climate condition ∉ Extension services ∉ Equipment ∉ Proper farm management ∉ Proper use of fertilizers and pesticides ∉ Increase in land ∉ Promising market ∉ Improved agricultural techniques ∉ Other, (Specify) 16. What do you think was the main reason for this change in production (decrease)? ∉ Bad climate condition ∉ Limited extensions services ∉ Limited equipment ∉ Decreased land ∉ Lack of agricultural inputs ∉ Weed and pests ∉ Lack of market ∉ Loss of soil fertility ∉ Diseases ∉ Other, (Specify) 17. What was your revenue (in FRW) for each of those crops per harvest four years ago? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 68 Questions Sub Question/Codes Answers 18. What was your revenue (in FRW) for each of those crops in your last harvest? ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 19. On what do you often use the revenue from the sales of your produce? ∉ Increasing household income ∉ Investing in business ∉ Increasing agriculture production ∉ Paying children’s school fees ∉ Buying livestock ∉ Helped to buy household items ∉ Helped buy food ∉ Helped in festive occasion ∉ Helped cover medical fees, including health insurance ∉ Helped better manage my business ∉ Attracted more customers in my business ∉ Started other off-farm activities ∉ Accessed a loan from formal financial institutions (Having collateral) ∉ Other, (Specify) 20. Who provided you with the extension assistance for following crops? (1=Cooperative, 2=Input supplier, 3=Agribusiness, 4=Other (list)] ∉ Maize ∉ Beans ∉ Irish potatoes ∉ Horticultural crops 21. For which of the following practices did you receive assistance? (1=Yes, 0=No) ∉ Inputs, (Fertilizers, Pesticides, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing, ∉ Other, (Specify) 22. Who provided you the assistance on inputs (fertilizers, pesticides, seeds)? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 69 Questions Sub Question/Codes Answers 23. Who provided you the assistance on extension services in production? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 24. Who provided you the assistance on tools and equipment for production? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 25. Who provided you the assistance on extension services for post￾harvest? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 70 Questions Sub Question/Codes Answers 26. Who provided you the assistance on tools and equipment for the post-harvest? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 27. Who provided you the assistance on marketing? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 28. Who provided you the assistance on others (specify)? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 29. How useful was the assistance you received? (1=Not at all useful, 2=Not very useful, 3=Useful, 4=Very useful) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 71 Questions Sub Question/Codes Answers 30. Did you adopt any of the following practices? (1=Yes, 0=No) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 31. From whom did you learn about the following practices? [1=Cooperative extension, 2=Input supplier, 3=Agribusiness, 4=Other (list)] ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 32. Where did you acquire the inputs/technologies to adopt the following practices? [1=Cooperative extension, 2=Input supplier, 3=Agribusiness, 4=Other (list)] ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 33. How effective were the following practices? (1=Not effective, 2=Low effectiveness, 3=Some effectiveness, 4= Highly effective) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 34. Will you continue the following practices in the next growing season? (1=Yes, 0=No) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 35. If no, why not? (1=Not effective, 2=Too expensive, 3=Not appropriate, 4=Too hard/too much work, N=Etc.) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 36. How has the availability of the following farming inputs changed in the past four years? (1=Worse, 2=Same, 3=Better, 4=Much better) ∉ Land, (Area) ∉ Labor, (Person/day) (Specify number of days) ∉ Fertilizer, (Kg specify types and quantity) ∉ Pesticides, (Specify types and quantity) ∉ Tools and equipment, (Specify types and quantity) ∉ Other, (Specify types and quantity) 72 Questions Sub Question/Codes Answers 37. From where did you receive market information? (list sources) ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 38. Was the information received new? (1=Yes, 0=No) ∉ 1=YES ∉ 0=NO 39. Did you use the marketing information you received to make decisions on (1=Yes, 0=No) ∉ Sales price of crops ∉ Timing of crop sales ∉ Location of crop sales ∉ Type of crop to sell ∉ Other, (Specify) 40. Do you have a reliable market (price, quantities) for your produce? ∉ 1= YES ∉ 0= NO 41. Since the project started have you sold your products in a new market? ∉ 1= YES ∉ 0= NO 42. How do you often sell you produce to the following market? (1=Not at all sell, 2=Rarely, 3=Often 4= Very often) ∉ Do not sell produce (consume all) ∉ Neighbors ∉ Cooperatives ∉ Local small village market ∉ Local large village ∉ Peri-urban market ∉ SMEs/Private sector with agreement ∉ Other, (Specify) 43. How long does it take, to reach the following in minutes (one way without loads)? ∉ Neighbors ∉ Cooperatives ∉ Local small village market ∉ Local large village ∉ Peri urban market ∉ SMEs/Private sector with agreement ∉ Other, (Specify) 44. What is the facilitation you have to reach the formal market? ∉ Market information ∉ Transport ∉ Equipment and tools ∉ Infrastructure ∉ Skills and knowledge ∉ Personnel 73 Questions Sub Question/Codes Answers 45. Who provide the market information? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 46. Who provide transport for the market? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 47. Who provide equipment and tools for the market? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other specify 74 Questions Sub Question/Codes Answers 48. Who provide infrastructure for the market? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other, (Specify) 49. Who provide skills and knowledge for the market? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other, (Specify) 50. Who provide the personnel for the market? ∉ Input suppliers ∉ Cooperatives ∉ Financial institutions, (SACCO, Bank, IMF) ∉ Informal saving and lending groups ∉ Sector/district agronomists or staff ∉ Government project, (Specify) ∉ Mass media, (Radio, TV, Newspapers) ∉ Distributors ∉ Customers ∉ Neighbors ∉ Agricultural promoters ∉ Other development project, (Specify) ∉ Other, (Specify) 51. In your opinion, what are the major factors that prevent you from accessing new markets? ∉ Transport cost ∉ Transport availability ∉ Bad Roads ∉ Markets are too far ∉ Markets are too small (not enough buyers) ∉ Lack of market information ∉ Low price ∉ Low production ∉ Other, (Specify) 75 Questions Sub Question/Codes Answers 52. To what extent was this information on market from different sources useful to increase your sales? 1=Not at all useful 2=Not very useful 3=Useful 4=Very useful 53. Did you use the market information to decide what or how to plant? ∉ 1=YES ∉ 0=NO 54. Did you receive any financing to help you adopt any of the following practices? (1=Yes, 0=No) ∉ Inputs (Fertilizers, Fertilizers, Seeds) ∉ Extension service in production ∉ Tools and equipment for production ∉ Extension services for post-harvest ∉ Tools and equipment for post-harvest ∉ Marketing ∉ Other, (Specify) 55. From whom did you receive financing? (1=Yes, 0=No) ∉ Microfinance institution ∉ Savings and credit union ∉ Bank ∉ Cooperative ∉ Input supplier ∉ Other, (Specify) 56. Have you received financing previously from the following? (1=Yes, 0=No)? ∉ Microfinance institution ∉ Savings and credit union ∉ Bank ∉ Cooperative ∉ Input supplier ∉ Other, (Specify)