Agribusiness Investment for Market Stimulation (AIMS) Program End-Line Assessment Final Report for Table of Contents Glossary ............................................................................................................................. 3 January 2022 AIMS End Line Assessment 1 Executive Summary ................................................................................................................... 4 1 Introduction & Research Questions ........................................................................................ 8 2 Methodology & Limitations.....................................................................................................11 2.1 Approach ....................................................................................................................11 2.1.1 Overview..............................................................................................................11 2.1.2 Outcome Harvesting Methodology .......................................................................13 2.2 Sampling.....................................................................................................................13 2.2.1 ASME Survey Sample Design .............................................................................13 2.2.2 ASME Survey Target Sample ..............................................................................15 2.2.3 ASME Survey Actual Sample...............................................................................16 2.2.4 Samples for Qualitative Data Collection...............................................................17 2.3 Limitations ..................................................................................................................17 3 Evaluation Findings ...............................................................................................................20 3.1 Program Indicator Targets and Results.......................................................................20 3.2 ASME Survey Summary .............................................................................................21 3.2.1 ASME Demographics...........................................................................................21 3.2.2 ASME Financial Activity .......................................................................................22 3.2.3 ASME Loan Application Experience (Borrowers and Applicants Only).................23 3.2.4 ASME EALGF Borrowing Experience (Borrowers Only).......................................24 3.2.5 ASME Experience of AIMs Support .....................................................................25 3.3 Research Question Findings.......................................................................................26 3.3.1 Relevance............................................................................................................26 3.3.2 Efficiency .............................................................................................................29 3.3.3 Effectiveness .......................................................................................................31 3.3.4 Impact..................................................................................................................36 3.3.5 Sustainability........................................................................................................39 4 Lessons Learned and Recommendations .............................................................................40 5 Summary and Conclusion .....................................................................................................44 Annex 1: Scope of Work ...........................................................................................................45 Annex 2: Detailed Methodology Description..............................................................................46 Data Sources & Collection Methods ......................................................................................46 Desk Research ..................................................................................................................46 Funding Partners ...............................................................................................................46 Program Implementors.......................................................................................................46 AIMS End Line Assessment 2 EALGF Partner Banks .......................................................................................................46 BASPs ............................................................................................................................46 ASMEs ............................................................................................................................47 Evaluation Questions and Information Sources .....................................................................48 Annex 3: Information Sources ...................................................................................................51 List of ASMEs Surveyed........................................................................................................52 List of Documents Reviewed .................................................................................................55 Other Data Sources................................................................. Error! Bookmark not defined. Annex 4: Evaluation Tools ........................................................................................................56 AIMS End Line Assessment 3 Glossary AIMS Agribusiness Investment for Market Stimulation ASMEs Agribusiness Small and Medium Enterprises BASPs Business Advisory Service Providers BDS Business Development Services DCA Development Credit Authority DFC United States Development Finance Corporation DFIs Development Finance Institutions EALGF East African Loan Guarantee Facility EIB European Investment Bank FAD Food Assistance Division FFPr Food for Progress FGD Focus Group Discussion FIs Financial Institutions GC Global Communities KBA Kenya Bankers Association M&E Monitoring and Evaluation NBFIs Non-Bank Financial Institutions NCE No-Cost Extension ODK Open Data Kit OECD Organization for Economic Cooperation and Development DAC Development Assistance Committee SoW Scope of Work TA Technical Assistance USAID United States Agency for International Development USDA United States Department of Agriculture USG United States Government AIMS End Line Assessment 4 Executive Summary Introduction The Agribusiness Investment for Market Stimulation (AIMS) program sought to increase agricultural trade through improving access to finance and markets for agribusiness small and medium enterprises (ASMEs) in Kenya, Tanzania, and Malawi. This report is an end-line evaluation the AIMS program in the no-cost extension (NCE) period from September 2019 to March 2022 1 . The core objective for this period was to promote and enable the utilization of a loan guarantee facility provided by the United States Development Finance Corporation (DFC), known as the East African Loan Guarantee Facility (EALGF), by two participating banks in Malawi (Ecobank) and Kenya (I&M Bank). The purpose of the EALGF was to stimulate lending to ASMEs to enable them to grow their business and through that to increase incomes and employment. Kadale Consultants (Malawi) and KCIC Consulting (Kenya) jointly conducted the end-line evaluation with the specific objectives as per the Scope of Work (SoW) to: Assess whether the activities under the EALGF achieved the intended results; Evaluate the impact of the EALGF on increased lending to the agricultural sector; Assess the likelihood that the EALGF results will be sustained; Document key successes, best practices, challenges, and lessons learned; and provide specific recommendations for future access to finance programs. To deliver these objectives, a set of research questions were designed to evaluate the Relevance, Efficiency, Effectiveness, Impact and Sustainability of the program. Methodology The evaluation used a participatory, mixed methods approach that aimed to provide a holistic view of program outcomes and causality. The quantitative element was a survey of 83 ASMEs (40 in Malawi and 43 in Kenya), sampled from a total population of 185 ASMEs (83 in Malawi and 102 in Kenya) that were involved in the AIMS program during the NCE phase. Qualitative methods included in-depth interviews with funding partners, implementing teams, participating banks, business advisory service providers (BASPs) and focus groups with ASMEs, as well as a desk review of relevant project documentation. Data collection took place between September and December 2021. Findings The finding from the evaluation were grouped according to which research area they informed. Relevance: Access to (formal) finance was identified as a constraint to growth by some ASMEs in both countries but may have been less relevant for smaller or less well established ASMEs, in Malawi in particular. Banks said that collateral was an important barrier for ASMEs in securing loans and that this was addressed by the EALGF to some extent. However, lack of collateral remained a barrier for many ASMEs to access finance, even with the guarantee in place. Elements of the design of the EALGF were less relevant for the operational context for banks in Malawi and Kenya. For example, ASME needs for specific working capital and overdraft facilities 1 NCE implementation activities were scheduled to end in September 2021 at the time when the Endline Assessment was commissioned, but were subsequently extended to March 2022. AIMS End Line Assessment 5 were not covered by EALGF. The EALGF was also not able to benefit the relatively high number of ASMEs that predominantly operate in informal markets and were not able to evidence their past or projected income in way acceptable to the banks. In terms of impact, there are significant gaps in ASMEs’ financial management capabilities that limited how effective access to formal finance was in promoting ASME growth. Furthermore, formal financial products are unlikely to have been able to meet ASMEs’ ‘opportunistic’ access to finance requitements. Efficiency: In terms of internal challenges, the program had difficulty in signing up banks to the EALGF in the first place, and there were further challenges in terms of its utilization, with DFC’s requirements for participation seen as onerous by target banks. Once signed up, the communication between the partner banks, AIMS and DFC was not always efficient. The main external factor that affected implementation was the Covid-19 pandemic. More generally, lending, in Malawi in particular, is hampered by a weak legal system, low trust, high default rates, donor dependency and thin markets for formal credit among ASMEs. There was also competition from other similar guarantee schemes available that limited the uptake and usage of the EALGF by partner banks. In terms of successful activities, the capacity building support and bank staff training in agri￾lending was reported by banks as being very effective in increasing the amount of loans they were able to make to ASMEs. Banks also highly valued the hands-on nature of AIMS support and links to new clients. Focusing BASP support on ASME loan application documents was also an effective change that the program made in response to earlier evaluation activity. AIMS coordinated with other actors during initial programme phase, however, there was limited scope for coordination in NCE phase. More work could have been done with other actors in the agriculture development sector to identify suitable ASME loan candidates for partner banks. Effectiveness: Factors that encouraged utilization of the EALGF included, alignment with group and country level bank strategies, product sensitization forums and other linkages facilitated between banks and ASMEs, and AIMS staff support to banks on sign up and getting loan approval from DFC. Both banks also said that the capacity building they received in agri-lending improved their engagement with ASMEs and encouraged them to lend into new sectors. Factors that discouraged utilization of the EALGF included, DFC’s requirement to review every loan application, DFC’s due diligence requirements on applications and general level of responsiveness, a lender suspension event at Ecobank that restricted lending in key period, and the difficulty of claiming on guarantee from the DFC in case of default. I&M Bank also felt the guarantee fee was too high and currency denomination constraints hampered utilization. In terms of the effectiveness of BASP support in helping ASMEs access loans, BASPs in both countries agreed that support with financial records was a priority need for ASMEs. ASMEs who had used BASPs were generally very positive about the services, and while BASP support for EALGF loan applications was effective in some cases, in others the supported ASMEs were still rejected. On a positive note, BASP support also helped to improve ASME access to other sources of finance in some cases. BASP support effectiveness was often limited by financial capacity gaps AMSEs, particularly in Malawi. Many ASMEs need ongoing effective financial management capability and not just a one￾off intervention. BASPs also commented they needed better understanding of banks/DFC’s requirements for their support to be consistently effective in enabling loan applications. AIMS End Line Assessment 6 Impact: The project clearly enabled partner banks to lend to new ASMEs who had less collateral than would normally be required and also clearly attracted new applicants for loans to the EALGF partner banks. Roughly two-thirds of ASMEs who had received loans covered by the EALGF said that they experienced some form of business growth as a result of the loan. In terms of attribution, most ASMEs that had received a loan said that AIMS direct support and/or the support they had received from BASPs (co-funded by AIMS) on loan applications had helped them to secure their loans. Most ASMEs said they were unlikely to have used BASP support in loan applications without AIMS support. In terms of the project’s unintended consequences, positive outcomes included, new complimentary products being developed for ASMEs in Kenya by I&M Bank, new products and payment schedules developed for ASMEs by BASPs in Malawi, and good exposure and networking opportunities for BASPs at AIMS organized events. Negative outcomes included the fact that the involvement of the guarantee slowed loan assessments and damaged Ecobank’s reputation, One BASP also felt their reputation was damaged by supporting unsuccessful loan applications. Banks also said that borrower awareness of donor involvement increased moral hazard risk. Sustainability: Regarding the extent to which banks have made internal changes to help increase access to credit for agribusinesses, Ecobank said that it did not make significant changes to their lending policies and procedures. By contrast, I&M Bank reported notable changes in resource allocation and in policies to embed and enable ASME lending. In terms of barriers to changing their service offering to better meet ASMEs' needs, Ecobank still feels that it requires a partner to mitigate the risk of continued ASME lending, without which they were not ready to make more substantial changes. Another issue identified was the potential mismatch between ASMEs’ cashflow cycles, which often follow agricultural seasons, and banks’ standard repayment schedules. Lessons and Recommendations Based on the findings of the evaluation, four key lessons, each with a set of recommendations, have been identified for consideration on future access to finance and loan guarantee programs: Lesson #1: The concept for the EALGF may need to be broadened to achieve the scale of lending, and subsequent growth in ASME production and trade, desired by the program. Recommendations # #1.1: Consider broader eligibility criteria for the businesses the guarantee scheme will target. #1.2: Consider including non-bank financial institutions in guarantee schemes targeting ASMEs. #1.3: Support banks to expand their digital offerings if this reduces transaction costs for ASMEs. #1.4: Build objectives for coordination with other access to finance initiatives in the target countries and sectors into guarantee programs to address the issue more holistically. #1.5: Build objectives for enhanced service offerings by Financial Institutions (FIs) to ASMEs alongside guarantee programs, to make the ASME market more attractive and financially sustainable for FIs. Lesson #2: The design of the EALGF could have been better suited to the target country context and could have enabled more efficient utilization by banks that did sign up. AIMS End Line Assessment 7 #2.1: Consider increasing the level of loan value coverage available when guaranteeing loans in less developed sectors and/or economies. #2.2: Enable partner banks to fully approve each loan to be covered by the facility, once their credit procedures have been approved. Adopt a spot-checking method for assurance. #2.3: Co-design the guarantee facility with financial institutions (FIs) and their associations in the target countries before entry. Lesson #3: The origination and contracting process for the EALGF took a lot of time and effort, this made it challenging for AIMS to sign up partner banks during the program #3.1: DFC should review and benchmark its internal origination and loan approval processes #3.2: Approach a longer list of banks from the start of the program to gauge interest widely, before focusing efforts on the most promising candidates. #3.3: Consider an extension to the EALGF guarantee period for I&M Bank now and consider using flexible guarantee periods from inception in future projects from inception. Lesson #4: There were significant capacity gaps in basic financial management in many ASMEs in the target countries #4.1: Consider including post-transaction support to ASMEs from BASPs, as well as loan application support, when working in less developed sectors and/or economies. AIMS End Line Assessment 8 1 Introduction & Research Questions The Agribusiness Investment for Market Stimulation (AIMS) program sought to increase agricultural trade through improving access to finance and markets for agribusiness small and medium enterprises (ASMEs 2 ) in Kenya, Tanzania, and Malawi. The AIMS program was implemented by Global Communities (GC) and funded by the United States Department of Agriculture (USDA). The AIMS program has four main objectives3 : Objective 1: Increased use of financial services--Leverage private and public investment: AIMS promotes the effective use of a $50 million loan guarantee fund provided by the United States Development Finance Corporation (DFC). Under this effort, Global Communities serves as a facilitator between commercial banks and DFC. The DFC guarantee is operationalized with two banks, Ecobank in Malawi and I&M Bank in Kenya, which have signed agreements with DFC. Objective 2: Increased use of financial services--Facilitate Agribusiness Lending: AIMS built the capacity of financial institutions, including banks and microfinance institutions, to increase lending to agribusiness SMEs through skill enhancement and developing new agribusiness loan products. Capacity building was done through trainings using the curriculum developed under the program and delivered by a third party and through the Eastern Africa Grain Council. AIMS also included banks in various forums to help build an understanding of the agriculture sector and build linkages to agriculture SMEs. Objective 3: Improved capacity of key organizations in the trade sector: AIMS worked with private business advisory service providers (BASPs) to build agribusiness SMEs' ability to access markets and financing and improve their business operations. AIMS also worked with apex organizations, including associations, large aggregators, and cooperative unions, to deliver services. Objective 4: Improved linkages between buyers and sellers: AIMS conducted buyer seller forums and linkages and supports access to ICT-based market information systems and marketing platforms to improve access to markets and market information. AIMS was originally designed as a five-year program, which began October 2014 and was scheduled to end in September 2019. AIMS received a two-year no-cost extension (NCE) from October 2019 to March 20224 to continue operations in Kenya and Malawi. Tanzania was not included in the NCE phase as the guarantee had not been operationalized with any banks there. The primary focus of the NCE phase was on Objective 1, to support the utilization of the DFC’s loan guarantee facility, known as the East African Loan Guarantee Facility (EALGF). Activities under objectives 2-4 were not part of the NCE period in full, however AIMS continued to implement elements of them on a small-scale to facilitate use of the EALGF. For example, approved business advisory service providers (BASPs) were incentivized to support ASMEs in preparing their loan 2 AIMS defines ASMEs as agricultural businesses that maintain annual revenues between $20,000 - $1million 3 The AIMS program contributed to the USDA’s Food for Progress’ Results Framework Strategic Objective 2 (FFPr SO2) – Expanded Trade of Agricultural Products (Domestic, Regional, and International. Specifically contributing to FFPr 2.2 – Increased Access to Markets to Sell Agricultural Products. 4 NCE implementation activities were scheduled to end in September 2021 but were extended to March 2022 after the commencement of the evaluation. AIMS End Line Assessment 9 applications to the EALGF partner banks and AIMS supported its partner banks in increasing outreach and marketing of products to the agricultural sector. The main activities conducted in the NCE phase in pursuit of Objective 1 were as follows: ➢ EALGF training for the AIMS team ➢ EALGF training for Ecobank & I&M Bank staff ➢ Sensitization forums with Ecobank & I&M Bank on their finance products for ASMEs, and with BASPs on their service offerings for ASMEs ➢ BASP consultative meetings with Ecobank & I&M Bank to understand the ASME loan eligibility requirements, in order to be able to provide effective support to ASME applications ➢ BASP support to ASMEs in the loan application process (co-funded & facilitated by AIMS) ➢ Events linking ASMEs to banks and other ASMEs for trade opportunities ➢ Loan assessment visits and follow up calls/meetings by AIMS in partnership with bank staff On behalf of the USDA, GC commissioned Kadale Consultants (Malawi) and KCIC Consulting (Kenya) to jointly conduct an end-line evaluation of the AIMS program, focusing on Objective 1, the utilization of the EALGF and its impact on agribusiness sector lending and growth in Kenya and Malawi.5 The specific objectives of the evaluation as per the Scope of Work (SoW) (see Annex 1: Scope of Work) were to: 1. Assess whether the activities under the EALGF achieved the intended results; 2. Evaluate the impact of the EALGF on increased lending to the agricultural sector; 3. Assess the likelihood that the EALGF results will be sustained; 4. Document key successes, best practices, challenges, and lessons learned; and 5. Provide specific recommendations for future access to finance programs. The evaluation adopted the Organization for Economic Cooperation and Development (OECD) Development Assistance Committee (DAC) Network on Development Evaluation criteria. This approach to evaluation poses questions that aim to assess the Relevance, Effectiveness, Efficiency, Impact and Sustainability6 of an intervention. Research questions for each element were proposed in the SoW and agreed in the inception period. These questions guided the focus of the evaluation. The complete list of research questions is given in Box 1 below. The evaluation's primary audience is GC, USDA, DFC, partner banks and BASPs. The ultimate purpose of this evaluation is to identify what lessons can be learned for future programming, based on evidence from the AIMS program experience. The evaluation will draw on the lessons learned to create a set of recommendations to support the design of future access to finance programs in the region. 5 Objectives 2-4 were assessed in a prior evaluation conducted in December 2019 6 The ‘Coherence’ criterion was not explicitly included in the SoW, but is in essence captured under the ‘Relevance’ criteria questions. A sixth element of ‘Lessons Learned and recommendations’ was included in the SoW. AIMS End Line Assessment 10 Box 1: Research Questions Relevance 1. Is the concept and design of AIMS EALGF suitable to the overall market situation and challenges faced by ASMEs in Kenya and Malawi? 2. To what extent were ASMEs’ needs for finance met? Efficiency 1. Has EALGF implementation been able to adhere to stated plans (i.e. work-plans, action-plans, results framework, and budget). If not, what internal challenges did the program face? 2. What were the main external challenges in the implementation of the EALGF? 3. What are the best/most efficient practices that contributed to program successes? (And how did the program adapt to insights from monitoring and evaluation conducted before and during implementation?) 4. How well is EALGF coordinating with other key actors (development actors, government, and private sector) working on similar or complementary programming? What could have been done to improve coordination? Effectiveness 1. What factors encouraged or discouraged banks from using the facility? 2. Were Technical Assistance (TA) / support interventions with the banks effective in increasing the use of the facility? If so, how? And if not, why not? What would have been more effective?' 3. How effective was the Technical Assistance provided via BASPs to ASMEs in increasing their readiness to receive loans? Impact 1. What was the impact of the EALGF-covered loans received by the ASMEs? (e.g. additional sales, employment, new business) 2. To what extent can changes in the outcomes of interest, including SME growth and trade, be attributed to the project? 3. What unintended consequences of the project were produced, positive or negative? Sustainability 1. To what extent have banks under the EALGF changed their internal resource allocation, adopted new lending policies or procedures or developed products that increase access to credit for agribusinesses? 2. What barriers exist that may prevent the full adoption of policies, procedures, and product lines to meet ASMEs' needs? Lessons Learned and Recommendations 1. What has worked especially well under EALGF and why? What has not worked well and why? 2. How have experiences and results differed between Kenya and Malawi? 3. What are preliminary lessons learned from implementing AIMS EALGF interventions targeting increasing financial inclusion to ASMEs? AIMS End Line Assessment 11 2 Methodology & Limitations This section gives a summary of the research methodology, broken down into sub-sections covering the overall methodological approach, the sample selection, and limitations. Further detail on the methodology can be found in Annex 2: Detailed Methodology Description. 2.1 Approach 2.1.1 Overview The evaluation used a participatory, mixed methods approach that aimed to provide a holistic view of program outcomes and causality. The research was designed to enable triangulation of information from multiple sources. The quantitative element of the evaluation was a survey of ASMEs that were involved in the AIMS program in Malawi and Kenya during the NCE phase. Qualitative methods included in-depth interviews with funding partners (USDA and DFC), implementing teams (GC and AIMS project staff), project participants (participating banks, ASMEs and BASPs) and other stakeholders (other banks engaged by AIMS7 ). The evaluation also reviewed all relevant program documentation and data to develop a detailed understanding of the program’s activities and outcomes. The documents that were reviewed are listed in Annex 3: Information Sources. Qualitative and quantitative research instruments were developed for each category of stakeholder in collaboration with the AIMS project team. Questions were designed to generate insights for each of the evaluations research questions (see Box 1). The research instruments are attached to this report as Annex 4: Evaluation Tools. Information sources were matched to each research question as detailed in Table 7 in Annex 2: Detailed Methodology Description. Furthermore, each question in the instruments, other than filter questions, were coded according to which research questions they were designed to inform. This was to ensure that only questions that added value were included in the data collection instruments as well as to improve the efficiency of data analysis and reporting. Data collection methods and the final sample sizes are summarized in Table 1 below. Table 1: Summary of Data Collection Data Source Collection Format Quant. Qual. Malawi # Kenya # Participating ASMEs, split into borrowers, applicants and non-applicants Survey  40 43 Participating ASMEs Focus groups  2 2 BASPs Interviews  2 4 Partner Banks In-depth interviews  1 1 7 After discussions with AIMS staff, it was established that there was minimal interaction with government stakeholders for the NCE phase activity, so they were not included in the data collection. AIMS End Line Assessment 12 Data Source Collection Format Quant. Qual. Malawi # Kenya # Non-partner Banks Interviews  0 0 Implementation Team - GC/AIMS Staff Guided discussions, desk research  Multiple Implementing Partner - DFC Interviews  2 Funding Partner - USDA Interview  1 The evaluation was conducted from August 2021 to February 2022. Following a kick-off meeting, the evaluation team prepared an inception report and presentation in collaboration with GC/AIMS project staff. This was followed by instrument design, including pre-testing, piloting, and training the data collection team for the ASME survey. The ASME survey instrument was programmed using Open Data Kit (ODK) software for tablet-based data collection. Other instruments were semi-structured question guides and captured qualitative data, with the researcher taking notes during the interviews and then writing these up into reports for subsequent analysis as soon after the interview as possible. Data collection took place in October and November 2021. Quantitative data was cleaned and analyzed using SPSS, with each question being analyzed by country at a minimum. Initial findings were presented to GC/AIMS in December 2021 leading to the drafting of the report. The report draft was reviewed by GC and USDA and feedback was incorporated in the final version. Figure 1 below is an overview of the evaluation workplan. Figure 1: High Level Summary of Project Workplan Inception: August 2021 •Desk research •Inception report and agree method and sample Design & Testing: September/October 2021 •Develop data collection instruments •Piloting instruments, train data collectors Data Colleciton: October/November 2021 •Qualitative interviews (in-person and by video call) •Quantitative survey (Phone interview and data recoreded using tablets) Analysis: Novermber/December 2021 •Quantitative data cleaning, analysis, graphs etc. •Qualitative analysis and triangulaton findings Reporting: Deceber 2021 - February 2022 •Present initial findings to GC/AIMS team for discussion •Draft and final versions of full report AIMS End Line Assessment 13 2.1.2 Outcome Harvesting Methodology The evaluation, particularly regarding the effectiveness, impact, and sustainability research questions, drew on the ‘Outcome Harvesting’ methodology. The key element of this approach is that it “reverses the logic of conventional monitoring and evaluation. Rather than tracking activities and outputs to see whether they are generating results as planned, harvesters (evaluators) first identify outcomes, whether planned or not, and then determine how the change agent contributed.”8 Key outcomes were identified from initial discussions with the ‘change agents’ i.e., the AIMS program team, and the ‘social actors’ i.e., the banks participating in the EALGF, the ASMEs seeking loans and the BASPs supporting them. For each outcome, a variety of data sources was used to assess the degree to which outcomes have occurred and the contribution of the change agent (AIMS) to that outcome. The key outcomes of interest for this evaluation are summarized in Figure 2 below. Figure 2: Summary of Key Program Outcomes Outcome harvesting seeks to understand the likely causes for these outcomes and also assess their sustainability. It also looks for any important unintended outcomes of program interventions and identifies their causes. From these insights, the key lessons for future access to finance programs in the region can be drawn, leading to the recommendations contained in this report. The aim of the approach is to provide evidence-based answers to the research questions, based on the information collected through the evaluation process. The report is not structured as a typical outcome harvest report, which lists the outcomes and provides descriptions for each. Rather, the outcomes are mainly discussed in the effectiveness section, the causes and AIMS contribution in the impact section and the sustainability of outcomes in the section of the same name. 2.2 Sampling 2.2.1 ASME Survey Sample Design The sample frame was 185 ASMEs (102 in Kenya and 83 in Malawi) that had been engaged by AIMS during the NCE period. This included ASMEs that had applied for loans covered by the EALGF and those that participated in the NCE phase activities in some way, but had not gone on 8 Taken from Outcome Harvesting, by Ricardo Wilson-Grau and Heather Britt, published May 2012. For further detail on this approach, this publication is available at https://usaidlearninglab.org/library/outcome￾harvesting-complexity-aware-monitoring-approach Changes in bank attitudes, capacity & processes for ASME lending Sign up and usage of the EALGF by two banks ASME readiness to apply for loans with BASP support Increased credit to ASMEs for investment ASME growth and increased trade AIMS End Line Assessment 14 to apply for an EALGF-covered loan9 . Based on the sample frame, the minimum sample size for each country was estimated using the modified Cochran formula, with a 90% Confidence Level and using a 10% Margin of Error. This resulted in a minimum ASME survey sample for each country as shown in Table 2 below. The target sample size was increased above the minimum by three respondents for each country, to account for potential non-usable responses: Table 2: ASME Survey Target Sample Sizes ASMEs Kenya Malawi Sample Frame 102 83 Minimum Sample 42 38 Target Sample (+3) 45 41 Within each country sample frame, there were different groups of interest based on their borrowing status under the EALGF, namely: ➢ Borrowers – ASMEs that received a bank loan that was covered under the EALGF ➢ Applicants – ASMEs that applied for a bank loan that would have been covered under the EALGF, but the application was pending, rejected, deferred, or withdrawn. ➢ Non-Applicants – ASMEs that were engaged by AIMS during the NCE, but had not made an application for a bank loan that would be covered by the EALGF The ASME sample can also be grouped by business type, using the categories adopted by AIMS throughout the program to date, namely: ➢ Agro-processor ➢ Cooperative (producers) ➢ Exporter ➢ Input Supplier ➢ Wholesaler/Retailer Respondents for each country sample were selected purposively. This was to ensure that the data gathered from the sample was able to provide the most useful information in relation to the evaluation’ research objectives. It was agreed with GC that the sample should include all borrowers from both countries, as these ASMEs would be able to provide information on their direct experience of receiving a loan under the EALGF and the impact it had on their business. Secondly, the sample should also include both applicants and non-applicants. Applicants were able to provide information on their experience of the application process and the reasons why the application did not ultimately lead to a loan disbursement, so were prioritized over non￾applicants. The final consideration was to include a mixture of all business types in the sample and borrowing status sub-samples as far as possible, and ideally in proportion to the respective 9 The engagement by AIMS with ASMEs that did not apply for loans was primarily facilitating attendance at sensitization forums with Ecobank & I&M Bank on their agribusiness finance products, and with BASPs on their service offerings for ASMEs. AIMS End Line Assessment 15 population sizes for each group. Finally, where more than one ASME fitted the desired criteria for the sub-sample, a random selection was made. It was agreed at inception that the ASMEs in scope for the end-line evaluation were highly unlikely to have been included in the baseline assessment, and that the two groups would not be statistically comparable. Thus, it was agreed that the sampling would not purposively attempt to include respondents who had been interviewed at the baseline. 2.2.2 ASME Survey Target Sample An up-to-date database of ASMEs, their borrowing status and business type was provided by the AIMS team. This information led to a further division of the sample into those that applied, but later dropped out, and those that applied and saw their application through to a decision point (rejection or deferral). For Malawi, there were 13 ASMEs that had borrowed and 21 that had applied, of which 4 later dropped out, giving 34 from these two categories. This meant that to reach the target sample size, 8 ASMEs who did not apply would also be included in the sample, with at least one respondent from each of the five business types targeted for inclusion. For Kenya, there were 4 ASMEs that borrowed and 43 applicants, of which 15 later dropped out, giving 47 from these two categories. As this was larger than the overall target sample size, it was agreed to include a sample of 4/15 from the applicants that had dropped out, leaving room for a sample of 9 ASMEs that did not apply, to reach the target sample size 45. The ‘dropped out’ applicants and non-applicants target sub-sample sizes were set to cover the maximum possible number of business types. Table 3 and Table 4 below shows the stratified ASME sample frame and target samples respectively, for each country: Table 3: ASME Sample Frame for Kenya and Malawi ASME Population Kenya Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 1 2 0 1 0 4 Applied 7 6 6 4 5 28 Applied but dropped out 1 3 4 7 0 15 Did not apply 5 10 5 5 30 55 Total 14 21 15 17 35 102 ASME Population Malawi Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 4 0 1 5 3 13 Applied 5 2 4 4 2 17 AIMS End Line Assessment 16 Applied but dropped out 2 0 1 1 0 4 Did not apply 11 1 6 17 14 49 Total 22 3 12 27 19 83 Table 4: ASME Target Samples for Kenya and Malawi ASME Target Samples Kenya Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 1 2 0 1 0 4 Applied 7 6 8 4 5 30 Applied but dropped out 1 1 1 1 0 4 Did not apply 1 1 1 1 3 7 Total 10 10 10 7 8 45 ASME Target Samples Malawi Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 4 0 1 5 3 13 Applied 5 2 4 4 2 17 Applied but dropped out 2 0 1 1 0 4 Did not apply 2 1 1 2 1 7 Total 13 3 7 12 6 41 2.2.3 ASME Survey Actual Sample 16 of the originally selected sample (11 in Kenya, 5 in Malawi) did not complete the survey and had to be either substituted or, if an appropriate substitution was not available, dropped from the sample. Reasons for non-completion included unwilling to participate, no response to calls and messages, and phone numbers not connecting. In these cases, a substitution took place. The substitution protocol is explained in Annex 2: Detailed Methodology Description. Towards the end of the data collection, appropriate substitutes were exhausted, so it was agreed with GC that the final sample size could be lower than the original target, provided it was still above the minimum, and this was the case. The final sample is summarized in Table 5 below and the list of ASMEs surveyed is in Annex 3: Information Sources Table 5: ASME Actual Samples, Kenya and Malawi ASME Actual Samples Kenya Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 1 2 0 0 0 3 AIMS End Line Assessment 17 Applied 6 3 3 2 4 18 Applied but dropped out 1 0 0 1 0 2 Did not apply 3 4 5 4 4 20 Total 12 9 7 7 8 43 ASME Actual Samples Malawi Agro￾Processor Exporter Input Supplier Wholesaler / Retailer Cooperative Total Borrower 4 0 1 3 3 11 Applied 5 2 4 4 2 17 Applied but dropped out 2 0 1 1 0 4 Did not apply 2 1 2 2 1 8 Total 13 3 8 10 6 40 2.2.4 Samples for Qualitative Data Collection For the qualitative data collection, the evaluation aimed to include: • Both Funding Partners (USDA and DFC) • Both EALGF Partner Banks (Ecobank in Malawi and I&M Bank in Kenya) • At least four other banks that were engaged by AIMS for EALGF participation that did not sign up (referred to as ‘Non-partner banks’), from a sample frame of seven (four in Kenya and three in Malawi). • All six BASPs (four in Kenya, two in Malawi) engaged by AIMS in the NCE period, noting that some have since ceased to participate. • Four ASME Focus Groups, one each for borrowers and for applicants for Malawi and Kenya respectively. Groups would consist of up to 12 ASMEs for that country from among those that had participated in the ASME survey. All qualitative interviews and focus groups were completed as planned, with the exception of the non-partner bank interviews. The AIMS teams recommended a shortlist of non-partner banks to interview who had progressed to some extent in the EALGF origination process. For some banks, the staff who had been involved at the time (as far back as 2015) had since moved on. For another, the bank had been taken over and no longer existed as a separate entity. For the remainder, the staff that were contacted, both by AIMS and the consultant, were either unwilling to provide an interview or did not respond to multiple attempts at contact. In the end it was agreed to rely on the AIMS and DFC staff who had been involved in the origination process for insights as to why other banks did not participate in the EALGF. 2.3 Limitations Covid-19 limited in-person interviewing. The risk posed by Covid-19 meant the evaluation team limited the number of in-person interviews, with no in-person interviews or focus groups with AIMS End Line Assessment 18 ASMEs. This had the potential to limit the quality of interactions with participants and the depth of insight obtained. For the ASME survey, this concern was mitigated by carefully designing the survey instrument to gather insightful quantitative data using well-structured questions and the use of virtual Focus Groups Discussions (FGDs) to explore more nuanced topics in detail. Reluctance of ASMEs to participate in survey and focus groups. The team anticipated that there would be reluctance particularly among ASMEs who had invested considerable time in applying for a loan that had been rejected or deferred. The team sought to mitigate this reluctance by designing a short, focused instrument that did not demand a significant amount of time, yet still captured valuable insights. Despite this, 16 of the originally selected sample of ASMEs were not willing/able to participate, and this necessitated a high number of substitutions, some of which had to be drawn from non-applicants, once all borrowers and applicants had been approached. The result was that the sample was skewed more toward non-applicants (a larger group) than borrowers or applicants than had been designed. The participation of some members of the virtual focus groups was also limited by network quality issues. Reluctance or inability of non-partner banks to give an interview. There was reluctance to participate by the non-partner banks who had engaged with AIMS in order to participate in the EALGF but did not ultimately do so. The team sought to mitigate this in advance by setting out to interview all shortlisted banks rather than just a sample, knowing some may not participate. In the event, it was not possible to interview any of the non-partner banks suggested by AIMS. Sample selection bias. The ASME survey sample was purposively selected, and so selection bias is inherent in the sample design. The evaluators have considered the potential influence of selection bias on the results on a topic-by-topic basis. Recall bias. For some respondents, there was a time-lag of up to two years between the research and the events in question. For example, Ecobank started making loans covered by the EALGF in 2018. This is more likely to result in the omission of certain relevant information, rather than a systematic skewing of the responses to any particular question in one direction or another. This was mitigated through careful selection of research questions. No single source of relevant information within organizations. Banks, and some ASMEs are multi-layered and decision-making ability and information on the business is often not held by one or two individuals, which might have limited the level of detail available from some interviews. This was mitigated by the AIMS team having good knowledge of the organizations in most cases and directing the evaluators to the most appropriate person(s) for the interviews. Linked to this point, some ASMEs were cooperatives, and information for cooperatives that are led by multiple officers can be difficult to obtain from one respondent. This was mitigated by having multiple officers from the cooperative present on speaker phone, so that input could be provided by the most qualified. Small ASME survey sub-sample sizes: Though the ASME survey did cover a high proportion of the sample frame, the relatively small number of ASMEs that participated in the NCE phase, and thus a small sample, resulted in smaller sub-samples for certain questions in the survey. Some questions were only relevant to sub-sets of the sample, for example, asking an applicant who was declined a loan, why they were declined. There is no mitigation for this, however, only results that show a clear pattern with a relatively high number of responses are included in this report and generally should be treated as indicative rather than as evidence of statistically significant variation. AIMS End Line Assessment 19 Limited Comparison with Baseline Assessment: It was agreed during the inception period that the baseline sample was designed to provide a picture of the countries’ ASME population as a whole, whereas the end-line sample was only to be drawn from the limited number of ASMEs engaged by AIMS during the NCE period. Across all project participants (Banks, ASMEs, BASPs), there was very little overlap between the organizations interviewed at baseline and the organizations that have benefited from the AIMS program activities during the NCE phase. As a result, the end-line sample was not directly comparable with the baseline sample. Furthermore, the focus of the research questions for the baseline was a needs assessment of the finance needs of ASMEs, whereas the focus of this end-line assessment is explicitly on the utilization of the EALGF. So, there was also very little overlap in the questions between the baseline and end-line survey, meaning time-series analysis of the results was not possible. AIMS End Line Assessment 20 3 Evaluation Findings 3.1 Program Indicator Targets and Results Through its own internal monitoring activities, the AIMS program set targets and collected data for eleven (11) performance indicators10 for the NCE phase activities. The indicators, targets and results (for the NCE phase only) are summarized in Table 6 below. The indicators give a quantitative picture of the program’s performance against its targets and provide a helpful context to the qualitative research findings that are the focus of this evaluation. Table 6: Program Indicator Targets and Results (NCE Phase) Indicator Target (NCE Phase) Achieved (NCE Phase) % Achieved (NCE Phase) Value of Loans provided as a result of USDA assistance ($) 7,678,077 1,530,424 19.9% Number of loans disbursed as a result of USDA assistance 55 18 32.7% Value of new public and private sector investment leveraged as a result of USDA assistance ($) 7,678,077 1,530,424 19.9% Value of agricultural and rural loans ($) 4,115,000 752,541 18.3% Number of individuals receiving financial services as a result of USDA assistance 1,055 764 72.4% Percentage of non-performing Loans advanced under the EALGF assistance 2.0% 1.3% 65.0% Total number of individuals benefiting directly as a result of USDA 1,590 975 61.3% Total number of individuals benefiting indirectly as a result of USDA 923 8,089 876.4% Number of SMEs, including farmers, receiving Business Development Services (BDS) from USG-assisted sources 47 12 25.5% Number of jobs attributed to USDA assistance 97 57 58.8% 10 Indicators include USDA Food Assistance Division (FAD) Standard Indicators. AIMS End Line Assessment 21 The number and value of loans disbursed were below the target, and this was the primary driver of the results for the dependent indicators also being below their targets11 , with the exception of the ‘Total number of individuals benefiting indirectly as a result of USDA’ 12 . The reasons for the below-target uptake and utilization of the EALGF are explored from here on. 3.2 ASME Survey Summary This section provides a brief overview of the sample of ASMEs surveyed and some pertinent insights into their attitudes and experience to borrowing in general and from EALGF partner banks specifically. This information provides additional context and evidence to the findings reported under each research question in the subsequent sections. 3.2.1 ASME Demographics Figure 3: ASME Demographics Among the sample, ASMEs in Kenya were most likely to be limited companies (60%) and ASMEs in Malawi were most likely to be sole proprietors (48%). Businesses were most likely to have a combination of male and female owners, at 67% in Kenya and 45% in Malawi. Only 5% of the ASME sample in Kenya and 10% in Malawi had female-only ownership. This meant that the valid 11 Other than ‘Percentage of non-performing Loans advanced under the EALGF assistance’, a result of less than 100% means that the program results fell short of their intended target. 12 This result was far higher than the target, primarily due to loans being given to a larger number of cooperatives than was expected, which had many indirect beneficiaries • 60% were limited companies, 16% cooperatives, 12% sole proprietros • 67% had mixed male/female (M/F) ownership, 5% female • Median number of permanent staff was 8 (4M, 4F) • Median number of casual staff was 6 (4M, 2F)* •Businesses had been operating for a mean average of 10.6 years • Common Value Chains: Dairy (37%), Vegetables (23%), Maize (20%), Fruits (16%) • 6 Borrowers (14%), 17 Applicants (39.5%) and 20 Non-Applicants (46.5%) Kenya • 48% were sole proprietors, 25% cooperatives, 18% limited companies • 45% had mixed M/F ownership, 10% female • Median number of permanent staff was 7 (4M, 3F) • Median number of casual staff was 13 (8M, 5F)* •Businesses had been operating for a mean average 9.9 yrs • Common Value Chains: Maize (57%), Soybean (48%), Groundnuts (35%), • 11 Borrowers (27.5%), 21 Applicants (52.5%) and 8 Non-Applicants (20%) Malawi AIMS End Line Assessment 22 number of responses (valid N) for the female owned ASME sample sub-set was too small for meaningful analysis by sex. The size, in terms of number of employees, and level of experience of the ASMEs in the sample was similar for Malawi and Kenya. There was a notable difference in the value chains in which the ASMEs were involved in, with maize being the only value chain among the most common in both. Among the 28 ASMEs surveyed (20 in Kenya and 8 in Malawi) who had not yet applied for a loan under the EALGF, 73% and 88% in Kenya and Malawi respectively said it was because they were not actively looking to borrow at the time. 3.2.2 ASME Financial Activity Figure 4: ASME Financial Activity In both countries, most ASMEs in the sample had borrowed in the last three years, and 35% and 50% in Kenya and Malawi respectively had borrowed at least once from a bank in that time. The mean value of loans received was three times as high in Kenya, although the median was almost identical to that of Malawi. Loan applicants in Malawi were roughly twice as likely to have got less than they applied for as applicants in Kenya. Though the samples and questions were not directly comparable with those surveyed at baseline, some comparisons with the endline ASMEs are possible and interesting. At the baseline (2015), for Kenya, 56% of cooperatives and 70% of other forms of ASME had borrowed before, compared to 63% for ASMEs (which included cooperatives13) at end-line who had borrowed in the last three years. For Malawi the baseline was 33% for cooperatives and 49% for ASMEs compared to 63% 13 The baseline study reported results for cooperatives separately to other legal forms of ASME. For the endline study, cooperatives were included with all other legal forms of ASME. • 63% had borrowed in last 3 years, including 55% of non-applicants • Of those that had borrowed, 56% had borrowed from banks in the last 3 years (35% of the total sample) •Value of loans taken in last 3 years: Median=$25k, Mean=$180k (n=64) •Applicants got less than they applied for in 16% of cases • 51% had new loan applications ongoing, (including 45% of non-applicants)… •…of which 32% were trying to borrow from I&M Bank, 55% from other banks Kenya • 63% had borrowed in last 3 years, including 25% of non-applicants • Of those that had borrowed, 80% had borrowed from banks in the last 3 years (50% of the total sample) •Value of loans taken in last 3 years: Median=$26k, Mean=$64k (n=44) •Applicants got less than they applied for in 33% of cases • 33% had new loan applications ongoing, (including 13% of non-applicants)… •…of which 31% were trying to borrow from Ecobank, 39% from other banks Malawi AIMS End Line Assessment 23 for ASMEs and cooperatives at end-line. At baseline, the average (mean) loan amount of the last loan taken for ASMEs in Kenya was $97,408, compared to a mean of roughly $180,000 at end line for all loans taken in the last three years. For Malawi the baseline figure was $51,360 and the end-line mean was roughly $64,000. 3.2.3 ASME Loan Application Experience (Borrowers and Applicants Only) Figure 5: ASME Loan Application Experience In both countries, getting a response to the loan application for loans under the EALGF took longer than what ASMEs said they were generally used to in other credit applications. The application process reportedly took particularly long with Ecobank in Malawi and was felt to be more demanding than other loans by just over half of ASMEs that had applied. This corresponded with a higher dropout rate of applicants in Malawi, mostly due to the length of the process. It is interesting to note that, even though I&M Bank said that the EALGF was only able to cover term • Of 23 applicants to I&M under the EALGF, 6 were successful (26%), 6 declined, 10 were still pending and 1 withdrew • 57% heard about the product from AIMS staff, (13% social media & fellow SMEs) • Most common intedned use of funds working capital for stock (48%) • 38% had a response within 8 weeks, vs 69% across all loans from all sources over past 3 years •I&M process was not clearly more or less demanding than other loans applied for • 6 applicants in the survey had been declined for 5 different reasons, with no clear pattern (1 applicant dropped out) • 88% of applicants would consider applying for a loan to I&M Bank again • Reason for not applying again was lack of communication from the bank (n=2) Kenya • Of 32 applicants to Ecobank under the EALGF: 11 were successful (34%), 12 declined, 2 were pending and 7 withdrew • 41% heard about the product from AIMS staff, (25% newspaper & 19% the bank) • Most common intedned use of funds was capital for expansion (50%) • 19% had a response within 8 weeks, vs 61% across all loans from all sources over the past 3 years, and… •…for 50% a response took over 4 months • 56% said Ecobank process was more demanding than other loans applied for • 12 applicants in the survey had been declined for 10 different reasons, with no clear pattern • 5 applicants dropped out because the process took too long. 1 said they got a loan from elsewhere and 1 said that the bank was ‘not supportive’ • 62% of applicants would consider applying for a loan to Ecobank again • Of the 8 that would not apply again, 7 said the process had demotivated them Malawi AIMS End Line Assessment 24 loans and not working capital facilities (I&M's definition for a working capital facility is a short-term facility of 1 year or less), the most common use of the loans in Kenya was still ‘working capital for stock/supplies/raw materials’. This response was common among ASMEs in Malawi as well, with 31% saying this was the intended use of funds. 3.2.4 ASME EALGF Borrowing Experience (Borrowers Only) Figure 6: ASME Borrowing Experience ASMEs who had borrowed under the EALGF from I&M Bank in Kenya were more likely to have had a positive experience than those borrowing from Ecobank in Malawi. This was ultimately indicated by all borrowers saying they would want to borrow from I&M Bank again while only half wanted to borrow from Ecobank again. The primary reason given for the lower result in Malawi was that the process of borrowing from Ecobank under the EALGF takes too long. Further explanations are explored in more detail in section 3.3 on Research Question Findings. • Reason for borrowers using I&M included easy process, low interest, BASP recommendation & flexible loan terms • On average, 81% of borrowers agreed with the series of positive statements about the loan they got from I&M Bank • 100% would borrow from I&M again • 75% of borrowers said bank staff support was the most helpful factor in their successful application Kenya • 54% of borrowers used Ecobank because it was the only loan available to them and 45% said it was because it was recommended by AIMS staff • On average, 52% of borrowers agreed with the series of positive statements about the loan they got from Ecobank • 55% of borrowers would borrow from Ecobank again. • Of those that would not, 80% said the process takes too long Malawi AIMS End Line Assessment 25 3.2.5 ASME Experience of AIMs Support Figure 7 below gives an overview of the different forms of support that ASMEs in the survey said they had received under the AIMS program. Figure 7: Support received by ASMEs in AIMS Program NCE Phase Figure 8: ASME Experience of AIMS Support In general, ASMEs surveyed in Malawi reported receiving more support from the AIMS program than those in Kenya. This is likely to be because the EALGF had been operational for longer in Malawi. In both countries, the support received was generally deemed ‘useful’ in respect to improving their access to finance. More detail on the support received by ASMEs, from BASPs in particular, is covered in section 3.3.3 on ‘Effectiveness’ below. • Roughly 75% said each form of support received from those selected in the list above was useful – aside from ‘links to banks’ which 35% ranked as useful • 44% of applicants said they got support from AIMS in the application process… •…100% of those that did said it was useful Kenya • Roughly 75% said each form of support received from those selected in the list above was useful • 63% of applicants said they got support from AIMS in the application process… •…70% of those that did found it useful Malawi AIMS End Line Assessment 26 3.3 Research Question Findings Drawing on all information sources consulted, this section presents the main findings of the evaluation as responses to each of the research questions in turn. 3.3.1 Relevance ➢ Is the concept and design of AIMS EALGF suitable to the overall market situation and challenges faced by ASMEs in Kenya and Malawi? The question above is best answered by considering the answers to three related sub-questions, as outlined in Figure 9 below. Figure 9: AIMS Concept and Design (Relevance) Sub-Questions (A) Access to (formal) finance was identified as a constraint to growth by some ASMEs in both countries. In the FGDs with ASME borrowers and applicants, there was general agreement that access to finance was, at times, a constraint on their growth. ASMEs in Malawi for example said that their businesses sometimes needed larger amounts of capital for certain purposes, such as a large stock purchase or high-value equipment purchase, that only banks (as opposed to say Microfinance Institutions (MFIs) or Savings and Credit Cooperative Organizations (SACCOs) are able to provide. (A) Access to (formal) finance was a less relevant constraint for smaller or less well established ASMEs, in Malawi in particular. BASPs in Malawi said many ASMEs were not interested in applying for loans. To quote one “There are basic activities businesses need to get the structure in place before they even come to the point of needing finance… (including) some basic compliance like tax registration.” When asked what most constrains ASMEs’ growth the BASP said “They would say it is access to finance, but it’s not really, it’s usually the underlying issues with their business.” From the ASME survey, 68% and 83% of respondents in Kenya and (A) Is access to (formal) finance a major barrier to ASME growth? •Overall Concept (B) Did the EALGF improve the availabilty of finance to ASMEs? •Supply Side (C) Are ASMEs able to access and make use of (formal) finance for growth? •Demand Side AIMS End Line Assessment 27 Malawi respectively said they were not currently applying for and had no plans to apply for a loan from a bank at the time of the survey. (B) The ASME collateral gap perceived by banks was addressed by the EALGF. Bank respondents in both countries confirmed that lack of collateral is a major constraint for them to lend to ASMEs, so the concept of a guarantee facility targeted to the ASME sector is relevant. (B) EALGF addressed the issue of banks’ credit risk for ASME, but not other constraints like profitability. AIMS and GC project staff said that lending to the agricultural sector in Africa is perceived as higher risk for banks, a view confirmed by the banks themselves. So, the existence of a guarantee facility targeted to this sector is well designed in this respect. However, addressing the risk element, while necessary, is unlikely to be sufficient to stimulate ASME lending at scale. Lending to ASMEs also needs to be profitable for both parties. The EALGF added cost in terms of compliance, so likely eroded the profitability of ASME lending for the partner banks. (B) Elements of the design of the EALGF were less relevant for the operational context for banks in Malawi and Kenya. The design of the EALGF made it challenging for banks to sign up and to get loans approved for coverage by the EALGF. Banks felt that DFC’s processes and requirements for signing up banks to the facility were onerous. Quotes related to this from across the mid- and end-line assessment included: ‘unnecessarily complicated’, ‘tedious’, ‘too detailed’, ‘time consuming’, ‘asking for too much confidential information’, ‘fatiguing’, ‘killed the initial interest that was there’, and ‘the assessment (by the DFC) was not commensurate with the risk or facility size’. One example given was the need to provide information not just on Ecobank Malawi’s shareholders, but also on the holding company and partner company shareholders. I&M Bank said that aligning the scheme with the participating bank’s policies would be beneficial as it would eliminate the need for the bank to develop new systems to fit the requirements of the EALGF scheme. (B) Loan application requirements could have been better suited to the ASME context in both countries, and Malawi in particular. Partner banks stated that the DFC needed to be more flexible in terms of matching its requirements to the context. Specific issues mentioned included by the Banks were: - The need for three years of historical financial reporting documents - The limitation that loans could only be made in local currency - Not permitting overdraft or working capital facilities, which the banks felt were low risk and in high demand among ASMEs - Level of detail required on application submissions (C) There are significant gaps in ASMEs’ financial management capabilities that limited how effective access to formal finance was in promoting ASME growth. Ecobank said that they feel they have sufficient access to capital and human resources to provide the level of credit required by the ASME market, and the presence of the guarantee helped mitigate the higher risk of doing so. Their view was that the issues constraining ASME growth were predominantly on the demand side, with ASME owners/managers lacking the capacity to manage their businesses in a way that enables them to apply for and successfully service loans. I&M also commented on the generally lower levels of education among ASME owners and their need for capacity building in financial management. It was also the view of the AIMS team and BASPs that ASMEs commonly lacked the capacity to manage and properly document their finances. Examples cited by AIMS End Line Assessment 28 stakeholder included the absence of basic financial management tools like management accounts, calculating gross and net profit margins on product or business units, expense control reports, balance sheets and annual profit and loss statements. Comparing feedback from various stakeholders suggests that the general level of finance and business acumen was higher in Kenya than Malawi. With better financial management, ASMEs in Kenya are better equipped to make use of finance as other aspects of their business were likely to be in better shape, compared to Malawi where access to finance is one of many gaps and not always the most critical to growth. ➢ To what extent were ASMEs’ needs for finance met? The guarantee successfully overcame the collateral hurdle for some ASMEs to access finance. Both banks said that the main function of the guarantee facility from their perspective was to lower the value of collateral that the ASMEs needed to provide to access a loan. Both Ecobank and I&M Bank said that the EALGF had resulted in loans being approved to ASMEs that would otherwise have been declined due to lack of collateral. They also said that the program had encouraged them to target the ASME market more actively and their lending to this sector has increased. Lack of collateral often remained an issue even with the guarantee in place. Banks, BASPs and ASME all acknowledged that many ASMEs were still not able to provide enough collateral to access loans, even though the guarantee reduced the requirement. The EALGF covered up to 70% of the loan value. Ecobank said it was often challenging to find ASMEs able to cover with collateral the balancing 50% of the loan value to make up the 120% coverage they required. By comparison, I&M Bank said they required 100% loan value coverage, but also noted that collateral could be a limiting factor for ASME loans. The existence of the guarantee seems to have helped certain types of ASME to access loans more than others, for example, commodity traders whose business models more readily lent themselves to the ownership of acceptable collateral. ASME needs for specific working capital and overdraft facilities were not covered by EALGF. Both banks felt the EALGF was too restrictive, particularly by not allowing overdraft products or working capital facilities to be covered. I&M Bank in particular felt that there was a substantial need for working capital facilities that were not eligible for coverage under the terms of the EALGF, but that did in some cases provide to the same ASMEs outside of the facility. The need for working capital facilities for ASMEs was mentioned by all the BASPs interviewed in Kenya as well, particularly as they need to manage uneven agricultural cashflows. The EALGF has not been able to benefit ASMEs that predominantly operate in informal markets. Ecobank expressed that many ASMEs in Malawi operate in mostly informal markets, in the sense that they did not have written agreements/contracts with suppliers and customers. The same was true, though to a lesser extent, in Kenya. Ecobank commented that the lack of formal contracts and secure markets exposes them to greater risk of default as they have no visibility (or control) over borrowers’ future income. The AIMS program and EALGF was very much targeted at enabling ASMEs to access formal sources of finance, however there is a disconnect here as formal finance providers find it challenging to work with businesses that engage in informal markets due to the banks’ internal controls and external regulatory constraints. Formal financial products are unlikely to have been able to meet ASMEs’ ‘opportunistic’ access to finance requitements. The ASME survey and FGDs revealed that many felt that the application and disbursement process was too slow to meet their business needs and AIMS End Line Assessment 29 opportunities when they arose, particularly in Malawi. Some said that between the time of applying and getting a decision, the need for the money had passed so they withdrew their application. 3.3.2 Efficiency ➢ Has EALGF implementation been able to adhere to stated plans (i.e. work-plans, action-plans, results framework, and budget). If not, what internal challenges did the program face? As noted in section 3.1 above, the overall quantitative results for the program fell short of the stated targets. Plans were impacted by a range of both internal and external challenges that prevented activities taking place or intended impacts being realized. In terms of internal challenges, the main challenge faced was signing up banks to the EALGF in the first place, and there were further challenges in terms of its utilization, with DFC’s requirements for participation seen as onerous by target banks. The main external factor that affected implementation was the Covid-19 pandemic. More detail on these points, as well as other challenges the program faced, are covered below. Communication & coordination between the partner banks, AIMS and DFC was a major internal challenge. This not only affected the speed with which banks were able to sign up to the EALGF, but also how quickly they could get loans approved for coverage. This resulted in lending amounts being well below the initial targets. Linked to this, banks and AIMS staff commented on the changing priorities of the DFC and inflexibility on the due diligence requirements. Interviews with the DFC suggest that their internal resources may have been stretched too thinly and staff were not able to respond to the issues that needed resolving in a timely manner. Dissatisfaction with the speed of the application process was clear in Malawi in the ASME survey and the FGDs. Matching the tenor of guarantee availability with the tenor of loans was challenging for I&M bank as they signed up late in the project period. The protracted process of signing up to the EALGF also impacted I&M with regards to the available tenor of the guarantee, since by the time they had signed up the availability period was short. This made matching the tenor of the loan to the tenor of the guarantee period impossible in some cases. ➢ What were the main external challenges in the implementation of the EALGF? The Covid-19 pandemic had wide-ranging and predominantly negative impacts on project implementation. The effect of Covid-19 included reduced demand for ASME produce, border closures and transport difficulties impacting profitability, leading to reduced appetite for credit risk among ASMEs. This may also have reduced ASMEs’ ability to service existing loans and so capacity to borrow more. Many planned in person product sensitization forums between banks, BASPs and ASME had to be cancelled, as did in-person visits to banks and ASMEs for loan assessments. These factors are likely to have contributed to reduced demand among ASMEs and the delays in the assessment process. BASPs said that their planned engagements with ASMEs were paused due to Covid-19. The economic uncertainty caused by Covid-19 also affected banks’ risk/lending appetite. Lending, in Malawi in particular, was hampered by a weak legal system, low trust, high default rates and donor dependency. Ecobank said there was a lack of trust between banks and SMEs, with a credit culture influenced by high defaults and little prospect for legal redress. Ecobank’s perception was that the Courts in Malawi had often sided with defaulters, they also AIMS End Line Assessment 30 noted the difficulty and cost of accessing legal recourse and shortfalls in the legal framework. The issue of donor dependency impacted ASMEs’ desire for private credit and also ASMEs’ willingness to pay for BASP support, as this was often provided for free by NGOs and other projects in Malawi. In Kenya, the issue of trust and perception also came in the ASME FGDs. ASMEs commented that there is a general perception that taking loans from banks carries a high(er) risk of losing property and assets as a result of defaulting on payments. Similar sentiments were also heard from ASMEs in Malawi. The markets for formal credit among ASMEs are thin, particularly in Malawi. A limited number of ASMEs with the ability to meet residual collateral requirements, keep accurate business records, manage business resources in a structured manner, meet commitments, adopt effective governance structures that protect wider stakeholders’ (like banks) interests. This was an issue in both countries, particularly in Malawi. This limited the number and size of loans that Ecobank could offer and so limited EALGF utilization. Ecobank expected to exhaust the $5m guarantee limit in 6-12 months, however the loans were much smaller than expected. They expected to be making loans of up to $400k, though the largest they made was $135k. The total amount loaned under the EALGF was $662,998 across 13 loans. Competition from other similar guarantee schemes available limited usage of the EALGF. AIMS project staff noted that guarantee schemes offered by other Development Finance Institutions (DFIs) presented a challenge to EALGF implementation. The banks had a choice between several different guarantee scheme offerings, and would rationally choose to devote their limited resources to participating in the ones that were most appealing to them. The process of signing up to the EALGF with DFC was relatively challenging and so banks were more likely to give their attention to other schemes, unless there were strategic or idiosyncratic reasons why they chose to participate in the EAGLF. For example, according to AIMS, some banks who were approached but did not participate in the EALGF, said that signing up to USAID’s DCA guarantee only took a few months, compared to nearly 2 years for the EALGF. The Development Credit Authority (DCA) also offered an umbrella guarantee which, according to AIMS staff, some banks preferred as it did not require them to share loan applicant details with guarantor, which has legal ramifications. Ecobank also had another guarantee in place from the European Investment Bank (EIB) which covered up to 80% of loan value (compared to the EALGF that only covered typically 50-60%), and they said the sign-up process took only a few months. ➢ What are the best/most efficient practices that contributed to program successes? And how did the program adapt to insights from monitoring and evaluation conducted before and during implementation? Capacity building support and bank staff training in agri-lending was a success. Both I&M and Ecobank reported that the capacity building support they received from AIMS for their agribusiness lending teams was well targeted and delivered. They both said it had been very helpful in being able to assess loan applications and contributed to them able to provide more loans to ASMEs. Banks highly valued the hands-on nature of AIMS support & links to new clients. Both banks very much appreciated having multiple contacts in-country that they could go to with EALGF issues. Both said the process would have been far more challenging if they had to deal directly with the DFC in the US. This covered signing up to EALGF and its utilization. Banks also appreciated the practical support from AIMS staff in sourcing potential ASME clients. Ecobank in AIMS End Line Assessment 31 Malawi felt that the public call for applications that AIMS supported and the screened was very effective in particular. Focusing BASP support on loan application documents was an effective change. The baseline identified a lack of access among ASMEs to financial management/record keeping support from BASP, 27% and 17% of the overall (all country) baseline sample for respectively – suggesting the focus of the NCE on this element was well-designed. The need for support in documentation preparation for ASMEs was also identified by banks in the mid-term assessment. In the end-line assessment, all BASPs in Kenya and Malawi said that this change in program focus was positive in terms of improving ASME access to finance. ➢ How well is EALGF coordinating with other key actors (development actors, government, and private sector) working on similar or complementary programming? What could have been done to improve coordination? AIMS coordinated with other actors during initial programme phase, however, there was limited scope for coordination in NCE phase. In the initial phase AIMS coordinated in the following ways: - Mw: Government. RBM & MCC - on getting credit rating for OPIC/DFC and trade fairs - Mw: USAID-funded AgDiv Programme - on BASP identification and WRS training) - Mw: USAID-funded DCA - on capacity building training for bank lending to ASMEs - Ke: Kenya Bankers Association (KBA) - on all other objectives, but not the EALGF - Ke: Kenya School of Monetary Studies - Identified a course organized by the central bank of Kenya, the Certificate in Agriculture Finance. Co-developed the course and facilitated its delivery in Kenya, Tanzania, and Malawi for 34 institutions and 400+ staff There was limited potential collaboration during NCE phase due to specific objective of supporting certain banks to utilize the EALGF. More work could have been done with other actors in the agriculture development sector to identify suitable ASME loan candidates. The AIMS team said that on reflection, they could have worked more with other organizations that work closely with ASMEs, such as NGOs, or large crop off-takers, on the identification of suitable and interested ASMEs to be linked to banks for loan applications. As noted, the market for formal credit among ASMEs, in Malawi in particular, is relatively small and there are information asymmetries between the demand and supply sides. The drop out or rejection rate was also relatively high, in Malawi especially, so having a larger pool of potential loan candidates to draw on and more information about those candidates up front from other organizations could have improved utilization. 3.3.3 Effectiveness ➢ What factors encouraged or discouraged banks from using the facility? Factors That Encouraged Utilization Alignment with group and country level bank strategies led to participation. Ecobank was keen to establish and pilot a relationship with DFC at the group level. At the country level, it was seen as a strategic necessity to have a risk-mitigation element in place before progressing with their intention to increase ASME lending. This was based on past experience of trying to move into the lower value loan sector. So, interests at the group and country level were well aligned AIMS End Line Assessment 32 with the EALGF offering. Similarly, I&M Bank had a strategy to diversify its portfolio, open more branches in rural towns and increase its agricultural lending, as this is the predominate sector in the targeted towns. It was actively looking for a partner to share the risk and so was keen to take the opportunity offered by the EALGF. Product sensitization forums and other linkages helped to create demand and more applications. These involved banks, BASPs, AIMS and ASMEs. These were very valuable both to the banks and the BASP in terms of understanding the product and creating interest and links with the ASMEs. This was stated by interviewees in both Kenya and Malawi. I&M Bank said a number of times that the leads to ASMEs provided by AIMS were ‘crucial’ in enabling utilization of the facility. The general structure of the guarantee was attractive to Ecobank. Ecobank was happy with an individual level guarantee as it allowed them to know with certainty what percentage of a transaction was covered. The fee structure and amount were reported to be acceptable. AIMS staff support on sign up and loan approval from DFC enabled more lending. Both banks said they appreciated the support they got from AIMS project staff in-country in the processed for both initial sign-up and commencement of lending to the EALGF. They valued that AIMS staff were present, hand-on and could advocate on their behalf as an independent party in the case of a disagreement or impasse with DFC. Factors That Discouraged Utilization I&M felt the guarantee fee was too high and currency denomination unhelpful. I&M said that the guarantee fee of 1.75% per annum was high, especially for term loans. This cost was passed on to ASME clients in the form of a higher interest rate, which the bank feel may have deterred some potential clients from applying. The bank also noted that the guarantee fee was charged in USD, while the loans are issued in Kenyan Shillings (KES), and was remitted to DFC quarterly, which makes the reconciliation hard for the bank. DFC’s requirement to review every loan application limited lending. Banks said they generally were gathering the same information that DFC required for their own internal credit review process. The need to put this data into DFC formats and wait for their review significantly delayed the loan approval process and added an administration burden, making it less attractive to make use of the facility. I&M Bank also felt that the interaction between DFC and partner banks needed to be ‘digitized’ or automated to improve efficiency. I&M Bank requested that DFC adopt an (improved) digitized/automated system that can interact with partner banks’ own systems. In their view this would improve the efficiency of the process of seeking approvals for facilities, reporting on the loans guaranteed, and accounting for the loan guarantee fees. All of which in turn would assist in speeding up loan approval times. DFC due diligence requirements on applications and responsiveness discouraged use. Banks said that the complexity and volume of legal and technical documents they were required to compile for DFC’s review of each loan application was demanding, both for them and ASMEs. As well as changing its requirements, DFC was also reportedly slow to respond when information was submitted, and would on occasions request the same information multiple times. Lender suspension event at Ecobank restricted lending in key period. Under the terms of the EALGF, two lender suspension events were triggered for Ecobank over the life of the program. AIMS End Line Assessment 33 These occurred when the percentage of non-performing loans exceeded the agreed maximum limit. Ecobank could not make loans covered by the EALGF in the suspension period. Ecobank said that the second of these events had been triggered and failed to be resolved over a period when they would have expected to make a large number of loans. This was during the post￾harvest season in Malawi when processors, aggregators and commodity traders would be looking to finance commodity purchases. The result was a lower utilization of the facility. Difficulty of claiming on guarantee from the DFC in case of default discouraged use. Although it had several cases of non-performing loans covered by the EALGF, at the time of the evaluation, Ecobank had yet to successful make a claim against the guarantee. They said that the processes they were required to follow and evidence to recover the debt before they could claim were too resource intensive to make claiming attractive. The difficulty of claiming against the guarantee discouraged EALGF utilization, as making these loans added cost and complexity without the certainty that a payout from DFC would be claimed and awarded in the case of default. ➢ Were Technical Assistance (TA) / support interventions with the banks effective in increasing the use of the facility? If so, how? And if not, why not? What would have been more effective? Both banks said that AIMS support was essential to them successfully signing up to and utilizing the EALGF. Ecobank valued that AIMS staff could act as a semi-independent agent on their behalf when facing issues with DFC, including providing clarifications and guidance on requirements. This applied to both the initial sign up and subsequently making loans under the guarantee facility. They especially valued having someone to meet with physically. I&M Bank staff said they valued the leads and the support in conducting due diligence on ASMEs. Support from AIMS was “very valuable and played a crucial role” in both sign up and utilization. One element they suggested they would have liked further support in relation to utilization was the perfection of security, as this often relied on input from third parties, such as the Land Registry, which could be time-consuming to have to deal with. Capacity building in agri-lending improved engagement with ASMEs and new sectors. Training in how to handle agricultural loans was reportedly well delivered and deemed very useful by both sets of bank staff. It helped them understand issues like commodity price volatility and ensure they request and collect the right information to make a well-informed lending decision. Trainings on how to administer the EALGF was also well received, there was some difficulty with the complexity of the subject matter, but they complemented the actual delivery by the AIMS teams. Both banks also said that participation in the EALGF had encouraged them to lend to new agricultural sub-sectors for the first time, such as horticulture and poultry, and to start lending to cooperatives for the first time, with positive results. Banks valued the product sensitization forums and saw an increase in demand. Ecobank mentioned this at the mid-term and end-line assessment. I&M Bank gave a similar report and said that AIMS, including via BASPs, provided them with good client leads. Ecobank would have valued more support from AIMS when it came to making a claim. As noted above, although it had defaults in its EALGF portfolio, Ecobank was yet to make a successful claim against the guarantee at the time of interview. They said this was due to the difficulty in making the claim from the DFC because some of the steps they are required to take before being able to claim are not practical to implement. They would have liked more support or explanation from AIMS earlier on in the process on how to go about ensuring that the guarantee AIMS End Line Assessment 34 can be claimed against when needed. While claims are not ideal, they are inevitable for a guarantee scheme that is extending loans to a higher risk market segment. ➢ How effective was the Technical Assistance (TA) provided via BASPs to ASMEs in increasing their readiness to receive loans? In the NCE period, the TA provided by BASPs to ASMEs focused on helping them to prepare documents in support of their loan applications, mainly on financial records. BASPs had provided other elements of business support, such as business planning, in the previous phase. BASPs provided support to between three and 20 ASMEs each. AIMs supported this by sharing the cost of BASP services with the ASMEs and by providing guidance to BASPs on the requirements of loan applications for the EALGF. BASPs in both countries said that support with financial records was a priority need for ASMEs. According to one BASP in Malawi, “Over 95% of the ASMEs we worked with don’t have management accounts, like a P&L, Balance Sheet or Cashflow Statement”. BASPs from both countries felt that the ASMEs they had worked with had needed and highly valued their support on preparing financial documents. All BASPs in Kenya said that the usefulness of their support was evidenced by the numbers of ASMEs who were willing to pay for BASP support outside of the AIMS program, though this was not the case in Malawi. ASMEs who had used BASPs were generally very positive about the services. This finding was reflected in both Malawi, where the bulk of BASP support to ASME loan applications had been provided in the NCE period, and in Kenya, where there had been limited support from BASPs required for ASME loan applications, but where ASMEs had received other forms of support earlier in the program. From the ASME survey, Of those who had received some form of BASP support, 69% in Kenya and 80% in Malawi said it was useful securing loans. BASP support for EALGF loan applications was effective, though not in all cases. Both the banks and the BASPs interviewed said that the support provided by BASPs to ASMEs in preparing financial documents for loan applications did have an impact and did lead to more applications being successful. They would like to see ASMEs being supported to access more of these types of services, though subsidized in some way due to affordability. BASPs in Malawi corroborated this. Ecobank feels there is a role for the public sector to support this work. I&M Bank similarly said that, after collateral, lack of financial and business information clarity was the major barrier for ASMEs to access loans. This suggests that the support provided by AIMS in this area was well targeted, though that there is still much to do in this area in both countries. BASP support effectiveness limited by financial capacity gaps in Malawian AMSEs. BASPs said that ASMEs do not have employees to record transactions or evaluate business performance, either due to resource constraints or because they did not feel it was critical to their business. Finance staff is also generally a monthly cost, whereas for many ASMEs their cashflow was uneven, so such costs are difficult to finance year-round. The lack of basic recording of transactions in some ASMEs meant that BASPs had to work with them to build up their records by looking at source documents before even getting to the production of reports. Malawian ASMEs had high number of sole proprietors and many businesses run by founders, so capacity is overstretched and not able to give financial management the focus it requires. BASP support was effective at addressing financial management capacity constraints to some extent, but the AIMS End Line Assessment 35 downside of this form of support is that is needs to be tailored to the needs of specific ASMEs and is time intensive for BASPs to provide one-on-one support. BASPs needed better understanding of requirements for support to be effective. One BASP in Malawi said it took them time to understand what Ecobank needed and so initially they were not effective at providing support as they did not fully understand credit requirements. They said that since they only interacted on a quarterly basis, it was not enough to feedback to applicants in a timely manner. The other felt there was a divergence in the approval criteria between what AIMS had said and what Ecobank were saying. BASPs were generally very positive about the guidance they received from AIMS, the critique was more around the inconsistency between that guidance and what they saw from the bank (Malawi especially). BASPs said that ASMEs they approached to apply for a loan became notably less interested once it was known that some applicants they had supported were rejected. Rejection of supported applications reduced appetite for BASP support in Malawi. When applicants they had supported were rejected, this demotivated one BASP from sourcing more applicants and deterred some ASMEs from wanting to engage them due to their unsuccessful track record. They said that ASMEs mainly wanted the support in order to get loans, hence the disappointment with the service if their application was rejected, regardless of other benefits realized. ASMEs need ongoing effective financial management and not just one-off intervention. BASPS in Kenya noted that support for financial statements is helpful, however it should not be a one-off exercise as they need continuous updating if they are to be used for loan applications and loan management. They said that a gap remains between the need and affordable availability for accountancy services. One BASP in Kenya said that having audited financial statements was the most important factor in getting a loan. However, many ASMEs are not legally required to have an external audit due to their size or registration status and so would be unlikely to be willing to incur such a considerable expense, especially if getting a loan is not certain even with a completed audit. BASP support also helped to improve access to other sources of finance. BASPs in Malawi and Kenya said that some of the ASMEs used the financial documents they helped to prepare to apply for loans from other sources besides Ecobank and I&M bank. BASP in Kenya said the program could be more effective if it expanded its reach. The BASPs in Kenya wanted greater sensitization of ASMEs on the program and services on offer and to have done forums in more regions to expand the number of ASMEs aware of the support on offer. Views were mixed on how likely it is that ASMEs will continue to use BASP support in future. Ecobank had positive view on sustainability of BASPs providing loan application support service to ASMEs, based on previous non-usage being, in part at least, due to lack of awareness of the facility. However, the mid-term assessment reported that only a very small portion of ASME continued to engage BASPs outside the program and that in general they indicated a reluctance to pay for such services. BASPs in Malawi also reported that some ASME did not pay their share, even after service had been delivered. BASPs in Malawi also said that their support with financials was used by the ASMEs for loan applications, but some failed to see the value of keeping records beyond that. In Kenya however, all four BASPs interviewed said they were confident that ASMEs AIMS End Line Assessment 36 would continue to use such services as many had done in the past and continued to do so without any support from AIMS. 3.3.4 Impact ➢ What was the impact of the EALGF-covered loans received by the ASMEs? (e.g. additional sales, employment, new business) ASMEs in the survey who had borrowed under the EALGF were asked to indicate what the impact was of that particular loan on their business. The results are show in Figure 10 below. Figure 10: Impact of the EALGF loan on ASME businesses Roughly two-thirds of ASME borrowers experienced growth as a result of their loan. As the chart above shows, high proportion of the ASMEs who had borrowed under the EALGF had experienced some form of positive impact on their business as a result. For example, 75% and 73% of ASMEs in Kenya and Malawi respectively said they had seen increased production/capacity as a result of using the loan. Only 27% of respondents in Malawi said they experienced no significant business benefits (none in Kenya). Even though the loan products were not designed as working capital facilities, many ASME borrowers said they had used their loans for working capital, such as the financing of an initial purchase of commodities for trading. Some also invested in capital equipment for business expansion, such as irrigation pumps. This finding should be caveated by the fact that the number of borrowers was small and the loan products they used were standard and not specifically tailored to supporting agribusiness growth. ➢ To what extent can changes in the outcomes of interest, including ASME growth and trade, be attributed to the project? This question is answered in two stages, as outlined in Figure 11 below: AIMS End Line Assessment 37 Figure 11: Impact Sub-questions (A) The project clearly enabled banks to lend to new ASMEs with less collateral. Provision of collateral coverage from the EALGF clearly resulted in both banks making loans they would not normally be willing/able to, provided all other requirements were in place. This has been discussed in earlier sections. There was some evidence from stakeholders of more general shifts in the market towards to improved access to finance for ASMEs. To quote one BASP in Malawi, “I have seen a lot of banks now bringing specific products for agriculture, which wasn’t the case in 2018. In the past three years we have seen more banks publicizing this (agri-lending)”. (A) The project attracted new applicants for loans to the EALGF partner banks. A large proportion of applicants and successful borrowers only knew about the product offering from partner banks through the AIMS project’s efforts, including via BASPs. The public call for proposals in Malawi was effective and stimulating demand. I&M reported that all clients who borrowed under the scheme were new clients, which strongly evidences this finding. (A) ASMEs were unlikely to have used BASP support in loan applications without AIMS. 72% of those receiving BASP support in both Kenya and Malawi were not aware that such services existed or were available to them before AIMS engagement and only 5% in Kenya and 10% in Malawi had used BASP services before. As has been detailed in sections above, ASMEs generally said that the BASP support they received had a positive impact on their access to finance, even if not all had successful loan applications under the EALGF. (A) AIMS direct support on loan applications to borrowers helped them secure loans. Among borrowers, 75% in Kenya and 82% in Malawi said AIMS staff support was useful in getting their loan. 55% in Malawi said it was the most helpful of all factors mentioned. (B) ASMEs said that getting the loan was a key factor in their growth, especially in Kenya. 100% of borrowers in Kenya and 46% in Malawi said the loan was the most important factor (27% in Malawi said it was the 2 nd most important factor) in realizing the benefits mentioned in Figure 10 above. This should be considered with the caveat that it was not possible to corroborate this finding from other sources or using other methods, which was beyond the scope of this evaluation. However, the strongly positive result from the ASME survey and other reasons stated above create a credible case that the small number of ASMEs who did borrow under the EALGF did experience growth and this growth can be attributed in some part to AIMS activities. ➢ What unintended consequences of the project were produced, positive or negative? Positive unintended effects are listed first, followed by negative. These are effects that were not directly part of the objectives of the activities in the NCE phase to improve EALGF utilization. New complimentary products developed for ASMEs in Kenya by I&M Bank. I&M Bank respondents said that the project gave them an opportunity to sell and develop other products targeted at the ASMEs, such as payment systems for businesses making many small payments (A) How important was the project in enabling access to credit for the ASMEs? (B) How influential was the loan on ASMEs' growth or other benefits experienced? AIMS End Line Assessment 38 to smallholder farmers. The bank respondents said the bank had benefitted from being able to explore the agricultural sector and benefited in being exposed to various agricultural value chains, such as nut processing and value addition, which it feels has great potential. They also said that the guarantee helped to enable them to provide support for enterprises across the value chain. For example, in dairy they are now supporting a cooperative of small-scale farmers who are producing milk and also supporting the dairy processor by enabling them to pay small-scale farmers immediately when they receive the milk for processing using the new payment product. New products & payment schedules developed for ASMEs in Malawi by BASPs. One BASP said they have also developed a new product offering as a result of AIMS participation. This was a subscription model where ASMEs pay a small amount to get accounts produced monthly throughout the year. This came “pretty much from this experience with GC”. They have two clients on board so far and say the outlook looks promising. They have also come up with a simple system in Excel that SMEs can use. It’s a plug and play product they have developed and they are selling it to SMEs, as well as offering training related to it. The other BASPs in Malawi said that they changed their repayment policies for ASME clients, in that they started accepting delayed payments in line with agricultural seasonal revenues. Good exposure and networking opportunities for BASPs at AIMS organized events. BASPs said that AIMS involvement helped to grow their profile and generate new business, particularly in Kenya. Similarly, one BASP in Malawi said that their participation in the AIMS project had built their capacity and enabled them to be selected for participation in a new donor funded program, providing similar services to potential investees of a development bank. Involvement of the guarantee slowed assessment and damaged Ecobank’s reputation. It was clear from the survey and FGDs that some ASMEs had a negative experience of applying to Ecobank. The main complaint was the process took too long, to the extent that the money applied for was no longer usable for the intended purpose by the time of decision. This impacted negatively on Ecobank’s reputation. The bank itself was aware of this and said that the requirement of having the loan assessed by AIMS and DFC was the main reason for the longer lead time on lending decisions. When applications supported by BASPs were rejected it damaged their reputation too. BASPs in Malawi said by being involved with AIMS and getting support from BASPs, ASMEs had a strong expectation that their loan applications would be successful. When applications were rejected, it created a lot of disappointment and even resentment towards the BASPs (as well to AIMS and the bank), especially since the ASME had paid (50%) of the cost of the BASP service and put their time into applying. “They feel like they paid for nothing” said one BASP. This had a negative reputational impact for them and deterred other ASMEs they approached from participating. Borrower awareness of donor involvement increased moral hazard risk. Ecobank said that despite doing their best not to mention that the loans it made were covered under the EALGF, the fact that AIMS interacted with ASMEs in other ways and followed up with applicants for their monitoring activity, means that ASMEs were aware of some form of donor involvement in the loan they got from Ecobank. Ecobank thought that this increased the risk that the borrower would believe that they could default without consequences or get favorable terms in some other way. I&M Bank raised a similar issue, noting that not informing the client of the guarantee facility had AIMS End Line Assessment 39 sometimes confused the client, which was compounded by the issue of having an extra charge on the processing of the loans under the guarantee. 3.3.5 Sustainability ➢ To what extent have banks under the EALGF changed their internal resource allocation, adopted new lending policies or procedures or developed products that increase access to credit for agribusinesses? Ecobank did not make significant changes to their lending policies and procedures. There has been no change in policies, standards, or product offering to ASME under the EALGF. Ecobank felt that their product offering was broad enough to meet the financing needs of most businesses, including ASMEs, though they acknowledged that their products were not designed with agribusiness models in mind specifically. Ecobank said they have only done transactions that they would normally want to do, the only thing the guarantee provided was coverage for the collateral that some ASMEs would not have been able to provide. However, Ecobank did say that that participation in the AIMS program did significantly improve the capacity of their existing staff to assess credit applications from agribusinesses. I&M Bank reported notable changes in resource allocation and in policies to enable ASME lending. I&M said that participation had increased their appetite for ASME lending as well as their internal capacity. They established lending teams in 12 branches and ensured that all teams had some experience in agricultural business lending. The bank also developed marketing material targeted at ASMEs. On the policy framework, the bank said that they have relaxed some policies to accommodate ASMEs supported under the EALGF. The bank relaxed the policy of not lending to clients that did not already have a savings account with them and that of only offering loans against security in form of a debenture. Under the EALGF, the bank also started considering offering moratoriums on repayment for 6 months on new loans. The bank also decided to reduce its own processing fee of 2.2% to accommodate the additional guarantee fee that ASMEs would otherwise have to pay. I&M bank expects these changes to become standard practice. The bank has also offered some of the clients who participated in the EALGF loan scheme additional products, like a working capital facility (not covered by EALGF) and internet banking. ➢ What barriers exist that may prevent the full adoption of policies, procedures, and product lines to meet ASMEs' needs? Ecobank still feels that it requires a partner to mitigate the risk of continued ASME lending. While the bank still wants to expand its book in the ASME sector, it said it is only looking to do so with a partner on board to share the risk. It still feels the risks of ASME lending are too high and so need some additional mitigation measures in place outside those they would normally seek for credit. The issue of profitability of ASME lending also remains a barrier for Ecobank, compounded by the additional processing requirements of having and needing a guarantee in place. Mismatch between ASMEs business cycle and banks’ repayment schedules. A potential barrier raised by the ASMEs in FGDs was the mismatch between ASMEs’ business cycles, which are often seasonal and banks’ repayment schedules, which are typically fixed monthly repayments. Some ASMEs said they would prefer to make fewer, but larger repayments aligned to their cashflow cycle and also to have a longer initial grace period. AIMS End Line Assessment 40 4 Lessons Learned and Recommendations Considering all the different aspects of the evaluation findings, what has worked well under the EALGF and what has not, as well as the differences between Malawi and Kenya, the main lessons from the evaluation are presented below. Recommendations relating to each lesson are then given in turn. Lesson #1: The concept for the EALGF may need to be broadened to achieve the scale of lending, and subsequent growth in ASME production and trade, desired by the program. The types of credit products that formal financial institutions are able to provide is limited by regulation and the relative cost of lending smaller amounts. The result is that the number of ASMEs that are able to access and benefit from the current offerings of formal finance providers may be quite small, particularly in a less-developed economy like Malawi. The transactions costs of providing and of accessing formal finance are relatively high on both sides, compared to other options that may be available for lending/borrowing. A guarantee facility may only be able to benefit a relatively small number of ASMEs for whom formal credit is appropriate and for whom the main barrier is the collateral requirement. These issues may be beyond the scope of one program to address and may require greater coordination with other players on improving access to finance in the target countries. Recommendation #1.1: Consider broader eligibility criteria for the types of businesses that the guarantee scheme will target. The purpose of the EALGF was to improve access to finance for ASMEs, hence only agribusinesses were eligible for coverage. There are valid reasons for targeting agribusinesses in terms of their impact potential. Ecobank understood this however they said that a guarantee that covered lending to SMEs across all sectors would be of greater value to them and the wider Malawian economy. I&M Bank recommended something similar in suggesting that the guarantee should also have been available for loans to businesses that provide supporting services to ASMEs, such as technology providers. This recommendation may be less applicable to USDA with its mandate to focus on agriculture/agribusiness, but could be an option for the DFC. Recommendation #1.2: Consider including non-bank financial institutions (NBFIs) in guarantee schemes targeting ASMEs. The average loan sizes in Malawi in particular mean that MFIs may be better placed to meet the finance needs of many ASME, as smaller loan sizes can be unprofitable for formal commercial banks to provide with their regulatory requirements and associated overheads. Furthermore, finance providers in less developed economies often need to judge the viability of ASMEs’ future business prospects without relying on formal documents like written contracts, and NBFIs may be better placed to do this from a regulatory and internal capacity perspective. Future programs might consider offering a bank and a NBFI guarantee facility targeting ASMEs concurrently. Recommendation #1.3 Support banks to expand their digital offerings where this may reduce transaction costs for ASMEs. The transactions costs of doing business with banks was reportedly too high for many ASMEs. One option to reduce these costs over time is for banks to digitize their products to make them more easily accessible to ASMEs, such as through partnership with mobile money operators. This could be a valid activity for donors to support in a way that benefits the whole banking sector by providing a means to reach ASMEs/SMEs more cost efficiently. AIMS End Line Assessment 41 Recommendation #1.4: Build objectives for coordination with other access to finance initiatives into guarantee programs to address the issue more holistically. Coordination across development programs is not straightforward, however targeted cooperation around a defined objective, such as ASME access to finance, could be achievable. This would help to address the barriers to ASME finance holistically. For example, the EALGF dealt with the issue of the lack of collateral among ASMEs, while other access to finance programs in the region had used matching grant schemes that aimed to enable financial institutions to offer business insurance tailored to agribusinesses. Collaboration could have enhanced the objectives of both types of programs. Recommendation #1.5: Build objectives for enhanced service offerings by FIs to ASMEs alongside guarantee programs, to make the ASME market more attractive and financially sustainable for FIs. The aim of a more broadly focused program would be to go beyond increased lending to ASMEs, to widen the overall service offering to ASMEs by (formal) finance providers, such as payment, savings and insurance services. I&M have taken steps in this direction by testing and now implementing a payment service to smallholder farmers for clients that run contract farming/buying operations. To deliver improved services, banks would need to invest in their agri / SME departments in terms of staff, coverage and product development, justified through offering a wider range of services that can generate revenues and offer ASMEs a more complete service package that will strengthen their overall business performance. Lesson #2: The design of the EALGF could have been better suited to the target country context and could have enabled more efficient utilization by banks that did sign up. While the EALGF clearly did enable access to credit for some, many ASMEs still did not have the collateral required to cover the balancing percentage of security required, even with the EALGF guarantee in place. Furthermore, the ‘double due diligence’ requirement on loan application assessments added administrative cost and time delays for the banks and ASMEs, which resulted delays in approving applicants and in lower utilization than would otherwise have been possible. This may have been exacerbated by a reported lack of automated processes and systems in place for sharing information between DFC and partner banks on loans covered by the facility. Recommendation #2.1: Consider increasing the level of coverage available when guaranteeing loans in less developed sectors and/or economies. Guarantors may need to consider covering an increased percentage of the loan value, perhaps up to 80%, due to the difficulty that many ASMEs face in providing collateral that is acceptable to banks. While guarantors may be concerned that this would encourage excessive risk taking on the part of the partner banks, they would still be required to follow their usual credit procedures for approval and recovery in the case of default, and evidence these to the guarantor14 , as a sufficient safeguard for the level of risk being taken. Recommendation #2.2: Enable partner banks to fully approve each loan to be covered by the facility, once their credit procedures have been approved by the guarantor. Utilization of the guarantee facility would likely improve if the guarantor did not also need to conduct its own credit assessment on every loan. The guarantor could instead conduct initial and then periodic 14 Assuming that the structure was similar to the EALGF in terms of the guarantor assessing each loan individually. AIMS End Line Assessment 42 due diligence in such a way that it gains confidence of the partner banks’ own credit approval processes. This could include periodic spot checking of a sample of credit applications. This would reduce the administrative burden on both partners and still provide the necessary assurance to the guarantor that the partner banks are able to and do carry out sufficiently robust assessments. Recommendation #2.3: Co-design the guarantee facility with financial institutions (FIs) and their associations in the target countries before entry. FIs in the target countries understand the risk profile of the target beneficiaries better than an external organization and can advise on how the facility might be structured to maximize impact whilst minimizing risk. This approach would also help the guarantor to tailor the individual loan assessment requirements to the country context. The guarantor will have certain elements of the design that are ‘non-negotiable’, but where there is some flexibility, it would likely benefit from inviting input from banks (and other FIs) in the target countries. For example, this approach could help to ensure that the guarantee scheme covered the types of finance needed by the target beneficiates, such as specifically designed working capital facilities for ASMEs, a need which came out clearly from this evaluation. There may be other minor issues and needs that are identified, such as the issue raised by I&M Bank of the guarantee fee being charged in USD on KES denominated loans, and remitted to DFC quarterly, making reconciliation hard for the bank. Or the need to make systems improvements that allow for more efficient information sharing between DFC and the targeted partner banks. Having the project spearheaded by an active industry association, such as the KBA15 , can also increase buy-in for the overall project. Lesson #3: The origination and contracting process for the EALGF took a lot of time and effort, this made it challenging for AIMS to sign up partner banks during the program. A major factor that limited impact of the program was the lack of take up of the EALGF by banks in the target countries. The perception was that this is mainly due to the strict terms and inflexibility of the DFC’s guarantee offering and the slow pace of communication. The due diligence requirements were perceived to be excessive in relation to the size of the facility and level of risk involved. There were also other more attractive guarantee schemes available for potential partners to choose from. Even with the banks that were ultimately successful in signing up, the origination and contracting process for the EALGF took almost two years. With the benefit of hindsight, the AIMS team also felt that they lost time, due to the required approach of engaging with only a small number of banks at a time in an iterative process of origination, as many such engagements were ultimately unsuccessful. A process that enabled all potential partners to be engaged at the outset, with a means to filter them down, would potentially have enabled earlier commencement and more partner banks to be signed up. Recommendation #3.1: DFC should review and benchmark its internal origination and loan approval processes. DFC may want to consider an internal review of its origination and loan approval processes and benchmark against similar DFIs. Banks and project staff said that requirements for participation and utilizations needed to have been more clearly set out at the beginning of the process, and to have remained consistent through the life of the project, as far as possible. DFC may also want to consider how it can improve communication. This may involve having more presence in the target countries, such as through budgeting for more travel for its 15 The Kenya Banker Association has a membership of 43 commercial banks in Kenya AIMS End Line Assessment 43 staff to visit potential partners, or through empowering partner organisations already based in country, such as USAID teams, to communicate and make decisions on its behalf. Recommendation #3.2: Approach a longer list of banks from the start of the program to gauge interest widely, before focusing efforts on the most promising candidates. It would have been more efficient to have engaged with a wide variety of banks from the start of the program then narrow down the selection when it was clear there was interest and a good match. Recommendation #3.3: Consider an extension to the EALGF guarantee period for I&M Bank now and consider using flexible guarantee periods from inception in future projects. I&M Bank requested that DFC consider extension of the EALGF guarantee period for onboarding new loans. This is because of the time it took to onboard the bank and the initial learning curve involved before utilization of the facility reaches its full potential. The bank said that they have built momentum now and have many ASMEs loan applicants in their pipeline. An extension period of 6 months or more was suggested. The recommendation extends more broadly to suggest that guarantee periods be for a set period of time starting from the point at which each partner bank signs up, rather than being set from the onset of the program. Lesson #4: There were significant capacity gaps in basic financial management in many ASMEs in the target countries. BASPs in both Malawi and Kenya noted the limited capacity in many ASMEs for basic financial management and record keeping, often citing a lack of suitably qualified accounting or bookkeeping staff. This was leading to challenges not only in securing financing, but also in ASMEs managing their cashflows to meet repayment schedules, and meeting tax and other compliance requirements. Recommendation #4.1: Consider including post-transaction support to ASMEs from BASPs, as well as loan application support. ASMEs would benefit from capacity building on how to effectively manage their finances, especially those that are accessing formal finance for the first time. Time-bound post transaction support by BASPs to ASMEs, such as on managing cashflows to meet repayment schedules, could be considered as part of future programs. AIMS End Line Assessment 44 5 Summary and Conclusion To summarize, the key area of success and areas to improve from the AIMS program end-line evaluation are listed below: Key Areas of Successes: ✓ Partner banks made loans to ASMEs that would otherwise be rejected. Both banks said that participation in the EALGF had enabled them to make loans to clients they would normally not be able to lend to due to lack of collateral, so this directly increased lending to a small number of ASMEs. ✓ Partner banks were connected into and lent to new clients in new sectors. This included lending for the first time to ASMEs in the horticulture and poultry sectors and to cooperatives, with mostly positive results. Both banks were very positive about the role of AIMS in connecting them to new ASME clients. ✓ I&M Bank made changes in resource allocation and policies to enable ASME lending. It expects these changes to be part of its business model into the future. I&M also developed complimentary products aimed at ASMEs, notably payment systems for produce buyers, and has also increased the bank’s appetite for ASME lending more generally. This has included making specific working capital loans to ASMEs outside the facility, in addition to the term loans covered by the EALGF. Key Areas to Improve:  The complexity and effort required from partner banks in signing up to the EALGF was too great. Effective collaboration between DFC, AIMS and partner banks has been challenging throughout the program. This led to lack of interest, delays and dropouts in signing up banks to the scheme and with its subsequent utilization.  Double due diligence by both the partner bank and DFC was a burden for the ASMEs. This also slowed the application decision process, particularly for Ecobank. This was an issue even with the valued support provided by AIMS project staff to both the partner banks and the ASME applicants.  Residual collateral requirements remain a barrier to credit for many ASMEs, even after guaranteed portion is covered by the EALGF.  Ecobank still feels the risk of lending to ASMEs is too high without a guarantee in place. The experience from participating in the EALGF did not convince them otherwise.  The complexity of processing default claims under the guarantee facility, as expressed by Ecobank, deterred usage. As a core feature of the guarantee product, the difficulty in claiming in the case of default deterred the bank from utilizing the facility further. To conclude, its clear that the EALGF did achieve its objective of increasing access to loans for ASMEs, that the loans did contribute to increases in ASME trade for the majority of businesses that received them, and that the activities of the AIMS program were important contributors to this impact. However, the scale of impact was well below the program’s targets. The range of both internal and external limiting factors have been discussed in this report, with recommendations presented for consideration in future access to finance programs for ASMEs. 1 Annex 1: Scope of Work/ Statement of Work (SOW) I. INTRODUCTION Background on Global Communities. Global Communities is an international non-profit organization that works closely with communities worldwide to bring about sustainable changes that improve the lives of vulnerable people. We envision a world where everyone has the freedom, means, and ability to live and prosper with dignity by creating a long-lasting, positive, and community-led change that improves the livelihood of vulnerable people worldwide. Global Communities believes that the people who understand their needs best are the people of the community itself. We make a difference by engaging with communities, governments, the private sector, andNGOs as partners for good—bringing together complementary strengths and shared responsibilitiesto work toward common goals. Global Communities has existed for more than 60 years. Until 2012, we were known as CHF International and, before that, the Cooperative Housing Foundation. For more details on Global Communities and its work, see https://www.globalcommunities.org/aboutus Project Background The United States Department of Agriculture (USDA) awarded Global Communities the Agribusiness Investment for Market Stimulation (AIMS), a five-year project starting in October 2014 and ending in September 2019, with a total estimated budget of $20,554,297.88. AIMS is a multi-country project supporting Kenya, Malawi, and Tanzania. The project aims to bolster agriculture trade by increasing the capacity of agribusiness small and medium enterprises (A-SMEs) to access finance and reach markets. The Project defines A-SMEs as businesses that maintain annual revenues between $20,000 - $1million; the project includes a $50 million loan guarantee facility (LGF) backed by the United States Development Finance Corporation (DFC), the size of the SME was determined to be in line with SME capacity to service the loan under LGF. AIMS received a four-year no-cost extension (NCE) from October 2019 to September 2023 to continue operations in Kenya and Malawi towards East Africa Loan Guarantee Facility (EALGF) utilization. Tanzania was dropped off in the NCE as the AIMS project had not managed to sign a bank under the LGF there. AIMS uses a market systems approach that involves building market actors' capacity on the supply and demand sides. The program works in several value chains, including horticulture, grains, dairy, and livestock. AIMS targets agribusiness SMEs across the value chains from production to aggregation to processing and exporting, including individual companies, cooperatives, and farmer-based organizations. AIMS also builds banks' capacity in the region to lend to small and medium agribusinesses and helps to offset risks of lending to this sector under the loan guarantee facility provided byDFC. The AIMS program has four main objectives, each with specific approaches to achieve them. Note that the end￾line evaluation will only assess Objective 1, covering activities starting in February 2018 when the first agreement was signed under the LGF. Objectives 2, 3 and 4 were assessed in a prior evaluation. While these activities have ended as free-standing objectives, AIMS continues to implement elements of them on a small-scale to facilitate use of the EALGF. For example, approved business advisory service providers are incentivized to support SMEs in developing loan applications, and AIMS supports its partner banks in increasing outreach and marketing of products to the agricultural sector. Please see Annex F for a list of LGF activities. • Objective 1: Increased use of financial services--Leverage private and public investment. During the NCE phase, AIMS promotes the effective use of a $50 million loan guarantee fund provided by the United States Development Finance Corporation (DFC). Under this effort, Global Communities serves as a facilitator between commercial banks and DFC. The DFC guarantee is operationalized with two banks, Ecobank- Malawi and I&M 2 Bank-Kenya, already signed agreements with DFC. • Objective2:Increaseduseoffinancialservices--FacilitateAgribusinessLending.AIMS built the capacityoffinancial institutions, including banks and microfinance institutions, to increase lending to agribusiness SMEs through skill enhancement and developing new agribusiness loan products. Capacity building was done through trainings using the curriculum developed under the program and delivered by a third party and through the Eastern Africa Grain Council. AIMS also included banks in various forums to help build an understanding ofthe agriculture sector and build linkages to agriculture SMEs. • Objective 3: Improved capacity of key organizations in the trade sector. AIMS worked with private business advisory service providers (BASP) to build agribusiness SMEs' ability to access markets and financing and improve their business operations. AIMS also worked with apex organizations, including associations, large aggregators, and cooperative unions, to deliver services. • Objective 4: Improved linkages between buyers and sellers. AIMS conducted buyer seller forums and linkages and supports access to ICT-based market information systems and marketing platforms to improve access to markets and market information. AIMS’ four objectives contributed to Food for Progress’ Results Framework Strategic Objective 2— Expanded Trade of Agricultural Products (Domestic, Regional, and International) FFPr SO2—specifically Result 2.2—Increased Access to Markets to Sell Agricultural Products (FFPr 2.2). The full AIMS Results Framework is provided in Annex E. During AIMS' first phase, October 2014-September 2019, the project tracked 24 performance (output and end-line) indicators that include USDA Food Assistance Division (FAD) Standard Indicators. In August 2019, USDA approved a revised Monitoring and Evaluation plan with a revised indicator list and targets for the NCE phase, October 2019- September 2021. The revised indicators focused on the loan guaranty facility utilization performance, the revised indicators are provided in Annex D. Global Communities had one implementing partner on AIMS between October 2014 and September 2019: the Eastern Africa Grain Council (EAGC). EAGC supported objectives 2, 3, and 4. Specifically, it provided training on warehouse receipt financing to banks under objective 2, training and mentoring under objective 3, and buyer-seller linkages and access to its market platform. AIMS took a market systems approach of working with and through large buyers, cooperative unions, membership, and trade associations and government entities, which it refers to as apex organizations as well as business advisory services providers and market information systems (MIS) providers to deliver services. AIMS had memoranda of understanding with these institutions. During the No-Cost Extension Phase, AIMS focused on Objective 1, establishing and utilizing a Loan Guarantee Facility component. The key types of partner organizations and the number served to date are listed below. Bidders should use the table below in proposing sample size, sampling strategy, and identifying appropriate data collection tools. Figure 1 - Number of Partner Organizations Served to Date Category of Partners # of Partner Organizations Served to Date Kenya Malawi Financial service providers (Commercial Banks) 1 1 Agribusiness SMEs (includes input suppliers, agro-processors, wholesalers/retailers, and exporters) 70 83 Business Advisory Service Providers 4 2 3 II. END-LINE EVALUATION A. Purpose and Objectives The purpose of this end-line evaluation is to critically and objectively review and examine the Loan Guaranty Facility experience and the implementing environment. The evaluation should assess whether the targeted beneficiaries have received services as expected and assess whether the Loan Guaranty Facility met its objective and goals. This evaluation will document any unintended outcomes, best practices, and lessons learned related to the LGF. The focus of this evaluation will be to assess how the guarantee facility has affected agricultural finance to A-SMEs in the participating banks, while also examining the sustainability of the impacts. The end-line evaluation will: • Assess whether the activities implemented under the LGF achieved the intended results and objectives; please see Annex F for activity descriptions. • Evaluate the impact of the LGF on increasing lending to the agriculture sector; • Assess the likelihood that LGF results will be sustained; • Document key successes, best practices, challenges, and lessons learned; and • Provide specific recommendations for future access to finance programs. The end-line evaluation will assess project performance toward objective one across the following general themes, as defined below, and for which specific research questions are proposed in Table 1 below. Relevance: the extent to which AIMS LGF interventions (i.e. training, services, etc.) meet the needs of the program beneficiaries, are aligned with the country’s financial inclusion strategy toward Agriculture -SME, and were designed to address technical and management barriers toward A-SMEs access to finance in Kenya and Malawi.. Effectiveness: the extent to which the EALGF has achieved its objectives. Effectiveness should also assess the extent to which the interventions contributed to the expected results or objectives. Efficiency: the extent to which the program resources (inputs), including time and cost, have led to the achieved results. Sustainability: evaluation of the likelihood that the LGF services and/or benefits of AIMS will endure beyond completion of the program. Sustainability should also assess the extent to which the program is planning for the continuation of program activities, such as through encouraging local ownership and sustainable partnerships. To the extent possible, the evaluation should assess the extent to which the effects are due to the project intervention and not to other factors. The evaluation's primary audience includes Global Communities, USDA, DFC, commercials banks, and BASPs. This evaluation will draw lessons learned to create a set of key findings and recommendations to support the design of future programs. These lessons learned will also be disseminated to the program stakeholder and partners, including USDA. The evaluation process is intended to be participatory and results-oriented in its findings and recommendations, which should be valid, insightful, and useful. While relevant program staff and key program stakeholders and partners will be involved cooperatively to the extent possible, as this is an independent, third-party evaluation, the selected evaluator will be responsible for management oversight and the determination and presentation of key findings and recommendations. B. End-Line Evaluation Questions To address the relevance, effectiveness, efficiency, and sustainability of AIMS EALGF, the evaluation team will be 4 guided by the list of Illustrative Research Questions in Table 1 below. Additional questions and revisions to these questions may be proposed by bidders and/or considered during the evaluation launch or in the inception report, based on input from the evaluation consultant, Global Communities, and/or key program partners involved in the finalization of the evaluation design. Notes that the key areas of inquiry outlined in the table below should be applied to AIMS Program Objectives 1. Table 1: Illustrative End-line Evaluation Questions: Focus Areas for End-line Evaluation Proposed Questions The relevance of the AIMS Program • Is the concept and design of AIMS EALGF suitable to the overall market situation and challenges faced by agriculture small and medium enterprises (A-SMEs) in Kenya and Malawi? • The extent to which A-SME needs for finance were met/unmet Program Efficiency • Has EALGF implementation been able to adhere to stated plans(i.e. work-plans, action-plans, results framework, and budget), and if not, why? • What were the challenges in the implementation of the LGF and how did we address them? What are the best/most efficient practices that contributed to program successes? • How well is EALGF coordinating with other key actors (development actors, government, and private sector) working on similar or complementary programming? What could have been done to improve coordination? • What has worked especially well under LGF and why? • What has not worked well and why? Program Effectiveness • What factors encouraged or discouraged banks from using the facility? • Were TA/support interventions with the banks effective in increasing the use of the facility? • What TA (via BASPs) provided to A-SME were most effective in increasing SME readiness to receive loans? What TA provided to A-SMEs was least effective in increasing readiness to receive loans, and why? • What has worked especially well under LGF and why? • What has not worked well and why? • Project Impact • To what extent can changes in the outcomes of interest, including SME growth and trade, be attributed to the project? • What unintended consequences of the project were produced, positive or negative? Program Sustainability • To what extent have banks under the LGF adopted new lending policies or procedures and developed products that increase access to credit for agribusinesses? Did AIMS technical assistance improve/enhance the lending process? What barriers exist that may prevent the full adoption of policies, procedures, and product lines to meet A-SMEs' needs? Lessons Learned & Recommendations • What has worked especially well under LGF and why? • What has not worked well and why? • How have experiences and results differed between Kenya and Malawi? • What are preliminary lessons learned from implementing AIMS LGF interventions targeting increasing financial inclusion to A-SMEs? 5 III. METHODOLOGY EVALUATION I. Evaluation Design The end-line evaluation should use statistically proven methods that incorporate quantitative and qualitative data collection techniques applied to partner banks, business advisory service providers and a representative sample of SME borrowers. Where time series data is available, endline analysis should include a time series analysis as well. The evaluation team will triangulate data through 1) document reviews; 2) key informant interviews; 3) focus group discussions, and 4) survey questionnaires as discussed in detail below. Document review: The evaluation team is encouraged to consult a broad range of background documents (both internal and external) related to the program in the three target countries where the program is being implemented. The package of briefing materials from the AIMS program includes, but is not limited to, the following materials: 1. Program Award Cooperative Agreement executed between Global Communities and USDA. 2. Nine (11) Semi-annual Performance Reports submitted by AIMS to USDA, which includesAIMS indicator results in FAIS; 3. AIMS Monitoring and Evaluation Plan 4. Revised Performance Monitoring Plan submitted to USDA on May 30, 2017. 5. Baseline Report (including Instruments) submitted to USDA on February 15, 2016 6. Outcome Assessment submitted to USDA on June 16, 2020. This assessment replaced the mid-term evaluation and evaluated the performance of Objectives 2, 3 and 4. 7. Bi-annual reports, program write-ups, special studies, and case studies. Research Methods: It is anticipated that the evaluation team will use a variety of research tools and methods for carrying out this end-line evaluation, including participatory methods like Outcome Harvesting, Most Significant Change, or Ripple Effects Mapping that involve key stakeholders as respondents and in providing information necessary to make evaluation activities effective and efficient. This might include Key Informant Interviews, Focus Group Discussions(FGDs),surveys, and secondary data. The evaluators should propose statistically sound methods most appropriate to address evaluation questions and plan to triangulate data obtained qualitatively with trends and findings derived from quantitative data obtained through the evaluation. To the extent that it is possible, the final evaluation should make an effort to target key stakeholders in the financial services, agribusiness SMEs, and business advisory service providers that were interviewed at baseline and have remained involved or benefitting from the AIMS program. As appropriate, the evaluation may also include additional questions that would be used to capture other indicators that had no baseline values. II. End-line Evaluation Management: Under USDA policy, this evaluation will be officially managed by Global Communities’ Monitoring & Evaluation (M&E) Unit in Silver Spring, MD, the organizational headquarters in the U.S. Nonetheless, the evaluation team will work closely with AIMS staff in East Africa, including the Program Director, Credit Director, M&E officer, Credit Specialists, and Technical Managers. Program Managers and Technical Managers will be available to answer technical questions about the program structure and implementation. At the same time, in-country teams can assist with logistics, arrangingmeetings, lodging,and othermatters. Theevaluation teamwill be freetodrawtheir conclusions free from organizational or political pressure. USDA will take an active role throughout the process including providing input on the TOR/SOW, participating in a stakeholder call with the evaluation team before fieldwork, and reviewing and approving the evaluation report. During the stakeholder call, USDA will communicate the key evaluation questions to the evaluator to be of the highest priority. The AIMS program staff will assist in making appointments with partners, program beneficiaries, government officials, and other stakeholders at the request of the evaluation team, and ensure that the consultant has full logistical support. However, AIMS program staff/representatives will not be present during the interview sessions or any other data collection activities performed by the evaluation team. Interview sessions will not occur at AIMS 6 offices. The end-line evaluation must be carried out in conformance with USDA’s Monitoring and Evaluation Policy. The policy states: "As specified in regulations (see 7 CFR Part 1499.13 and 7 CFR Part 1599.13), evaluations will be independent and conducted by a third party. Specifically, the regulations specify that the third party conducting the evaluation: • Is financially and legally separate from the participant's organization; • Has staff with demonstrated knowledge, analytical capability, language skills, and experience in conducting evaluations of development programs involving agriculture, education, andnutrition; • Uses acceptable analytical frameworks such as comparison with non-project areas, surveys, the involvement of stakeholders in the evaluation, and statistical analyses; • Uses local consultants, as appropriate, to conduct portions of the evaluation; and, • Provides a detailed outline of the evaluation, major tasks, and specific schedules before initiating the evaluation." IV. CONTRACTOR AND GLOBAL COMMUNITIES TASKS AND RESPONSIBILITIES I. Contractor Responsibilities: a) Understand the scope, orientation, structure, inputs, outputs, and results of the program and obtain secondary data by conducting a desk review of documents provided by Global Communities, including but not limited to: i. Program Statement of Work; ii. Program M&E plan iii. Program reports. b) Develop a detailed Inception Report and work plan for the overall evaluation. The plan must be presented to Global Communities M&E and Program Management teams. Due to COVID-19, the inception report will be presented and discussed through an online video conferencing platform. After Global Communities approves the inception report and the implementation plan, and the research instruments, the evaluation team will start the field work and the data collection. The presentation and plan should include identification of the following: o Proposed data sources and collection methods; o Proposed plan and methodology for responding to each evaluation question outlined in Table 1 o Quality assurance plan o Proposed sampling method(s) and location of communities, organizations, and individuals to include in research that allows for comparison with thebaseline; o Limitations in data collection and analysis; and o Draft research instruments (i.e. surveys, protocols/guides for focus groups, key informant interviews (KII), and interviews). Data collection instruments should be designed to allow for comparison with the baseline. c) Develop a sampling methodology (including random sampling method and sample size calculations) for the quantitative survey that is statistically representative of the project participant population and consistent with the baseline, if possible. d) Develop, in English, scientifically sound research tools, and instruments that would help measure the effects of the program’s interventions covering all components highlighted underthe evaluation's purpose, described above. The evaluators should apply quantitative and qualitative methods,includingrelevantsurvey tools/questionnaires, key informant interviews, and focus group protocol and observation checklists. These should be shared with and approvedbyGlobal Communities before theiruse. 7 e) Whererelevant, employmixedmethods of gatheringdata, particularly for purposes of triangulation. f) Train research assistants/enumerators in the application of selected research methods and instruments. May include training on how to conduct an in-depth interview and focus group discussions and testing their knowledge before data collectionstarts. Research assistants/enumerators must understand and speak the local Language. g) Pre-test research instruments before their use with a small sample of respondents to ensure questions are understood, respondents are willing to provide the information requested and questions will yield the type of information the evaluators are seeking. h) Collect data through identified research methods. Primary data sources are likely to include surveys, interviews, and focus groups of key individuals and organizations involved in A-SME lending, production/processing, and agricultural marketing and trade as well as program staff and key stakeholders. Host government policy stakeholders should also be included if deemed relevant to the AIMS program implementation. The evaluation firm will collect and secure information consents for all data collected. A final sampling framework will be developed in collaboration with Global Communities following the submission of the Inception Report and before the commencement of data collection and finalization of research instruments. The evaluation team must have adequate safety plans in place during field data collection. Data collection should not exacerbate the spread of COVID-19. i) Conduct all data entry, cleaning, and analysis and generate a summary offindings. j) As relevant, disaggregate data by target community, gender, ethnicity, and age. k) Analyze and triangulate quantitative and qualitative findings as described in the evaluation objectives and key research sections. l) Facilitate a half-day validation workshop in-country. The AIMS program and technical staff will provide the Contractor with additional input to validate initial findings and recommendations before they are made final. m) Provide two drafts and one final copy of the evaluation report that is free of personally identifiable information (PII) and proprietary information. Body of final report (excluding annexes) should not exceed 50 pages and incorporate feedback from Global Communities and USDA on earlier drafts. The final report should include thefollowing: 1. Visual presentations(e.g. charts, graphs, and/or tables) to most appropriately display key findings; 2. Analysis of key findings; 3. Executive Summary; 4. List of Key Findings and Recommendations; 5. Contact details of persons interviewed/key informant interviews(annex); 6. Final copies of all survey instruments(annex); 7. Final Copies of all raw data sets (both qualitative and quantitative). The United States Government is the primary owner of the data, and it is not to be used by the consultant for any purpose beyond this evaluation. 8. Indicator tables (annex) 9. Evaluation Scope of Work (annex) 8 Final versions of evaluation reports ready for publication should be accessible to persons with disabilities. For guidance on creating documents accessible to persons with disabilities, please see the following resources: https://www.section508.gov/create/documents https://www.section508.gov/create/pdfs n) Provide weekly phone updates to Global Communities HQ M&E team and AIMS Program Leadership and M&E Manager on the evaluation's progress throughout the entirety of the performance period. o) Present final evaluation report to Global Communities and USDA stakeholders. Global Communities will arrange the session through an online video conferencing platform. p) A 2-3 page stand-alone brief describing the evaluation design, key findings and other relevant considerations. It will serve to inform any interested stakeholders of the midterm evaluation, and should be written in language easy to understand by non-evaluators and with appropriate graphics and tables. II. Global Communities Responsibilities: a. Providing the evaluator with copies of all AIMS program documents (proposal, grant agreement, reports, work plans, M&E special studies, program approaches and write-ups, success stories, etc.). b. Introducing evaluators to key beneficiaries and informants as needed. c. Assisting with logistics as needed.Unless otherwise agreed, all in-country travel and logistical arrangements will be made by Global Communitiesstaff. d. Reviewing all contractor deliverables and providing feedback on time. e. Making the final version of the evaluation report publicly available. f. Providing payments to the contractor based on work completed and accepted byGlobal Communities. V. TIMEFRAME The assignment is scheduled for an estimated period of 12 weeks (3 months), with an estimated start date of March 15, 2021. The proposed schedule is as follows: (*) denotes deliverable to be submitted to Global Communities Task Name Duration Start Finish Responsible Award and Contract Signing 0 days Fri 3/12/21 Fri 3/12/21 OGC Kickoff meeting to discuss overall project goals 6 days Mon 3/15/21 Mon 3/22/21 MELKM, Technical Director, Field Technical Team, PMT, Contractor 9 Desktop Research and review of program documents, reports, program results, existing data collection tools, revised M&E plan, program approaches and strategies, database systems, program write ups, M&E systems and processes, etc. 11 days Tue 3/23/21 Tue 4/6/21 Contractor with support of Global Communities Develop and submit draft Inception Report/Research Plan 14 days Tue 3/23/21 Fri 4/9/21 Contractor Review inception report and provide feedback 5 days Mon 4/12/21 Fri 4/16/21 MELKM, Technical Director, Field Technical Team, PMT Submit final inception report 2 days Mon 4/12/21 Tue 4/13/21 Contractor Launch of Evaluation (review SOW and address questions or concerns, hold meetings with assessment team and Global Communities staff, USDA, and key partners, and finalize research questions and plans, etc.) 5 days Wed 4/14/21 Tue 4/20/21 Global Communities & Contractor Develop and submit draft data collection instruments 5 days Wed 4/21/21 Tue 4/27/21 Contractor Develop field team training materials 10 days Wed 4/28/21 Tue 5/11/21 Contractor Training for data collectors 4 days Wed 5/12/21 Mon 5/17/21 Contractor Pre-test (desk testing the tool by internal staff) 4 days Wed 5/12/21 Mon 5/17/21 Global Communities and Contractor Piloting and refinement of research tools 3 days Wed 5/12/21 Fri 5/14/21 Global Communities Modify data collection instrument 4 days Mon 5/17/21 Thu 5/20/21 Contractor Approval of All Plans and Instruments by Global Communities 6 days Fri 5/21/21 Fri 5/28/21 Contractor USDA call with selected evaluator 1 day Fri 5/21/21 Fri 5/28/21 USDA and Contractor Carry out data collection and analysis 20 days Mon 5/31/21 Fri 6/25/21 Contractor Gather information through focus group and in-depth interview with key informants - Qualitative 10 days Mon 5/31/21 Fri 6/11/21 Contractor Survey Data collection – Quantitative 10 days Mon 5/31/21 Fri 6/11/21 Contractor 10 Transcribe qualitative interviews 10 days Mon 5/31/21 Fri 6/11/21 Contractor Data entry, cleaning, and analysis 10 days Mon 6/14/21 Fri 6/25/21 Contractor Contractor provides debriefing to GC and Key Stakeholders that includes a Power Point presentation approved by GC prior to debriefing 8 days Fri 7/2/21 Tue 7/13/21 Contractor Develop and Submit *Draft 1 Report to GC for review 4 days Wed 7/14/21 Mon 7/19/21 GC Global Communities Response to Draft 1 Report 5 days Tue 7/20/21 Mon 7/26/21 Contractor Submit *Draft 2 Report to GC and USDA for review 5 days Mon 8/2/21 Fri 8/6/21 Contractor Global Communities and USDA review and provide feedback to Draft 2 Report 5 days Mon 8/9/21 Fri 8/13/21 GC and USDA Submit *Final (Interim) Report, including all instruments and appendices, and final dataset, to Global Communities for subsequent submission to USDA for review and comments (to be submitted to USDA within 60 days of fieldwork completion) 5 days Mon 8/16/21 Fri 8/20/21 Global Communities USDA Response to Evaluation Report 5 days Mon 8/23/21 Fri 8/27/21 USDA Final Report, including all Tools and Appendices submitted to USDA 5 days Mon 8/30/21 Fri 9/3/21 Global Communities VI. Evaluator Qualifications A. Team Composition and Qualifications: The Bidder shall identify in summary format the names and anticipated positions of the individual proposed to perform the requirements described above. The Bidder shall indicate the level of effort for each proposed person who will perform under the contract. The Bidder shall submit a complete and current resume for each proposed professional employee (not exceeding 3 pages each) who will be carrying out the work if award is made. These resumes must clearly describe the individuals’ education, experience, and professional credentials. The Contractor is expected to have a team comprised of individuals with strong leadership skills; experience in research design and methods, qualitative and quantitative tool development, and data collection and analysis; as well as technical competence in relevant agricultural marketing, finance and trade. In particular, the evaluation consultant (or key members of the evaluation team) will have the following qualifications: 11 - Must have at least five years of demonstrated experience in evaluating agricultural finance/marketing programs (required) and hold an academic qualification at least of B.S. in agriculture, monitoring and evaluation, international development or a related field. - Experience in conducting evaluations in Eastern Africa Region is desirable, and ability to work in multi￾cultural teams - Expertise of AG finance in Eastern Africa is desirable. - Experience in evaluating USDA, USAID, or other US-government-funded projects is desirable. - Experience in statistics, quantitative and qualitative instrument design, plus data collection and analysis. - Demonstrated Experience and capability in producing high-quality, visually appealing evaluation reports with a strong attention to data visualization and effective graphing of survey data. - Fluency in English and outstanding English writing and presentational skills. - Experience recruiting, supervising and training enumerators and evaluation team members. - Gender balance B. Organizational Capabilities: The Bidder shall describe and demonstrate the following: Part 1. Capacity: a. Specialized competence the organization possesses with regards to the requirements described in this STATEMENT OF WORK. If firm has knowledge of and experience working in Kenya and Malawi, and experience with assignments of a similar nature that should be described. b) Capabilities in the timely mobilization, management and training of short-term technical assistance experts and teams. c) Organizational systems and procedures are adequate related to: personnel policies and ability to comply with recruitment, travel policies; financial management; project management; contract administration; progress reporting; and other areas in order to successfully comply with contract requirements and accomplish the expected results. Part 2. Past Performance: The Bidder shall submit a list of all current evaluation contracts and subcontracts and those completed within the last three (3) years that are similar in size, scope and complexity to this STATEMENT OF WORK. Additionally, for the three most recent similar contracts, the Bidder shall provide the client’s name, contract number, contract value, current postal and e-mail addresses, and telephone number for a currently available point of contact. The list of current and previous contracts/sub-contracts shall be attached as an annex to the Technical Proposal, and will not count against the page limit. 12 VII. DELIVERABLES 1. Participation in an evaluation kick-off meeting. 2. Beforethe commencementoffieldwork, a detailedInception Report, work plan, andmethodologiestobeused for evaluation. 3. Before the commencement of fieldwork, a refined set of plans (tools and all instruments) to be used for quantitative and qualitative evaluations and observation checklists. USDA and Global Communities will retain ownership of instruments used for interviews, surveys, and any other data collection efforts developed by the Contractor. 4. Weekly phone updates to Global Communities HQ M&E team and AIMS COP and M&E Manager on the progress of the evaluation throughout the entirety of the performance period. 5. Two in-person validation workshops, one per country, validation workshop with key AIMS program and technical staff, and other relevant stakeholders. DC-based teams will join the workshops online. Global Communities will assess the COVID-19 situation in the field and adapt the plan to ensure our staff and partners' health and safety. 6. The final presentation of evaluation findings and recommendations to Global Communities and USDA stakeholders. 7. Submission of a first and second draft report with relevant attachments and files. The report should represent a thorough analysis and presentation of data and findingsrelated to the relevance, efficiency, effectiveness, and sustainability of AIMS, from which the evaluators identify key lessons learned and recommendations for strengthening the project over its remaining life. Evaluation findings should be presented as analyzed facts linked to evidence. 8. Submission of a final report (and raw data set in Excel-compatible software)). Data should be appropriately coded and be accompanied by analysis, guidance on all evaluation methods, sampling designs, survey instruments/questionnaires, and related instruments used in the evaluation. 13 VIII. REPORTING The report, not to exceed 50 pages (not including annexes), should be concise and to the point, utilizing charts, graphs, and diagrams where appropriate. In producing the evaluation report, the evaluation team will link findings to the data analyzed (all findings must be evidence-based). Those links must be made clear in the report. Additionally, the conclusions will clearly relate to the findings. It may require back and forth consultation, payment will not be rendered until the report is approved. The following table displays requirements for the final report: Final Report Requirements Report Length Maximum of 50 pages, excluding the Table of Content and Annexes Executive Summary Include a 3 to 5-page Executive Summary that provides a brief overview of the evaluation purpose, project background, evaluation questions, methods, findings, and conclusions. Research Questions Address all evaluation questions in the SOW Research Meth ods and Limitations • Explain the evaluation methodology in detail. • Disclose evaluation limitations, especially those associated withthe evaluation methodology (e.g. selection bias, recall bias,etc.). NOTE: A summary of methodology can be included in the body of the report, with the full description provided as an annex. Evaluation Findings • Present findings covering AIMS relevant countries, Kenya and Malawi, as analyzed facts, evidence, and data supported by strong quantitative and/or qualitative evidence and not anecdotes, hearsay, or people’sopinions. • Include findings that assess end-lines on males and females. Recommendations • Support recommendations with specific findings. • Provide recommendations that are action-oriented, practical, specific, and define who is responsible for the action. • Provide recommendations for midcourse corrections, if needed. Annexes Include the following as annexes, at a minimum: • Scope of Work. • Full description of evaluation methods. • All evaluation tools(questionnaires, checklists, discussion guides, surveys, etc.). • A list ofsources of information (key informants, documentsreviewed, other data sources). • Indicator tables comparing progress at midterm to baseline valuesand targets Quality Control Assess deliverables for quality by including an in-house peer technical review with comments provided to the evaluation team. ANNEX D: LIST OF INDICATORS TRACKED VIA PERFORMANCE MONITORING PLAN Note: The PMP has been modified from project start-up and baseline and is significantly shorter than the original. Per the M&E Plan approved by USDA on August 16, 2019, the evaluation firm will use the yellow-highlighted list of indicators for this end-line evaluation. This assessment will focus on Objective 1 and Loan Guaranty facility utilization and performance between October 2019 and September 2021. • Indicator # 1: Value of sales ($) by project beneficiaries • Indicator # 2: Volume of commodities (metric tons) sold by project beneficiaries. • Indicator # 3: Value of Loans provided as a result of USDA assistance. • Indicator # 4: Number of loans disbursed as a result of USDA assistance. • Indicator # 5: Value of new public and private sector investment leveraged as a result of USDA assistance. • Indicator # 6: Value of agricultural and rural loans. • Indicator # 7: Number of individuals receiving financial services as a result of USDAassistance. • Indicator # 8: Percentage of non-performing Loans advanced under the LGF assistance. • Indicator # 9: Totalnumberofindividualsbenefitingdirectlyas a resultofUSDA. • Indicator # 10: Totalnumberofindividualsbenefiting indirectlyas a resultofUSDA. • Indicator # 11: Number of SMEs,including farmers,receiving BDS from USG-assisted sources. • Indicator#12:Number of individuals who have received short-term agricultural sector productivityorfood securitytrainingas a resultofUSDAassistance • Indicator # 13: Number of private enterprises, producer organizations, water users associations, women’s groups, trade, and business associations, and CBOs that applied improvedtechniquesandtechnologiesasa result ofUSDAassistance. • Indicator # 14: Numberof jobs attributed to USDA assistance • Indicator # 15: Number of ICT based marketing services supported. • Indicator # 16: Numberoftargetbeneficiariesaccessing supportedmarketingservices. • Indicator # 17: Number of B2B sessionssupported. • Indicator # 18: Number of B2B participants. • Indicator # 19: Numberof SMEs thathavesignedagreements/contractsestablished betweenbuyersandsellersas a resultofUSDAassistance(custom). • Indicator # 20: Number of new agreements/contracts established between buyersand sellers as a result ofUSDA assistance (custom). • Indicator # 21: Valueofnewagreements/contractsestablishedbetweenbuyersand sellers ($). • Indicator # 22: Percentofsalesundercontracts/agreementsamongtargetbeneficiaries. • Indicator # 23: Number of public-private partnerships formed as a result ofUSDA assistance. ANNEX E: Project-Level Results Framework1 1 The End-line evaluation will only assess activity 1 and 2 under IR: Increased use of Financial Services (FFPr 2.2.3.1, 2.3.1.1) ANNEX F: Activity Description Activity Who are the intended participants? Individuals and organizations Where the Activity will be implemented? What is the nature of the activity? Describe the activity being performed What is the intended outcomes of the activity? Guarantee Facility Agreement (GFA) training for Ecobank/I&M bank staff Bank staff involved in agribusiness loan underwriting process including, relationship officers, heads of agribusiness, branch managers, loan recovery team, legal staff and business development staff Activity targets current EALGF banks i.e. Ecobank Malawi and I&M Bank Kenya This is an orientation session for better understanding of the guaranty facility agreement provisions to facilitate compliance. The session outlines the roles of each of the three parties to the GFA i.e. OPIC, Global Communities and Bank for better administration of the facility Increased utilization of EALGF through establishing a clear EALGF assessment process. It is also meant to impart the participants with requisite cash flow assessment skills. Key areas discussed include: loan eligibility and origination process, loan assessment, reporting, recovery process, need for GFA compliance, guaranty availability and opportunities offered by guaranty LGF training for AIMS team AIMS team involved in the guaranty administration including the Trade Specialists and M&E Officer Malawi and Kenya This is a training on the guaranty requirements. It builds competences on EALGF assessment skills and overall understanding of the guaranty requirements in a way that guarantees imparting acquired skills by participants to partnering EALGF banks Better management of the EALGF banks partnership. Enhanced loan assessment skills to facilitate onboarding credit worthy loans on the guaranty managing credit risk. Increased utility of the guaranty by effectively passing on value and opportunity the guaranty presents to the bank for buy in. Develop staff skills to improve on staff morale by sharpening their current competencies Loan assessment visits and follow up This involves visiting bank borrowers at their business who are EALGF prospects. Participants in the visit include bank relationship officers and business development staff and AIMS Credit and Trade staff Malawi and Kenya across regions where the ASMEs operations are based. ASME business visits to inform the loan assessment process and validate information furnished to the bank and AIMS for purposes of a quality loan assessment that is accurately representative of the business operations Gathering business information from borrower, comparing observed site business operations with data already furnished by prospective borrower. Have a one on one discussion with borrower that leads to evaluation of character, ability, means, purpose loan repayment and loan structuring needs leading to a quality loan Activity Who are the intended participants? Individuals and organizations Where the Activity will be implemented? What is the nature of the activity? Describe the activity being performed What is the intended outcomes of the activity? SME sensitization forums with Ecobank & I&M Bank on new opportunities EALGF eligible ASMEs, bank relationship and business development staff, AIMS staff, Business Advisory Service Providers (BASPs) and other agribusiness finance value chain stakeholders Malawi and Kenya across agribusiness rich regions EALGF banks pitch their agribusiness finance products to SMEs to arouses their interest to apply for loans. BASPs and other stakeholders complement banks by addressing gaps that lead to ASMEs' loan eligibility Increased utility of EALGF. Linkages to other industry actors including BASPs. Enhanced ASMEs' capacity to qualify them for loan consideration. Position bank as an agribusiness oriented financial institution and help scaling up of agribusiness portfolio. BASP consultative Meetings BASPs, EALGF banks' relationship and business development staff and AIMS staff Malawi and Kenya A forum involving AIMS prequalified BASPs to discuss expected AIMS support under the cost sharing approach where BASPs will work with ASMEs to address capacity gaps, mostly lack of financial reports would help meet EALGF loan access requirement Development of financial reports in conformity with standards outlined on the GFA. Consideration of expression of loan borrowing interest to ASMEs that would have been disqualified on account of incapacity to demonstrate loan repayment capacity. Ultimately encourage and influence adoption of best practices in business management by ASMEs that improve their eligibility for loans in future BASP support to SMEs ASMEs with potential to request for loans under LGF; SMEs meeting AIMS standards of annual turn￾over range of $5000- $1million. Malawi and Kenya across regions where the ASMEs operations are domiciled Development of financial reports to support EAGLF eligibility. The service is to those ASMEs that the bank's preliminary analysis demonstrates a heightened probability of capacity to service loan being sought and missing information would provide needed assurance in making the decision Compliance with GFA provisions on provision of historical financial reports and projections for the tenure of loan facility. Basis of EALGF loan eligibility evaluation through ASMEs operations and repayment capacity assessment. Bank adoption of the ASME support process through engagement of BASPs EALGF SMEs capacity development on market dynamics for improved finance access EALGF beneficiaries and prospective EALGF ASMEs, BASPs, EALGF bank relation and business development staff and AIMS staff Malawi and Kenya EALGF beneficiary ASMEs receive guidance by BASPs through a forum discussing key market strategies that can enhance ASMEs access to finance Increased access to finance through SMEs improved capacity to negotiate market contracts/agreements Activity Who are the intended participants? Individuals and organizations Where the Activity will be implemented? What is the nature of the activity? Describe the activity being performed What is the intended outcomes of the activity? Market linkage events ASMEs (buyers & sellers), EALGF bank relation and business development staff, AIMS staff, BASPs and other agribusiness stakeholders Malawi and Kenya Linkage forum targeted at identifying new markets for SMEs. Buyers and sellers are identified and brought together in a forum that explores new partnerships to facilitate trade Growth in agriculture trade and increased access to finance by participating ASMEs. Upscaling of EALGF's banks agribusiness portfolio through identification of new ASMEs that leads to increased utility of the guaranty. AIMS End Line Assessment 46 Annex 2: Detailed Methodology Description Data Sources & Collection Methods A mix of qualitative and quantitative data was collected from project stakeholders. The sources and collection methods used are presented here in turn. The research was mainly conducted in English for all participants. However, researchers for the ASME survey were also fluent in Swahili or Chichewa and conducted the interviews in these languages as and when it was needed for specific respondents. Potentially challenging elements to translate in the ASME survey instrument were translated in advance by group consensus among the data collectors to ensure consistency. Desk Research The evaluation team completed a desk review of the program documents shared by GC/AIMS during the inception period. These provided an understanding of the broader AIMS Monitoring and Evaluation (M&E) approach, in particular the M&E plan, baseline, and outcome (mid-term) assessments. The semi-annual reports to USDA and quarterly reports to DFC provided an overview of the activities conducted and results achieved to date, plus the challenges faced over the course of the program and how it has responded to each. As well as being useful background understanding for the evaluation, knowledge of these documents also makes clear what information has already been reported to USDA/GC, and therefore did not need to be repeated in the evaluation. Funding Partners Interviews were held with both USDA and DFC. The interview with USDA covered the objectives, rationale, and key areas of interest from the evaluation from their perspective. Two separate interviews with DFC officers were conducted to better understand the approach taken for the EALGF, their rationale, the intended performance and other factors that have influenced design and delivery. These interviews took the form of guided discussions by the lead consultant. Program Implementors Several interviews were held with the in-country GC/AIMS project staff for both Malawi and Kenya. These interviews were a mixture of semi-structured interviews and guided discussions, with points of interest raised in earlier sessions followed up in detail during subsequent sessions. EALGF Partner Banks The two commercial banks were pivotal institutions in the project, and the evaluation team sought to interview their key team members in depth. Research with the EALGF partner banks involved a semi-structured interview that aimed to capture the outcomes observed from their participation in the EALGF and then work back to identify enabling and or constraining factors that contributed to these outcomes. BASPs A semi-structured interview guide was developed, and interviews were held with all six BASPs engaged by AIMS during the NCE period. These focused on their engagement with AIMS and ASMEs under the program, to understand the impact on both their business and the businesses of the ASMEs they have serviced. AIMS End Line Assessment 47 ASMEs Quantitative (Phone) Survey Quantitative data was collected from a sample of ASMEs that AIMS has engaged with during the NCE using a telephone survey. This was the agreed approach due to the risk posed by Covid-19, which was mitigated by limiting the number of people that the research team had face-to-face contact with. The sample included three categories of ASMEs: those that have successfully borrowed under the EALGF; those that that applied for a loan but were not (yet) successful; and, those that have not (yet) applied. It was agreed that the survey should be fully quantitative for a number of reasons. A quantitative instrument makes it possible to extract useable insights, while also keeping the time required manageable for the interviewee. This is especially important for phone interviews give the challenges of mobile battery, signal, and inconvenience. Furthermore, owner-managers of ASMES are very pressed for time and may not be willing to engage in a long call, especially if they have not ‘benefitted’ from the program in the form of receiving a loan. The research team called the potential respondent first to arrange a time, then called back to conduct interview, unless the respondent wished to go ahead on first contact. Consent was sought before proceeding and the interview terminated if not given. Responses were recorded using a pre-programmed questionnaire on tablets using the Open Data Kit (ODK) software. Completed questionnaires were uploaded daily from the data collector’s tablets to a central server on the ODK platform. This data was checked daily by the research manager for quality and consistency. The completed output was a data table in a .csv file that could be imported directly into SPSS for cleaning and analysis. Instrument Testing and Training Desk testing and field testing of the instruments was conducted by the teams in both countries prior to training data collectors. Training covered the AIMS project, ASME finance, sampling protocols (including substitution protocols), the instrument, use of ODK/tablets, research ethics, communication protocols and quality controls. The training involved practice with researchers interviewing each other, then some ‘live’ practice interviews by phone of ASMEs that were not in the final sample. Substitution Protocol At least three attempts will be made to reach each of the ASMEs included in the proposed sample using the contact details provided. If the contact details provided do not appear to be working, then the research team will ask AIMS for assistance in reaching that organization. If after both of these steps have been taken and contact is still not possible, or if the respondent is reached, but is unwilling to participate in the survey, then a substitution will occur. The aim will be to substitute the target ASME with another from the same sub-population, which will be selected at random. As all borrowers and applicants that were declined or deferred by the banks are included in the target sample already, it will not be possible to substitute ASMEs in these sub-populations. For these substitutions, an alternative respondent will be selected at random from the unsampled population, which will be made up of those who did not apply (and/or applied but dropped out for Kenya only). AIMS End Line Assessment 48 Focus Group Discussions The quantitative survey was supplemented by qualitative (virtual) Focus Group Discussions (FGDs) with selected ASMEs. These discussions focused on more nuanced insights into topics such as usefulness of BASP support, outcome attribution and recommendations for future similar projects and products. ASMEs were grouped into non-competing businesses to allow them to speak freely. Evaluation Questions and Information Sources The evaluation addressed questions of the program’s relevance, effectiveness, impact, efficiency, sustainability and identified lessons learned and recommendations. A set of high-level research questions that address each of these factors was included in the SoW and were reviewed and revised during the inception phase. The final list of evaluation questions and the primary sources of information for answering each is presented in Table 7 below. AIMS End Line Assessment 49 Table 7: Research Questions and Information Sources Focus Areas for End-line Evaluation Proposed Questions Main Information Source(s) The relevance of the AIMS Program • Is the concept and design of AIMS EALGF suitable to the overall market situation and challenges faced by ASMEs in Kenya and Malawi? > Primary doc Review - compare to BL suggestions and other documented ways program has adapted to changes > Secondary docs on ASME challenges in Ke & Mw or relevant policy papers > Discussions with AIMS team > Bank Interviews on agri-lending market > ASME survey and FGDs • To what extent were ASMEs’ needs for finance met? > ASME survey and FGDs Program Efficiency • Has EALGF implementation been able to adhere to stated plans (i.e. work-plans, action-plans, results framework, and budget). If not, what internal challenges did the program face? > Discussions with AIMS team > Discussion with DFC > Primary doc review (Outcome Assessment & others) > Partner and non-partner bank interviews > Triangulate with participants > Discussions with AIMS team > Discussion with DFC > Primary doc review > Partner and non-partner bank interviews • What were the main external challenges in the implementation of the EALGF and? • What are the best/most efficient practices that contributed to program successes? How did the program adapt to insights from monitoring and evaluation conducted before and during implementation • How well is EALGF coordinating with other key actors (development actors, government, and private sector) working on similar or complementary programming? What could have been done to improve coordination? > Discussions with AIMS team > Primary doc review > (Other stakeholders such as Govt. if relevant) Program Effectiveness • What factors encouraged or discouraged banks from using the facility? > Discussions with AIMS team > Partner and non-partner bank interviews AIMS End Line Assessment 50 • Were Technical Assistance / support interventions with the banks effective in increasing the use of the facility? If so, how? And if not, why not? What would have been more effective?' > Discussions with AIMS team > Partner bank interviews • How effective was the Technical Assistance provided via BASPs to ASMEs in increasing their readiness to receive loans? > Discussions with AIMS team > Partner bank interviews > BASP interviews > ASME survey and FGDs Project Impact • What was the impact of EALGF-covered loans received by the ASMEs? (e.g. additional sales, employment, new business) > ASME Survey (Borrower version) • To what extent can changes in the outcomes of interest, including SME growth and trade, be attributed to the project? > Partner bank outcome harvesting (lending) > ASME FGDs for business growth outcomes (triangulated with survey insights) > Secondary docs on related programs or other business environment changes • What unintended consequences of the project were produced, positive or negative? > All participant interviews > Discussions with AIMS team Program Sustainability • To what extent have banks under the EALGF changed their internal resource allocation, adopted new lending policies or procedures or developed products that increase access to credit for agribusinesses? > Partner bank interviews > Discussions with AIMS team What barriers exist that may prevent the full adoption of policies, procedures, and product lines to meet ASMEs' needs? > Partner bank interviews > Discussions with AIMS team Lessons Learned Recommendations • What has worked especially well under EALGF and why? > All sources • What has not worked well and why? > All sources • How have experiences and results differed between Kenya and Malawi? > All sources • What are preliminary lessons learned from implementing AIMS EALGF interventions targeting increasing financial inclusion to ASMEs? > All sources AIMS End Line Assessment 51 Annex 3: Information Sources Table 8: Key Informant Consulted Type Organization Country Partner Bank Ecobank Malawi Partner Bank I&M Bank Kenya BASP OB advisory Kenya BASP Afribusiness LLP Kenya BASP One Africa Advisory Limited Kenya BASP Sweet and Dried Kenya BASP Equip Consulting Group Malawi BASP ICT & Local Development Consultancy Malawi Program Team AIMS - Program Director Malawi Program Team AIMS - Credit Director Kenya Program Team AIMS - M&E Manager Malawi Program Team AIMS - Credit Specialist Malawi Program Team AIMS - Credit Specialist Malawi Program Team AIMS - Credit Specialist Kenya Program Team Global Communities - M&E Lead USA Program Team Global Communities - Program Manager USA Program Team Global Communities - Senior Director USA Guarantor DFC USA Guarantor DFC USA AIMS End Line Assessment 52 List of ASMEs Surveyed Table 9: ASME Survey Sample List - Kenya No. Category Organisation Business Type Country 1 Applicant Imenti Community Based Organization Cooperative Kenya 2 Borrower Batian Nuts Limited Exporter Kenya 3 Applicant Spring Fresh Exporter Kenya 4 Applicant Gem Grain Amaranth Millers Agro-processor Kenya 5 Non￾Applicant Keringet Food Ltd Agro-processor Kenya 6 Borrower Bravoken Fresh Exporter Kenya 7 Applicant Sorghum Pioneer Agencies Wholesaler/Retailer Kenya 8 Applicant Cherobu Dairy Cooperative Kenya 9 Non￾Applicant Agrofoods Trading and Consultancy Group Limited Wholesaler/Retailer Kenya 10 Non￾Applicant Western Fresh Industries (WEPI) Exporter Kenya 11 Non￾Applicant MELSOPs Dairy Limited Cooperative Kenya 12 Applicant Kamumo products limited Agro-processor Kenya 13 Non￾Applicant Central Vet Supplies Input supplier Kenya 14 Non￾Applicant Sunbelt Farm Input supplier Kenya 15 Applicant Alpharxerd Enterprises Led Agro-processor Kenya 16 Applicant Mowlem Fish and Allied Products limited Wholesaler/Retailer Kenya 17 Applicant By Grace Farm Limited Agro-processor Kenya 18 Non￾Applicant Suka Cooperative Cooperative Kenya 19 Borrower Faulu Flour Mills Limited Agro-processor Kenya 20 Applicant Charisma Stores Agro-processor Kenya 21 Applicant Jampalink Company Agro-processor Kenya 22 Non￾Applicant Ex-lewa dairy farmers cooperative society Cooperative Kenya 23 Applicant Eunidrip irrigation Input supplier Kenya 24 Applicant Avoveg Health Exporter Kenya 25 Applicant Mission S. H. G Dairy Cooperative Kenya 26 Non￾Applicant Sego Gaa Poultry Farm Wholesaler/Retailer Kenya 27 Non￾Applicant Maragrow Limited Exporter Kenya AIMS End Line Assessment 53 No. Category Organisation Business Type Country 28 Non￾Applicant Smart Shamba Input supplier Kenya 29 Applicant Kongeluke Cooperative Cooperative Kenya 30 Applicant Meru Central Coffee Union Agro-processor Kenya 31 Non￾Applicant Smart Logistics Solutions ltd Wholesaler/Retailer Kenya 32 Non￾Applicant Ahavah Farms Wholesaler/Retailer Kenya 33 Non￾Applicant Procurement Technical Agribusiness Centers (PTAC) Input supplier Kenya 34 Applicant Global organic chicken processors ltd Agro-processor Kenya 35 Applicant Paksons Enterprises ltd Input supplier Kenya 36 Applicant Ibunga Enterprises Wholesaler/Retailer Kenya 37 Non￾Applicant Ambango Fresh Exporter Kenya 38 Non￾Applicant African Farms and Markets Ltd Input supplier Kenya 39 Non￾Applicant Kiplombe F.C.S Cooperative Kenya 40 Non￾Applicant White Bird Millers Agro-processor Kenya 41 Non￾Applicant Soy Afric Agro-processor Kenya 42 Applicant Grey Consolidated Ltd Exporter Kenya 43 Non￾Applicant ISQCL ltd Exporter Kenya Table 10: ASME Survey Sample List - Malawi No. Category Organisation Business Type Country 1 Borrower Chitsanzo Cooperative Cooperative Malawi 2 Applicant Happier Foods Limited Wholesaler/Retailer Malawi 3 Borrower Global Seeds Agro-processor Malawi 4 Borrower RW Traders Wholesaler/Retailer Malawi 5 Applicant Worthy General Supplies Input supplier Malawi 6 Borrower Mgwirizano Copperative Cooperative Malawi 7 Borrower Legume D'lite Agro-processor Malawi AIMS End Line Assessment 54 No. Category Organisation Business Type Country 8 Applicant Walotanji Investment Agro-processor Malawi 9 Applicant Hara Cooperative Cooperative Malawi 10 Applicant Mawelera Enterprise Input supplier Malawi 11 Non￾Applicant Likasi Cooperative Cooperative Malawi 12 Applicant Donija Investments Wholesaler/Retailer Malawi 13 Borrower Mwandama Cooperative Cooperative Malawi 14 Applicant Chilumba Farm Agro-processor Malawi 15 Applicant Multiseed Company Agro-processor Malawi 16 Non￾Applicant Aliko Trading Wholesaler/Retailer Malawi 17 Non￾Applicant Fairway Enterprise Exporter Malawi 18 Applicant Agwenda Investments Input supplier Malawi 19 Non￾Applicant Fresh Point Agro-processor Malawi 20 Applicant Double Estella Investments Exporter Malawi 21 Applicant Chikwendeni General Dealers Input supplier Malawi 22 Borrower Blacpad Investments Agro-processor Malawi 23 Applicant North East Foods Exporter Malawi 24 Applicant Mother's Choice Agro-processor Malawi 25 Borrower Tatin Trading Input supplier Malawi 26 Applicant Kambewu Organic Fertilizer Investments Input supplier Malawi 27 Applicant Berca Investments Agro-processor Malawi 28 Applicant Chatangwa Enterprises Wholesaler/Retailer Malawi 29 Borrower Yontanga General Dealers Wholesaler/Retailer Malawi 30 Borrower JAT Investments Agro-processor Malawi 31 Applicant Hortinet Foods Limited Agro-processor Malawi 32 Non￾Applicant Eden Group Agro-processor Malawi AIMS End Line Assessment 55 No. Category Organisation Business Type Country 33 Non￾Applicant Hewe Agrodealers Input supplier Malawi 34 Non￾Applicant First Irrigation House Wholesaler/Retailer Malawi 35 Non￾Applicant Triple G Enterptise Input supplier Malawi 36 Applicant Tanthwe Enterprise Agro-processor Malawi 37 Applicant Takaaful Cooperative Society Ltd Wholesaler/Retailer Malawi 38 Applicant Nyanja cooperative Cooperative Malawi 39 Applicant Freight and General Dealers Wholesaler/Retailer Malawi 40 Borrower Lonnie Livestock Farm Wholesaler/Retailer Malawi List of Documents Reviewed 1. AIMS Program Overview and Results Slide Deck 2. Program Award Cooperative Agreement executed between Global Communities and USDA. 3. Semi-annual Performance Reports submitted by AIMS to USDA 4. AIMS Monitoring and Evaluation Plan 5. Revised Performance Monitoring Plan submitted to USDA 6. Baseline Report (including Instruments) submitted to USDA 7. Outcome Assessment submitted to USDA (This assessment replaced the mid-term evaluation and evaluated the performance of Objectives 2, 3 and 4.) 8. EALGF Quarterly reports to DFC 9. Credit Assessment Documentation for Faulu Flour Mills, Kenya 10. Bridging the Financing Gap: Unlocking the Impact Potential of Agricultural SMEs in Africa, Summary Report, Aceli Africa 11. Outcome Harvesting, by Ricardo Wilson-Grau and Heather Britt, May 2012 12. Testing Tools for Assessing Systemic Change: Outcome Harvesting, USAID