September 2016 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by the USAID/Jordan Monitoring and Evaluation Support Project, implemented by Management Systems International, a Tetra Tech Company. EVALUATION Final Performance Evaluation of the Jordan Loan Guarantee Facility (JLGF) Final Evaluation Report – Jordan Loan Guarantee Facility iii FINAL PERFORMANCE EVALUATION REPORT JORDAN LOAN GUARANTEE FACILITY (JLGF) 4 September 2016 Prepared by: Dr. Miguel Cabal, Team Leader, MSI Sam Hargadine, MSI Hisham Yasin, MSI Contracted under: AID-278-C-13-00009 USAID/Jordan Monitoring and Evaluation Support Project (MESP) Cover Photo: Ma’daba, Jordan – Innovation Packaging Co., a Jordanian small enterprise, obtained a loan under the Jordan Loan Guarantee Facility in 2013. Photo Credit: Abed Shmais DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Final Evaluation Report – Jordan Loan Guarantee Facility iv CONTENTS Acronyms................................................................................................................................. vi Executive Summary............................................................................................................... vii Introduction..............................................................................................................................1 Background...............................................................................................................................2 Economic and Political Environment................................................................................................................. 2 JLGF Approach ....................................................................................................................................................... 5 Evaluation Methodology ........................................................................................................12 Findings and Conclusions.......................................................................................................16 EQ1: How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs, and the tailored/generic in-bank training) in achieving the activity’s objectives?...............16 EQ2: What are the impacts of the activity on the key target beneficiaries? .........................................31 EQ3: What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why?..........................................35 EQ4: What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance?..............................................................................................................................37 Recommendations .................................................................................................................38 Annexes...................................................................................................................................41 Annex A: Evaluation Statement of Work.......................................................................................................42 Annex B: Evaluation Design with Instruments..............................................................................................47 Annex C: References and Works Cited.......................................................................................................106 Annex D: Implementation Timeline...............................................................................................................107 Annex E: Performance Mangement Indicators............................................................................................108 Annex F: Statistical Annex ...............................................................................................................................111 Final Evaluation Report – Jordan Loan Guarantee Facility v TABLE OF TABLES Table 1: Jordan annual GDP per capita growth and FDI inflows (annual percent of GDP) ..................... 3 Table 2: Comparison of OPIC and CBJ enterprise definitions........................................................................ 6 Table 3: Loan Principle and OPIC Commitment by Partner Bank ................................................................. 7 Table 4: JLGF Original Assumptions Compared with Understandings at the end of FY 2013..............11 Table 5: JLGF Performance Evaluation: Data Collection, Methods and Instruments by Respondent Type....................................................................................................................................................................13 Table 6: Bank partner average loan principle.....................................................................................................18 Table 7: SME and start-up owner level of satisfaction with JLGF technical assistance ............................20 Table 8: GFA Signing Date by Institution............................................................................................................24 Table 9: Proportion and value of loans year on year going to fixed assets and working capital...........28 Table 10: Average expected jobs generated by loan type ..............................................................................28 Table 11: Survey responses regarding JLGF contribution to financial access ............................................32 Table 12: Survey responses regarding perceptions of permanence to business practices.....................37 Table 13: JLGF SME Beneficiaries Expected to be Included in the Evaluation by Data Collection Method...............................................................................................................................................................53 Table 14: JLGF Performance Evaluation: Data Collection Instruments by Respondent Type ...............53 TABLE OF FIGURES Figure 1: Proportion of SME loan applications by bank type............................................................................ 4 Figure 2: JLGF theory of change.............................................................................................................................. 6 Figure 3: Number and Concentration of Active and Settled SME Loans by Governorate....................... 8 Figure 4: Prominent economic sectors in the JLGF portfolio.......................................................................... 9 Figure 5: Distribution of overall lending by economic sector........................................................................10 Figure 6: Cumulative Number of Loans Generated, FY 2015-2016.............................................................17 Figure 7: Cumulated Value of Loans Generated, FY 2015-2016...................................................................17 Figure 8: Total number (left axis) and value (right axis) of loans by bank partner...................................18 Figure 9: Yearly average loan principle (US$) by bank activity level ............................................................19 Figure 10: Constraints to enterprise growth by gender of owner...............................................................21 Figure 11: Percentage of SME Workshop Attendees Comprised of Women...........................................23 Figure 12: Percentage of loan guarantee applications exhibiting deficiencies across 16 points of analysis................................................................................................................................................................26 Figure 13: Loan guarantee approval rate by year..............................................................................................27 Figure 14: Cumulative Number of Persons Receiving New or Improved Employment..........................33 Final Evaluation Report – Jordan Loan Guarantee Facility vi ACRONYMS CBJ Central Bank of Jordan EQ Evaluation Question FDI Foreign Direct Investment FY Fiscal Year GDP Gross Domestic Product GFA Guarantee Facility Agreement IBS Institute for Banking Studies JEDCO Jordan Enterprise Development Corporation JLGF Jordan Loan Guarantee Facility JLGC Jordan Loan Guarantee Corporation JOD Jordanian Dinar JWIC Jordan Women Industrial Community MESP USAID/Jordan Monitoring and Evaluation Support Project MSI Management Systems International OPIC U.S. Overseas Private Investment Corporation SME Small and Medium-sized Enterprise SOW Statement of Work USAID United States Agency for International Development Final Evaluation Report – Jordan Loan Guarantee Facility vii EXECUTIVE SUMMARY INTRODUCTION Activity Summary This report presents findings, conclusions, and recommendations from the final performance evaluation of the Jordan Loan Guarantee Facility (JLGF). JLGF is a five-year cooperative agreement between the United States Agency for International Development (USAID), the U.S. Overseas Private Investment Corporation (OPIC), and Global Communities. USAID awarded Global Communities approximately US$9.3 million in October 2011 to provide technical assistance to partner banks and Small and Medium-sized Enterprises (SME) to increase access to finance for the Jordanian SME sector. OPIC provided the $250 million guarantee facility. JLGF’s approach therefore is to lower the risk of lending to Jordanian SMEs through the loan guarantee while also building the capacity of banks to properly review and approve qualified SME loan applications in line with international best practices. Evaluation Purpose This report responds to the USAID Statement of Work (SOW), provided as Annex A, requesting the USAID/Jordan Monitoring and Evaluation Support Project (MESP)1 to conduct an evaluation of JLGF performance across the activity’s FY 2012 through FY 2016 term.2 The purpose of this evaluation is to provide the Agency with findings and strategic recommendations relating to (1) the effectiveness of JLGF interventions, mainly the technical assistance that USAID/JLGF provided to banks and SMEs; and (2) lessons learned and best practices to be utilized by USAID in the future. This report is meant to inform USAID’s design and implementation decisions for similar future programming. Evaluation Questions Effectiveness 1. How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? 2. What are the impacts of the activity on the key target beneficiaries? Sustainability 3. What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? Learning 4. What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance? BACKGROUND Economic and Political Environment According to activity documents, secondary data sources, and in-depth interviews with finance sector stakeholders, the enabling environment for economic growth and employment in Jordan has been gravely impacted by regional instability. Table 1 in the body of the report illustrates declining Gross Domestic Product (GDP) growth rates and Foreign Direct Investment (FDI) levels over the 1 MESP is implemented by Management Systems International, a Tetra Tech Company. 2 U.S. government fiscal years begin October 1 and end September 30. Final Evaluation Report – Jordan Loan Guarantee Facility viii past decade.3 Beginning on Page 2, this report provides greater coverage of how the conflicts in Iraq and Syria and political revolution across the region have impacted the Jordanian economy. SME Sector Approximately 98 percent of private businesses in Jordan fall into the category of SMEs,4 yet bank lending to SMEs constituted only 11 percent of total lending in 2011 when JLGF began. 5 Over the intervening years and due to the saturation of the market for larger firms, banks are pivoting towards SMEs as a source of growth. Of the 25 banks operating in Jordan, 15 have confirmed departments specializing in SME financing.6 However, despite wide-spread awareness of the potential of the SME market, collateral based credit underwriting practices skew the approval rate of loan applications causing otherwise qualified SMEs to be denied access to finance. JLGF Approach JLGF was designed in part to address this credit underwriting gap. Figure 2 in the body of the report outlines the theory of change across JLGF’s intervention arms. Two objectives organize the approach. The first is designed to mobilize bank lending through strategic partnerships of banks and sector stakeholders, as well as build capacity of SMEs to comply with bank and OPIC lending requirements. Objective 2 is concerned with building bank capacity to sustainably lend to SMEs. This assistance takes the form of workshops and trainings tailored to close partner bank capacity gaps relating to credit underwriting for SMEs, as well as the development of an SME lending diploma through the Amman based Institute for Banking Studies (IBS). EVALUATION METHODOLOGY This evaluation used a mixed method approach to answer the evaluation questions. Secondary data was sourced from JLGF reports and internal databases, as well as literature and statistics relevant to the Jordanian SME sector. Qualitative and quantitative primary data were collected from JLGF, its stakeholders, banks, government institutions, SMEs, and relevant SME associations. Further description of this evaluation’s timeline, data sources, collection procedures, and analysis plan begins on Page 12. FINDINGS AND CONCLUSIONS This executive summary shares conclusions for the top-level questions outlined above. Findings and conclusions for all of the questions and sub-questions are provided beginning on Page 16. Question 1: How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? JLGF holds formal partnerships with seven Jordanian banks. These seven banks were able to utilize the loan guarantee facility based on the provisions of their Guarantee Facility Agreement (GFA). Of these seven banks, the three most active constituted roughly 70 percent of the number of loans guaranteed under the facility. The activity was notably successful at reducing the proportion of loan guarantee applications exhibiting deficiencies in analyzing the borrower’s (1) risk and mitigation factors, (2) market competitiveness, (3) strategy of differentiation, and (4) cash flow projections. The rejection rate of loan guarantee applications fell from 47 percent in 2012 to 2 percent in 2016. At the time of writing this report, $71 million has been leveraged through the facility for Jordanian SMEs, against a current combined obligated ceiling of $170 million across the seven banks. This 3 GDP: 4.9 percent in 2005 and -0.02 percent in 2015. FDI: 15.8 percent in 2005 and 3.4 percent in 2015 (as a proportion of GDP). 4 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. 5 Based on figures provided in Global Communities’ 2011 Technical Proposal. 6 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. Final Evaluation Report – Jordan Loan Guarantee Facility ix constitutes 42 percent utilization of the obligated ceiling and 28 percent of the maximum $250 million. It should be noted that the facility is designed to be demand-driven, therefore banks are not required to use it. Also, OPIC GFA criteria are strict, thereby limiting the number of eligible banks able to gain coverage through the facility. Seventy-five percent of the approximately 330 enterprises/start-up owners who received technical assistance were satisfied with the training they received; however, 37 percent of established business owners and 53 percent of start-up entrepreneurs said that training did not address the specific constraints to growth for their business. Outreach mechanisms to SMEs were haphazard which constrained the ability of the activity to tailor training to specific demographics or enterprise tiers. Question 2: What are the impacts of the activity on the key target beneficiaries? The activity’s key beneficiaries were banks and those that own/control SMEs. More than 90 percent of SME training participants said that they have made effective long-term or short-term changes to their business plan as a result of JLGF training. While owners/controllers from approximately 330 unique businesses participated in training, only 18 of these obtained a loan through the facility. This disconnect leads to two considerations: 1. The activity was ineffective at channeling SME training participants towards the facility; and yet, 2. More than 60 percent of SME participants said that the training increased their ability to access finance (i.e., through sources other than JLGF). JLGF was recognized by financial institutions as having high technical standards and the banks who were most active in their facility utilization benefitted the most from the technical assistance provided. Based on in-depth interviews and focus groups with bank employees, the institutionalization of changes in SME credit underwriting practices hinges on bank senior management perceptions regarding the value and risk of SME lending. JLGF recognized that more work will need to be done in order to institutionalize the culture shift achieved among these decision makers. JLGF trainers suggested gradually working more with senior managers, for instance, working with them on changing various forms associated with cash-flow based analysis for more junior employees to utilize. Taking this approach, it is suggested, would create buy-in from senior managers by incorporating their vision for their specific institution. Question 3: What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? JLGF contributions to the SME lending sector have not been sufficiently transferred to local partners, nor institutionalized within partner banks to the degree necessary for outcomes to be sustained. The OPIC facility, absent of USAID’s decision, will continue until 2022 under some bank GFAs. Should the JLGF activity end in September 2016, the servicing of this facility will end as well. There is currently no plan to transfer facility day-to-day management or the delivery of technical support to a local partner. If the activity is terminated without a proper wind-down, the reputation of USAID and OPIC is likely to be hurt in the eyes of the Jordanian finance sector. Question 4: What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance? JLGF’s three pronged approach was effective in the Jordanian context; however, outreach to SMEs could have been better targeted to direct potential clients to the facility. On the other hand, the activity was successful at improving the capacity of SMEs to access finance through sources other than the facility. Greater clarity in future activities relating to the aim (i.e., facility utilization or increasing access to finance generally) would likely streamline efforts and improve the scale of contribution these types of investments leverage to achieve development impact. Final Evaluation Report – Jordan Loan Guarantee Facility x RECOMMENDATIONS Recommendation 1 For a potential follow-on activity or for activities of similar scope, USAID should include in the activity’s design explicit enterprise targeting criteria, including gender-related and geographic targets, and articulate how outreach to these enterprises will be conducted and monitored. This may include establishing tiers of enterprise beneficiaries and developing separate outreach protocols for each type. Recommendation 2 For a potential follow-on activity, USAID should include in the design actions to gradually transfer ownership of technical assistance delivery to a local partner. Likely candidates for aspects of this include:  JLGC due to their extensive bank partnerships;  The CBJ due to their regulatory and policy role; and  SME Associations due to their potential capacity to raise awareness of the facility and provide trainings to enterprise owners/controllers Recommendation 3 For a potential follow-on activity, USAID should include in the design actions to pilot the gradual and responsible wind-down of JLGF credit underwriting reviews. This would likely start with the most active banks utilizing the facility. It is expected that this gradual wind-down would support long-term sustainability by testing the institutionalization of cash-flow based credit underwriting best practices among partner banks. Recommendation 4 For a potential follow-on activity, USAID should ensure that the servicer/implementing partner establishes dedicated units concerned with (1) outreach to enterprises and new partners, especially outside Amman; (2) managing relationships with bank senior managers, specifically addressing requests and concerns so as to improve senior-level buy-in; and (3) building institutional capacity within banks and partners targeted to takeover technical assistance components after the end of USAID support.7 Recommendation 5 Should the JLGF servicing agreement continue, USAID should revise the activity’s M&E plan in the following ways:  Include only goal level indicators relating to access to finance. For example, percent of bank credit lent to SMEs, likely utilizing the Jordanian definition.  Disaggregate all relevant indicators relating to the target beneficiaries.  Formally monitor credit underwriting changes within partner banks by monitoring the proportion of deficiencies in guarantee application requests.  Include indicators specifically designed to track performance towards sustainability targets. Recommendation 6 In an effort to better understand its target clients and provide tailored assistance, USAID should integrate into its activities of similar scope periodic assessments capturing SME characteristics and needs. Relatedly, stakeholder mapping would be useful to better understand needs and potential linkages with local partners and improve the likelihood of sustaining outcomes. This could take the 7 Or identify a separate entity that would be able to continue the delivery of trainings and develop training curricula. Final Evaluation Report – Jordan Loan Guarantee Facility xi form of conducting a longitudinal study of loans provided and could contribute to Agency and stakeholder learning about the durability of jobs created when Jordanian SMEs gain access to finance. Periodic assessments could also inform future efforts of similar scope by collecting SME perspectives relating to loan repayment, specifically why specific borrowers have required loan restructuring. Recommendation 7 Should start-ups and smaller enterprises remain a target beneficiary in future activities of similar scope, USAID should (1) contribute to stronger and more capable support networks, likely through existing enterprise associations (such as JWIC), and (2) implement follow-up procedures to ensure beneficiary needs are met, thereby tailoring additional support as businesses increase in size and sophistication. These two adaptions would improve the likelihood of sustaining outcomes by transferring JLGF’s current knowledge, capacity, and responsibility to capable enterprise associations and high-growth SMEs. Final Evaluation Report – Jordan Loan Guarantee Facility 1 INTRODUCTION ACTIVITY SUMMARY This report provides findings, conclusions, and recommendations based on the final performance evaluation of the Jordan Loan Guarantee Facility (JLGF). JLGF is a five-year cooperative agreement between the United States Agency for International Development (USAID), the U.S. Overseas Private Investment Corporation (OPIC), and Global Communities. Global Communities is the partner servicing the facility and implementing the activity, which began at the start of Fiscal Year (FY) 2012. The purpose of JLGF is to catalyze investment in Small and Medium-sized Enterprises (SMEs) by coupling partial loan guarantees with technical assistance to approved banking partners, non-bank financial institutions, and SME entrepreneurs.8 EVALUATION PURPOSE, AUDIENCE, AND INTENDED USES This report responds to the USAID Statement of Work (SOW), provided as Annex A, requesting the USAID/Jordan Monitoring and Evaluation Support Project (MESP)9 to conduct an evaluation of JLGF performance across the activity’s FY 2012 through FY 2016 term.10 The purpose of this evaluation is to provide the Agency with findings and strategic recommendations relating to (1) the effectiveness of JLGF interventions, mainly the technical assistance that USAID/JLGF provided to banks and SMEs; and (2) lessons learned and best practices to be utilized by USAID in the future. This report is meant to inform USAID’s design and implementation decisions for similar future programming. The audience for this report is expected to be: 1. USAID, specifically the Jordan Mission, but also those working on SME capacity building activities more broadly in the region and in Washington; 2. OPIC; 3. Jordanian stakeholders (e.g., those working within the government and banking sectors on SME development); and 4. The wider development community engaged in private sector development in the Middle East, upper-middle income countries structurally similar to Jordan, or those who are interested in the effectiveness of loan guarantee facilities coupled with technical assistance schemes. EVALUATION QUESTIONS Effectiveness 1. How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? a. To what extent were steps taken to address gender differences and/or gaps? b. What gaps or approaches should the activity have addressed or adopted, if any, in order to be more effective and efficient in implementation? c. Was the cost and effort expended on the achieved results appropriate? Were there opportunities for cost and implementation efficiencies, and what are they? 2. What are the impacts of the activity on the key target beneficiaries? a. What was the effect of the activity on: 1) the financial sector, 2) SMEs and women￾owned SMEs, and 3) employment? b. What strategies or approaches were key in the activity’s success in achieving its goals? And what strategies or approaches hindered it? 8 Global Communities. Jordan Loan Guarantee Facility Technical Proposal. 9 MESP is implemented by Management Systems International, a Tetra Tech Company. 10 U.S. government fiscal years begin October 1 and end September 30. Final Evaluation Report – Jordan Loan Guarantee Facility 2 c. What do the beneficiaries perceive as critical elements of the support they received through the activity? Why did they find them critical? Sustainability 3. What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? Learning 4. What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance? REPORT STRUCTURE This report first provides background on (1) the economic and political environment in Jordan, (2) capacities of the Jordanian financial and SME sectors, and (3) more detail regarding the JLGF implementation approach and associated milestones. Second, this report provides an overview of methodological considerations that describe what, how, when, where, and why data were collected; as well as how the evaluation team analyzed these various data streams to produce the conclusions and recommendations provided in this report. Third, this report outlines findings and conclusions organized by evaluation question. Per USAID guidance and to reduce repetition, this report treats the four high-level questions as organizational guideposts and provides findings and conclusions for sub-questions under the larger discussion contained in Question 1 or 2 as appropriate. Finally, beginning on page 38 the report offers recommendations for specific use by USAID, but also for use by OPIC, Global Communities, and other stakeholders to improve future programming of similar scope and context. This report further provides as Annexes the Evaluation’s Statement of Work (A), Design (B), References (C), Timeline of Implementation (D), Activity Performance Metrics (E) and a Statistical Annex (F), which provides cross-tabulations of the various data streams. BACKGROUND ECONOMIC AND POLITICAL ENVIRONMENT According to activity documents, secondary data sources, and in-depth interviews with finance sector stakeholders, the enabling environment for economic growth and employment in Jordan has been gravely impacted by regional instability. Annual growth in Gross Domestic Product (GDP) in 2005 was close to five percent, seeing a steady decline to negative growth rates after 2010 as major export markets experienced political revolution and civil conflict. In addition, net inflows of Foreign Direct Investment (FDI) dropped precipitously from a 2006 peak of 23.5 percent relative to GDP to 3.4 percent in 2015.11 Starting in 2011, Jordan was forced to spend an additional US$2.5 billion per year to secure fuel from international markets at costly rates, due to a steep drop in fuel supplies from Egypt, accounting for what was then 80 percent of the local energy supply. 12 11 World Development Indicators. World Bank Group. 2015. 12 Fakih, A. and M. Ibrahim. (2016). “The Impact of Syrian Refugees on the Labor Market in Neighboring Countries: Empirical Evidence from Jordan.” Discussion Paper No. 9667. Forschungsinstitut zur Zukunft der Arbeit. Final Evaluation Report – Jordan Loan Guarantee Facility 3 Table 1: Jordan annual GDP per capita growth and FDI inflows (annual percent of GDP) Year GDP Growth Rate FDI 2005 4.89% 15.8% 2006 4.24% 23.5% 2007 3.87% 15.3% 2008 2.76% 12.9% 2009 1.15% 10.1% 2010 -1.60% 6.4% 2011 -1.12% 5.2% 2012 -0.78% 4.9% 2013 -0.31% 5.4% 2014 0.30% 5.6% 2015 -0.02% 3.4% Source: World Development Indicators Jordan hosts the largest proportion of refugees in the world relative to the size of the country’s population. The United Nations High Commissioner for Refugees estimates that 1.4 million Syrians have crossed the border and now reside in Jordan. In addition, Jordan hosts an estimated 140,000 Palestinians who do not hold Jordanian citizenship and an estimated 30,000 refugees from Iraq. 13 An estimate from 2013 calculates various expenses borne by the central government for hosting refugees, including, but not limited to:  Infrastructure expansion for electricity, water, and municipal services (est. US$1.7 billion);  Health services (est. US$167.9 million);  School enrollment of approximately 80,000 children (est. US$81.4 million);  Provision of water (est. US$62 million);  Lodging/bedding (est. US$4 million); etc.14 According to the Jordan Department of Statistics, in areas populated by refugees Jordanian unemployment rates can be upwards of 15 percent. Further, a 2013 United Nations Food and Agriculture Organization study provides evidence that Jordan domestic employment opportunities and wages have been driven down in the agriculture sector, which accounts for 60 percent of employment in rural areas.15 Interviews with government and industry associations for this evaluation provided additional perspectives on the economic and labor environment in Jordan. A common theme was that public sector employment, which hovers between 35 and 38 percent of total employment,16 has reached its 13 “Refugees constitute third of Jordan population – World Bank official.” Jordan Times. 19 December 2015. 14 Fakih, A. and M. Ibrahim. (2016). “The Impact of Syrian Refugees on the Labor Market in Neighboring Countries: Empirical Evidence from Jordan.” Discussion Paper No. 9667. Forschungsinstitut zur Zukunft der Arbeit. 15 “Agricultural Livelihoods and Food Security Impact Assessment and Response Plan for the Syria Crisis in the Neighboring Countries of Egypt, Iraq, Jordan, Lebanon and Turkey. (2013). United Nations Food and Agriculture Organization. 16 Taghdisi-Rad, S. (2012). “Macroeconomic policies and employment in Jordan: Tackling the paradox of job￾poor growth.” Employment Sector Employment Working Paper No. 118. International Labor Office. Final Evaluation Report – Jordan Loan Guarantee Facility 4 full capacity to absorb new workers. According to the Central Bank of Jordan (CBJ) and the Jordan Loan Guarantee Corporation (JLGC), “the only way to create new jobs in Jordan is through the private sector.” These respondents further confirmed that despite intense pressure on the national accounts caused by regional economic shocks, “SMEs are the key to growth in Jordan.” Further, SME and start-up owners who participated in JLGF trainings recognize that the economic and political environment is a limitation to their business’ growth.17 Small and Medium Sized Enterprises A recent survey published by the Association of Banks in Jordan notes that 98 percent of private businesses in Jordan fall into the category of SMEs,18 yet a 2011 JLGF activity document estimated that only 11 percent of bank financing went to the SME sector in Jordan.19 The CBJ has provided a definition of SMEs that is compared with OPIC criteria in Table 2. But in essence, the CBJ classifies SMEs as having revenues or assets between approximately US$1.4 to $4.2 million and employing between five and 100 workers. Enterprises smaller than this threshold are considered micro. 20 Of the 25 banks operating in Jordan, 15 have confirmed departments specializing in SME financing.21 This trend has risen in recent years and is expected to continue among both commercial and Sharia (Islamic law)-compliant institutions.22 Figure 1 depicts the proportion of SME loans generated by bank type. Figure 1: Proportion of SME loan applications by bank type Source: Association of Banks in Jordan survey According to the same survey, there are six consistent reasons banks deny SME loan applications: 23 17 40 percent of SME interview respondents state that regional instability is one of their top two constraints to growth. 18 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. Note: this figure likely includes micro enterprises as well. An additional report by the European Investment Bank breaks the figure down further, with SMEs, absent of micro enterprises, constituting 93 percent of private businesses. See: Betz, F., and G. Frewer. (2016). Jordan: Neighbourhood SME Financing. European Investment Bank. 19 Based on figures provided in Global Communities’ 2011 Technical Proposal. 20 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. 21 Ibid. 22 Interest rate based fees are not utilized by Sharia-compliant institutions, rather deals are usually structured with an upfront fee. Further, rates charged by commercial institutions to SMEs range between 8 and 14 percent. The SME loan application rejection rate among banks is 10 percent; and banks that have below average rejection rates tend to have the largest market share in terms of SME lending. 23 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. Commercial (75%) Islamic (20%) Foreign (5%) Final Evaluation Report – Jordan Loan Guarantee Facility 5 1. Weak administrative and marketing capacities of SMEs; 2. Insufficient collateral for the level of risk; 3. High debts from other loans; 4. Weak financial data provided by SMEs – relatedly, a 2011 census of economic enterprises in Jordan confirmed that 92 percent of Jordanian enterprises do not use regular accounting books; 5. Insufficient bank awareness of SMEs; and 6. Insufficient data on the SME sector as a whole. Eleven of 24 banks surveyed by the Association of Banks responded that they are targeting an increase in SME lending by 26 to 50 percent in the near future, while another six banks say a target up to 25 percent may be possible. The remaining banks stated that they do not have a fixed limit for the volume of financing planned for SMEs, as it depends on SME eligibility. JLGF APPROACH The JLGF implementation approach is three pronged: 1. Improve and expand SME access to finance through the loan guarantee facility; 2. Mobilize bank lending to SMEs through a network of partner banks, sector stakeholders, and improved capacity of SME entrepreneurs to meet loan application standards; and 3. Build the capacity of banking sector partners to meet international standards for SME credit underwriting, track loan performance, and better integrate SME-centric lending into existing lending models. Central to JLGF’s model is institutionalizing best practices of SME financing following a cash-flow centric method of credit underwriting, which is part of an SME lending toolkit made up of seven “Cs.” These Cs are: Capacity, Cash-flow, Capital, Collateral, Character, Conditions, and Commitment.24 According to JLGF, Jordanian banks traditionally put emphasis on collateral and credit history based lending. This approach, the theory implies, constrains SME access to finance because smaller enterprises, especially start-ups, typically are not large enough to (1) offer sufficient sources of collateral, (2) do not have lengthy credit histories, nor (3) have strong linkages to banks. “In an economic environment of two and a half percent growth, it simply makes sense to do cash-flow based loans. When Jordan was growing at six percent and everyone was making money, collateral based lending perhaps made more sense for banks.” – JLGF Senior Manager While not omitting the other six Cs, JLGF’s approach attempts to leverage the OPIC facility with targeted technical assistance to expose and shift banks to this alternative cash-flow based approach to credit underwriting for the SME sector. Theory of Change This report provides Figure 2 to illustrate the overarching theory of change of the activity’s implementation. Note that Figure 2 is the analysis of the evaluation team because implementation priorities shifted over time, as confirmed by JLGF interviews and activity documentation.25 The effectiveness of this approach is discussed under Evaluation Question (EQ) 1; however, in broad terms:26 24 “Seven ‘C’s’ of Credit.” SME Toolkit. International Finance Corporation: World Bank Group. 25 Compare JLGF Work Plans FY2012, 2013, 2014, and 2015. 26 Use of the term “broad” is intentional in the sense that specific JLGF initiatives shift between the two objectives in the first two years of the activity, solidifying in FY 2014. The FY 2014 onwards approach is what is presented as Figure 2 and is the focus of the discussion in this section. Final Evaluation Report – Jordan Loan Guarantee Facility 6  Objective 1 is concerned with partnerships (banks and associations) and SME capacity to meet loan approval requirements; and  Objective 2 is concerned with partner bank capacity to meet international lending standards and effectively lend to qualified SMEs. Figure 2: JLGF theory of change Source: Analysis of the evaluation team based on JLGF activity documents and in-depth interviews OPIC and CBJ SME definitions differ; note that JLGF follows OPIC guidelines in its assistance efforts. Table 2 compares the differences between the two sets of criteria. Table 2: Comparison of OPIC and CBJ enterprise definitions Criteria Enterprise Size Institution OPIC Central Bank of Jordan Value of Assets Small Less than US$3 million Less than US$1.4 million Medium US$3 to $15 million US$1.4 to US$4.2 million Sales/Revenue Small Less than US$3 million Less than US$1.4 million Medium US$3 to $15 million US$1.4 to US$4.2 million Number of Employees Small Less than 50 5 to 20 Medium 50 to 300 21 to 100 Source: OPIC and CBJ websites Scope and Breadth of JLGF Activities Global Communities was awarded US$9,344,720 by USAID to implement the JLGF activity from FY 2012 through FY 2016. OPIC’s role was the establishment of the guarantee facility, with a coverage ceiling of US$250 million. Global Communities, in addition to providing technical assistance to banks and SMEs, served as the day-to-day servicer of the loan guarantee. In practice this meant that OPIC provided the guarantee in case covered loans defaulted, and USAID paid for the servicing of the facility and implementation of the technical assistance. Servicing in this context refers to reviewing loan guarantee applications and ensuring compliance with OPIC requirements. Final Evaluation Report – Jordan Loan Guarantee Facility 7 At the time of writing this report, the ceilings established for the partner banks was US$170 million of the maximum $250 million OPIC facility. As of FY 2016 Quarter 2, partner banks utilized approximately $71 million, constituting 42 percent of the combined, obligated ceiling and 28 percent of total coverage possible under the facility. Table 3 illustrates the significant differences in both ceilings and the utilization of the facility among partner banks. Table 3. Loan Principle and OPIC Commitment by Partner Bank Bank Number of Loans Total Value of Loans Current OPIC Commitment Arab Bank 2 $1,478,814 $15,000,000 Bank al Etihad 128 $22,474,011 $30,000,000 Cairo Amman 22 $13,369,709 $40,000,000 Capital Bank 42 $16,026,836 $15,000,000 Housing Bank 22 $3,258,475 $40,000,000 Jordan Ahli 83 $12,616,949 $15,000,000 Jordan Kuwait 3 $1,721,751 $15,000,000 Total 302 $70,946,545 $170,000,000 Source: OPIC website Note: Bank al Etihad and Capital Bank have received ceiling increases beyond their original agreement. Review and approval of loan guarantee applications from partner banks A key aspect of JLGF’s approach is its process for reviewing consent requests (i.e., loan guarantee applications) from partner banks. Per the terms of the Guarantee Facility Agreement (GFA) – signed by each partner bank, OPIC, and JLGF – each loan request is sent by the bank to JLGF for guarantee approval.27 This stage allows JLGF to review the credit underwriting analysis undertaken by the bank and return the application should any information be missing or deficient. From FY 2012 through FY 2015 of the activity, guarantee applications exceeding US$750,000 were referred to OPIC for final approval of the guarantee; however, this threshold was increased in FY 2016 to US$1.5 million, according to JLGF staff. Geography Figure 3 provides an illustration of the geographic spread of JLGF support, including the number and value of each governorate’s portfolio. Amman accounts for more than two-thirds of the overall portfolio (213 loans), with Irbid, Zarqa, and Balqa hosting 33, 23, and 7 loans respectively. Together these four governorates host 91 percent of the total. Notably, Aqaba in the south, the seat of a Special Economic Zone, hosts eight loans. 27 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2013. Final Evaluation Report – Jordan Loan Guarantee Facility 8 Figure 3: Number and concentration of active and settled SME loans by governorate Gender JLGF collected sex disaggregated performance indicator data since start-up; however, activity documents explicitly target supporting female entrepreneurs as part of its FY 2015 strategic positions statement.28 This coincides with the hiring of a gender specialist in FY 2014. JLGF’s articulation of the issue suggests that other Jordanian female targeted assistance programs support women-owned microenterprises; but women that own more sophisticated businesses have little 28 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2015. <3,000 3,001-10,000 10,001-30,000 >30,000 Final Evaluation Report – Jordan Loan Guarantee Facility 9 resources available to them. JLGF states that it endeavors to “fill this gap with specialized resources devoted to helping women-owned SMEs access finance and know-how.”29 The effectiveness of this approach and resources leveraged for women are discussed as part of Evaluation Question (EQ) 1. Sector/Economic Activity Loans guaranteed under JLGF went to SMEs in roughly 35 sectors30 with a degree of overlap (e.g., “accommodation and food services” is coded separately than “full service restaurants”). Yet, four broad economic activity types represent approximately three-quarters of the portfolio. These sectors are provided in Figure 4. Figure 4: Prominent economic sectors in the JLGF portfolio31 Source: JLGF portfolio database According to the survey conducted by the Association of Banks in Jordan, the distribution of credit in the overall economy by economic activity is similar; however not identical. The top four sectorial destinations are provided in Figure 5. 29 Ibid. 30 Sectors were coded using the North American Industry Classification System 31 Calculations based on JLGF records. Wholesale Trade (32%) Retail Trade (23%) Manufacturing (13%) Accommodation and Food Services (7%) Other (25%) Final Evaluation Report – Jordan Loan Guarantee Facility 10 Figure 5: Distribution of overall lending by economic sector32 Source: Association of Banks in Jordan survey Partner Institutions The original technical proposal and start up report for JLGF33 stressed that the activity would complement rather than replace or compete with local or other foreign donor assisted loan guarantee programs designed for Jordanian SMEs. The degree to which this occurred is discussed in the section addressing EQ1. Partner Banks Of the 25 banks operating in Jordan, JLGF partnered with seven. All seven banks are commercial rather than Islamic. These banks are: 1. Arab Bank; 2. Cairo Amman; 3. Bank al Etihad; 4. Housing Bank; 5. Jordan Ahli; 6. Capital Bank; and 7. Jordan Kuwait. Publicly Backed Institutions The Jordan Enterprise Development Corporation (JEDCO) is a Jordanian government sponsored institution meant to modernize and expand enterprises in Jordan to harness gains from trade agreements and lessen negative externalities associated with globalization. It was intended that JEDCO would serve as an outreach and referral mechanism for JLGF; however, according to USAID this agreement was cancelled due to concerns over JEDCO’s capacity. This left JLGF without a formal partner to refer qualified SMEs towards the JLGF facility. The Jordan Loan Guarantee Corporation (JLGC) was identified as a complimentary institution to JLGF at the outset; however, there was a lack of coordination throughout the course of the activity. The reasons for this are discussed in the section addressing EQ1. Based on interviews and activity documents, however, JLGC and JLGF share the same overarching goals and there are opportunities to support, rather than compete for, different segments of the SME market. 32 Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. 33 Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2012, Quarter 2. Construction (23%) General Trade (18%) Public Services (15%) Industry (10%) Other (34%) Final Evaluation Report – Jordan Loan Guarantee Facility 11 JLGF has partnered with the Jordan Institute for Banking Studies (IBS), an Amman based institution associated with the CBJ, to develop an SME Financing Diploma program. JLGF’s main support is in the form of implementing the cash-flow analysis module of the program, as well as assisting IBS to obtain accreditation for its diploma program. Implementation Specific Challenges and Adaptations Global Communities’ original design prioritized working through bank partnerships to mobilize SME lending; however, by the end of FY 2013, it became clear to JLGF that an adapted approach was necessary to effectively address capacity constraints. Table 4 details JLGF’s original assumptions with the activity’s assessment of the situation at the end of the activity’s second year. Annex D provides greater detail of the timeline of JLGF implementation activities. Table 4: JLGF original assumptions compared with understandings at the end of FY 2013 Original Assumption34 Situational Assessment Banks would originate SME loans. Banks lack capability in SME loan origination and require technical assistance in SME business development. Banks have minor credit underwriting deficiencies that would be addressed through coaching and workshops. Banks have significant credit underwriting deficiencies, requiring more aggressive technical assistance efforts. JLGF would not have contact with borrowers. JEDCO would be the main contact between the facility and enterprises and serve as an outreach/referral mechanism for bankable SMEs to the facility. JLGF must have contact with SME borrowers in order to underwrite guarantees, as banks lack the capability to gather the necessary credit information. JEDCO could not be relied upon to direct sufficient borrowers to the facility. JLGF would not provide technical assistance to SMEs. JLGF needs to provide training and technical assistance to SMEs to make them abler to access credit, strengthen their financial management and enable them to comply with JLGF credit informational requirements. Source: JLGF Work Plan FY 2014 Activity documents state that in more than 80 percent of cases banks were not able to provide monthly cash-flow projections with assumptions, which is the basis for cash-flow-based lending. Therefore, JLGF found this accepted international best practice in commercial finance very challenging to implement in the Jordanian context. According to JLGF, “despite all the years of training banks have received in commercial lending, Jordanian banks are still basing credit decisions largely on collateral rather than repayment capacity determined through cash-flow analysis. This shortcoming is due to both the lack of capacity of credit staff and the fact that banks do not require this as part of their credit analysis process. While not ideal, these gaps have necessitated JLGF staff having direct contact with prospective SME borrowers.”35 The revised approach included: 34 An overarching JLGF design assumption was that bank and SME capacities were higher than they were which led to substantial management adaptations of the course of the activity. 35 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2016. Final Evaluation Report – Jordan Loan Guarantee Facility 12 1. Assisting partner banks in gathering and analyzing the information needed to underwrite the loan guarantees; and 2. Monitoring active loans for compliance with OPIC requirements and consistency with international best practices in SME credit analysis and portfolio management.36 Reflecting back, JLGF documents from 2016 state, “this increased engagement with both banks and SMEs, which JLGF initiated during 2014, constituted a substantial change in the way the program operates, as compared to the original plan… necessitating the shift to increased engagement with banks and SMEs.”37 EVALUATION METHODOLOGY This section describes the overall methodology that was used to answer the evaluation questions. The evaluation’s design is provided as Annex B, which includes final qualitative and quantitative data collection instruments. JLGF performance monitoring data included in this report is inclusive through FY 16 Quarter 2, unless otherwise stated. This report was written in August/September 2016. TIMELINE AND PROCESS OVERVIEW The evaluation team’s initial review of JLGF activity documentation began on June 6, 2016. Over the course of June, the evaluation team developed the study’s design and work plan, including the generation of open ended topic guides for key informant interviews and a phone survey questionnaire for JLGF SME training participants. An initial plan to also survey loan borrows was scrapped because (1) the focus of this evaluation was centered on the technical assistance aspect of the activity, and (2) banks would need to first reach out to their clients before the evaluation made contact, delaying an already tight data collection schedule. Primary data was collected over two phases. Phase I was conducted from June 20 through July 4 and focused on in-depth interviews with key informants from USAID, the JLGF implementing partner, partner banks and purposively-selected SME training participants. SME in-depth interview respondents were selected to provide the evaluation team with a range of perspectives, including those who (1) participated in training and borrowed under the facility, (2) just participated in training, (3) just received a loan, as well as those representing (4) different governorates, (5) genders, (6) economic activities, and (7) varying sized businesses. Phase 1 concluded at the start of the Eid-al Fitr national, week-long holiday. Phase II began on July 12 and was concluded on August 11. Phase II consisted of additional in-depth interviews with partner banks, SME associations, SME training participants, OPIC staff, and a follow￾up interview with JLGF implementer staff. Also included in Phase II was the implementation of the study’s quantitative phone survey of JLGF training participants. The phone survey was implemented by subcontractor Mindset, a market research firm based in Jordan. MESP and Mindset pre-tested the survey instrument on July 14 among 29 initial respondents. Fourteen of these initial respondents reported to not have a current business; thus a separate questionnaire was developed for start-up entrepreneurs to complement the one already developed for established SME owners. Mindset implemented the phone survey from July 25 through August 5 and provided the cleaned dataset on August 15. Analysis of the various data streams was an ongoing process and a preliminary report summarizing qualitative findings to date and closed-ended responses to the survey questionnaire was delivered to USAID on August 7. This version of the final report incorporates the full analysis of all data streams discussed in more detail below. 36 Ibid. 37 Ibid. Final Evaluation Report – Jordan Loan Guarantee Facility 13 DATA COLLECTION METHODS The evaluation team collected, reviewed, and analyzed both secondary and primary data for this evaluation. Secondary Data Sources Major secondary sources consulted for this evaluating included the following activity and context documents. These documents were the source of the activity monitoring data used in this report. For a full list of documents used to ground the evaluation in the overarching regional and Jordanian context, see Annex C. 1. JLGF Technical Proposal 2. JLGF Work Plans 3. JLGF’s Quarterly Reports 4. JLGF Databases: a. SME loan guarantee database b. Workshops attendees for both SME and banks staff c. Workshop participant exit surveys of the training activities d. Rejected applications for loan guarantee 5. Survey study of SMEs in Jordan conducted by the Association of Banks Primary Data Collection The evaluation team collected primary data from both supply and demand side actors (i.e., banks “supplying” and SMEs “demanding” access to finance). The team developed 14 instruments to collect both qualitative and quantitative data. Table 5 shows how each of these instruments were utilized for each respondent type. The instruments are provided in this report in Annex B and a matrix is provided as Appendix 1 of Annex B that details how each information source was used to answer each evaluation question. Table 5. JLGF performance evaluation: data collection, methods and instruments by respondent type Respondent Type Number of Interviews or Focus Groups Data Collection Method Instrument Heads of SME Department/Senior Management 7 In-depth interview 2. Bank Head of SME/Upper Management topic Guide Training Participants/Credit officers 7 Focus group discussion/open￾ended questionnaire 3. Bank Trainees Focus Groups- Discussion Guides 3A. Bank Trainees Questionnaire Implementing Partner 4 In-depth interview 1. IP Topic Guide Trainers including IBS 5 In-depth interview 4. Trainers Discussion Guide USAID JLGF 1 In-depth interview 5. USAID Topic Guide USAID Projects 3 In-depth interview 5. USAID Topic Guide JEDCO 1 In-depth interview 6 JEDCO Topic Guide JLGC 1 In-depth interview 7. JLGC Topic Guide OPIC 1 In-depth interview 8. OPIC Topic Guide CBJ 1 In-depth interview 9. CBJ Topic Guide Final Evaluation Report – Jordan Loan Guarantee Facility 14 Respondent Type Number of Interviews or Focus Groups Data Collection Method Instrument Supply Side Qual. 31 - - Associations and Chambers of Industry and Commerce 4 In-depth interview 10. SME Associations Topic Guide SME/Start-up beneficiaries 25 In-depth interview 11. SMEs In-depth Interview Topic Guide Demand Side Qual. 29 - - Total Qualitative In-depth/Focus Group Interviews 60 - - SME Training Participants (established SMEs) 84 Phone Survey 11.A. SMEs Survey Questionnaire SME Training Participants (start-ups) 64 Phone Survey 11.B. Start-up Survey Questionnaire Total Survey Respondents 148 - - Sampling Considerations Demand Side  Qualitative Sampling Approach for SMEs: In-depth interviews were used to provide rich detail to (1) compare with the quantitative findings and (2) explore differences in experience and perception between sub-groups (e.g., different types of activity participation). The purposive qualitative sampling approach ensured that respondents were roughly equally distributed among the following SME/Start-up owners: o Those who only received training from JLGF; o Those who received training and loan guarantees from JLGF, and o Those who only received the loan guarantee. Differences in gender, governorate, and business size were later used as valuable points of comparison; however, not explicitly part of sampling approach used for the qualitative portion of the evaluation.  Quantitative Sampling Approach for SMEs: The quantitative sample is based on the attendance sheets of those SMEs/start-ups owners that directly participated in the trainings offered by JLGF. Of these attendees, the evaluation team was able to identify contact information for 293 participants. This sampling frame represents as close to a census as possible given the limitations of attendance records. Based on activity records and databases, the evaluation team estimates that approximately 300 to 330 unique enterprises/start-ups participated in trainings sponsored by JLGF over the life of the activity.  Organizations that represent SMEs: The evaluation conducted in-depth interviews with key informants from the following associations that work closely with SMEs and represent the sector: o Zarqa Chamber of Commerce; o Irbid Chamber of industry; o Irbid Chamber of Commerce; o Jordan Women Industrial Community (JWIC); and o Jordan Industrial SME Association  Banks: While the banks are on the supply side when it comes to lending, they also consumed JLGF technical assistance. The evaluation included interviews with bank staff that participated in JLGF training mainly through focus group discussions. Final Evaluation Report – Jordan Loan Guarantee Facility 15 Supply Side  Banks: The evaluation team interviewed key informants from all seven partner banks. This included bank senior management or SME department heads (if applicable). The evaluation also interviewed credit officers who participated in JLGF bank oriented training through focus group discussions.  OPIC: The evaluation team interviewed three staff (group phone call) from OPIC who oversee JLGF.  Trainers: The evaluation team captured the views of five training providers including those responsible for JLGF’s involvement with the Institute of Banking Studies.  JLGF and USAID Team: As implementers and designers of JLGF, the JLGF IP and USAID teams were interviewed and consulted to ensure that their critical roles and perspectives were adequately understood.  Related USAID activities: The evaluation team interviewed key informants from within relevant USAID activities that support SMEs in Jordan. The evaluation team conducted interviews with, USAID/JCP, USAID/BEST, and USAID/ESCB. USAID/LENS was not responsive to interview requests.  Government programs targeting SMEs: The evaluation team interviewed senior managers within JLGC, JEDCO and the CBJ for their views of the SME sector and of JLGF specifically. DATA ANALYSIS PROCEDURES This evaluation primarily relied on three types of analysis: 1. Before and After: The unit of observation under this type of analysis was at the beneficiary level (i.e., banks and SMEs). This line of analysis explores patterns and trends that can be observed from the primary and secondary data collected. 2. Planned versus Actual: This line of analysis focuses on tracking the setting and achieving of targets, outcomes and outputs laid out from the performance management plan, and monitoring and evaluation framework. 3. Performance versus Benchmarking: This type of analysis is at the activity level. The focus is how JLGF’s intervention model and processes were positioned to address and relate to local, regional, and international opportunities and challenges. The evaluation team further conducted content analysis of in-depth interviews and focus group discussions. Descriptive statistics are provided based on survey data from SME owners/entrepreneurs. When applicable, this report provides cross-tabulation of survey responses against key variables of interest, for example gender, size of the SME, purpose and type of loan, industry, location, etc. DATA STORAGE AND TRANSFER Data storage procedures for this evaluation are governed under the provisions set out in the MESP contract signed by USAID and MSI. Survey data collected for this evaluation will be cleaned for submission to the Development Data Library in a machine readable format. Respondent identifying information will be redacted in accordance with MSI and MESP ethical guidelines. STUDY LIMITATIONS This evaluation’s timeline included a fast turnaround schedule and arranging interviews during Ramadan, which limited the number of in-depth interviews that were possible outside of Amman. The evaluation team saw value in surveying loan recipients in addition to SME training participants; however, decided against it because multiple layers of permissions (specifically partner banks) would have been required and this was considered unfeasible given the timeframe. Final Evaluation Report – Jordan Loan Guarantee Facility 16 TEAM COMPOSITION Dr. Miguel Cabal – Team Leader Dr. Cabal has 25 years’ experience working as a researcher, evaluator, and policy planner throughout the Americas. Dr. Cabal’s key technical expertise lies in an understanding of SMEs and financial and non-financial support services, quantitative and qualitative data collection methodologies, and econometric analysis. Hisham Yassin – Jordanian Financial Sector Specialist Mr. Yassin has more than 25 years’ experience working in both the Jordanian and regional finance and banking sectors. He is experienced in longstanding bank lending practices in Jordan as well as investment analysis. Sam Hargadine – Evaluation Specialist Mr. Hargadine is an evaluation specialist focusing primarily on economic growth and poverty reduction programming. He has managed teams and coordinated studies in South Asia, East Africa and Latin America and has consistently delivered evidence-based findings and conclusions in an effort to improve policy-guidance. Ragheb Fityan - Monitoring & Evaluation Specialist Mr. Fityan is a monitoring and evaluation specialist with experience in the private sector in Jordan. He holds a B.S. in Management Information Systems and is able to combine information technology with real world applications to performance measurement. MESP and MSI Support Both the MESP Project team and MSI Home Office provided invaluable technical oversight and logistical support. The team was overseen by the MESP Senior Monitoring and Evaluation Specialist who served as the main contact with USAID/Jordan. Adhering to MESP and MSI organizational conflict of interest policies, no potential conflicts were noted for the implementation of this performance evaluation. Each of the core team members signed a disclosure form relating to real or potential conflicts. These files are available upon request. FINDINGS AND CONCLUSIONS EQ1: HOW EFFECTIVE IS THE IMPLEMENTATION APPROACH (SUCH AS CREDIT ANALYSIS, TECHNICAL ASSISTANCE TO THE SMEs, AND THE TAILORED/GENERIC IN-BANK TRAINING) IN ACHIEVING THE ACTIVITY’S OBJECTIVES? Findings and Conclusions for Objective 1: Mobilize bank lending for SMEs using loan guarantees At the time of writing this report, 302 unique loans were guaranteed through the facility totaling approximately US$71 million in SME credit. Fifty of these loans were obtained by female borrowers corresponding to approximately $5.4 million in credit. Figure 6 and Figure 7 illustrate the cumulative target and achievement across these measures. See Annex E for more detail across all formal performance monitoring indicators. Final Evaluation Report – Jordan Loan Guarantee Facility 17 Figure 6: Cumulative Number of Loans Generated, FY 2015-2016 Source: JLGF Quarterly and Annual Reports. Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. Figure 7: Cumulated Value of Loans Generated, FY 2015-2016 Source: JLGF Quarterly and Annual Reports. Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. According to JLGF and USAID, 12 of the 302 loans have been rescheduled and only 5 have activated the facility though a default.38 Bank Activity Partner bank utilization of the facility varied. Throughout this report, the evaluation team breaks up bank activity level by grouping together the three most active banks (Etihad, Jordan Ahli, and Capital), middle-two (Cairo Amman and Housing), and bottom two (Jordan Kuwait and Arab). These groupings allow for a more precise discussion of trends between banks that have utilized the facility differently. Activity level is based on total number of loans generated per bank, but roughly corresponds with total value as well. Table 6 provides the average loan principle per partner bank. Bank al Etihad and Jordan Ahli (the two most active), have the third and second lowest average loan principles per transaction, respectively; yet, account for roughly 70 percent of the number of loans generated and 50 percent of the total value dispersed. Figure 8 visualizes the relative number and value of loans per partner bank. This variance provides insight into how different banks approach SME lending. Arab Bank for instance is one of the largest banks in Jordan and sees its section of the market at the higher end of the SME size distribution. Bank al Etihad on the other hand made clear during interviews that it sees itself as a market leader in SME lending and has organized its human￾resources in a way that facilitates a greater volume of smaller sized loans suiting the types of clients 38 Note that the majority (222) of loans are still active and may remain covered by the guarantee until 2022, depending on the partner bank’s GFA. 0 100 200 300 400 Number of loans generated (FY12-13) Number of loans generated (FY14) Number of loans generated (FY15) Number of loans generated (FY16) Total Target Total Generated Women Owned Target Women Owned Generated $- $20,000,000.00 $40,000,000.00 $60,000,000.00 $80,000,000.00 $100,000,000.00 $120,000,000.00 Value of loans generated (FY12-13) Value of loans generated (FY14) Value of loans generated (FY15) Value of loans generated (FY16) Total Target Total Generated Women Owned Generated Women-Owned Generated Total Target Total Generated Women-Owned Target Women-Owned Generated Final Evaluation Report – Jordan Loan Guarantee Facility 18 the JLGF facility was designed to support (i.e., those meeting the OPIC definition of an SME, see Table 2). Table 6: Bank partner average loan principle Bank Average Loan Principle No. of Loans Bank al Etihad $175,578 128 Jordan Ahli $152,011 83 Capital Bank $381,591 42 Cairo Amman $607,714 22 Housing Bank $148,112 22 Jordan Kuwait $573,917 3 Arab Bank $739,407 2 Source: JLGF Portfolio Database Figure 8: Total number (blue, left axis) and value (red, right axis) of loans by bank partner Source: JLGF Portfolio Database Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. Across the portfolio, the average value of loans generated under the facility was approximately US$235,000; however, unique loan values ranged widely ($25,000 to $3.25 million). Figure 9 provides the average loan principle over time across the three bank activity levels. As seen, the middle and least active bank tiers typically utilize the guarantee for loans on the higher end of the portfolio’s distribution than that of the three most active banks. $0 $5,000,000 $10,000,000 $15,000,000 $20,000,000 $25,000,000 0 20 40 60 80 100 120 140 Bank el Etihad Jordan Ahli Capital Bank Cairo Amman Housing Bank Jordan Kuwait Arab Bank Number of loans Value of loans (USD) Final Evaluation Report – Jordan Loan Guarantee Facility 19 Figure 9: Yearly average loan principle (US$) by bank activity level Source: Evaluation Team Calculations Based on JLGF Loan Guarantee Dataset Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. Notably, the average loan principle of the entire JLGF portfolio is approximately double the average loan size that SME training participant survey respondents say they need. This observation corresponds with the fact that most of the enterprises that received loans guaranteed by the facility were already clients of a partner bank, while 43 percent of SME training participants did not have an established enterprise. The next sub-section discusses this disconnect in greater detail. SME Facility Utilization The evaluation team used JLGF SME training attendance records to estimate the number of unique enterprises that participated in training. This estimate is between 300 and 330.39 Eighteen of these enterprises obtained loans guaranteed by the facility (between five and six percent). According to the SME survey, 57 percent of SME training participants have a current business and 43 percent either have a start-up and/or are classified as an entrepreneur. Fifty-one percent of training participants were women. The low rate of facility utilization among SME training participants is plausibly related to a number of contextual considerations: 1. According to the activity’s Technical Proposal and FY2014 Work Plan, JLGF’s original design did not expect for the activity implementer to work directly with SMEs. Because a cooperative agreement with JEDCO, who was meant to refer potential clients towards the facility was cancelled by USAID due to capacity concerns, JLGF eventually developed materials/curriculum to work with SMEs directly.40 39 The evaluation team analyzed attendance records, including sign in sheets and exit surveys to ascertain how many unique enterprises participated in JLGF trainings. The range provided is a best estimate because attendance was tracked at the individual level, not by enterprise. It was also possible for attendees to participate in more than one training, although this was uncommon. 40 A partial explanation for why JLGF was originally meant to remain removed from direct SME involvement was a concern about moral hazard. It was thought, based on OPIC guidance (and an opinion shared by JLGC), that if borrowers knew their loan was backed by a foreign actor, default may be more likely. Over the course 0 200,000 400,000 600,000 800,000 1,000,000 FY2012 FY2013 FY2014 FY2015 FY2016 Mean Loan Value Most Active Mid-Activity Least Active Final Evaluation Report – Jordan Loan Guarantee Facility 20 2. According to implementing partner interviews and activity documentation, JLGF direct outreach consisted of partner bank referrals, media advertisements, and partnerships with SME associations. 3. There was not a dedicated unit within JLGF that systematically targeted SMEs who would likely be eligible for facility utilization. Those charged with credit analysis and management duties took this role on. 4. Opinions differed among stakeholders on whether or not the SME training component should have been utilized to feed the facility’s pipeline. OPIC’s perception of JLGF is that the activity has been “wildly successful.”41 That approximately $71 million has been utilized against a current obligated ceiling of $170 million is explained by the facility’s demand-driven approach. Further, USAID, while wanting to maximize access to finance, noted that while facility utilization was a priority, so was increasing access to finance generally, even for those that would not qualify for the OPIC guarantee or preferred other finance providers beyond the seven partner banks (e.g., those who may prefer to use an Islamic bank). Of those 18 borrowers who received both technical assistance and a loan guarantee under the facility, 16 had available loan information in the portfolio database. Based on this loan information this report provides the following characteristics for consideration: 1. The average loan principle value among these 16 borrowers was approximately $367,500, approximately $130,000 more than the average loan principles for the wider portfolio. This suggests that these borrowers likely owned some of the larger enterprises that participated in training; 2. Twenty-five percent of these 16 borrowers were female; 3. Nine were located in Amman, three in Aqaba, two in Irbid, one in Zarqa, and one in Karak; 4. Seventy-five percent of these loans (12) originated in FY 2014 to 2015; and 5. All but three of these loans were issued by one of the three most active banks. The evaluation team spoke with two Amman-based SME associations that have been highly active with JLGF, the Jordan Women’s Industrial Community (JWIC) and the Jordan Industrial SME Association. These associations praise the quality of the services provided and the benefits obtained by their associates. Another two chambers, both based outside of Amman (in Zarqa and Irbid), were not aware of JLGF services but were very receptive to hear about them and hope to become involved. These organizations did insist that JLGF should take a more systematic and institutional approach to get in contact with the various chambers and increase their involvement. SME Satisfaction with JLGF Assistance A large majority of SME training participants were satisfied with the trainings they attended. Table 7 shows the frequency and percentage of SME and start-up responses to the question: “what is your overall level of satisfaction with the services provided by JLGF?” Table 7: SME and start-up owner level of satisfaction with JLGF technical assistance What is your overall level of satisfaction with the services provided by JLGF? Response Frequency SMEs Percent SMEs Frequency Start-ups Percent Start-ups Very satisfied 17 20.2 40 62.5 Somewhat satisfied 37 44.0 18 28.1 of implementation, JLGF found the high performance of the portfolio to be a justification to test the direct interaction with SMEs. While the performance of the portfolio remains high at the time of writing this report, note that a majority of loans disbursed under the facility are still in repayment. 41 Referring to the low default rate of the portfolio and strong relationships the facility has with partner banks. Final Evaluation Report – Jordan Loan Guarantee Facility 21 What is your overall level of satisfaction with the services provided by JLGF? Neither satisfied nor dissatisfied 17 20.2 3 4.7 Somewhat dissatisfied 11 13.1 2 3.1 Not satisfied at all 2 2.4 - - Refused to answer - - 1 1.6 Total 84 100 64 100 Source: SME/Start-up Phone Survey Despite high-levels of satisfaction, 37 percent of established SME and 53 percent of start-up owners stated that JLGF training did not address their specific needs. These findings are not necessarily contradictory. Based on in-depth interviews with SME training participants, respondents cited the high value of training, even if the curriculum did not specifically address their top constraint to growing their business. When asked about anticipated constraints to the growth of their business, SME/start-up survey respondents provided their top two constraints. Figure 10 provides these responses by sex. Men most often cited policy and institutional issues and women reported more acute constraints from supply problems and lack of access to finance. Figure 10: Constraints to enterprise growth by sex of owner Source: SME/start-up survey Responses were open-ended then post-coded Conclusion 1: The loan guarantee represents the central value proposition of the implementation approach. Bank staff and industry respondents all cited the guarantee as the central value proposition contributing to JLGF progress toward the activity’s stated aims. The guarantee is designed to share risk with banks, thereby creating the enabling environment to experiment with new lending policies and procedures that represent greater applicability for small and medium-sized enterprises. While the technical assistance provided to banks and SMEs was likewise considered important by respondents, the guarantee enabled the most active banks under the facility to test and incorporate international SME credit underwriting best practices. 0% 10% 20% 30% 40% 50% 60% Lack of financial services, no access to loans, no access to finance Economic or political environment Low profitability Lack or problems with the market Supply problems Policy and institutional issues Men Women Shared Proprietorship Final Evaluation Report – Jordan Loan Guarantee Facility 22 Conclusion 2: Tailored bank assistance was a critical element of the approach because it complemented the facility’s value proposition. Tailored assistance to banks to close credit underwriting gaps was seen by the most active banks as added value to the risk mitigation elements implicit in the guarantee. The technical assistance on offer therefore, was a critical element of the support, and in the view of partner bank respondents, was a key component of success. Participating banks maintained high standards of portfolio quality while adopting new SME lending practices. While JLGF trainers and banks alike suggest that reform targets will only be achieved gradually, lessons learned from partnerships, especially those with Bank al Etihad, Jordan Ahli, and Capital banks, present a model that can inform future efforts to tailor services to less active partner banks and for those yet to sign a GFA. Conclusion 3: SME outreach was haphazard and constrained the effectiveness of the activity. While SMEs found capacity building efforts helpful, low facility utilization by these participants suggest that outreach was haphazard. SME associations have proven to be successful partners of JLGF (e.g., JWIC); however, the Irbid and Zarqa chambers of commerce and industry were unaware of the full range of JLGF support. The unsystematic nature of the SME outreach, either in terms of directing potential borrowers towards the facility or the lack of tailored training to address specific constraints form a wider pool of enterprise owners, constrained the ability for JLGF to provide direct access to finance (through the facility) or indirect access (through greater SME capacity to obtain it elsewhere). Findings and Conclusions Relating to Gender: EQ1A – To what extent were steps taken to address gender differences and/or gaps? EQ1A is answered under Objective 1 because it refers to gender differences and gaps for SME beneficiaries. Women-led SMEs that have received guarantees under JLGF typically work with the most active banks (Etihad, Jordan Alhi, Capital). Based on JLGF databases, these female owned/controlled enterprises are typically more inclined than their male counterparts to be: 1. Operating in the wholesale trade or personal services sectors; 2. Be located in Amman; 3. Be on the smaller end of the size distribution; 4. Use loans for fixed assets (e.g., additional floor space); and 5. Be involved in start-up enterprises. JLGF has targeted both established women-owned SMEs as well as start-ups since the outset; however, FY 2015 saw a renewed focus. From 2012 to the time of writing this report, 9 of 21 formal workshops were specifically designed for businesswomen. Four of these were conducted in 2016.42 While the number of unique loans generated for women did meet performance monitoring targets, no such formal target was included in the activity performance management plan for the cumulative dollar value of such loans. Annex E provides more detail regarding activity targets. JLGF has made an effort to reach women outside Amman; four of the workshops designed to businesswomen were conducted outside the capital region and a strong partnership has been built with the Jordan Women Industrial Community (JWIC). JWIC was the focal point for five of the businesswomen workshops, meaning that they were in charge of outreach and materials were developed specifically for their members. JWIC noted that women entrepreneurs, especially those outside of Amman, require additional technical assistance and support in order to gain equal access to finance. This has to do with the lack of collateral women typically are able present to banks, as well as less experience in negotiating with the financial actors. 42 Based on JLGF training records provided to the evaluation team. Final Evaluation Report – Jordan Loan Guarantee Facility 23 On average there was gender equity among workshop attendees (51 percent female), however this is uneven across project years, as illustrated by Figure 11. Notably, an observed increase corresponds with the hiring of a gender specialist in FY 2014.43 The evaluation team further notes that for workshops designed for bank staff, 19 percent of attendees were female; however, gender parity for these workshops was not a stated objective of the JLGF activity. Figure 11 Percentage of SME Workshop Attendees Comprised of Women Source: JLGF Training Records Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. In FY 2015, JLGF increased its guarantee coverage from 60 percent to 75 percent of the loan value for female SME owners. Also in FY 2015, financial products were developed specifically for female entrepreneurs by Etihad and Jordan Ahli banks.44 These services were designed to specifically target female SME clients, as opposed to micro enterprises, which have traditionally received more support, according to both JLGF and JLGC staff. JLGF’s Gender Specialist noted that women in Jordan confront additional challenges in obtaining access to finance. An example of a gendered obstacle includes conservative bank practices that require a male relative to co-sign loan applications with a female entrepreneur, where this would not be the case in reverse. JLGF stated that these practices were changing rapidly, but not eradicated. Women entrepreneurs expressed satisfaction with the training programs provided by JLGF. Specifically, women business owners across sectors, including in the retail, manufacturing, and food services sectors, said that financial analysis and projection training was especially helpful in dealing with banks. Based on in-depth interviews, women more than men requested that follow up workshops be held to assist in implementing the tools and methods discussed in trainings. JLGF trainers further concluded that for the most part, women participants in training were more engaged and asked more questions than their male counterparts. Furthermore, the SME/start-up survey results show a higher level of overall satisfaction with JLGF assistance among businesswomen than their male counterparts: 84 percent of female participants expressed satisfaction compared to 76 percent of men. "I am very satisfied with JLGF support because they understand my kind of business, and they give me support in all levels. Also the way that JLGF worked with me was respectful and they give me an equal opportunity. [JLGF] are very experienced people and they know what they are doing." – Female SME owner Supply Side Perceptions JLGF partner banks without specific women SME products stated that they analyze business loan applications the same way regardless of the sex of the owner. They see the difference in the coverage of JLGF guarantee as the main difference between the two (75 percent instead of 60). Of the banks that have been involved in developing products specifically for women, gender awareness is more pronounced. These banks recognized the differences between Amman and other 43 Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2015, Quarter 2. 44 Ibid. 0% 20% 40% 60% 80% 100% 2012 2013 2014 2015 2016 Final Evaluation Report – Jordan Loan Guarantee Facility 24 governorates and the more sophisticated demands of women entrepreneurs in the capital. Partner banks further recognized that women-owned enterprises are more likely to be smaller than their male counterparts, therefore, have less ability to access finance using collateral emphasized underwriting. Conclusion 4: The activity’s focus on providing targeted assistance to women SME owners came too late. While the activity exceeded its target of 43 total number of unique loans to women-owned SMEs (50 loans were generated through FY 2016, Quarter 2), the dollar-value of these loans account for approximately seven percent of the total portfolio. It is the conclusion of the evaluation team that the increased focus shown in FY 2015 on women-owned SMEs has been effective over the last year; however, this focus came too late to greatly increase the accessibility of finance for female-owned enterprises at a large scale. A critical contributing factor of this increased focus on women-owned SMEs was the hiring of a gender specialist in FY 2014. Findings and Conclusions for Objective 2: Strengthen Bank Capacity for Sustainable SME Lending Objective 2 targets a strengthening in bank capacity to sustainably lend to SMEs. This objective assumes that by assisting banks with practical skills training, JLGF will contribute to an increase in SME access to finance.45 Bank Partnerships Five banks signed GFAs with JLGF in the first year of the activity. The total number of partnerships holding GFAs rose to seven in FY 2013, which was the cumulative target for the JLGF activity. Note that all seven partner banks received OPIC approval in FY 2012. Table 8 details which year each of the seven partner banks signed their GFA. Table 8: GFA Signing Date by Institution Financial Institution Fiscal Year GFA Signed Arab Bank 2012 Cairo Amman 2012 Bank al Etihad 2012 Housing Bank 2012 Jordan Ahli 2012 Capital Bank 2013 Jordan Kuwait 2013 Source: JLGF Work Plan FY2014 Although JLGF continues to work with OPIC and commercial banks to secure additional partnerships, this effort has not been successful to date. Based on interviews with OPIC, USAID, and JLGF, all parties noted that completing a GFA is a challenging task. To sign a GFA, partner banks must comply with a wide-range of OPIC requirements. These include environmental and labor standards, bank governance and incorporation criteria, and financial compliance with OPIC’s standard legal framework. All parties agreed that GFAs take a considerable amount of time to finalize. For instance, if a bank already complies with the labor standards set out by OPIC, the bank’s board of directors must certify this compliance. Because board meetings may only occur once a quarter, it may be several months before an affidavit can be formally signed. All sources, including USAID, JLGF, JLGC, and the CBJ, stated that including a Sharia (Islamic law)- compliant financial institution in the activity would be beneficial; however, negotiations and 45 Global Communities. Jordan Loan Guarantee Facility Technical Proposal. Final Evaluation Report – Jordan Loan Guarantee Facility 25 bureaucratic requirements have proved to be insurmountable. OPIC partly attributes this to the cost it would take to draft Sharia-compliant legal documents which would need to be approved by OPIC’s senior management body. OPIC further noted that reaching an agreement with an Islamic bank is more complicated than simply repackaging the 1.65 percent interest fee as a fixed return repayment46 and would involve the sanctioning of such an agreement by an Islamic scholar. Despite these challenges JLGF and its stakeholders continue to work towards the inclusion of an Islamic bank under the facility. At the time of writing this report, negotiations with Islamic International Arab Bank were ongoing. Notably, 25 percent of SME/start-up survey respondents stated that their preferred source of finance was Sharia-compliant institutions. This response was second only to donor funded mechanisms at 42 percent. Conclusion 5: The lack of an Islamic bank partnership likely constrained the facility’s utilization. According to the SME beneficiary survey, Islamic banks are preferred to commercial banks as a place to apply for a loan, especially for women and respondents outside of Amman. JLGF staff echoed this sentiment. Thus, the evaluation team speculates that the lack of an Islamic bank partner acted as a limitation to facility utilization, or at the very least, limited options for those that prefer alternative sources of finance than commercial entities. Credit Underwriting A key component of JLGF’s assistance to banks was in the improvement of credit underwriting gaps. JLGF’s FY 2016 Work Plan provides a detailed account of partner bank credit underwriting analysis deficiencies. According to JLGF, “it was anticipated that partner banks would originate, underwrite, and monitor loans with minimal need for JLGF to have contact with the SME borrowers.” “In more than 80 percent of cases the banks were not able to provide monthly cash￾flow projections with assumptions. This is the basis for cash-flow-based lending, which is an accepted international best practice in commercial finance. Despite all the years of training banks have received in commercial lending, the Jordanian banks are still basing credit decisions largely on collateral rather than repayment capacity determined through cash-flow analysis. This shortcoming is due to both the lack of capacity of credit staff and the fact that banks do not require this as part of their credit analysis process.” – JLGF Work Plan FY 2016 JLGF responded to these limitations by assisting partner banks in gathering information needed to underwrite the loan and comply with international standards pertaining to cash-flow based lending. This included becoming more active in monitoring loans for compliance with OPIC requirements and international best practices associated with SME portfolio management.47 Figure 12 provides a comparison between the percentage of loan guarantee applications submitted to JLGF by partner banks in 2013 and 2015. The percentage exhibiting deficiencies dropped significantly across key cash flow based lending points of analysis such as the analysis of the market and the borrower’s competitors, the mitigation of risk, and the borrower’s strategy for differentiating themselves in their sector. Notably, there were few deficiencies in the analysis of collateral and the clarity of purpose of the proposed loan, as these underwriting techniques were already familiar to partner banks – relating to the use of collateral based tools. Because the most active banks constitute such a large majority (70 percent) of the number of loan guarantee application requests reviewed by JLGF, these findings largely apply to them. 46 Interest based payments being haram (forbidden under Islamic law). 47 Global Communities. Jordan Loan Guarantee Facility Work Plan FY 2016. Final Evaluation Report – Jordan Loan Guarantee Facility 26 Figure 12: Percentage of loan guarantee applications exhibiting deficiencies across 16 points of analysis Source: Evaluation team approximations based on JLGF Work Plans FY 2014, 2015, 2016 Conclusion 6: Assumptions regarding the capacity of partner banks initially underestimated the support necessary to incorporate credit underwriting best practices promoted by JLGF. Based on findings from activity documents and interviews with JLGF staff, the initial activity start-up assessments of potential partner bank capacity were insufficient to accurately calibrate capacity building assistance at early stages of the activity. This report concludes that targeted capacity building support to the banking sector could have been better tailored to specific bank credit underwriting limitations if assessments conducted at the start would have more accurately measured institutional capabilities, thereby matching implementation resources to a more accurate understanding of baseline capacity. Effectiveness of Bank Trainings According to in-depth interviews and focus group discussions with partner banks, bank trainings were perceived as being of high quality, especially as individual bank partnerships with JLGF matured. According to the most active banks, initial concerns about the timeliness of loan guarantee application approvals were assuaged as the turnaround time dropped to under one week. For banks that have presented fewer applications for the guarantee, the legacy of long turnaround times in the early years of the activity remained a criticism. According to USAID, a choice was made to provide capacity building assistance only for banks partnered under a formal GFA. This limited the opportunity for banks that were either unable or uninterested in meeting OPIC requirements to participate in tailored technical assistance. In addition to the decrease in loan application deficiencies illustrated in Figure 12, the rate of rejection of loan 0 10 20 30 40 50 60 70 80 90 100 Proposed loan term Collateral and guarantee for the proposed loan Purpose of proposed loan - list of assets to be financed Owner reputation and experience Owner cash contribution Proof of use of loan proceeds Description of key management, experience, and qualifications Analysis of historical financial performance Summary of opportunity Proposed use of loan proceeds Historical financial analysis for the last three years Borrower's strategy for differentiation. Monthly financial projections Detail assumption for financial projections Risks and mitigating factors Analysis of market and competition % of guarantee applications with deficiences 2015 % of guarantee applications with deficiences 2013 Final Evaluation Report – Jordan Loan Guarantee Facility 27 guarantee applications decreased precipitously over the course of the activity. Figure 13 illustrates the approval rate across time. As seen, 46 percent of loan guarantee applications were rejected in the activity’s first year. The proportion dropped to two percent in FY 2016 (as of Quarter 2). Figure 13: Loan guarantee approval rate by year Source: JLGF portfolio database Conclusion 7: Tailored bank training was highly effective among the most active banks utilizing the facility. The drop in rejection rates, illustrated in Figure 13 and the drop in credit underwriting deficiencies, all while maintaining the overall financial performance of the portfolio suggests that for those who participated in JLGF bank tailored training, it was highly effective. Findings and Conclusions Relating to Implementation Challenges (Across Both Objectives) Regional Instability Universally, JLGF, JLGC, the CBJ, SME Associations, and qualitative and quantitative responses from enterprise owners all confirm that regional instability is a constraint on the Jordanian economy and thus the level of risk banks and SMEs are willing to take on in terms of investment. Based on the JLGF portfolio database, an increasing proportion of loans year on year have been used to invest in working capital rather than fixed assets. This is important because fixed assets (e.g., new equipment, new floor space) are associated with greater increases in employment. The logic underscoring this is that businesses expanding their manufacturing floor space, for instance, are more likely to hire additional workers to utilize that space than borrowers looking to restock their shelves (an example of working capital). Table 9 and Table 10 are based on the JLGF portfolio database and illustrate the decline in fixed asset investment across the portfolio (correlated with a conservative investment climate) and the average number of jobs generated for each type of investment, respectively. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2012 2013 2014 2015 2016 Approval Rate Average “Entering its fifth year of operation, the main challenge that JLGF faces is the deteriorating business environment resulting from disruptions and uncertainties caused by regional conflicts. As a result of the closure of the Iraqi border, the manufacturing sector is losing more than $280 million a month, per the Amman Chamber of Industry. Further, the continuing threat of Islamic State in Iraq and Syria has significantly exacerbated the already growing economic and political uncertainty in the region due to the unrest and conflicts in Yemen, Gaza, Syria, Egypt, Lebanon, and Iraq.” – JLGF Work Plan FY2016 Final Evaluation Report – Jordan Loan Guarantee Facility 28 Table 9: Proportion and value of loans year on year going to fixed assets and working capital Year Type of Loan Declining / Fixed Assets Revolving / Working Capital Total Value % Total Value % 2012 $1,624,011 79% $430,791 21% 2013 $2,515,537 36% $4,540,960 64% 2014 $7,915,946 37% $13,262,712 63% 2015 $8,789,356 32% $18,401,130 68% 2016 $2,752,825 20% $10,713,277 80% Source: JLGF portfolio database Values for 2016 are preliminary based on the number of active and settled loans at the time of writing this report. Table 10: Average expected jobs generated by loan type Loan Type Average Expected Jobs Generated Declining / Fixed Assets 11.09 Revolving / Working Capital 4.73 Source: Evaluation team calculation based on JLGF portfolio database. Conclusion 8: Regional instability posed a challenge to JLGF’s effectiveness Secondary and primary data sources, including background reports, activity documents, and interviews with JLGF staff, partner bank employees, stakeholder associations, and SME owners all identify regional instability as a constraint to the Jordanian business environment. These sources, as well as the evaluation team, further conclude that the weakening of regional economic integration and greater levels of uncertainty was a net constraint on the investment climate for SMEs in Jordan. JLGF Partnership with JLGC and CBJ JLGC JLGC is another prominent guarantee facility in Jordan and is supported by the CBJ. Both facilities have the overarching aim to increase access to finance for SMEs; however, coordination and formal partnership have proven elusive over the life of the activity. Activity documents make clear that the two organizations did not see eye to eye in the first half of the award’s implementation. From the perspective of JLGF at the end of FY 2014, “the portfolio guarantees offered by JLGC and through JEDCO’s programs are damaging to any effort to positively change SME credit underwriting practices of the banks as these programs, by default, accept current SME credit underwriting practices thereby perpetuating bad banking practice.”48 Interviews with JLGF and JLGC staff for this evaluation reflect a milder tone, and that opportunities for collaboration are robust. When interviewed, JLGC stated that it sees JLGF as an organization that compliments its own efforts. JLGC senior staff mentioned that various efforts to collaborate had been undertaken over the years, with little to show for it; however, recent talks have been promising – specifically with respect to joint outreach to the sector at large. In the past, JLGC has discussed providing joint guarantees; i.e., JLGC would guarantee a loan up to its ceiling of JOD100,000 and JLGF would guarantee the remainder up to its ceiling. JLGC understood that the likelihood of this formal joint structure was unlikely, citing the inflexibility of OPIC requirements and that “[JLGF] has a text book approach.” 48 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2015. Final Evaluation Report – Jordan Loan Guarantee Facility 29 CBJ JLGF found the CBJ to be relatively supportive; however, is hopeful for greater promotional efforts to increase the pipeline. JLGF staff also hope for more assistance in promoting the cash-flow analysis approach within the banking sector. Respondents stated that the CBJ understands that there is a financing gap for SMEs and the programs it supports are unlikely to fill this gap alone; therefore, JLGF hopes to compliment rather than compete with CBJ initiatives. Further, the CBJ confirmed the vital importance of SME access to finance and holds a positive view of the current JLGF management team. Yet the CBJ also stressed the need for JLGF to better coordinate with JLGC in order to improve performance across the board. OPIC Facility Fee According to a former senior manager within JLGF, there are two major issues in mobilizing access to finance. The first is sufficiently guaranteeing against risk to the bank’s assets and the second is that interest rates are too high. “The activity was designed to solve the collateral problem, but not the interest rate one,” a manger responded. OPIC’s 1.65 percent fee was seen by partner banks as too high, which the manager did not agree with. “Even when you factor in the 1.65 percent fee, against the 60 to 75 percent guarantee the banks should be able to lend at the same or even lower interest rate.” Interviews with the current JLGF leadership elaborated that there are negotiations underway to lower the facility fee to as low as one percent and JLGF believes this will help the overall effectiveness of the activity.49 CBJ managers concurred that a decrease in the facility fee would likely decrease interest rates charged to borrowers. The CBJ also said that in conjunction with the current downward trend in interest rates there will likely be a positive impact on SME credit demand. SMEs and SME associations also expressed their concern with high interest rates more generally, stating that it was too high and does not compare with rates seen elsewhere in the region Findings and Conclusions for EQ1B: What gaps or approaches should the activity have addressed or adopted, if any, in order to be more effective and efficient in implementation? Identified Gaps Several gaps are evident based on the findings and overall context presented thus far: 1. Achieving sustained and productive partnerships; 2. Performance monitoring; 3. Facilitating access to finance for women-owned enterprises; 4. Facilitating access to finance outside of Amman and the surrounding governorates; and, 5. Institutionalizing credit underwriting improvements among senior bank managers. This section provides conclusions relevant to gaps 1, 2, and 5 listed above. Gaps 3 and 4 have been discussed previously (see conclusions 4 and 5). Conclusion 9: Weak partnerships limit the effectiveness of the activity and will likely constrain the sustainability of outcomes. JLGF had difficulty in (1) expanding the number of banks covered under a GFA and (2) coordinating with local institutions to expand the reach of the facility to additional qualified SMEs. This is important because current trends suggest that the activity is likely to underutilize the facility’s potential coverage. As mentioned, the current combined ceiling for the partner banks is US$170 million of the $250 million facility and the banks have utilized approximately US$71 million at the time of writing this report. 49 At the time of writing this report, OPIC had recently responded to feedback about the facility fee and is currently developing a new fee structure. This new structure will most likely be rolled out before the end of the 2016 calendar year. Final Evaluation Report – Jordan Loan Guarantee Facility 30 An example of a successful partnership model includes that of JLGF and the women’s industry group JWIC. JWIC found JLGF support helpful for its members and, according to respondents, the partnership successfully introduced clients to the activity’s training and loan guarantee arms. While JWIC representatives underscored the additional support female entrepreneurs require with respect to accessing finance in comparison to their male counterparts, the engagement strategy, whereby JLGF staff tailor assistance to the needs of a specific membership, is a model worth studying in greater depth and exploring with other target demographic groups (e.g., geographically or by sector). Conclusion 10: Performance monitoring limitations constrained the activity’s ability to quickly address implementation challenges and revise previously held assumptions. Based on activity documents and in-depth interviews with stakeholders, the evaluation team concludes that adaptations to the implementation approach were appropriate; however, came late. It took two years for initial assessments and performance data to trigger an adaptation to the approach at the end of FY 2013. After JLGF began interacting directly with SMEs, performance improved. Further, as JLGF began tailoring assistance to banks to close SME credit underwriting deficiencies, economies of efficiency began to improve activity results. While there is no counterfactual to compare, more direct performance monitoring – better tailored to activity aims – may have improved the responsiveness of the activity to the challenges confronted. Annex E provides performance monitoring data across the activity’s selected indicators; however, a few examples of inconsistences are provided here to support the conclusion. 1. The goal of JLGF is to improve SME access to finance; however, the only goal level indicator tracks jobs created. 2. The causal logic of the activity’s approach puts intense pressure on Objective 2: Strengthening bank capacity. Yet, the performance indicators only track the number of bank partnerships and the person hours of training provided to bank partners. A formal measure tracking underwriting deficiencies was not included in the performance management plan. 3. While monitoring data was disaggregated by sex when data was person-specific, greater disaggregation for geography, size of business, and type of loan would have likely improved SME assistance and raised the proportion of training participants who felt that assistance addressed their specific needs. Notably, JLGF warned that due to its demand-driven approach, over-disaggregation of targets could be counter-productive. Pursuant to the goal level indicator, the measure tracks an indirect objective well-beyond what JLGF was actually targeting (i.e., jobs created instead of a measure of SME financial access). Further, other performance indicators overly emphasized facility utilization (i.e., number of loans issued) rather than outcomes for target beneficiaries. This report speculates that because of the demand-driven structure of the OPIC facility, over emphasis on facility utilization indicators may have been less immediately helpful for performance management than if performance data had been collected and reviewed relevant to the size, type, location, etc. of enterprises participating in capacity building assistance or accessing finance through the facility or elsewhere. Conclusion 11: Senior bank managers temper the speed of JLGF reforms. Respondents from banks and JLGF trainers cited senior bank managers as a constraint, not necessarily to the idea of changing credit underwriting practices, but to the speed in which reforms should take place. It is a bank senior manager’s responsibility to hold public confidence and conservative lending practices are not without merit from this perspective. Continuing to present the value proposition that cash flow based lending is not risky and has resulted in a high quality SME portfolio presents a powerful case for continued reform. Granted, some banks simply see their target market as businesses considerably larger than the ones JLGF was designed to support. Final Evaluation Report – Jordan Loan Guarantee Facility 31 Findings and Conclusions for EQ1C: Was the cost and effort expended on the achieved results appropriate? Were there opportunities for cost and implementation efficiencies, and what are they? The JLGF activity’s budget ceiling was US$9.3 million. Under that ceiling several adaptations occurred after realizing that bank capacity gaps were larger than originally envisaged, and direct interaction with SMEs was in the activity’s interest. Table 4 in the introduction of this report details these assumptions in greater detail and Annex D provides a timeline of major implementation events. These adaptations, however, did not come with additional resources. JLGF clarified that over time, economies of efficiency developed because bank capacity was improving and tailored modules were a natural outgrowth of the support JLGF ought to be providing. A former JLGF senior manager noted that from his perspective, the activity experienced intense human-resource pressure around the end of FY 2013 as it was attempting to (1) build relationships with banks and (2) pivot its approach towards direct interaction with SMEs. In-depth interviews with SME training participants, especially female respondents, recommended that JLGF conduct follow-up trainings or check-ins to improve the effectiveness of its SME capacity building efforts. When asked if post-training follow-ups to SME owners were considered, JLGF staff acknowledged that this would be a good approach to maintaining relationships; however, this was not at the top of priorities given limited human resources. Industry associations, also agreed that follow-up measures would bring greater value to JLGF SME support efforts. Conclusion 12: The original scope of JLGF and the resources applied to it were likely too small to fully address the stated objectives. The initial strategy of JLGF was to increase access to finance for SMEs through the provision of technical assistance to partner banks. JLGF began directly delivering technical assistance to existing enterprise-owners and entrepreneurs only after it became apparent that JLGF would be better able to achieve its stated objectives through this type of direct relationship with end-beneficiaries seeking access to finance. The trainings that were provided to businesses-owners and entrepreneurs were received positively (according to the SME/start-up survey), yet the number of enterprises that were aware of JLGF’s services and had access to said trainings was insufficient to achieve the activity’s higher level goals relating to the overarching SME enabling environment. Based on the relative success of the outreach that did occur, a greater contribution to SME financial access would likely have been achieved if additional resources and a more targeted approach to training enterprise￾owners and entrepreneurs had been employed, thereby achieving scale at a greater magnitude. EQ2: WHAT ARE THE IMPACTS OF THE ACTIVITY ON THE KEY TARGET BENEFICIARIES? Findings and Conclusions for EQ2A: What was the effect of the activity on: 1) the financial sector, 2) SMEs, and 3) employment? Financial Sector/Banks JLGF underwriting assessments As discussed as part of EQ1, JLGF contributed to large improvements in credit underwriting gaps among partner banks, especially among those who were most active in utilizing the facility. Supporting this conclusion is the rise in approval rates previously illustrated in Figure 13. Showing an increase in the rate of approval from 54 percent in FY 2012 to 98 percent in FY 2016, to date. Conclusion 13: JLGF contributed to the narrowing of credit underwriting gaps among the most active banks participating in the activity. Comparing results from the SME survey with the drastic drop in loan rejection rates and continued high performance of the overall JLGF portfolio suggests that one of the largest contributions of the activity was an improvement in credit underwriting schemes. Improvements such as this increased Final Evaluation Report – Jordan Loan Guarantee Facility 32 the likelihood that enterprise owners who would have otherwise been qualified but still rejected for a loan were able to access finance for their business. Institutionalization of cash flow based analysis Banks who used the facility less actively than Bank al Etihad, Jordan Ahli, and Capital show fewer signs of institutional change. In-depth interviews with loan officers within the partner banks as well as with JLGF and OPIC suggest that bank senior management controls the processes by which a bank will or will not develop unique products for SMEs or change the manner in which borrower risk is assessed. OPIC sees the facility as a demand driven tool, thus recognizes that some of the largest banking institutions, such as the Arab Bank, are less inclined to expend resources on attracting smaller enterprises in the distribution. Conclusion 14: While capacity building initiatives received positive participant feedback, changing the culture of SME lending, especially among senior bank managers, will take additional time. The effectiveness of bank capacity building efforts was limited despite high praise given to training modules (as found from in-depth interviews and participant exit surveys). This is because not all senior bank managers are convinced that cash flow based lending is appropriate for their institution. Interviewees provided that bank manager opinions are shifting and the evaluation team notes a range in the aggressiveness of partner banks to implement JLGF practices. Yet the culture shift away from collateral weighted lending will take more than the five years covered under the current activity to sustainably take hold. SME Beneficiaries Based on in-depth interviews and survey results, SME loan applicants and training participants indicate recognition of JLGF’s main aim – extending access to finance – and note improvements after interacting with one or several of the activity’s intervention arms. SME training participant survey responses indicate that more than 60 percent of both established SME and start-up respondents agree or strongly agree that JLGF helped them to increase their access to finance. Table 11 provides frequencies for how each respondent group answered the question. Table 11: Survey responses regarding JLGF contribution to financial access JLGF Technical assistance has increased my access to finance Response Frequency SMEs Percent SMEs Frequency Start-ups Percent Start-ups Strongly Agree 22 26.2 16 25 Agree 33 39.3 24 37.5 Neither Agree or Disagree 13 15.5 16 25 Disagree 11 13.1 5 7.8 Strongly Disagree 3 3.6 - - Refused / Don’t Know 2 2.4 3 4.7 Total 84 100 64 100 Source: Survey of SME training participants Based on the survey, a significant contribution of technical assistance has been raising awareness of how to access finance: 73 percent of SMEs and 81 percent of start-ups considered that this was the case. Seventy-three percent of established SME and 80 percent of start-up respondents indicated that they have changed their business practices mainly by improving their (1) accounting procedures, (2) financial statements, and (3) cash-flow projections. Approximately 85 percent of those that have changed their business practices consider those improvements to be permanent. Seventeen percent Final Evaluation Report – Jordan Loan Guarantee Facility 33 of approximately 330 established SME owners (who participated in JLGF trainings/workshops) responded that they applied for a loan, either through the JLGF facility or elsewhere, as a result of JLGF support. One-third of these respondents (18 SMEs) stated that they have been successful in obtaining the loan covered by the OPIC guarantee they applied for. Of this sub-group, those who the evaluation team interviewed each expressed that the combination of the guarantee and technical assistance (i.e., trainings/workshops) empowered them to seek out the financing they needed and appropriately negotiate with banks. Conclusion 15: SMEs who should have otherwise qualified, or were close to qualifying, found technical assistance to be helpful in making their case to banks when seeking finance. Despite relatively few enterprises participating in both technical assistance workshops and the loan guarantee, those who did attribute success to the intervention model (i.e., loan guarantee with technical assistance). This suggests that JLGF is a model that works for those business owners who would otherwise qualify for financing if they were only able to better articulate their business plan and financial situation to lenders and if those lenders were better placed to review their cash flow. Results for business owners who had structural constraints (less ability to show financial viability) or deeper skills deficits were not conclusive. The evaluation team’s speculation, which deserves further study, is that JLGF performed well for SMEs “close to the line.” In other words, it helped push SMEs with strong qualifications over the financing gap (either through the facility or elsewhere); however, many SMEs lower down the capacity distribution will need greater amounts and variety of assistance, especially those outside the main population and banking centers. Employment The activity monitored the number of jobs created as a result of USG assistance. 50 At the time of writing, approximately 2,880 new jobs were created and filled against a target of 6,482.51 Figure 14 illustrates activity progress over the life of the activity. As discussed under EQ1, no formal measure was included that tracked access to finance beyond the loans guaranteed as part of the activity’s portfolio. Figure 14: Cumulative Number of Jobs Created Source: JLGF Quarterly and Annual Reports 50 According to the JLGF Performance Indicator Reference Sheet, this measure tracks the number of new jobs created through SME access to finance activities. “Jobs created” is defined as a new position created by a start￾up or an existing company facilitated by bank financing mobilized by JLGF loan guarantees. Data for this measure are collected through baseline information required as part of the credit analysis form, and per the GFA, banks submit annual report information corresponding to each guaranteed loan with updated employment data. 51 Activity documents provide a target of 7,718; however, the convention of this report has been to accept “actual figures” for FY2012 and 2013 as the target because targets for these years are not reported in the activity’s performance monitoring plan. 0 1000 2000 3000 4000 5000 6000 7000 Number of persons (FY12) Number of persons (FY13) Number of persons (FY14) Number of persons (FY15) Number of persons (FY16) Jobs Targeted Jobs Created Final Evaluation Report – Jordan Loan Guarantee Facility 34 Values for 2016 are preliminary based on figures provided in Quarter 2, FY 2016 Findings and Conclusions for EQ2B: What strategies or approaches were key in the activity’s success in achieving its goals? And what strategies or approaches hindered it? Factors supporting success Conclusion 16: The Jordanian banking sector already recognizes the value of SME lending. According to interviews with partner banks, as well as sector-wide actors (i.e., JLGC and the CBJ) there is intense pressure on banks to seek out new sources of growth. Macro-economic pressure, as well as market saturation of banking services targeted to large firms, makes the SME market a prime target for bank lending. As presented in the Association of Banks survey, there is wide-recognition in the financial sector that SMEs are vital to sustained growth. Market imperfections, though, have constrained the ease in which banks can pivot to smaller enterprises. In essence, the case is pre￾made for initiatives such as JLGF to assist the Jordanian banking sector to mobilize lending for SMEs. Views on how and in what forms that assistance should be implemented, however, is not universal (e.g., the nuanced differences between OPIC/JLGF and CBJ/JLGC approaches). Conclusion 17: The perception of high technical standards gave JLGF strong credibility among partner banks. Partner bank employees continuously cited the high quality of JLGF technical assistance and the thoroughness of its review procedures. While some banks, especially those who are less active in their use of the facility, viewed the guarantee approval process to be too slow, none of the seven partner institutions thought JLGF had insufficient talent to provide expert guidance. Bank in-depth interview respondents overwhelmingly found that JLGF’s credibility was a contributing factor to the activity’s successes. Hindrances Conclusion 18: The 1.65 percent OPIC facility fee is too high. All sources outside of JLGF, to varying degrees, said that the OPIC fee was too high. JLGC and the CBJ were two of its largest critics suggesting that it turned away potential partners and lowered the overall effectiveness of the JLGF activity. Banks echoed this sentiment to varying degrees, and JLGF documents and interviews indicate an increasing realization that the sustained quality of the portfolio justifies a reduction in the fee down from 1.65 percent. The final interview with JLGF suggested that talks are ongoing with OPIC to lower the fee to one percent. Findings and Conclusions for EQ2C: What do the beneficiaries perceive as critical elements of the support they received through the activity? Why did they find them critical? The implementation approach was presented earlier under conclusions 1 and 2 as a critical re￾enforcing element unto itself that supported activity progress. Yet the discussion below elaborates on this point as it relates specifically to SME directed technical assistance. Conclusion 19: SME assistance was vital to the success of the activity, despite the small scale of support. While the implementation adaptation to interact directly with SMEs was a controversial one, the continued high financial quality of the facility’s portfolio supports the notion that this move was appropriate. That said, vital Jordanian partner institutions, such as JLGC and the CBJ, do not share this view. If recent efforts to revitalize JLGF and JLGC coordination were to take hold, intermediary organizations (e.g., JEDCO) may be needed to continue, and hopefully scale up, SME outreach, which the evaluation team concludes is vital. The critical nature of SME training is evident based on (1) the stated need of SMEs and SME associations, (2) the lagging performance of the facility outside of Final Evaluation Report – Jordan Loan Guarantee Facility 35 Amman, and (3) the greater level of assistance SMEs will need once those closest to the qualification standard gain access to finance. Further, the results from the SME training participant survey suggest that the most critical skills entrepreneurs have learned are the ability to develop a budget (30 percent), project management (20 percent) and financial management analysis (18 percent). EQ3: WHAT TECHNICAL INTERVENTIONS AND PRACTICES OF THE ACTIVITY CAN USAID EXPECT TO CONTINUE WITHOUT ADDITIONAL SUPPORT? AND WHICH ARE UNLIKELY TO CONTINUE AND WHY? To address this question, this report recognizes that loan guarantee facilities are not meant to be sustained indefinitely – facilities are a means to an outcome. Based on JLGF’s implementation approach, the outcome in this case is a sustained increase in access to finance for SMEs with knock￾on impacts for SME sector growth and increased job opportunities for the Jordanian people. Findings and Conclusions Relating to Local Ownership and Activity Termination JLGF’s original design52 expected robust partnerships with JEDCO and JLGC. JEDCO was to act as an intermediary between the activity and SMEs by providing outreach and technical assistance support to SME loan applicants. According to JEDCO, contracting requirements on the part of establishing a formal partnership with JLGF stalled, then failed. USAID described concerns about whether JEDCO would be able to deliver on the terms set out in the potential cooperative agreement and chose not to continue with this grant. JLGC’s raison d’être is essentially the same as JLGF’s; however, according to interviews with the CBJ and JLGC, the activity’s relationship with JLGC also stalled, then failed. According to both JLGF and JLGC in-depth interviews, high level meetings in FY 2016 have revived efforts to partner; albeit at the time of writing this report, USAID’s continued support for JLGF after September 2016 is still to be determined. During interviews with JLGF staff and as detailed in activity documentation,53 close out plans do not include initiatives to sustainably transfer the technical assistance portion of the activity to a local partner should USAID support end in FY 2016. Both JEDCO and JLGF staff said that JLGC is the heir apparent in terms of a local organization that can take on much of the support provided by JLGF. However, this is not to say that JLGF thinks this transition can happen immediately or even soon. The CBJ agreed in the sense that itself, JLGF, and JLGC “are the biggest players in the market,” and better coordination between all three should be a key consideration going forward. Relatedly, JLGC remains concerned about the moral hazard of interacting directly with SMEs, suggesting that an actor like JEDCO and governorate associations are still well placed to take on that role. JLGF, JLGC, and the CBJ each independently noted the different sectors the two facilities tend to cater to. The CBJ/JLGC SME definition is smaller than that of OPIC/JLGF’s, as well are their guarantee ceilings. Based on this, JLGF said a partnership with JLGC could take the form of bisecting the market and coordinating assistance. JLGC and the CBJ suggested that instead of separate guarantees, deals could be structured whereby JLGC covered the guarantee for larger SMEs up to its guarantee ceiling (recently raised to JOD300,000/US$422,500) and OPIC/JLGF could cover the remainder. The CBJ stated that “sitting with the banks and understanding their needs” would be the number one way for a donor like USAID to effectively design programs that sustain results for the SME sector. The consensus between JLGC and the CBJ was that JLGF could do more listening to bank perspectives. 52 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2012, FY2013. See also JLGF Technical Proposal. 53 Global Communities. Jordan Loan Guarantee Facility Work Plan FY2016. Final Evaluation Report – Jordan Loan Guarantee Facility 36 Conclusion 20: The impact of JLGF is unlikely to be sustained if (1) the USAID activity closes in September 2016 and (2) a follow-on award does not specifically target the sustainable transfer of ownership to a local partner. This report has provided findings addressing the effectiveness of JLGF as part of EQ1 and its impact in EQ2; and the overall conclusion is that a shift in the Jordanian credit underwriting paradigm towards a cash-flow prioritized approach makes implicit sense in terms of contributing to the stated goals and JLGF has been largely successful, albeit with some important considerations for the future. However, the evaluation team further concludes that gains made to this effect have not been sufficiently transferred to local partners, nor institutionalized within partner banks to the degree necessary for outcomes to be sustained. The OPIC facility, absent of USAID’s decision, will continue until 2022 under some bank GFAs. Should the JLGF activity end in September 2016, the servicing of this facility will end as well. OPIC stated that they do not have the capacity to bring JLGF support of the facility in-house. “Should our cousin agency end its support for this very successful program, we would be heartbroken.” – Senior Counsel, OPIC There is currently no plan to transfer facility servicing nor the delivery of technical support to a local partner. This is reputational risk for both USAID and OPIC and a sufficient wind-down scheme would likely improve the chance of sustaining the progress achieved over the last five years. Findings and Conclusions Relating to the Sustainability of Capacity Building Efforts Banks Respondents cited senior management uptake challenges when discussing the effectiveness of institutional change schemes targeted to financial institutions in Jordan. It is not a leap therefore to assume that institutionalizing a paradigm shift towards cash-flow based underwriting would take longer than the normal five-year activity cycle. Relatedly, a context-specific evidence base has been and will continue to be helpful in increasing confidence among senior bank managers who may be skeptical about cash-flow based lending. After all, it is these senior managers whose responsibility it is to maintain public confidence. According to the Association of Banks survey and corroborated by in-depth interviews, Jordanian banks see SME lending as a high-growth market; and as JLGC put it, “SME lending is necessary to make profit in this economic climate.” However, bank interviews suggest that a loan guarantee is still required to maintain upward lending trends to SMEs. “I think JLGF has made a jump in the market, in terms of a shift of mentality… The guarantee is a bridge. It is not to compensate if the customer is broke.” – Respondent from a top three SME lender Among more conservative banks, and for those predominantly working with enterprises outside of Amman, SME lending is viewed as having a relatively high level of risk. Although, as JLGC stated, large corporations already have established relationships with banks, “the best way to find a new large client is to help an SME grow into one.” JLGF trainers were interviewed for their perspective on the sustainability of bank capacity building achievements. The consensus was that SME client facing employees are receptive to new approaches to bank lending; however, banks are bureaucratic organizations with lending decisions made at top management levels. These trainers, based on their experience working directly with the banks, suggest working through the CBJ to institutionalize new approaches to lending due to the level of respect it has among bank senior managers. Trainers also suggested working gradually, for instance, working with senior bankers on changing various forms associated with cash-flow based analysis for Final Evaluation Report – Jordan Loan Guarantee Facility 37 more junior employees to utilize. Taking this approach, it is suggested, would create buy-in from senior managers by incorporating their vision for their specific institution. Conclusion 21: While the trend in SME lending among current and likely JLGF partners suggests the SME financing sector will continue to grow, cash-flow based underwriting has not yet been institutionalized. Based on the Association of Banks survey, and perspectives of banks and JLGF trainers interviewed by the evaluation team, the outlook for SME lending in Jordan is positive; however, sustaining the trend towards cash-flow based underwriting, instead of collateral, which is harder on small enterprises and start-ups, is far from certain. In order to maximize the sustained impact for would￾be SME borrowers, the shift away from collateral focused underwriting will need to be adopted by the highest levels of the Jordanian financial sector. SME owners JLGF records indicate that approximately 300 enterprises participated in capacity building training. Based on the survey of SME training participants and borrowers of JLGF guaranteed loans, more than 80 percent of SME and start-up respondents say that changes they have made to their business will be permanent. Table 12 illustrates response frequencies on the topic of how likely respondents believe changes to their business practices will be following JLGF assistance. Table 12: Survey responses regarding perceptions of permanence to business practices When it comes to permanence and sustainability, do you expect the changes you have made to your business as a result of JLGF technical assistance to be: Response Frequency SMEs Percent SMEs Frequency Start-ups Percent Start-ups Permanent and effective in the long-run. 52 83.9 44 86.3 Temporary and effective in the short-run. 8 12.9 5 9.8 Don’t Know 2 3.2 2 3.9 Total 62 100 51 100 Based on in-depth interviews, the consensus was that these permanent changes revolve around reading and preparing financial statements, record keeping, and effectively negotiating/communicating with banks. EQ4: WHAT LESSONS LEARNED AND RECOMMENDATIONS CAN BE DEDUCED FROM THIS ACTIVITY FOR FUTURE SIMILAR PROGRAMMING TO BE MORE EFFECTIVE AND FOR SUSTAINING USAID’S INTENDED RESULTS OF INCREASING ACCESS TO FINANCE? Conclusion 22: Loan guarantees partnered with SME and banking sector technical assistance is a sound intervention model for the Jordanian context. The JLGF implementation approach, post FY 2013, was an effective one in principle. It was made up of three components: (1) a loan guarantee of sufficient coverage to change risk calculations, (2) technical assistance to build bank capacities to meet international SME lending practices, and (3) direct interaction with SMEs, aiding them in meeting OPIC and bank lending criteria. While conclusions regarding scale, time line, and implementation challenges were presented previously, this report further concludes that this three pronged intervention “package” does effectively contribute to improved access to finance for SMEs. The evaluation team caveats this conclusion, noting that contribution is not the same as attribution, and testing the impact of individual intervention arms, as compared with the package, was beyond the scope of this evaluation. Furthermore, there were Final Evaluation Report – Jordan Loan Guarantee Facility 38 significant economies of efficiency created when partnering with banks that were already eager to expand their SME portfolios. Conclusion 23: Outreach efforts and limited linkages with industry associations could be improved. Systematic institutional awareness and follow up approaches are highly valued by respondents from SMEs, SME associations, and chambers of commerce and industry. As noted previously, a large scale effort to improve the capacities of SMEs, especially those outside of Amman, was beyond the scope of this activity; however, improving the qualification of SMEs to meet bank requirements is seen as a constraint going forward to rapidly scaling SME access to finance across Jordan. Furthermore, all SME associations interviewed for this evaluation agreed that the level of awareness of the services JLGF provides is generally low.54 Conclusion 24: The time to forge sustainable partnerships is now. The central aim of the JLGF activity was to increase access to finance for SMEs. This in theory would lead to growth in the SME sector and thus more job opportunities for the Jordanian people. However, to accomplish these higher level objectives, the scale of JLGF will need to be expanded. The two other major actors besides JLGF in this space are JLGC and the CBJ. All three parties seek greater cooperation and can complement the other’s comparative advantages. Linkages to other major funds, such as those by various European donors, the World Bank, and International Monetary Fund are also ripe for collaboration. This is due to the level of foreign interest in the immediate and medium term stability of the Jordanian economy given the intense macro-economic and political factors the country faces. Conclusion 25: Limitations in performance monitoring constrain adaptive management. Improvements to JLGF’s performance management plan would have likely contributed to better activity results, chiefly: (1) caught problems in the initial approach earlier, (2) improved JLGF’s value proposition to senior bank managers (i.e., ability to present strong evidence assuaging perceptions of risk), and (3) improved JLGF’s understanding of partner perceptions and thus contributed to improved linkages. RECOMMENDATIONS Per the USAID Evaluation Policy, recommendations should be action-oriented, practical, and specific. Based on the findings and conclusions provided in this report, recommendations were generated collaboratively with USAID in order to ground truth their applicability to Agency bureaucratic and resource realities. The below list is meant to represent manageable but vital considerations for future Agency investments of similar scope to JLGF. Recommendation 1 For a potential follow-on activity or for activities of similar scope, USAID should include in the activity’s design explicit enterprise targeting criteria, including gender-related and geographic targets, and articulate how outreach to these enterprises will be conducted and monitored. This may include establishing tiers of enterprise beneficiaries and developing separate outreach protocols for each type. 54 Five SME associations were interviewed. Two of the three outside of Amman had not heard of JLGF. The third had only heard about JLGF through another mechanism. The two associations located in Amman were aware of JLGF; however, recognized that counterparts outside of the capital region were unlikely to know of the activity, based on their perception of JLGF outreach efforts. Final Evaluation Report – Jordan Loan Guarantee Facility 39 Recommendation 2 For a potential follow-on activity, USAID should include in the design actions to gradually transfer ownership of technical assistance delivery to a local partner. Likely candidates for aspects of this include:  JLGC due to their extensive bank partnerships;  The CBJ due to their regulatory and policy role; and  SME Associations due to their potential capacity to raise awareness of the facility and provide trainings to enterprise owners/controllers Recommendation 3 For a potential follow-on activity, USAID should include in the design actions to pilot the gradual and responsible wind-down of JLGF credit underwriting reviews. This would likely start with the most active banks utilizing the facility. It is expected that this gradual wind-down would support long-term sustainability by testing the institutionalization of cash-flow based credit underwriting best practices among partner banks. Recommendation 4 For a potential follow-on activity, USAID should ensure that the servicer/implementing partner establishes dedicated units concerned with (1) outreach to enterprises and new partners, especially outside Amman; (2) managing relationships with bank senior managers, specifically addressing requests and concerns so as to improve senior-level buy-in; and (3) building institutional capacity within banks and partners targeted to takeover technical assistance components after the end of USAID support.55 Recommendation 5 Should the JLGF servicing agreement continue, USAID should revise the activity’s M&E plan in the following ways:  Include only goal level indicators relating to access to finance. For example, percent of bank credit lent to SMEs, likely utilizing the Jordanian definition.  Disaggregate all relevant indicators relating to the target beneficiaries. For example: a. Both number and value of loans for women-owned or controlled enterprises; b. Both number and value of loans for enterprises by governorate; c. Training hours for each tier of enterprise (start-up, small, medium) d. Value of credit accessed by training participants (utilizing activity portfolio database for loan information within the facility and conduct a periodic assessment among participants to understand if they have accessed finance elsewhere).  Formally monitor credit underwriting changes within partner banks by monitoring the proportion of deficiencies in guarantee application requests.  Include sustainability indicators which may include the following: a. Percent of bank senior managers citing permanent and positive changes in SME credit underwriting procedures. b. Number of SME Associations that have hosted a joint training with JLGF. c. Percent of guarantee applications that have fewer underwriting gaps than in an established baseline. Recommendation 6 In an effort to better understand its target clients and provide tailored assistance, USAID should integrate into its activities of similar scope periodic assessments capturing SME characteristics and 55 Or identify a separate entity that would be able to continue the delivery of trainings and develop training curricula. Final Evaluation Report – Jordan Loan Guarantee Facility 40 needs. Relatedly, stakeholder mapping would be useful to better understand needs and potential linkages with local partners and improve the likelihood of sustaining outcomes. This could take the form of conducting a longitudinal study of loans provided and could contribute to Agency and stakeholder learning about the durability of jobs created when Jordanian SMEs gain access to finance. Periodic assessments could also inform future efforts of similar scope by collecting SME perspectives relating to loan repayment, specifically why specific borrowers have required loan restructuring. Recommendation 7 Should start-ups and smaller enterprises remain a target beneficiary in future activities of similar scope, USAID should (1) contribute to stronger and more capable support networks, likely through existing enterprise associations (such as JWIC), and (2) implement follow-up procedures to ensure beneficiary needs are met, thereby tailoring additional support as businesses increase in size and sophistication. These two adaptions would improve the likelihood of sustaining outcomes by transferring JLGF’s current knowledge, capacity, and responsibility to capable enterprise associations and high-growth SMEs. Final Evaluation Report – Jordan Loan Guarantee Facility 41 ANNEXES Annex A: Evaluation Statement of Work Annex B: Evaluation Design with Instruments Annex C: References and Works Cited Annex D: Implementation Timeline Annex E: Performance Monitoring Targets and Achievements Annex F: Statistical Annex Final Evaluation Report – Jordan Loan Guarantee Facility 42 ANNEX A: EVALUATION STATEMENT OF WORK USAID Jordan Loan Guarantee Facility (JLGF) Performance Evaluation Statement of Work (SOW) Introduction USAID/Jordan requires an external final performance evaluation of the USAID/ Jordan Loan Guarantee Facility (JLGF) Activity, a five-year cooperative agreement with Global Communities, with a total value of $9,344,720, covering the performance period of October 2011- October 2016. Background and Activity Description The Overseas Private Investment Corporation (OPIC) established the US$250 million Jordan Loan Guarantee Facility (JLGF) whereby USAID provided technical assistance to help set up the Facility and assist OPIC in the negotiation with the banks that signed the Guarantee Facility Agreements (GFAs). The US$250 million OPIC/JLGF was intended to improve access to finance for small and medium enterprises (SMEs) in Jordan, through partnering with local financial institutions. Following the establishment of OPIC/JLGF, USAID decided to design a cooperative agreement called USAID/Jordan Loan Guarantee Facility (JLGF) Activity to provide technical assistance to both SMEs and financial institutions to encourage the use of the up-to US$250 million OPIC backed loan guarantees. This Activity (USAID/JLGF), coupled with the OPIC/JLGF (worth US$250 million), provides customized technical assistance for partnering banks, Non-bank Financial Institutions (NBFIs), and SMEs. The USAID/JLGF Activity was designed to also strengthen the capacity of the Jordanian financial services sector to continue serving the credit needs of the SME market in a sustainable manner. Details of the activity to be evaluated: Development Objective: USAID/Economic Development and Energy Office Activity Title: USAID – Jordan Loan Guarantee Facility (JLGF) Award Number: AID-278-A-12-00001 Award Dates: 13 October 2011 – 12 October 2016 Funding: $9,344,720.00 Implementing Partner: Global Communities AOR: Shatha Al-Haj Purpose of the evaluation The objective of this evaluation is to provide USAID with findings and strategic recommendations related to a) the effectiveness of the Activity’s interventions, mainly the technical assistance that USAID/JLGF provided to banks and SMEs, and b) the lessons learned and best practices to be utilized by USAID in the future. The evaluation results will help inform USAID’s design and implementation decisions for similar future programming. USAID/JLGF is promoting long-term sustainable development through high-impact partnerships with the private and financial sectors in Jordan to facilitate the introduction and application of international best practices with respect to SME financial management and SME credit underwriting standards. Final Evaluation Report – Jordan Loan Guarantee Facility 43 Description of the problem Small and medium enterprises (SMEs) continue to be the key to innovation and the engine of economic growth and job creation in Jordan. In total, there are more than 100,000 SMEs in Jordan, representing roughly 95 percent of all registered companies (Jordan Department of Statistics). These enterprises are core to the national economy as they contribute 50 percent or more to the Gross Domestic Product (GDP); and provide employment to an estimated 60 percent of the Jordanian workforce. Despite the crucial role SMEs play in Jordan’s economic growth, they have great difficulty accessing the finance needed for start-up and growth. The SME financing gap in Jordan has been estimated at more than $3 billion. In particular, SMEs in Jordan are frequently unable to obtain financing from banks. Although more than half of the formal SMEs in Jordan maintain bank accounts, most do not have access to credit. Women-owned SMEs are particularly challenged in accessing finance. While many women-owned businesses have successfully accessed microcredit, graduation to SME loans from commercial banks has been limited. The volume of bank financing for SMEs is limited due to a lack of financial information and weak financial infrastructure, including lack of credit information, weak creditor rights, and deficient secured transactions (collateral) infrastructure. Bankers are unable to access standard credit history information and typically rely on real estate collateral and personal relationships to make credit decisions. Banks also suffer, by varying degrees, from inadequate credit underwriting practices and a lack of familiarity with financial service marketing opportunities in the SME marketplace. Objectives and Theory of Change JLGF’s goal is to improve and expand sustainable access to financial services for SMEs in Jordan, facilitating SME growth and job creation. JLGF was developed to support USAID/Jordan’s efforts to strengthen small businesses as drivers of job creation and maintenance of employment in order to generate more balanced economic growth that creates jobs in Amman and other urban areas where 75 percent of the Jordanian population resides. JLGF’s Theory of Change: JLGF’s goal (to improve and expand sustainable access to finance for SMEs facilitating SME growth and job creation) is supported by two program objectives: Objective 1: Mobilize bank lending for SMEs using loan guarantees: Access to financing is crucial to facilitate upgrading and expansion of the equipment, facilities and production of Jordanian SMEs. JLGF will use partial loan guarantees to incentivize Jordanian banks to increase lending to the underserved SME sector. This objective covers includes two technical interventions: a) the utilization of the OPIC $250 million loan guarantee, and b) the technical assistant provided to SMEs to become bankable and access the guarantee. Objective 2: Strengthen bank capacity for sustainable SME lending JLGF will build the capacity within the main partnering Jordanian financial institutions to efficiently and effectively meet the debt financing needs of local SMEs. JLGF will provide extensive classroom and on-the-job training to build the ability of Jordanian bankers to profitably lend to SMEs. By developing this internal capacity, JLGF will enable a sustainable increase in lending, facilitating long-term SME growth and significant job creation. Final Evaluation Report – Jordan Loan Guarantee Facility 44 USAID JLGF Results Framework 56 Year 4 actuals as of August 30, 2015 57 The number of person hours of training completed in private sector productive capacity supported by USG assistance may appear less than previously reported as it has been adjusted to reflect the workshops held for SMEs only as it previously included awareness sessions as well. Indicator / Definition Units Year 1 Actuals 2012 Year 2 Actuals 2013 Year 2 Target 2013 Year 3 Actuals 2014 Year 3 Target 2014 Year 4 Actuals56 2015 Year 4 Target 2015 Year 5 Target 2016 Goal: Improve and expand sustainable access to finance for SMEs facilitating SME growth and job creation Number of persons receiving new employment or better employment as a result of participation in USG￾funded workforce development programs. Number Persons 66 498 1,800 924 2,400 974 2,400 1,000 Objective 1: Mobilize bank lending for SMEs using loan guarantees Number of loans guaranteed by JLGF. (Number of SMEs that successfully accessed bank loans or private equity as a result of USG assistance.) Number Loans / SMEs 6 41 40 80 60 108 80 110 Volume of loans guaranteed by JLGF. $ 2,985,876 8,215,819 20,000,000 19,560,734 30,000,000 26,529,314 19,200,000 26,840,000 Number of firms receiving USG assistance to invest in improved technologies. Number Firms 5 22 27 37 24 14 15 17 Number of woman owned businesses financed. Number Businesses 0 7 3 7 5 17 7 18 Person hours of training completed in private sector productive capacity supported by USG assistance. Person Hours 1,206 2,016 720 1,992 1,620 75057 1,620 900 Objective 2: Strengthen bank capacity for sustainable SME lending Number of financial institutions receiving USG assistance in extending services to micro and small businesses. (# Partner banks) Number Institutions 5 7 7 7 8 7 8 8 Person hours of training completed by financial sector professionals on international standards supported by USG assistance. Person Hours 1,456 876 500 1,145 1,120 1,844 1,120 675 DRAFT Final Evaluation Report – Jordan Loan Guarantee Facility 45 Evaluation Questions In addressing the stated purpose, the evaluation will explicitly answer the questions stated below. Effectiveness: 1. How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? a. To what extent were steps taken to address gender differences and/or gaps? b. What gaps or approaches should the activity have addressed or adopted, if any, in order to be more effective and efficient in implementation? c. Was the cost and effort expended on the achieved results appropriate? Were there opportunities for cost and implementation efficiencies, and what are they? 2. What are the impacts of the activity on the key target beneficiaries? a. What was the effect of the activity on: 1) the financial sector, 2) SMEs and women￾owned SMEs, and 3) employment? b. What strategies or approaches were key in the activity’s success in achieving its goals? And what strategies or approaches hindered it? c. What do the beneficiaries perceive as critical elements of the support they received through the activity? Why did they find them critical? Sustainability: 3. What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? Learning: 4. What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance? EVALUATION DESIGN AND METHODOLOGY The external evaluation will use quantitative and qualitative approaches to assess how effective the Activity was in meeting its objectives and to provide recommendations on how to make future interventions with similar objectives more effective and more sustainable. The evaluation will focus particular attention on both the demand and supply sides in the access to finance sector, i.e. both the seven partnering banks and SMEs who received the USAID/JLGF technical assistance. Up until the date of drafting this SOW the USAID/JLGF Activity supported 270 SMEs in accessing finance and taking out loans worth a total of $62,628,749 (JOD 44,341,154) which is about 25 percent of the $250,000,000 maximum guarantee provided by OPIC. EVALUATION TEAM COMPOSITION In order to meet the requirements of team composition, ensure data quality, and contribute to building capacity of local evaluation specialists, the following is suggested for team composition: 1. Team Leader (position combined with Evaluation Specialist or Technical Specialist) 2. Jordanian Technical Specialist 3. Research Coordinator/Note taker (2) The evaluation team will also be supported by the MESP Evaluation Manager, Senior M&E Specialist and Evaluation Assistant. Final Evaluation Report – Jordan Loan Guarantee Facility 46 PERFORMANCE PERIOD The evaluation will be conducted from April 2016 through July 2016 with data collection conducted in May to Mid-July and final report submitted by the end of July 2016. Logistics for the assessment and evaluation will be provided by MESP. DELIVERABLES AND TIMELINE MESP finalize SOW, begin desk review of JLGF, develop work plan. April MESP develop evaluation methodology and tools, finalize work plan and submit evaluation design report Second week of May Field Work May – Mid July Debriefing presentation for USAID and JLGF on evaluation findings, initial conclusions and recommendations Mid/End of July USAID and Evaluation team collaboration on developing the recommendations  USAID Participation in the team FCR Session  Meeting/s between the evaluation team and USAID to develop actionable recommendations Mid/End of July MESP submit draft report End of July  The evaluation report will not exceed 30 pages and must adhere to USAID’s Evaluation Policy; it must include a table of contents, list of acronyms, and executive summary as well as a copy of the SOW and data collection instruments;  The report will address each of the key questions identified in the relevant sections of the SOW and any other factors the team considers to have a bearing on the objectives of the evaluation;  The key evaluation questions must be answered, and recommendations must be stated in an actionable way with defined responsibility for the action;  Sources of information will be properly identified and listed in an annex;  The assessment and evaluation reports will be published on USAID’s Development Experience Clearinghouse at edec.usaid.gov.  Upon request from USAID or closure of MESP, both electronic and hard copy data files will be transferred to USAID. In the meantime, electronic files are on the MESP file and hard copies are warehoused at MESP. MESP submit final evaluation report Mid of August DRAFT Final Evaluation Report – Jordan Loan Guarantee Facility 47 ANNEX B: EVALUATION DESIGN WITH INSTRUMENTS A. INTRODUCTION Project Summary JLGF’s goal is to improve and expand access to financial services for SMEs in Jordan, facilitating SME growth and job creation. JLGF contributes to USAID development objective 1: Broad-based, Inclusive, Economic Development Accelerated and 4 Gender Equality and Female Empowerment Enhanced. JLGF supports USAID/Jordan’s efforts to strengthen small businesses as drivers of job creation and maintenance of employment in order to generate more balanced economic growth that creates jobs in Amman and other urban areas where 75 percent of the Jordanian population resides. The objective of this evaluation is to provide USAID with findings and strategic recommendations related to a) the effectiveness of the activity’s interventions, mainly the technical assistance that USAID/JLGF provided to banks and SMEs, and b) the lessons learned and best practices to be utilized by USAID in the future. The evaluation results will help inform USAID’s design and implementation decisions for similar future programming. USAID/JLGF is promoting long-term sustainable development through high-impact partnerships with the private and financial sectors in Jordan to facilitate the introduction and application of international best practices with respect to SME financial management and SME credit underwriting standards. Evaluation Questions Effectiveness: 1. How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? a. To what extent were steps taken to address gender differences and/or gaps? b. What gaps or approaches should the activity have addressed or adopted, if any, in order to be more effective and efficient in implementation? c. Was the cost and effort expended on the achieved results appropriate? Are there opportunities for cost and implementation efficiencies, and what are they? 2. What are the impacts of the activity on the key target beneficiaries? a. What was the effect of the activity on: 1) the financial sector, 2) SMEs and women￾owned SMEs, and 3) employment? Particular attention is placed in assessing the impact of JLGF activities on MSE’s access to finance. b. What strategies or approaches were key in the activity’s success in achieving its goals? And what strategies or approaches hindered it? c. What do the beneficiaries perceive as critical elements of the support they received through the activity? Why did they find them critical? Sustainability: 3. What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? Learning: 4. What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of increasing access to finance? B. EVALUATION DESIGN Final Evaluation Report – Jordan Loan Guarantee Facility 48 This section summarizes the evaluation team’s general understanding and approach to answering the evaluation questions. Understanding of the Evaluation Questions Two of the four evaluation questions focus on the concept of effectiveness, while the concepts of sustainability and learning are captured by the remaining two questions. Below we provide an outline of how the evaluation team will capture and operationalize these concepts to answer the four evaluation questions. Effectiveness: For examining effectiveness the team will measure the performance of the activity against its objectives by focusing at the output and the outcome level. In this effort the team will combine both secondary and primary data and include the point of view of different actors. The aim will be to assess the extent to which the JLGF strategy and intervention has been effective in providing training and technical assistance to both banks and SMEs in order to facilitate access to tailored financial services, in particular to women-owned enterprises and outside Amman. The effectiveness of the JLGF can be assessed by determining the extent to which its strategy and intervention have been successful in addressing the miss-match between the banks processes and requirements for extending a loan and the capacity of SMEs to respond to those requirements. From the banks stand point, the degree of success of the JLGF intervention is measured by the change in approach of the bank’s lending methodology from one based on the quality and value of a collateral to one based on the business cash flow. Also the success may be assessed by the changes made by banks for tailoring their products to the specific characteristic of SMEs, including women-owned enterprises. From the SMEs stand point the degree of success depends upon the capacity of the business owner and SMEs’ management to present reliable financial and commercial information of the business and its expansion plan. The final test of accomplishment materializes on a successful approval and repayment of a loan and in resulting effect on business growth and the capacity for generating employment. To better capture the effectiveness of the JLGF the design includes both quantitative and qualitative data, from primary and secondary sources and an analysis that comprises before and after comparisons, degree of achieving targets, the team will review similar interventions in the region and assess the usefulness of comparing JLGF with those identified as the most relevant. The team will also examine the implementation approach used by JLGF in comparative perspective (comparing it against other similar interventions) to assess how well suited the approach was to achieve the specific objectives as well as whether there are more efficient approaches to accomplishing the same objectives. Similarly, the evaluation will compare the data for the recipients of the loan guarantee against the data of SMEs who were declined such guarantees to identify patterns and assess whether there were any systematic gaps in the implementation approach. The effectiveness of the activity will be examined at both the aggregate and dis-aggregate level, for example by gender, location, economic activity, etc. This will be facilitated by critically examining the existing JLGF beneficiary dataset as well as through the qualitative data collected in IDI with SMEs; KII with partner banks’ high and medium level management; focus group discussions with bank’s loan officers and costumer representatives trained by JLGF. In addition, quantitative data collected through a representative survey of SMEs and Star-ups that have received services from JLGF will be incorporated to the analysis. Other stakeholders’ perspective such as SME’s associations and chambers of commerce and industry, trainers and Institute of Banking Studies are taken into account through KII. The perspectives of USAID JLGF, other USAID relevant projects and the implementing partner will also be captured to ensure that considerations associated with activity design and implementation is sufficiently factored in the overall analysis. Sub-groups of interest, for example women-owned SMEs and SMEs outside of Amman would be oversampled for qualitative data collection and fully covered in the quantitative survey. This will allow Final Evaluation Report – Jordan Loan Guarantee Facility 49 the evaluation team to understand their experiences as well as to capture potential areas of improvement for these sub-groups. Lastly, in gauging effectiveness, the evaluation will also look at the broader context and factors beyond the JLGF’s sphere of influence. In this regard, the evaluation team will review the existing rules and regulations, as well as take into consideration the general profile of SMEs and the banking sector, and particularly supply and demand issues of SME’s finance in Jordan. Sustainability: For the question focusing on sustainability the aim is to assess the extent to which the project has contributed to creating the conditions for expanding and sustaining access to SMEs financing. The key sustainability issues will be addressed from different perspectives (supply and demand side) highlighting the approaches and practices that are viewed as more effective. The evaluation will also examine some of the earlier beneficiaries to gauge their experiences and performance overtime. On the supply side, the evaluation team will examine whether and the extent to which the JLGF intervention impacted the willingness of lending institutions to extend loans to SMEs in the future and to incorporate credit analysis and tools that emphasize cash flow projections rather that collateral. Using primary and secondary data, the evaluation team will also review system and sector wide incentives and constraints that effect prospects of sustainability. Lessons Learned: The research effort will capture the most significant lessons that different actors identify as contributing to developing SME lending in Jordan. Particular attention will be directed towards drawing actionable recommendations from these lessons. The representative survey of SMEs and start-ups trained by JLGF and the IDI to the seven partner banks will allow the team to adequately capture the different perspectives. This will also allow the evaluation team to assess whether and the extent to which their perspectives converge or diverge by respondent type. Approach to Answering the Evaluation Questions This section details the evaluation team’s approach to answering the evaluation questions. JLGF provides partial loan guarantees and technical assistance to mobilize bank financing for creditworthy but underserved SMEs. This includes customized technical assistance for partnering banks and SMEs. To capture effectiveness, sustainability and lessons learned this evaluation will start by critically mapping and reviewing the intervention model and all of its associated processes. The evaluation team will utilize quantitative (representative survey) and qualitative (key informant interviews and focus groups) data collection approaches to capture the opinions and perspectives of the different stakeholders in the activity intervention model. The evaluation team will focus on stakeholders on both the demand and supply side of access to finance. Demand side: The evaluation team will be conducting both qualitative and quantitative research with the SMEs that benefitted from JLGF technical assistance. The sampling approach has been developed based on the following key considerations: 1. JLGF Technical Assistance focus 2. General feasibility, in terms of the process required to contact the SMEs, whether through JLGF or the Banks themselves 3. Implementing partner and Bank’s feedback Final Evaluation Report – Jordan Loan Guarantee Facility 50  Qualitative Sampling Approach: The qualitative approach covers a number of different beneficiaries, including: o SMEs who only received training from JLGF, o SMEs who received training and loan guarantees from JLGF, and o SMEs who only received the loan guarantee.  Quantitative Sampling Approach: The quantitative sample is based on the list of those SMEs that directly participated in the trainings offered by JLGF. Some of these SMEs also benefitted from the loan guarantees, though there number is relatively small. The survey approach was deemed as the most suitable to cover this group, given their direct contact with the JLGF team. In terms of feasibility, it was also easier to approach these SMEs in a relatively efficient manner, since, in accordance with the feedback provided by the JLGF team, the evaluation team does not need to coordinate and get approval from the banks. Another justification for focusing the survey on this group is because of the overall importance of trainings (technical assistance) to the JLGF model.  Organizations that represent JLGF target Group: JLGF also worked with associations and chambers of industry and commerce to obtain a list of SMEs that could be potential beneficiaries of JLGF. As a link between JLGF and SMEs they will likely have relevant insights on the extent to which the services offered respond to the needs of their associates and ways in which those services may be more effective in supporting SMEs growth. The evaluation team included the following key informants from within these associations: o Zarqa Chamber of Commerce, o Irbid Chamber of industry, o Irbid Chamber of Commerce o Jordan Women Industrial Community (JWIC) o Jordan Industrial SMEs Association  Banks: While the banks are on the supply side when it comes to lending, within the context of this intervention as beneficiaries of JLGF’s technical assistance (trainings) we will also assess the training and technical assistance needs as articulated by the banks themselves. The evaluation will include the bank staff that receive the JLGF training mainly through focus group discussions although the evaluation team will consult with the banks to gauge the feasibility of this option given the research time frame and offer alternatives to capture trainees insights such as in-depth interviews and questionnaires. Supply Side: On the supply side the evaluation team will consult with banks, trainers and JLGF and USAID team.  Banks: The evaluation team will consult with all seven participating banks that JLGF works with and interview either high management or SMEs departments. The evaluation team will compare the extent to which JLGF interaction varies by banks. The evaluation will focus on assessing JLGF contributions to changing the banks’ approach to lending practices (cash flow based lending methodology versus the traditional collateral based). The team will review the activity indicators and examine banks’ infrastructure for SME services.  Trainers: In the assessment of the supply side of the technical assistance, the evaluation will also capture the views of the training providers including the Institute of Banking Studies (IBS) and will also examine the training content.  JLGF and USAID Team: As implementers and designers of JLGF, the JLGF and USAID team will also be covered to ensure that their critical roles and perspectives are adequately understood.  Related USAID projects: The evaluation team will be interviewing key respondents within the relevant USAID projects who support the SMEs in Jordan. The optimal targets for interviewing are Chief of Party and / or Senior Technical Advisors (Specialists) in the relevant USAID SMEs supporting projects. The evaluation team found (4) relevant USAID projects. USAID LENS and USAID JCP are related to development and competitiveness of SMES; and USAID BEST and USAID ESCB are sectoral in nature but focus on addressing gender issues. Final Evaluation Report – Jordan Loan Guarantee Facility 51  Government program targeting SMEs: The evaluation team will interview the Jordan Enterprise Development Corporation-JEDCO, a governmental organization, seeks to increase the competitiveness of the SMEs in industry, service and agricultural sectors. In addition to examining the supply and demand side of JLGF, the evaluation will also assess the broader context including the different regulations affecting SME’s financing in Jordan. C. DATA COLLECTION METHODS This evaluation will rely on secondary and primary data collection. Secondary Data Collection The team will review and incorporate into the analysis relevant secondary data sources, following are some key pieces of secondary data that they team has considered:  JLGF Technical Proposal  JLGF Work Plans  JLGF’s Quarterly Reports  SMEs beneficiaries data base  Workshops attendees for both SMEs and banks’ staff  Database of workshop attendees evaluation of the training activities  Database of rejected loan applicants  Reports and statistics of the Central Bank of Jordan (CBJ)  A Survey Study of SMEs in Jordan: Analysis in Supply-side and Demand-side focusing on bank financing, conducted by the Studies Department at the Association of Banks in Jordan. Primary Data Collection For primary data collection the evaluation team will rely on using mixed methods, qualitative and quantitative research. Qualitative Research: Under qualitative research the evaluation team plans on doing key informant interviews (KII), in-depth interviews (IDIs) and focus groups. Below is a summary of the data collection method by respondent type:  Key informant in-depth interviews will be conducted with the following key informants. The team anticipates a total of no less than 30 such interviews: o JLGF Staff  Chief of Party  Monitoring and Evaluation team  Gender and Training specialist  Credit and Risk Specialist o USAID Team o Banks  SME department  Upper Management  Loan officers o Trainees for SMEs o Trainers for SMEs and Banks o Other relevant USAID projects. o Associations and chambers of industry and commerce  Focus Groups with bank staff trained by JLGF. It is anticipated that a FGD session will be conducted in all 7 partner banks. Although focus group discussion is the main method to be Final Evaluation Report – Jordan Loan Guarantee Facility 52 used for bank staff trained by JLGF the team will consult the banks and accommodate to the feasibility of this option and offer to conduct IDI or submit an open ended questionnaire covering the same subjects than the ones discussed in a FGD. The evaluation team also assessed the feasibility of doing focus group discussions with SMEs but based on the feedback received from the implementing partner (specifically related to the difficulty involved in bringing them together), the team has decided to do in-depth interview and a phone survey of this group. Sampling Approach for SME Data Collection The evaluation team will be conducting both qualitative and quantitative research with the SMEs that benefitted from JLGF technical assistance. The sampling approach has been developed based on the following key considerations: 1. JLGF Technical Assistance focus 2. General feasibility, in terms of the process required to contact the SMEs, whether through JLGF or the Banks themselves 3. Implementing partner and Bank’s feedback  Qualitative Sampling Approach: The evaluation team plans on conducting around 30 in￾depth interviews with SMEs. The qualitative approach covers a number of different beneficiaries, including: o SMEs who only received training from JLGF, o SMEs who received training and loan guarantees from JLGF, and o SMEs who only received the loan guarantee. In terms of process, the evaluation team identified these beneficiaries and requested JLGF to contact them directly to gauge their availability and interest in participating in this research. Those who agreed to participate in the research were then independently contacted by the evaluation team to set meeting appointments and to conduct in-depth interviews. The evaluation team is also following up with Banks to arrange interviews with beneficiaries who only received the guarantee from JLGF. This distinct approach is being used as both the banks and JLGF team recommended it as the most appropriate way to contact SMEs belonging to this category.  Quantitative Sampling Approach: The quantitative sample is based on the list of those SMEs that directly participated in the trainings offered by JLGF. Some of these SMEs also benefitted from the loan guarantees, though there number is relatively small. The survey approach was deemed as the most suitable to cover this group, given their direct contact with the JLGF team. In terms of feasibility, it was also easier to approach these SMEs in a relatively efficient manner, since, in accordance with the feedback provided by the JLGF team, the evaluation team does not need to coordinate and get approval from the banks. Another justification for focusing the survey on this group is because of the overall importance of trainings (technical assistance) to the JLGF model. In terms of process, the evaluation team has built a sampling frame using attendance sheets from the JLGF trainings. The evaluation team then obtained the contact information for these SMEs by going through the registration forms for the trainings. The sample is being shared with the JLGF team who makes quick phone calls to the SMEs to gauge their interest and availability for participating in the study. Those who agree to participate in the survey are then referred to the mindset team who will conduct the phone surveys with the SMEs. This approach of having the JLGF team reach out to the SMEs first is particularly useful, as it enhances the possibility of SMEs participating in the phone survey. This effects the overall response rate for this survey. The evaluation team had initially considered the possibility of setting up the survey sample using the database consisting of SMEs that benefitted from loan guarantees. However, only a small percentage of those who received guarantees actually received any training from JLGF. Moreover, Final Evaluation Report – Jordan Loan Guarantee Facility 53 in terms of process, conducting a survey with the loan guarantee sample would also require getting permissions from individual banks, something that is not feasible within the current time frame for this evaluation. In consultation with the JLGF team it was decided to cover this group through qualitative research, which does not require a large sample and is more manageable in terms of coordination. Lastly, for the SMEs that benefitted from the loan guarantees, JLGF already has a very comprehensive database. The evaluation team will analyze this data and compare it against the data obtained from the survey focused on the training beneficiaries. The evaluation team constructed a dataset containing 293 registries of different SMEs/start-ups with contact information which became the target for the survey. This dataset was grounded on the Global Communities database covering 524 registries of SMEs and Start-ups registered to attend the JLGF and selecting only the ones whose signatures appear in the attendance sheets. In addition, the evaluation team will contact 45 SMEs to conduct in depth interviews. The Table below shows the number of SMEs in the population and to be contacted by method of data collection and the expected percentages of the population targeted by evaluation team. The actual percentage that will be effectively reach by the different methods will be determined by the rejection rates. Table 13. JLGF SME Beneficiaries Expected to be Included in the Evaluation by Data Collection Method JLGF Exposure Unit Population Data Collection Method In￾Depth Survey JLGF Data Base Total Included In the Evaluation Target as percent of Population Training and Guarantee loan SME 18 18 18 100% Training Only with contact information SME 282 10 256 266 94% Guarantee loans Guarantee loans 302 13 302 302 100% The survey results will allow for capturing a representative snapshot of SME perceptions related to effectiveness, sustainability and opportunities to improve such interventions. Given the near universal coverage, it will also allow the evaluation team to disaggregate the survey results by key variables of interest. In case we encounter a low response rate for the phone survey, the evaluation team will use the available statistics on the beneficiaries (beneficiary dataset) to understand what component of the overall beneficiary population is represented in the survey and the in-depth interview. This will inform how the team reports the results. Data Collection Instruments Content The team developed the 14 instruments shown in the Table to collect both qualitative and quantitative data (ANNEX). All instruments were developed addressing the research questions and tailored to the type of respondent and the data collection method. The following table shows the different instruments and the annex file contains the instrument. Table 14. JLGF Performance Evaluation: Data Collection Instruments by Respondent Type Types of Respondents Data Collection Method Instruments Supply Side Implementing Partner IDI 1. IP-Topic Guide Final Evaluation Report – Jordan Loan Guarantee Facility 54 Types of Respondents Data Collection Method Instruments Banks - Heads of SME Department / Upper Management IDI 2. Banks Head of SME /Upper Management topic Guide - Trainees - Credit officers FGD Open-ended questionnaire 3. Banks Trainees Focus Groups- Discussion Guides 3A. Bank Trainees Questionnaire Trainers including IBS IDI 4. Trainers Discussion Guide USAID JLGF IDI 5. USAID Topic Guide USAID Projects IDI 5. USAID Topic Guide JEDCO IDI 6 JEDCO Topic Guide JLGC IDI 7. JLGC Topic Guide OPIC IDI 8. OPIC Topic Guide CBJ IDI 9. CBJ Topic Guide Demand Side Associations and Chambers of Industry and Commerce IDI 10. MSEs Associations Topic Guide SMEs Training Beneficiaries IDI 11. SMEs In-depth Interview Topic Guide Survey SMEs / Start-ups Beneficiaries Phone Survey 11.A. SMEs Survey Questionnaire 11.B. Start-up Survey Questionnaire D. DATA ANALYSIS METHODS This evaluation will primarily rely on three types of analysis: 1. Before and After a. The unit of analysis under this type of analysis will be at the beneficiaries (Banks and SMEs) level. This line of analysis explores patterns and trends that may be observed in the data and the primary and secondary data collected. 2. Planned versus Actual a. The unit of analysis for this type of analysis will be at the implementing partner and USAID level. This line of analysis focuses on tracking the setting and achieving of targets on products, outcomes and outputs laid out on the Monitoring and Evaluation Plan and the changes introduced on the yearly work plan during the execution of the project. The analysis will reflect the learning process of all the actors involved regarding specific socio-economic and institutional conditions in Jordan’s SMEs learning sector. 3. Performance versus Benchmarking a. The unit of analysis for this type of analysis will be at the activity level (JLGF or other similar projects). Understanding how the JLGF’s intervention model and processes are positioned on the local, regional and international arena will enrich the interpretation of the lessons learned and gives the team additional insights for recommending actionable and specific improvements. The team will be very selective on the kind of program is going to be included in the comparison. The above-mentioned analysis will be undertaken using the data collected through qualitative and quantitative research. The evaluation team will carry out content analysis of key informant interviews, in-depth interviews and focus group discussions. Using the secondary and primary data, the team will make planned (strategies and implementation approaches) versus actual (performance) Final Evaluation Report – Jordan Loan Guarantee Facility 55 comparisons. The survey data will provide descriptive statistics, as well as cross-tabulation of survey responses against key variables of interest, for example Gender, size and purpose of loan, location etc. Appendix I. Getting to Answers (G2A) II. Work plan III. Data Collection Instruments IV. Roles & Responsibilities Final Evaluation Report – Jordan Loan Guarantee Facility 56 Appendix I: Getting to Answers Program or Project: JLGF Performance Evaluation Team Members: Miguel Cabal, Hisham Yasin, Ragheg Fityan Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, e.g., FGDs, Survey, Key Informant Interviews Sampling or Selection Approach (if one is needed) Data Analysis Methods Data Source(s) Method Effectiveness 1. How effective is the implementation approach (such as credit analysis, technical assistance to the SMEs and the tailored/generic in-bank training) in achieving the activity’s objectives? 1.a To what extent were steps taken to address gender differences and/or gaps? 1.b What gaps or approaches should the activity have addressed or adopted, if any, in order to be more effective and efficient in implementation? 1.c Was the cost and effort expended on the achieved results Yes/No - Performance monitoring plan - JLGF work plan - JLGF Database - Activity reports - Banks ( Managers and credit officers) - SMEs organizations - Trainers(IBS, others) - Secondary data sources - In depth interviews - Survey - Document & data review - Focus group discussions - Around 40 SMEs for in depth interviews taking in account (gender, size of loans, banks, and governorate); the rest (256) will be reached by phone surveys. - Interview key staff members in banks(SME department or upper management and the trainees) - 4 trainers - Associations and chambers of industry and commerce - Content analysis of KIIs - Comparison of the identified strategy and implementation approaches as stated in the activity documents. - Survey content analysis - Primary document analysis * Description * Comparison58 * Explanation59 58 Comparison – to baselines, plans/targets, or to other standards or norms 59 Explanation – for questions that ask “why” or about the attribution of an effect to a specific intervention (causality) Final Evaluation Report – Jordan Loan Guarantee Facility 57 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, e.g., FGDs, Survey, Key Informant Interviews Sampling or Selection Approach (if one is needed) Data Analysis Methods Data Source(s) Method appropriate? Were there opportunities for cost and implementation efficiencies, and what are they? Sustainability 2. What are the impacts of the activity on the key target beneficiaries? 2.a What was the effect of the activity on: 1) the financial sector, 2) SMEs and women-owned SMEs, and 3) employment? 2.b What strategies or approaches were key in the activity’s success in achieving its goals? And what strategies or approaches hindered it? 2.c What do the beneficiaries perceive as critical elements of the support they received Yes/No - Performance monitoring plan - JLGF work plan - JLGF Database - Activity reports - Banks (Managers and credit officers) - SMEs organizations - Trainers (IBS, others) Secondary data sources - In depth interviews - Survey - Document & data review - Focus group discussions - Around 40 SMEs for in depth interviews taking in account (gender, size of loans, banks, and governorate); the rest (256) will be reached by phone surveys. - Interview key staff members in banks (SME department or upper management and the trainees) - 4 trainers - Associations and chambers of industry and commerce - Content analysis of KIIs - Comparison of the identified strategy and implementation approaches as stated in the activity documents. - Survey content analysis - Primary document analysis * Description Comparison * Explanation Final Evaluation Report – Jordan Loan Guarantee Facility 58 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, e.g., FGDs, Survey, Key Informant Interviews Sampling or Selection Approach (if one is needed) Data Analysis Methods Data Source(s) Method through the activity? Why did they find them critical? Learning 3. What technical interventions and practices of the activity can USAID expect to continue without additional support? And which are unlikely to continue and why? Yes/No - Performance monitoring plan - JLGF work plan - JLGF Database - Activity reports - Banks ( Managers and credit officers) - SMEs organizations - Trainers(IBS, others) - Secondary data sources - In depth interviews - Survey - Document & data review - Focus group discussions - Around 40 SMEs for in depth interviews taking in account (gender, size of loans, banks, and governorate); the rest (256) will be reached by phone surveys. - Interview key staff members in banks(SME department or upper management and the trainees) - 4 trainers - Associations and chambers of industry and commerce - Content analysis of KIIs - Comparison of the identified strategy and implementation approaches as stated in the activity documents. - Survey content analysis - Primary document analysis - Qualitative analysis techniques ( analyze open ended questions, identify categories and coding) Description Comparison * Explanation 4. What lessons learned and recommendations can be deduced from this activity for future similar programming to be more effective and for sustaining USAID’s intended results of Yes/No - Performance monitoring plan - JLGF work plan - JLGF Database - Activity reports - Key Informant Interviews - Survey - Document & data review - Focus group discussions - Around 40 SMEs for in depth interviews taking in account (gender, size of loans, banks, and governorate); the rest (256) will be - Content analysis of KIIs - Comparison of the identified strategy and implementation approaches as stated in the activity documents. - Survey content analysis * Description Comparison * Explanation Final Evaluation Report – Jordan Loan Guarantee Facility 59 Evaluation Questions Type of Answer/ Evidence Needed (Check one or more, as appropriate) Methods for Data Collection, e.g., FGDs, Survey, Key Informant Interviews Sampling or Selection Approach (if one is needed) Data Analysis Methods Data Source(s) Method increasing access to finance? - Banks (Managers and credit officers) - SMEs organizations - Trainers (IBS, others) - Secondary data sources reached by phone surveys. - Interview key staff members in banks(SME department or upper management and the trainees) - 4 trainers - Associations and chambers of industry and commerce - Primary document analysis - Qualitative analysis techniques (analyze open ended questions, identify categories and coding) Final Evaluation Report – Jordan Loan Guarantee Facility 60 Appendix II: Work plan DESCRIPTION DELIVERABLE RESPONSIBLE DATES Inception Phase Desk review of project documents  Desk review  List of evaluation reference documents Team Leader Evaluation Team June 6 – 20 2016 Work plan submitted to USAID for approval  Work plan/ schedule Team Leader June 8 2016 USAID approves work plan TBD Design of the evaluation: methodology and tools  Research design report  Draft instruments in English  Draft instruments in Arabic Team leader Evaluation Team June 15 2016 USAID approves evaluation research design report TBD Implementation Phase Data Collection Phase 1 Data Collection through key informant interviews (KII) and in depth interviews, and focus groups (FG)  KII notes and summary  Focus group notes  Notes from the quarterly reports  Data Review Matrix Evaluation Team June 20 – July 4 2016 EID Holiday  Review of material  Review of data collection  Review of instruments  Review of selection criteria Team Leader July 5 – 9, 2016 USAID First Consultation: data collection challenges and opportunities  Presentation Team Leader & Evaluation team and MESP July 11, 2016 Final Evaluation Report – Jordan Loan Guarantee Facility 61 DESCRIPTION DELIVERABLE RESPONSIBLE DATES Data Collection Phase 2 Data Collection through key informant interviews (KII) and in depth interviews, focus groups (FG), complementary surveys (phone interviews) , and database  KII notes and summary  In-depth interview notes and summary  Focus group notes Team leader Evaluation Team July 12 – 21, 2016 Data Analysis  KII notes and summary  Focus group notes  Survey data (tentative) Team leader Evaluation Team July 21 – 25, 2016 USAID Second Consultation: Initial findings and actionable recommendations  Presentation Team leader Evaluation Team and MESP July 25, 2016 Analysis, De-briefing and Reporting Phase Final data analysis and drafting of the report  FCR Table  Draft report Team Leader & Evaluation team July 25 – 31, 2016 De-briefing of findings to USAID  PPT presentation Team Leader, Evaluation team and MESP July 31, 2016 USAID comments on draft evaluation report  Draft report with comments USAID Week of 7 August, 2016 Response to the USAID comments and update report  Updated report and response to comment table Team Leader, Evaluation team and MESP Week of 14 August, 2016 Final evaluation report incorporating USAID comments  Final evaluation report MESP, Team Leader and Evaluation team August 2016 DRAFT Final Evaluation Report – Jordan Loan Guarantee Facility 62 Appendix III: Data Collection Instruments Data Collection Instruments Types of Respondents Data Collection Method Instruments Supply Side Implementing Partner IDI 1. IP-Topic Guide Banks - Heads of SME Department / Upper Management IDI 2. Banks Head of SME /Upper Management topic Guide - Trainees - Credit officers FGD Open-ended questionnaire 3. Banks Trainees Focus Groups- Discussion Guides 3A. Bank Trainees Questionnaire Trainers including IBS IDI 4. Trainers Discussion Guide USAID JLGF IDI 5. USAID Topic Guide USAID Projects IDI 5. USAID Topic Guide JEDCO IDI 6 JEDCO Topic Guide JLGC IDI 7. JLGC Topic Guide OPIC IDI 8. OPIC Topic Guide CBJ IDI 9. CBJ Topic Guide Demand Side Associations and Chambers of Industry and Commerce IDI 10. MSEs Associations Topic Guide SMEs Training Beneficiaries IDI 11. SMEs In-depth Interview Topic Guide Survey SMEs / Start-ups Beneficiaries Phone Survey 11.A. SMEs Survey Questionnaire 11.B. Start-up Survey Questionnaire Final Evaluation Report – Jordan Loan Guarantee Facility 63 (A)Questions on Effectiveness/Impact of JLGF’s Contributions to Closing the Gap on SMEs Lending 1. To what extent was the technical assistance that was received relevant to the bank policies for providing loans to SMEs? Were all the relevant themes included? Do you think that there was a key theme that was not covered that is important in closing the SME lending gap? (B) Questions on the training of Bank Employees: 1. To what extent has the training provided by JLGF contributed to enhance the capacity of the SME’s credit officers? How? Alignment with Bank’s Strategy and Products  To what extent do the services provided by JLGF align with the overall bank strategy? How?  How does JLGF’s activity align with your other bank’s products? What about its alignment with other SMEs products?  What are the key features of the JLGF intervention that are most useful for the bank strategy and policy towards SMEs financing?  Would you please highlight any instrumental role you think JLGF has played in your banks SMEs lending and/or in dealing with the SMEs financing gap?  How are the activities provided by JLGF reflected in the banks’ operations related to SME lending?  Would you please state some examples in the areas (Policy, Structural, procedural, functional…) of the bank that were affected by the JLGF activity?  Did the implementation of JLGF activities affect any changes in your bank’s organizational structure with regard to SMEs financing specifically? Effectiveness in Changing Bank’s Credit Analysis Policies and Process  Would you please specify any changes in your bank’s lending policies and procedures that accompanied the implementation of JLGF?  How do you evaluate the effectiveness of the JLGF activity on your bank’s credit analysis process?  What types of obstacles and/or constraints did you face in the implementation of your work with JLGF?  How do you view the impact of the JLGF intervention on your bank’s performance, ways of doing business, credit appetite to SMEs, and credit view to SMEs, etc? Suggestions for improvements  Would you suggest ways in which JLGF services could be more effective in supporting the banks efforts to extend loans to SMEs? Final Evaluation Report – Jordan Loan Guarantee Facility 64 Relevance of training  How do you evaluate the training provided by JLGF to the bank employees?  Do you think that the training topics covered satisfy your needs as an SME credit officer?  Were the training topics relevant to your needs in applying the SMEs financing?  Which training topic gives you the most added value on SME financing?  Do you think that there are other training topics that were missed and not covered in the JLGF training? Training Impact  How do you think that the training activities contributed in the following: o Change the bank’s credit mentality o Improve the SME’s access to banks’ finance o Employment Training quality  How do you evaluate the trainers who delivered the JLGF training in terms of: o Qualification and experience? o Delivery approach (i.e. the training approach and the use of suitable training tools) Applicability of training in the Bank  How applicable to your day to day work do you find the training topics offered?  Are the training materials offered consistent with your work procedure at the bank?  To what degree do you see harmony between the JLGF training material on SMEs financing and your bank’s lending policies and procedures?  How easy do you find it to operationalize the training topics at your bank/department? Applicability to SME financing  How applicable is the training to the needs of the SMEs in different sectors or business lines?  Do you find that the training material reflects the changing needs of SMEs in Jordan? Final Evaluation Report – Jordan Loan Guarantee Facility 65 2. Comparing with other intervention models and procedures: o How the JLGF intervention model differs from other models, e.g. the JLG Corp. model? Suggestions for Improvement  Would you comment on How the training activities could be more effective and/or more efficient for future sessions of the training?  Do you suggest any specific changes in the JLGF training program to better fit the needs of:  Jordanian Banks  SME’s in Jordan  During your training, what is the best practical experience you went through and would like to capitalize on in future training?  What is / are the most important skills that you developed out of the JLGF training on SMEs financing and would like to capitalize on for the future? (C)Questions on sustainability: 1.Can you think of some best practices that you came across during the implementation of JLGF that worked towards the benefits of SMEs and/or your bank, or the benefit of any other beneficiary? 2.What are your thoughts on SMEs lending in the near future?  Is the financing to start-up SMEs available as an option at your bank now?  At your bank, do the current policies and procedures allow for flexibility in approving loan collaterals on the SMEs loans other than the traditional collaterals? Please specify if any? 3.What are the specific practices of the JLGF activity that your bank is likely to continue in the near future? And what are those that might not continue? Why? Questions on lessons learned: 1. What do you think is the most significant lesson you learned from participating in the JLGF experience? 2. Can you name the most important skills that you and/or other people at your bank learnt from the JLGF technical assistance? 3. Do you have any specific suggestion or recommendation to improve the work of the JLGF activity? Final Evaluation Report – Jordan Loan Guarantee Facility 66 BANKS’ HEAD OF SME /UPPER MANAGEMENT TOPIC GIDE (A) Questions on Effectiveness of JLGF’s Contribution to Closing the Gap on SMEs Lending? 1. To what extent the technical assistance received was relevant regarding the bank policies for providing loans to SMEs? Were all the relevant themes included? Do you think that there was a key theme that was not covered that is important in closing the SME lending gap? (B) Questions on the training on Bank Employees:  To what extent do the services provided by JLGF align with the overall bank strategy? How?  What are the key features of the JLGF intervention that are most useful for the bank strategy and policy towards SMEs financing?  How the activities provided by JLGF are reflected in the banks’ operations related to SME lending?  Would you please state some examples on the areas of the bank that affected by the JLGF activity?  How do you evaluate the effectiveness of the JLGF activity on your bank’s credit analysis process?  Would you please specify any changes in your bank’s lending policies and procedures that accompanied the implementation of the JLGF?  What types of obstacles and / or constraints you faced in the implementation of your work with JLGF?  Would you suggest ways in which JLGF services could be more effective in supporting the banks efforts to extend loans to SMEs?  How do you view the impact of the JLGF intervention on your bank’s performance, way of doing business, credit appetite to SMEs, credit view to SMEs,..etc.  Did the implementation of the JLGF affected any changes in your bank’s organizational structure with regard to SMEs financing specifically?  Is the financing to start-up SMEs available as an option at your bank now?  Do your current bank’s policies and procedures allow for flexibility in approving loan collaterals on the SMEs loans other than the traditional collaterals? Please specify if any? Final Evaluation Report – Jordan Loan Guarantee Facility 67 1. To what extent the training provided by JLGF contributed to enhance the capacity of the SME’s credit officers? How?  How do you evaluate the training provided by JLGF to the bank employees?  Do you think that the training topics covered satisfy your needs as an SME credit officer?  Were the training topics relevant to your needs in applying the SMEs financing?  Which training topic gives you the most added value on SME’s financing?  Do you think that there are other training topics that were missed and not covered in the JLGF training?  How do you think that the training activities contributed in the following: o Change the bank’s credit mentality o Improve the SME’s access to banks’ finance o Employment  How do you evaluate the trainers who delivered the JLGF training in terms of; o Qualification and experience. o Delivery approach (i.e. the training approach and the use of suitable training tools?  How do you find the applicability of the offered training topics to your day to day work?  Is / are the training material offered consistent with your work procedure at the bank?  To what degree you see a harmony between the JLGF training material on SMEs financing and your bank’s lending policies and procedures?  How easy you find it to operationalize the training topics at your bank / department?  How flexible is the training topics to the needs of the SMEs at different sectors or business lines?  Do you see the training material capturing the changing SMEs needs in Jordan?  How do you evaluate the effectiveness of the training to the benefit of the SMEs’  Can you think of the training activities to be more effective and/or more efficient in the future runs of training? Do you suggest any specific change in the JLGF training program to better fit the needs of the  Jordanian Banks  SME’s in Jordan What is the best practical experience you went through in the training and would like to capitalize on for future training? What is / are the most important skills that you developed out of the JLGF training on SMEs financing and would like to capitalize on for the future? Final Evaluation Report – Jordan Loan Guarantee Facility 68 Questions on sustainability: 1. Would you think of some best practices that you came across during the implementation of the JLGF to the benefits of the SMEs and/or your bank, or the benefit of any other beneficiary? 2. What are your thoughts on the SMEs lending in the near future? 3. What are the specific practices of the JLGF activity that your bank is likely to continue in the near future? And what are those that might not continue? Why? Questions on lessons learned: 1. What do you think is the most significant lesson you learned from participating in the JLGF experience? 2. Can you name the most important skills that you and or other people at your bank learnt from the JLGF technical assistance? 3. Do you have any specific suggestion or recommendation to improve the work of the JLGF activity? BANKS TRAINEES FOCUS GROUPS- DISCUSSION GUIDE Questions on Quality and Applicability of the Training received from JLGF 1. To what extent was the content of the training received relevant to your role in the bank? Were all the relevant themes included? Do you think that there was a key theme that was not covered by the training? 2. How do you value the overall experience of the training taking into consideration the ability of the trainer, the training methodology and the training tools?  How do you evaluate the training provided by JLGF to the bank employees?  Do you think that the training topics covered satisfy your needs as an SME credit officer?  Were the training topics relevant to your needs in SMEs financing?  Which training topic gives you the most added value on SME financing?  Do you think that there are other training topics that were missed and not covered in the JLGF training?  How do you evaluate the training provided by JLGF to the bank employees?  How do you evaluate the trainers who delivered the JLGF training in terms of: o Qualification and experience? o Delivery approach (i.e. the training approach and training tools)?  How do you find the applicability of the offered training topics to your day to day work?  Are the training materials offered consistent with your work procedure at the bank?  How easy do you find it to operationalize the training topics at your bank / department?  Do training topics flexible enough to serve the needs of the different SMEs economic activities? Final Evaluation Report – Jordan Loan Guarantee Facility 69 2. To what extent were the concepts, processes and tools easy or difficult to apply in your job? Why? Questions on Effectiveness & Impact: (A) JLGF services and the effect on Loans and Loan Guarantees: 1. To what extent do the services provided by JLGF align with the overall bank strategy? How? 2. What are the key features of the JLGF intervention that are most useful for the bank strategy to extend loans to SMEs? 3. How are the interventions provided by JLGF reflected in the banks’ operations related to SME lending? 4. Were there any significant changes in any area of the process or procedures of extending loans that changed because of the JLGF activity? Could you give me an example? How did it change? 5. To what extent have the changes introduced an improvement or hindered the banks credit analysis? 6. What was the most significant challenge of applying the methodology and tools to your work? 7. Has your work with JLGF affected any changes in the organization of the bank? 8. To what extent has the training provided to SMEs had an effect on their ability to apply for a loan? Can you distinguish between SMEs that received training and the ones that did not? 9. How do you see the value of the JLGF activity on closing gaps in the following:  SMEs financing gap in Jordan?  Gender gaps (if exist)? 10. Could you suggest ways in which JLGF services could be more effective in supporting the banks efforts to extend loans to SMEs? 11. How do you think that the training activities contributed in the following: o Change the bank’s credit mentality o Improve the SME’s access to banks’ finance o Employment 12. Comparing with other intervention models and procedures: o How the JLGF intervention model differs from other models, e.g. the JLG Corp. model?  How do you find the applicability of the offered training topics to your day to day work?  Is / are the training material offered consistent with your work procedure at the bank?  How easy do you find it to operationalize the training topics at your bank / department?  How applicable are the training topics to the needs of the SMEs in different sectors or business lines? Final Evaluation Report – Jordan Loan Guarantee Facility 70 (B) Questions on sustainability: 1. What are your thoughts on extending loans to SMEs in the near future? What are the specific practices leaned in the training that have been or are going to be incorporated into the banks’ lending procedures? 2. Are you or the bank planning on seeking additional training or technical assistance for improving SMEs lending procedures? Why? 3. Which best lending practice of the JLGF activity is your bank likely to continue in the near future? And what are those that might not continue? Why? (C) Questions on lessons learned: 1. What do you think is the most significant lesson you learned from participating in this project? 2. Can you name the most important skills that you and/or other people at this bank learnt from the JLGF technical assistance? 3. Do you have any specific suggestion or recommendation to improve the work of the JLGF? BANKS’ TRAINEES QUESTIONNAIRE Bank Name: ____________________________________________________ Date, Month/ Day / Year. _________ / ________/ ____________ Time, ________________ City: ________________ Background Information: As part of the continuous USAID protocol to keep check in the effectiveness of the cooperation and in thinking on ways of improving JLGF and future similar programs, the evaluation comes to provide USAID with findings and strategic recommendations related to Activity’s interventions effectiveness, mainly on the technical assistance that USAID/JLGF provided to banks and SMEs, and the lessons learned and best practices to be utilized by USAID in the future. The evaluation team is a third party to keep the objectivity of the evaluation, this evaluation is not on you or your bank, but on JLGF and in particular the extent in which the JLGF training has been effective on supporting your banks’ efforts of extending loans to SMEs and how well has responded to your and your bank’s expectations and needs. The information gathered will be used to better understand the program and analysis to learn an improve this and future programs.  Can you think of the training activities to be more effective and/or more efficient in the near future?  Do you suggest any specific change in the JLGF training program to better fit the needs of the following:  Jordanian Banks  SMEs in Jordan (2) What is the best practical experience you went through in the training, and would like to capitalize on for future training? (3) What is / are the most important skill/s that you developed out of the JLGF training and would like to capitalize on for the future? Final Evaluation Report – Jordan Loan Guarantee Facility 71 No information or data from this interview will be attributed to you individually. Your information will be kept confidential and the findings from this discussion will be reported in aggregate, rather than at the individual level. You do not have to answer any questions that you are not comfortable answering. We greatly appreciate your willingness to share your thoughts in both positive aspects that should be kept or enhanced as well as issues that need improvement. Thank you  Feel free to write your answers in English or Arabic  If you need more space to answer a question feel free to write in the back or add more paper as needed. B.1. Name of the Respondent: _______________________________________________ B.2. Position/ Title: _________________________________________________________ B.3. Could you please comment in your current involvement with SME (Small Medium Enterprise) lending? Do you work full or part time on analyzing SMEs loan application? B.4. Did you attend one or more training with JLGF? (Check the one that applies to you) 1. One training ______ (if one, fill out the description of the first row) 2. More than one __________ (fill all that apply, if don´t remember write, don’t remember) B.5. Do you remember the year (s), Month (s), the general subject (s) and location (s) of the JLGF training? Main subject Year Month Location Training 1 Training 2 Training 3 Training 4 I. Questions on Quality and Applicability of the Training received from JLGF 1. To what extent the content of the training received was relevant to your role in the bank? Were all the relevant themes included? What are the subject with the most contribution to your knowledge of SME financing? Do you think that there was a key theme that was not covered by the training? 2. How do you value the overall experience of the training taking into consideration the ability of the trainer, the training methodology and the training tools? Final Evaluation Report – Jordan Loan Guarantee Facility 72 3. To what extent where the concepts, processes and tools easy or difficult to apply in your job? Why? II. Questions on Effectiveness & Impact: (D) JLGF services and the effect on Loans and Loan Guarantees: 1. To what extent do the services provided by JLGF are useful for the bank strategy to extend loans to SMEs? 2. Please comment on how the training provided by JLGF is reflected or not in the banks’ operations related to SME lending? There was any significant change in any area of the process or procedures of extending loans that changed because of the JLGF activity? How did it change? Could you give an example? 3. If changes have been made. to what extent the changes introduced relates to the improved or hindered the banks credit analysis? Please feel free to comment on:  the capacity and techniques used by trainers  The relevance of the topics and examples  The materials and teaching tools  The applicability of easiness to operationalize the training topics an tools into your day to day work.  The flexibility to fit the characteristic or SMEs of different sector or business lines Final Evaluation Report – Jordan Loan Guarantee Facility 73 4. What was the most significant challenge of applying the methodology and tools into your work? 5. Has your work with JLGF affected any changes in the organization of the bank? (E) JLGF services and the effect on SMEs 6. Are you aware that JLGF provides training to SMEs? 7. If yes, to what extent the training provided to SMEs had an effect on their ability to apply for a loan? Can you distinguish between SMEs that received training and the ones that did not? 8. How do you see the value of the JLGF activity on closing the following:  SMEs financing gap in Jordan?  Gender gaps (if exist)? 9. Could you suggest ways in which JLGF services could be more effective in supporting the banks efforts to extend loans to SMEs? III. Questions on sustainability: 1. What are your thoughts on extending loans to SMEs in the near future? What are the specific practices leaned in the training that have been or are going to be incorporated into the banks’ lending procedures? Final Evaluation Report – Jordan Loan Guarantee Facility 74 2. Are you or the bank planning on seeking additional training or technical assistance for improving SMEs lending procedures? Why? 3. Which best lending practice of the JLGF activity that your bank is likely to continue in the near future? And what are those that might not continue? Why? IV. Questions on lessons learned: 1. What do you think is the most significant lesson you learned from participating in JLGF? 2. Can you name the most important skills that you and or other people at this bank learnt from the JLGF technical assistance? 3. Do you have any specific suggestion or recommendation to improve the work of the JLGF? Trainers; IP Staff, Sub-contracted trainers, Institute of Banking Studies Date, Time, Location: Name of Rapporteur: Introduction and Overview: - Short introductory background information on the research and its objectives and interview protocols. Background: 2. Are you training the SMEs, or the Banks’ Credit Officers, or both? 3. From the JLGF material, what are the exact training modules you use for training? 4. For how long you have been training using the JLGF training material? 5. Are you aware of the other technical assistance other than training offered by JLGF? Final Evaluation Report – Jordan Loan Guarantee Facility 75 (A) Questions on Effectiveness/Impact of JLGF’s Contribution to Closing the Gap on SMEs Lending? 1. To what extent was the technical assistance that was received relevant to addressing the SME’s lending gap? Were all the relevant themes included? Do you think that there was a key theme that was not covered that is important in closing the SME lending gap? 2. How do you evaluate the capacity (knowledge, skills) of the following before and after training: a. SMEs? b. Banks’ Credit Officers? (B) Questions on the Training 2. To what extent did the training provided by JLGF contribute to enhancing the capacity of the following; a. Banks’ SME’s credit officers? How? b. SMEs? How? Relevance  To what extent did the training offered by JLGF interrelate and complement other aspects of the technical assistance?  Do you think that the subjects covered in the JLGF training were tailored to the characteristics and needs of the following: o Jordanian SMEs? o Jordanian Banks?  Do you think that the trainees had similar expectations of the purpose and scope of the training, as well as the level of knowledge required? o Banks Trainees o SME Trainees Quality  What are the most important factors that affected the quality of the JLGF training provided to the following: o Banks’ Credit Officers? o SMEs?  What are the most important skills that a JLGF trainer should have?  Do you think that the training materials (tools, guidelines, cases, examples...etc.) were tailored to serve the needs of the following: o SMEs? o Jordanian Banks?  Do you think that the logistics (location, organization, timing and schedule… etc.) were suitable and convenient? Applicability  How applicable are the training topics to the needs of the following: o SMEs in different sectors or business lines? o Banks?  How do you evaluate the capacity and willingness of the following trained beneficiaries to apply the concepts and applications presented in the training courses o Banks Credit Officers? o SMEs? (C) Questions on sustainability: 4. Can you think of some best practices that you came across during the implementation of JLGF that worked towards the benefits of the following: Final Evaluation Report – Jordan Loan Guarantee Facility 76  SMEs?  Jordanian banks?  Any other beneficiary? 5. What are your thoughts on the Jordanian banks’ SMEs lending in the near future?  Is the financing to start-up SMEs currently available as an option at the Jordanian banks?  Do the current policies and procedures at the Jordanian banks allow for flexibility in approving loan collaterals on the SMEs loans other than the traditional collaterals? (D) What do you think are the specific practices of the JLGF activity that the Jordanian banks are likely to continue in the near future? And what are those that might not continue? Why? Questions on lessons learned: 1.What do you think is the most significant lesson you learned from participating in the JLGF experience? 2.Can you name the most important skills that the Jordanian Credit officers at banks and the SMEs learnt from the JLGF technical assistance? 3.Do you have any specific suggestion or recommendation to improve the work of the JLGF activity? USAID SMEs Support Projects Date, Time, Location: Name of Rapporteur (At the end of KII) Titles, affiliation, phone, emails of those interviewed. Introduction and Overview: - Short introductory background information on the research and its objectives and interview protocols. Background information:  What are the main types of services that your project offers to SMEs? And how does it contribute to facilitating SMEs access to finance?  What is your main target clientele?  Which definition for the SMEs are you using? And how is that affecting your services to the SMEs in Jordan? (A) Questions on Effectiveness / Impact of JLGF’s Contribution to Closing the Gap on SMEs Lending? 6. How does your project and JLGF interrelate? Where do you see that your project and JLGF mainly complement and interrelate in serving the SMEs in Jordan? Any contradiction on the complementary role? 7. How do you evaluate the effectiveness of the activities provided by the JLGF? How well they respond to the needs of the SMEs in Jordan? Alignment with the project’s Strategy and objectives  How do you the compare between your project’s strategy and objectives to those of the JLGF in serving the SMEs in Jordan?  Where do you see the major link between your project and JLGF? Who serves the other? How?  To what extent do the services provided by JLGF align with the exact needs of the SMEs in Jordan? How?  What are the key features of the JLGF intervention that are most useful for the Jordanian banks’ strategy and policy towards SMEs financing?  How do you view JLGF’s role in the Jordanian banks SMEs lending and on filling the SMEs financing gap?  How would you view the impact of the JLGF activities provided by on the Jordanian banks’ operations related to SME lending? In particular relating to: o lending policies and procedures o credit analysis process o performance, way of doing business, credit appetite to SMEs, and credit view to SMEs  As a result of JLGF activities, do you see any impact on the organizational charts of the Jordanian banks?  How do you evaluate the flexibility of the Jordanian banks to accept guarantees on the SMEs loans other than the traditional collaterals? Final Evaluation Report – Jordan Loan Guarantee Facility 77 Suggestions for improvements  Would you suggest ways in which JLGF services could be more effective in supporting the Jordanian banks efforts to extend loans to SMEs in order to minimize their financing gap? (B) Questions on sustainability: 6. Are you aware of any best practices that you came across during the implementation of JLGF that worked towards the benefits of SMEs and/or the Jordanian banks, or the benefit of any other beneficiary? 7. What are your thoughts on the Jordanian banks SME’s lending in the near future? 8. What are the specific practices of the JLGF activity that the Jordanian banks are likely to continue in the near future? And what are those that might not continue? Why? (C) Questions on lessons learned: 4. Do you think that there is a significant lesson to be learned from the JLGF experience? 5. Do you have any specific suggestions or recommendations to improve the effectiveness the JLGF activities in closing the SMEs’ financial Gap? JEDCO – CEO or Senior Technical Advisors / Specialists Date, Time, Location: Name of Rapporteur: Introduction and Overview: - Short introductory background information on the research and its objectives and interview protocols. Background information:  What are the main types of services that your projects offer to SMEs? And how does it contribute to facilitating SMEs access to finance?  Would you highlight the technical and financial services offered by JEDCO to its clients? What is the delivery approach?  What is your main target clientele?  Which definition for the SMEs are you using? And how is that affecting your services to the SMEs in Jordan? (A) Questions on Effectiveness / Impact of JLGF’s Contribution to Closing the Gap on SMEs Lending? 1.How does your project and JLGF interrelate? Where do you see that your projects and JLGF mainly complement and interrelate in serving the SMEs in Jordan? Any contradiction on the complementary role? 2.How do you evaluate the effectiveness of the activities provided by the JLGF? How well they respond to the needs of the SMEs in Jordan? Alignment with the project’s Strategy and objectives  How do you the compare between your projects’ strategies & objectives to those of the JLGF in serving the SMEs in Jordan?  Where do you see the major link between your projects and JLGF? Who serves the other? How?  To what extent do the services provided by JLGF align with the exact needs of the SMEs in Jordan? How?  What are the key features of the JLGF intervention that are most useful for the Jordanian banks’ strategy and policy towards SMEs financing?  How do you view JLGF’s role in the Jordanian banks SMEs lending and on filling the SMEs financing gap? Final Evaluation Report – Jordan Loan Guarantee Facility 78  How would you view the impact of the JLGF activities provided by on the Jordanian banks’ operations related to SME lending? In particular relating to: o Lending policies and procedures o Credit analysis process o Performance, way of doing business, credit appetite to SMEs, and credit view to SMEs  As a result of JLGF activities, do you see any impact on the organizational charts of the Jordanian banks?  How do you evaluate the flexibility of the Jordanian banks to accept guarantees on the SMEs loans other than the traditional collaterals?  Comparing with other intervention models and procedures: o How the JLGF intervention model differs from other models, e.g. the JLG Corp. model? o How the JLGF training approach differs from other training for SMEs and banks provided by other institutions?  Suggestions for improvements  Would you suggest ways in which JLGF services could be more effective in supporting the Jordanian banks efforts to extend loans to SMEs in order to minimize their financing gap? (B) Questions on sustainability: 1. Are you aware of any best practices that you came across during the implementation of JLGF that worked towards the benefits of SMEs and/or the Jordanian banks, or the benefit of any other beneficiary? 2. What are your thoughts on the Jordanian banks SME’s lending in the near future? 3. What are the specific practices of the JLGF activity that the Jordanian banks are likely to continue in the near future? And what are those that might not continue? Why? (C) Questions on lessons learned: 1. Do you think that there is a significant lesson to be learned from the JLGF experience? 2. Do you have any specific suggestions or recommendations to improve the effectiveness the JLGF activities in closing the SMEs’ financial Gap? JLGC Date: Time: Location: Interviewer: Introduction: Objective of the Performance Evaluation I. Background and Context: 1. Please describe your role within your organization. 2. Please describe how your organization defines SMEs. a) Have these definitions changed over time? 3. Please describe the services offered by JLGC to banks and any other providers of financial services. 4. Please describe Jordanian Bank lending practices to SMEs. a) To what extent are banks moving towards considering SMEs as an important target group? Why? b) What are some of the challenges and opportunities the banks face when extending financial services to SMEs? II. Effectiveness / Impact of JLGF 1. Please describe to what degree JLGC and JLGF have interacted. 2. From your perspective how would you describe the effectiveness of JLGF activities? How well do these activities respond to challenges in increasing financial access? Final Evaluation Report – Jordan Loan Guarantee Facility 79 3. How do you the compare JLGC strategies and objectives to those of the JLGF in terms of improving SME access to finance? 4. To what extent do the services provided by JLGC align with the needs of the SMEs in Jordan? How does JLGF differ? 5. Please compare the key features of the JLGC and JLGF activities that are most useful for Jordanian banks relating to SME financing. 6. Please describe the impact of the JLGC and JLGF activities on the provision of financial services. Please compare if the approaches differ. a) Have you seen changes in bank organizational/management structures? 7. How have banks processes changed in reviewing SMEs loans requests? 8. Would you suggest ways in which JLGF services could be more effective in supporting the Jordanian banks efforts to extend loans to SMEs in order to minimize their financing gap? III. Questions on sustainability: 1. What predictions can you make with regard to bank lending to SMEs? a) Increase/decrease? Why? 2. Are there any practices related to the JLGF activity that you think are likely to become institutionalized within the banking sector? 3. More generally, in the effort to sustain growth in the SME sector, what lending practices do you think should be sustained or scaled up? IV. Questions on lessons learned: 1. Do you think that there is a significant lesson to be learned from the JLGF experience? Please describe. 2. Please provide any specific suggestions or recommendations to improve the effectiveness of JLGF activities as they relate to your work. 3. Would it be possible to follow up with you should we have any further questions or clarifications? JLGC Date: Time: Location: Interviewer: Introduction: Objective of the Performance Evaluation 1. Comparison of JLGF processes with other OPIC facilities (generally, the region, in Jordan specifically). 2. What is unique about JLGF from OPIC’s perspective? Do these factors contribute to the achievement of key objectives? 3. OPIC’s processes for partnering with specific banks and reviewing loans. Cross reference JLGF’s description of the intervention with OPIC’s. 4. How does OPIC view activity outreach efforts? Discuss 2012-2013 period as compared to later activity efforts. How did the approach change over time if at all, or how did priorities shift? Are these shifts explained simply by moving on to additional phases? 5. Efforts and results for approaching Islamic Banks. Best practices in Jordan, lessons from the region. 6. Efforts and results for approaching other financial institutions/stakeholders that target SMEs. 7. Discussion of constraints that prevent financial institutions from signing an agreement with JLGF. 8. What is OPIC’s view on the impact of JLGF? 9. Discussion of opportunities for expanding the impact of JLGF. Including sustaining gains made and/or scaling the approach, if applicable. 10. What changes in process and requirements make sense from OPIC’s perspective to better tailor the facility to the Jordan banking sector? Relatedly, what makes sense to better serve Jordanian SME needs? Category: Government Programs and Regulators: Final Evaluation Report – Jordan Loan Guarantee Facility 80 Date: Time: Location: Interviewer; Introduction: Objective of the Performance Evaluation I. Background and Context: 1. Please describe your role within the CBJ. a) How is your role related to SME development in Jordan? 2. Please describe how the CBJ defines SMEs, and how this definition may have changed over time. a) Why do you think the CBJ has changed these definitions? 3. From the perspective of the CBJ, please describe the regulatory and monetary environment affecting SMEs in Jordan. a) How have these regulations changed over time? b) What was the effect of this change? c) How does the CBJ’s role in monetary policy impact the SME enabling environment? 4. Are there plans to introduce new regulations or policies to change the existing ones with regard to SME lending in the near future? In what manner? Why? 5. From your perspective within the CBJ, please describe to what degree the institution views SMEs as an important target group. a) Why is this the case? 6. What are prominent challenges the banking sector faces when extending financial services to SMEs. a) Please discuss opportunities as well. 7. Follow up about perceptions of financial access for SMEs if it does not come up. II. Effectiveness / Impact of JLGF’s Assistance 1. How and to what degree have the CBJ and JLGF interacted? o What approval processes, or standard procedures, are expected for programs like JLGF in working with the CBJ or operating in Jordan generally? 2. Please describe your opinion of the services provided by JLGF. o To what extent do you feel JLGF responds to the challenge of providing financial access to SMEs in Jordan? 3. Are there specific changes in the CBJ’s regulations to the Jordanian banks that came as a result of Jordanian Banks partnering with JLGF? 4. How do you the compare CBJ strategies and objectives to those of the JLGF? o Follow up about financial access if it does not come up. 5. From your perspective, to what degree do the services provided by JLGF align with the needs of the SMEs in Jordan? In what specific ways? o Follow up about financial access if it does not come up. 6. To what degree are JLGF services helpful to the Jordanian banking sector? o Please discuss JLGF’s impact on bank lending policies and procedures. o Please discuss JLGF’s impact on credit analysis processes. 7. Please describe your sense of how bank processes have changed over the past few years relating to the provision of loans to SMEs? o Follow up about the focus on collateral if it does not come up. 8. Have you observed changes in bank organizational/management structures as they relate to providing SME services? o Do you think JLGF has played a role in this? How, why, or why not? 9. Please discuss differences you have observed between JLGF and other models/programs with similar objectives. o If needed, provide JLGC as an example. o How does the JLGF training approach differ from other training initiatives for SMEs? Final Evaluation Report – Jordan Loan Guarantee Facility 81 o And training initiatives for banks? 10. In what ways can JLGF services be made more effective in supporting Jordanian bank efforts to improve access to finance for SMEs? III. Sustainability 1. What predictions can you make with regard to bank lending to SMEs? o Increase/decrease? Why? 2. Are there any practices related to the JLGF activity that you think are likely to become institutionalized within the banking sector? 3. More generally, in the effort to sustain growth in the SME sector, what lending practices do you think should be sustained or scaled up? IV. Lessons learned and recommendations 1. Do you think that there is a significant lesson to be learned from the JLGF experience? Please describe. 2. Please provide any specific suggestions or recommendations to improve the effectiveness of JLGF activities as they relate to your work. 3. Would it be possible to follow up with you should we have any further questions or clarifications? SMEs Associations and Chambers of Commerce Date, Time, Location: Name of Rapporteur (At the end of KII) Titles, affiliation, phone, emails of those interviewed. Introduction and Overview: - Short introductory background information on the research and its objectives and interview protocols. Background information: 1. How much do you know about JLGF? Have you or your organization been involved with JLGF or have participated in any way in JLGF’s activities? Have JLGF have done awareness or training sessions with JLGF? About your organization (Association, or Chamber) and your associates, 2. About your associates 2.1. What is your main target clientele (or associates)? How important are the SMEs among your associates? Are your associates business from all economic activities (trade, manufacturing and services) or from a specific sector? Which definition for the SMEs are you using? And how is that affecting your services to the SMEs in Jordan? 3. About the organization services 3.1. Does your organization provide services to its associates? What kind of services your organization provides to its associates? 4. How important access to finance is for the development of the SMEs, and in particular to your associates? (A) Questions on Effectiveness / Impact of JLGF’s Contribution to Closing the Gap on SMEs Lending? 1. How does your project and JLGF interrelate? Where do you see that your project and JLGF mainly complement and interrelate in serving the SMEs in Jordan? Any contradiction on the complementary role? 2. To what extent the components of the JLGF are consistent with the MSEs’ needs and the Banks strategy to extend loans to SMEs? 3. To what extent the JLGF interventions on the banks and SME side respond to the expectations and needs of banks and SMEs? 4. How do you evaluate the effectiveness of the activities provided by the JLGF? How well they respond to the needs of the SMEs in Jordan? Alignment with the project’s Strategy and objectives Final Evaluation Report – Jordan Loan Guarantee Facility 82 5. How do you the compare between your project’s strategy and objectives to those of the JLGF in serving the SMEs in Jordan? 6. Where do you see the major link between your project and JLGF? Who serves the other? How? 7. To what extent do the services provided by JLGF align with the exact needs of the SMEs in Jordan? How? 8. What are the key features of the JLGF intervention that are most useful for the Jordanian banks’ strategy and policy towards SMEs financing? 9. How do you view JLGF’s role in the Jordanian banks SMEs lending and on filling the SMEs financing gap? 10. How would you view the impact of the JLGF activities provided by on the Jordanian banks’ operations related to SME lending? In particular relating to: o lending policies and procedures o credit analysis process o performance, way of doing business, credit appetite to SMEs, and credit view to SMEs 11. As a result of JLGF activities, do you see any impact on the organizational charts of the Jordanian banks? 12. How do you evaluate the flexibility of the Jordanian banks to accept guarantees on the SMEs loans other than the traditional collaterals? 13. Comparing with other intervention models and procedures, including both training of SMEs and loan guarantees: How the JLGF intervention model compares with the s services offered by others (in the training side support programs and in the guarantee side JLG Corp. model? Suggestions for improvements 14. Would you suggest ways in which JLGF services could be more effective in supporting the Jordanian banks efforts to extend loans to SMEs in order to minimize their financing gap? (B) Questions on sustainability: 1. Are you aware of any best practices that you came across during the implementation of JLGF that worked towards the benefits of SMEs and/or the Jordanian banks, or the benefit of any other beneficiary? 2. What are your thoughts on the Jordanian banks SME’s lending in the near future? 3. What are the specific practices of the JLGF activity that the Jordanian banks are likely to continue in the near future? And what are those that might not continue? Why? 4. How do you see the willingness of banks to extend loans to SMEs? Do you see them willing to innovate and adopt new practices for adapting to SMEs needs and characteristics? 5. Do you think that they are willing to make changes to their way to study the loans applications and change their traditional underwriting processes? (C) Questions on lessons learned: 1. Do you think that there is a significant lesson to be learned from the JLGF experience? 2. Do you have any specific suggestions or recommendations to improve the effectiveness the JLGF activities in closing the SMEs’ financial Gap? SME In-Depth Interview Introduction Management Systems International (MSI) is conducting an independent evaluation of the Jordan Loan Guarantee facility funded by USAID. The purpose of this evaluation is to assess how well the program responds to the SME needs in accessing financial services, and to develop recommendations and best practices specific to the effectiveness of JLGF’s technical assistance. In this survey we will inquire about the services received by your business, as well as your perceptions related to the services received and their effectiveness. Some questions will ask general characteristics of your business to give us some understanding of the context of your business. This is not an evaluation of your business in any way but rather an evaluation of JLGF’s ability to effectively support you in accessing financing. Final Evaluation Report – Jordan Loan Guarantee Facility 83 No information or data from this interview will be attributed to you individually. Your information will be kept confidential and the findings from this discussion will be reported in aggregate, rather than at the individual level. You do not have to answer any questions that you are not comfortable answering. You can also stop this survey at any point. We greatly appreciate your participation in this study. Thank you very much for your cooperation. Do you have any questions before start this interview? Do I have your permission to start this interview? We will start by asking you some general questions about your business: I. Section I - SME Characteristics 1. Name of Business: _____________________ Location: ________________________________ 2. Could you please tell us a little bit about your business: when did you start your business; your main products and services; and your customers? 3. Could you please tell us about your workers: Beside yourself, how many permanent workers do you have today? How many of them receive a salary or there are some family members that receive some compensation but not a salary? 4. Do you own or do you rent the space for your main business location? Separate from your business do you own any property/real-estate? 5. When it comes to sales, does your business have a high or low season (seasonal) or are your sales consistent throughout the year? Now I would like to ask some questions about the general context within which your business operates II. Section II – General Context 1. When it comes to the overall outlook of your sector over the last year, would you say that things are moving in the right direction or in the wrong direction? Why do you say that? 2. Thinking of your sales and profit over the last year, would you say that your sales and profits have improved, remained the same or worsened? Why is this the case? 3. Also over the last year, has the number of permanent workers of your business increased, remain the same or decreased? 4. Thinking of your sales and profit over the next year, would you say that your sales and profits will improve, remain the same or worsen? Why do you say that? 5. What are the most important constraints and/or limitations to growth of your business, today? Why do you say that? 6. How important access to finance is for the growth and performance of your business? Why do you say that? 7. What are the most important opportunities of growth for your business, today? Why do you say that? 8. What are the primary financial needs of your business? Final Evaluation Report – Jordan Loan Guarantee Facility 84 9. What are the two most important banking services for your business? 10. When it comes to the specific banking needs of your business, including obtaining credit and loans, what is your overall level of satisfaction with the banking sector in Jordan? Why do you say that? 11. What are the most important constraints that your business faces in accessing finance (obtaining loans)? Why do you say that? III. Section III- Future plans and Financial Needs 1. Are you planning on undertaking a specific action/s in order to grow or improve your business in the following year? If so, please share what you are planning to do and why? (if the answer is no, skip to question 4, if yes continue) 2. Does this plan require additional workers, or more skilled labor or you can manage with the same workers that you have today? 3. What is your primary plan to finance the implementation of the mentioned actions or projects? 4. Beyond financing, what type of assistance does your business need in order to grow and/or to implement the action/s you just mentioned? 5. Are you planning on starting a new business in the near future (next couple of years)? Is this new business related to the current business or a completely different business? (if the answer is no, skip to Section IV, if yes continue) 6. What type of assistance does your business need in order to start the new business you just mentioned? IV. Section IV – Exposure and Perception of JLGF Technical Assistance 1. How did you first learn about the JLGF project? 2. When was your first interaction with the JLGF team? 3. What type/s of technical assistance did you receive from the JLGF project? (have not received technical assistance is an option) 4. Could you please comment on the importance of the training received from JLGF on facilitating your business access to finance? Where do you see the exact value from the JLGF training in facilitating your business’ access to finance (obtaining loan)? 5. Could you please comment on the importance of the training received from JLGF and its effect on in the level of awareness of how to access finance? 6. Have you received an OPIC-JLGF loan guarantee? A single or multiple times? (IF HAVE NOT RECEIVED SKIP TO 8) 7. Please comment on the importance of the loan guarantee received from JLGF-OPIC on facilitating your business’s access to finance (obtaining the loan)? Where do you see the exact value of the JLGF￾OPIC loan guarantee in facilitating your business’s access to finance (obtaining a loan)? Final Evaluation Report – Jordan Loan Guarantee Facility 85 8. What is the most critical element and/or skill you have learned as a result of the JLGF Technical Assistance? Why do you say that? 9. What is the long-term impact of the assistance you have received from JLGF, in case any? 10. What is your overall level of satisfaction with the services provided by JLGF? Why? 11. When it comes to your specific business needs, to what extent did the JLGF project cover those needs? 12. Is there any specific additional technical assistance that was not provided by the JLGF project that you would like for them to provide? 13. Do you have any recommendations on how the technical assistance provided by the JLGF project may be improved? How or why would this help your business? 14. Do you know of any other project, program or institution that offers technical assistance like the JLGF project? 15. Did you benefit from any other project, program or institution that offers technical assistance like the JLGF project? (if the answer is no, skip to Section 18, if yes continue) 16. What is your overall level of satisfaction with similar type of technical assistance received from the other project/s you mentioned? 17. How do you compare the services provided by the other project, program or institutions that offer technical assistance with the one provided by JLGF? 18. Do you know of any other project, program or institution that offers loan guarantee like the JLGF project? 19. Did you benefit from any other project, program or institution that offers loan guarantee? (if the answer is no, skip to Section V, if yes continue) 20. What is your overall level of satisfaction of the guarantee received from the other project/s you mentioned? 21. How do you compare the loan guarantee provided by the other project, program or institutions with the JLGF-OPIC guarantee? V. Section V – Changes in Business Practices and Loan Application (access to capital) 1. Has your approach to business practices changed as a result of the JLGF technical assistance? How? (if the answer is no, skip to question 4, if yes continue) 2. What is the main business practice that you changed as a result of the JLGF technical assistance? Why? 3. Do you expect the changes you have made to your business as a result of JLGF technical assistance to be permanent and effective in the long-run or temporary and only effective in the short-run? Why do Final Evaluation Report – Jordan Loan Guarantee Facility 86 you say that? 4. Did you apply for a loan as a result of the technical assistance provided by JLGF? (if the answer is no, skip to question 8, if yes continue) 5. What was the purpose of that loan? 6. Did you succeed in obtaining the loan? (if the answer is yes, skip to question 8, if no continue) 7. Could you comment on the reason for the lack of success in obtaining the loan? 8. Are you planning to apply for a loan over the next year? (if the answer is no, finish the interview, if yes continue) 9. What is the purpose of that loan? 10. Are you considering using the JLGF-OPIC guarantee? (if the answer is yes finish the interview, if no continue) 11. Could you comment on the reasons for not considering using the JLGF-OPIC guarantee? Finish Thank the respondent and end the interview. Final Evaluation Report – Jordan Loan Guarantee Facility 87 Introduction: Management Systems International (MSI) is conducting an independent evaluation of the Jordan Loan Guarantee facility funded by USAID. The purpose of this evaluation is to assess how well the program responds to the SME needs in accessing financial services, and to develop recommendations and best practices specific to the effectiveness of JLGF’s technical assistance. In this survey we will inquire about the services received by your business, as well as your perceptions related to the services received and their effectiveness. This is not an evaluation of your business in any way but rather an evaluation of JLGF’s ability to effectively support you in accessing financing. No information or data from this interview will be attributed to you individually. Your information will be kept confidential and the findings from this discussion will be reported in aggregate, rather than at the individual level. You do not have to answer any questions that you are not comfortable answering. You can also stop this survey at any point. We greatly appreciate your participation in this study. Thank you very much for your cooperation. Do you have any questions before start this interview? Do I have your permission to start this interview? We will start by asking you some general questions about your business: 0. Do you currently have a business activity or an ongoing business? (If the answer is no apply questionnaire start up with no current business, if yes continue) a. Yes b. No I. Section I - SME Characteristics 1. Name of Business: _____________________ 2. Location: 2.1. City/Town: _____________________ 2.2. Governorate: ___________________ 3. Description of the Economic Activity Business: 3.1. Main Products and Services 3.1.1. ________________________ 3.1.2. ________________________ 3.1.3. ________________________ 3.2. Sector 3.2.1. ____________________________ 3.3. Main Market (Primary Market): 3.3.1. Governorate Level 3.3.2. National Country-wide (PLEASE RECORD THE ANSWER AS PRECISELY AS POSSIBLE) Final Evaluation Report – Jordan Loan Guarantee Facility 88 3.3.3. Regional and International Level 4. Gender of and type of Owner-ship: 4.1. Woman Sole Proprietor 4.2. Man Sole Proprietor 4.3. Group of Women 4.4. Group of Men 4.5. Women and Men shared proprietorship 5. Is your business in one location or do you have different locations (i.e. branches, point of sale, point of services) 5.1. Single Location 5.2. Multiple Locations 5.2.1. Number of Locations: _______________ 6. When did you start this business? 6.1. Year: ______________ Month: ___________ 7. Currently how many workers do you have? _____________ (number of permanent workers) 8. Do you own or do you rent the space for your main business location? 8.1. Own 8.2. Rent 8.3. Other (specify) ___________________________________________________ 9. Separate from your business do you own any property/real-estate? 9.1. Yes 9.2. No 10. When it comes to sales, does your business have a high or low season (seasonal) or are your sales consistent throughout the year? 10.1. Seasonal 10.2. Sales are consistent throughout the year 10.3. Don’t Know (DO NOT READ) Now I would like to ask some questions about the general context within which your business operates II. Section II – General Context 1. When it comes to the overall outlook of your sector over the last year, would you say that things are moving in the right direction or in the wrong direction: 1.1. Right Direction 1.1.1. What is the main reason : __________________________________________ (RECORD VERBATIM AND POST CODE) 1.2. Wrong Direction 1.2.1. What is the main reason : __________________________________________ (RECORD VERBATIM AND POST CODE) Final Evaluation Report – Jordan Loan Guarantee Facility 89 1.3. Don’t Know (DO NOT READ) 2. When it comes to the sales and profits (performance) of your business, over the last five years (ASK SINCE YOUR BUSINESS STARTED IF LESS THAN FIVE YEARS OLD), would you say that your sales and profits have: 2.1. Significantly improved 2.2. Somewhat improved 2.3. Remained the same 2.4. Somewhat worsened 2.5. Significantly worsened 2.6. Refused to Answer (DO NOT READ) 2.7. Don’t Know (DO NOT READ) 3. Thinking of your sales and profit over the last year, would you say that your sales and profits have: 3.1. Significantly improved 3.2. Somewhat improved 3.3. Remained the same 3.4. Somewhat worsened 3.5. Significantly worsened 3.6. Refused to Answer (DO NOT READ) 3.7. Don’t Know (DO NOT READ) 4. Thinking of your sales and profit over the next year, would you say that your sales and profits will: 4.1. Significantly improve 4.2. Somewhat improve 4.3. Remain the same 4.4. Somewhat worsen 4.5. Significantly worsen 4.6. Refused to Answer (DO NOT READ) 4.7. Don’t Know (DO NOT READ) 5. What are the two most important constraints and/or limitations to growth of your business, today? (OPEN ENDED) 5.1.1. ____________________________________(RECORD VERBATIM AND POST CODE) 5.1.2. ____________________________________(RECORD VERBATIM AND POST CODE) 5.1.3. Don’t Know (DO NOT READ) THE BELOW TABLE CONTAINS PRECODES. DO NOT READ THEM TO THE RESPONDENT. IF THE RESPONDENT ANSWER MATCHES A PRECODE BELOW, WRITE THE NUMBER FOR THE PRE-CODE. IF THE ANSWER DOES NOT MATCH ANY OF THE PRECODES, NOTE AS OTHER (22) AND RECORD VERBATIM 1 Low profits 2 Lack of market, difficult access to market 3 Problems with the site: venue too small, inadequate floor space or layout 4 Lack of financial services 5 No access to loans / No access to finance 6 Low product, services quality 7 Clients don’t pay or take long time for them to pay 8 Low motivation, not reliable or low skilled workers 9 High workers turnover Final Evaluation Report – Jordan Loan Guarantee Facility 90 10 Inadequate, outdated machinery, frequent malfunction, difficulties getting spare parts, expensive repair services 11 Legal issues 12 Inadequate policies, insufficient incentives 13 General country situation 14 Issues related to utilities, expensive or un reliable energy, water, communication services 15 Issues with inputs: difficulties accessing, low input or merchandise quality 16 Lack of time for the business have multiple jobs, have multiple responsibilities 17 Personal issues affecting the business 18 Security concerns, thieves 19 Too much paper work, time consuming and expensive processes to keep the business formal 20 High taxes and government fees 21 Harassment of authorities local or national authorities 22 Other (write the verbatim in 5.1.1 or 5.1.2) 5.2. When it comes to the performance and growth of your business, would you say that access to finance is: 5.2.1. Very Important 5.2.2. Important 5.2.3. Not at all important 5.2.4. Don’t know (DO NOT READ) 6. What are the two most important opportunities of growth for your business, today? (OPEN ENDED, post coded) 6.1. ____________________________________(RECORD VERBATIM AND POST CODE) 6.2. ____________________________________(RECORD VERBATIM AND POST CODE) 6.3. Don’t Know (DO NOT READ) 7. What are the two most important financial needs of your business? 7.1. ________________________________(RECORD VERBATIM AND POST CODE) 7.2. ________________________________(RECORD VERBATIM AND POST CODE) 7.3. Don’t Know (DO NOT READ) 8. What are the two most important banking services for your business? (Precoded, DO NOT READ, to record the answer choose and write the code from the table below; if answer does not fit in a code use the 14. other and write the verbatim) 8.1. ________________________________(RECORD VERBATIM AND POST CODE) 8.2. ________________________________(RECORD VERBATIM AND POST CODE) 8.3. Don’t Know (DO NOT READ) 1 Chequing account 2 Savings Account 3 Credit Card 4 Business loans 5 Fixed term saving certificates 6 ATM and Debit cards 7 Life, accident and other insurance 8 Mortgage 9 Wire transfer national and international Final Evaluation Report – Jordan Loan Guarantee Facility 91 10 Electronic Payment 11 Phone transactions 12 Services for receiving payments domestically and internationally (help here) 13 Savings-Credit products 14 Other (write the verbatim in 8.1. or 8. 2) 9. When it comes to the two most important banking needs of your business, what is your overall level of satisfaction? 9.1. Very satisfied 9.2. Somewhat satisfied 9.3. Neither satisfied nor dissatisfied 9.4. Somewhat dissatisfied 9.5. Very Dissatisfied 10. When it comes to accessing finance (obtaining loans) what are the two most important constraint faced by your business? (OPEN ENDED – RECORD VERBATIM) 10.1. ________________________________(RECORD VERBATIM AND POST CODE) 10.2. ________________________________(RECORD VERBATIM AND POST CODE) III. Section III- Future plans and Financial Needs Yes No May be/ not sure 1. Are you thinking of implementing or starting any kind of plan or project for your business during the following year? (If the answer is no go to question Nº 5) 2. Are you planning to do de following actions related to our business in the next 12 months? I will read for you some actions that you may be or may be not considering on doing. Please respond yes, no or not sure as it applies to you Action Yes No May be/ not sure a) Increase the number of workers? b) Get machinery or equipment? c) Increase the amount of raw material or merchandise you buy? d) Buy, remodel or increase the size of the shop or factory? e) Change the location or buy an additional shop, warehouse, sales point? f) Include new products, services or markets on you production/trade/service line? g) Start up a new business? h) Improve the promotion campaign? i) Adopt new or improved technologies? j) Buy e vehicle for the business? Final Evaluation Report – Jordan Loan Guarantee Facility 92 k) Are you planning on doing any project or changing your business in any significant way, not mentioned above? Specify _______________________________________________ (If the answer is no in all the questions go to next section) 3. How much do you think will it cost implementing the above mentioned plan(s)? (999. don’t know, 888. don’t want to answer) JOD Yes No May be/ not sure a) Would you consider applying for a loan for financing the above mentioned plan(s)? (If the answer is no go to question Nº 4) b) How much money would you consider asking for to financial source? (999. don’t know, 888. don’t want to answer) JOD c) Where do you prefer applying for the loan? Codes: 1. Family and friends 2. Lenders 3. Commercial Banks 4. Islamic banks 5. Donor funded programs (NOGs,) 6. Microfinancing institution 7. Suppliers 8. Other financial institution Specify_______________________________________ d) Why do you prefer this source? Codes: 1. Better service, 2. Better interest rate, 3. Convenient (location) 4. I have worked with it before, 5. Have better conditions, 6. Have products or services that fit my needs, 7. office hours, 8. Better repayment schedule 9. other (Specify) ___________________________________ 4. Beyond financing, what type of assistance does your business need in order to grow and/or to implement the action/s you just mentioned? 4.1. Please specify the most important assistance need: ___________________________________(RECORD VERBATIM AND POST CODE) 5. Are you planning on starting a new business in the near future (next couple of years)? 5.1 Yes 5.2 No (SKIP TO SECTION IV) 5.3 Refused to Answer (DO NOT READ) (SKIP TO SECTION IV) 5.4 Don’t Know (DO NOT READ) (SKIP TO SECTION IV) 6. Is this business related to the current business or a completely different business? 6.1. Yes, it is related to the current business 6.2. No, it is a completely different business 6.3. Refused to Answer (DO NOT READ) 6.4. Don’t Know (DO NOT READ) 7. What type of assistance do you need for starting the new business you just mentioned? 7.1. Please specify the most important assistance need: ___________________________________(RECORD VERBATIM AND POST CODE) Final Evaluation Report – Jordan Loan Guarantee Facility 93 IV. Section IV – Exposure and Perception of JLGF Technical Assistance 1. How did you first learn about the JLGF project? 1.1. Through a newspaper advertisement 1.2. Through a business association 1.3. Through a bank 1.4. Was contacted directly by JLGF 1.5. Other (Please Specify): __________________________ 2. When was your first interaction with the JLGF team? Please Specify: __________year 3. Of the following, please specify the type of technical assistance service you received from JLGF? (SELECT ALL THAT APPLY) 3.1. Trainings 3.1.1. SME Financing 3.1.2. Financial Analysis 3.1.3. Budgeting 3.1.4. Financial Management and Costing 3.1.5. Other (Specify): _______________ (IF DID NOT RECEIVE TRAINING, SKIP TO Q3.2) 3.2. Loan Guarantee, through OPIC 3.2.1. Single 3.2.2. Multiple (IF DID NOT RECEIVE GUARANTEE, SKIP TO Q4) 3.3. When it comes to facilitating access to finance for your business, would you say that the OPIC-JLGF guarantee was? 3.3.1. Very Important 3.3.2. Important 3.3.3. Not at all important 3.3.4. Don’t know (DO NOT READ) 4. What is the most important element and/or skill you have learned as a result of the JLGF Technical Assistance? 4.1. ____________________________________(RECORD VERBATIM AND POST CODE) 4.1.1. Why do you say that: ____________________________________(RECORD VERBATIM AND POST CODE) 5. What is the long-term impact of the assistance you have received from JLGF? 5.1. ____________________________________(RECORD VERBATIM AND POST CODE) 6. What is your overall level of satisfaction with the assistance provided by JLGF? 6.1. Very satisfied 6.2. Somewhat satisfied 6.3. Somewhat dissatisfied 6.4. Very dissatisfied Final Evaluation Report – Jordan Loan Guarantee Facility 94 6.5. Refused to answer (DO NOT READ) 7. When it comes to your specific business needs, to what extent did the JLGF assistance cover those needs? 7.1. Adequately covered my specific business needs 7.2. Partially covered my specific business needs 7.3. Did not cover my specific business needs 7.4. Refused to answer (DO NOT READ) 8. To what degree do you agree or disagree with the following statements: 8.1. JLGF Technical assistance has increased my access to finance 8.1.1. Strongly Agree 8.1.2. Agree 8.1.3. Neither Agree or Disagree 8.1.4. Strongly Disagree 8.1.5. Refused / Don’t Know (DO NOT READ) 8.2. JLGF Technical Assistance has increased my awareness of how to access finance 8.2.1. Strongly Agree 8.2.2. Agree 8.2.3. Neither Agree or Disagree 8.2.4. Strongly Disagree 8.2.5. Refused / Don’t Know (DO NOT READ) 8.3. JLGF Technical Assistance has supported me to meet banks and OPIC requirements for obtaining a loan 8.4. Strongly Agree 8.5. Agree 8.6. Neither Agree or Disagree 8.7. Strongly Disagree 8.8. Refused / Don’t Know (DO NOT READ) 9. Is there any specific additional technical assistance that was not provided by the JLGF project that you would like for them to provide? 9.1. Please specify the most important additional technical assistance need: 9.1.1. ____________________________________(RECORD VERBATIM AND POST CODE) 10. Do you have any recommendations on how the technical assistance provided by the JLGF project may be improved? If yes please specify 10.1. ____________________________________(RECORD VERBATIM AND POST CODE) 11. Do you know of any other project, program or institution that offers technical assistance like the JLGF project? (if answer is no, skip to question 16; if yes please provide the name of the institution/program) 11.1. Yes Specify the name of the program __________________________________ 11.2. No (skip to question 16) 11.3. Don’t Know (DO NOT READ – skip to question 16) 12. Did you benefit from any other project, program or institution that offers technical assistance like the JLGF project? 12.1. Yes 12.1.1. Please specify the type of assistance received: __________________________________ (RECORD VERBATIM AND POST CODE) Final Evaluation Report – Jordan Loan Guarantee Facility 95 12.2. No (skip to question 16) 12.3. Don’t Know (DO NOT READ – skip to question 16) 13. What is your overall level of satisfaction with similar type of technical assistance received from project/s other than JLGF? You were: 13.1. Very satisfied 13.2. Somewhat satisfied 13.3. Neither satisfied nor dissatisfied 13.4. Somewhat dissatisfied 13.5. Very dissatisfied 13.6. Refused to Answer (DO NOT READ) 13.7. Don’t Know (DO NOT READ) 14. How do you compare the assistance provided by the other project, program or institutions that offer technical assistance like the JLGF project? 14.1. JLGF project assistance is significantly better than other similar projects 14.2. JLGF project assistance is slightly better than other similar projects 14.3. JLGF project assistance is the same as other similar project 14.4. JLGF project assistance is slightly worse than other similar projects 14.5. JLGF project assistance significantly worse than other similar projects. 14.6. Refused to Answer (DO NOT READ) 14.7. Don’t Know (DO NOT READ) 15. Why do you say that? _________________________________ (RECORD VERBATIM AND POST CODE) 16. Do you know of any other project, program or institution that offers Loan Guarantee like the JLGF project? (If the answer is no, skip to section V; if yes please provide the name of the institution/program) 16.1. Yes Specify the name of the program __________________________________ 16.2. No (SKIP TO SECTION V) 16.3. Don’t Know (DO NOT READ – SKIP TO SECTION V) 17. Did you benefit from any other project, program or institution that offers Guarantee loans like the JLGF project? 17.1. Yes 17.1.1. Please specify the type of guarantee received: __________________________________ (RECORD VERBATIM AND POST CODE) 17.2. No (SKIP TO SECTION V) 17.3. Don’t Know (DO NOT READ – SKIP TO SECTION V) 18. What is your overall level of satisfaction of the guarantee received from project/s other than JLGF? You were: 18.1. Very satisfied 18.2. Somewhat satisfied 18.3. Neither satisfied nor dissatisfied 18.4. Somewhat dissatisfied 18.5. Very dissatisfied 18.6. Refused to Answer (DO NOT READ) 18.7. Don’t Know (DO NOT READ) Final Evaluation Report – Jordan Loan Guarantee Facility 96 19. How do you compare the guarantee provided by the other project, program or institutions? 19.1. JLGF -OPIC guarantee is significantly better than other similar projects 19.2. JLGF -OPIC guarantee is slightly better than other similar projects 19.3. JLGF -OPIC guarantee is the same as other similar project 19.4. JLGF -OPIC guarantee is slightly worse than other similar projects 19.5. JLGF -OPIC guarantee significantly worse than other similar projects. 19.6. Refused to Answer (DO NOT READ) 19.7. Don’t Know (DO NOT READ) 19.7.1. Why is better of worst? _________________________________ (RECORD VERBATIM AND POST CODE) V. Section V – Changes in Business Practices and Loan Application (access to capital 1. Has your approach to business practices changed as a result of the JLGF technical assistance? 1.1. Significantly changed 1.2. Somewhat changed 1.3. Remained the same (SKIP TO Q4) 1.4. Refused to Answer (DO NOT READ – SKIP TO Q4) 1.5. Don’t Know (DO NOT READ – SKIP TO Q4) 2. What is the main business practice that you changed as a result of the JLGF technical assistance? 2.1. Financial statements 2.2. Improved accounting 2.3. Improved cash-flow projections 2.4. Development of new products and services 2.5. Other (SPECIFY): _____________________________________________________ 3. When it comes to permanence and sustainability, do you expect the changes you have made to your business as a result of JLGF technical assistance to be: 3.1. Permanent and effective in the long-run 3.2. Temporary and effective in the short-run 3.3. Don’t Know (DO NOT READ) 4. Did you apply for a loan as a result of the technical assistance provided by JLGF? 4.1. YES 4.1.1. How many, if more than one: ___________________________ 4.2. No 4.2.1. Why not: ___________________________________ (SKIP TO QUESTION 6) 5. Did you succeed in obtaining the loan? (if yes continue if no skip to section 5.2) 5.1. Yes 5.1.1. Purpose of loan/s 5.1.1.1. _____________________ 5.1.1.2. _____________________ 5.1.1.3. _____________________ 5.1.2. Size of loan/s 5.1.2.1. _____________________ Final Evaluation Report – Jordan Loan Guarantee Facility 97 5.1.2.2. _____________________ 5.1.2.3. _____________________ 5.1.3. Status of loan 5.1.3.1. Settled 5.1.3.2. Active 5.1.3.3. Canceled 5.1.3.4. Other _______________________ 5.1.4. Primary benefit of the loan 5.1.4.1. _____________________ 5.1.5. Employment Growth 5.1.5.1. Yes 5.1.5.1.1.Number of full-time jobs created: _______________ 5.1.5.1.2.Number of part-time jobs created: ______________ 5.1.5.1.3.Percentage of jobs created for women: __________ 5.1.5.2. No 5.1.6. Has the loan allowed you to increase the salary and/or social security coverage for your employees? 5.1.6.1. Yes 5.1.6.2. No 5.1.7. Have your business hours of operation increased as a result of the JLGF intervention? 5.1.7.1. Yes 5.1.7.2. No 5.2. No 6. Are you planning to apply for a loan over the next year? 6.1. Yes 6.2. No 6.3. Maybe 6.4. Refused to Answer (DO NOT READ) 6.5. Don’t Know (DO NOT READ) (IF ANSWER IS NO, END THE SURVEY) 7. What is the purpose of that loan? 8. Would you use the JLGF for this loan? 8.1. Yes 8.2. No IF THE ANSWER IS NO 8.2.1. Please provide the reason. _______________________________ 8.3. Maybe 8.4. Refused to Answer (DO NOT READ) 8.5. Don’t Know (DO NOT READ) , Finish Thank the respondent and end the interview. Final Evaluation Report – Jordan Loan Guarantee Facility 98 Start-Ups with no Current Business Introduction: Management Systems International (MSI) is conducting an independent evaluation of the Jordan Loan Guarantee Facility (JLGF) funded by USAID. The purpose of this evaluation is to assess how well the program responds to SME needs in accessing financial services, and to develop recommendations and best practices specific to the effectiveness of JLGF’s technical assistance. In this survey we will inquire about the services received, as well as your perceptions about the services received and their effectiveness. This is not an evaluation in any way but rather an evaluation of JLGF’s ability to effectively support you to access financing. No information or data from this interview will be attributed to you individually. Your information will be kept confidential and the findings from this discussion will be reported in aggregate, rather than at the individual level. You do not have to answer any questions that you are not comfortable answering. You can also stop this survey at any point. We greatly appreciate your participation in this study. Thank you very much for your cooperation. Do you have any questions before we start this interview? Do I have your permission to start this interview? We will start by asking you some general questions: I. Section I – Start Up and Entrepreneurship 1. To which of the following age groups do you belong to? Or What is your Age Group? 1.1. _20 years old or younger 1.2. 21-25 years 1.3. 26 to 35 years 1.4. 36 to 45 years 1.5. 46 to 55 1.6. 56 to 65 1.7. 66 or older 1.8. Refuse to answer DO NOT READ 2. Are you located in Amman or in another Governorate? 2.1. Amman 2.2. Specify Governorate_________________________ 3. As far as your current occupation is concerned, would you say you are self-employed, an employee, a manual laborer or would you say that you are without a professional activity? 3.1. Self-employed, run your own business (specify) ________________________________ 3.2. Employee (specify)________________________________________ 3.3. Manual laborer (specify) ________________________________________ 3.4. Without a professional activity (specify) ________________________________________ 3.5. Refuse to answer DO NOT READ Final Evaluation Report – Jordan Loan Guarantee Facility 99 3.6. Don’t Know (DO NOT READ) 4. If you could choose between different kinds of jobs, would you prefer to be an employee or self-employed (have your own business)? 4.1. An employee 4.1.1. Why would you prefer to be an employee rather than be self-employed (have your own business)? 4.1.1.1. Regular, steady income (versus irregular variable income) 4.1.1.2. Job security 4.1.1.3. Fixed working hours 4.1.1.4. Covered by social welfare/insurance 4.1.1.5. No business idea 4.1.1.6. Not enough capital/resources to be self-employed 4.1.1.7. Not enough skills to be self-employed 4.1.1.8. It is a major step to take/hard to turn back 4.1.1.9. Afraid of red tape, administrative difficulties 4.1.1.10. Afraid of legal and social consequences if I fail 4.1.1.11. Other Specify ________________________________ 4.1.1.12. Don’t Know (DO NOT READ) 4.2. Self-employed 4.2.1. Why would you prefer to be self-employed (have your own business) rather than an employee? _________ ________________________________________ 4.2.1.1. Personal independence/ self-fulfilment 4.2.1.2. Exploiting a business opportunity 4.2.1.3. Better income prospects 4.2.1.4. Freedom to choose place and time of working 4.2.1.5. Lack of attractive employment opportunities/lack of employment opportunities 4.2.1.6. Members of family / friends are self-employed 4.2.1.7. Favourable economic climate 4.2.1.8. To avoid the uncertainties related to paid employment 4.2.1.9. To contribute to society 4.2.1.10. Other Specify______________________________________________ 4.2.1.11. Don’t know (DO NOT READ) 4.3. None (DO NOT READ) 4.4. Don’t Know (DO NOT READ) 5. Regardless of whether or not you would like to become self-employed (have your own business), would it be feasible for you to be self-employed within the next year? 5.1. Very feasible 5.2. Fairly feasible 5.3. Not very feasible 5.4. Not feasible at all 5.5. Don’t know (DO NOT READ OUT) 6. Why would it not be feasible for you to be self-employed within the next year? 6.1. No business idea 6.2. Not enough capital/financial resources to be self-employed 6.3. Not enough skills to be self-employed 6.4. Burden of red tape /Administrative difficulties 6.5. The risk of failure and its legal and social consequences are too big 6.6. It would be difficult to reconcile with my family commitments (Precoded, DO NOT READ, to record the answers choose and write the codes from the table below; if answer does not fit in a code use the 11. other and write the verbatim) (Precoded, DO NOT READ, to record the answers choose and write the codes from the table below; if answer does not fit in a code use the 10. other and write the verbatim) If not very feasible or not feasible at all go to Question 7 If very feasible and fairly feasible Skip to Question 8 Final Evaluation Report – Jordan Loan Guarantee Facility 100 6.7. The current economic climate is not good for a start-up 6.8. Other 6.9. Don’t know (DO NOT READ OUT) (Skip to section II) 7. Please describe the business that you were planning to start? _____________________ 8. Are you still going to establish the business? (if no, continue. if yes, skip to question 10) ___________________ 9. Please identify the primary reason that prevented you from starting the business. _____________________ 10. What type of assistance do you need for starting the new business you just mentioned? 10.1. Please specify the most important assistance needed: ___________________________________(RECORD VERBATIM AND POST CODE) II. Section II – Exposure and Perception of JLGF Technical Assistance 1. How did you first learn about the JLGF project? 1.1. Through a newspaper advertisement 1.2. Through a business association 1.3. Through a bank 1.4. Was contacted directly by JLGF 1.5. Other (Please Specify): __________________________ 2. When was your first interaction with the JLGF team? Please Specify: __________year 3. Of the following, please specify the type of technical assistance service you received from JLGF? (SELECT ALL THAT APPLY) 3.1. Trainings 3.1.1. SME Financing 3.1.2. Financial Analysis 3.1.3. Budgeting 3.1.4. Financial Management and Costing 3.1.5. Other (Specify): _______________ (IF DID NOT RECEIVE TRAINING, SKIP TO Q3.2) 3.2. Loan Guarantee, through OPIC 3.2.1. Single 3.2.2. Multiple (IF DID NOT RECEIVE GUARANTEE, SKIP TO Q4) 3.3. When it comes to facilitating access to finance for your business, would you say that the OPIC-JLGF guarantee was? 3.3.1. Very Important 3.3.2. Important (PLEASE RECORD THE ANSWER AS PRECISELY AS POSSIBLE) Final Evaluation Report – Jordan Loan Guarantee Facility 101 3.3.3. Not at all important 3.3.4. Don’t know (DO NOT READ) 4. What is the most important element and/or skill you have learned as a result of the JLGF Technical Assistance? 4.1. ____________________________________(RECORD VERBATIM AND POST CODE) 4.1.1. Why do you say that: ____________________________________(RECORD VERBATIM AND POST CODE) 5. What is the long-term impact of the assistance you have received from JLGF? 5.1. ____________________________________(RECORD VERBATIM AND POST CODE) 6. What is your overall level of satisfaction with the assistance provided by JLGF? 6.1. Very satisfied 6.2. Somewhat satisfied 6.3. Somewhat dissatisfied 6.4. Very dissatisfied 6.5. Refused to answer (DO NOT READ) 7. When it comes to your specific business needs to start a business, to what extent did the JLGF assistance cover those needs? 7.1. Adequately covered my specific business needs 7.2. Partially covered my specific business needs 7.3. Did not cover my specific business needs 7.4. Refused to answer (DO NOT READ) 8. To what degree do you agree or disagree with the following statements: 8.1. JLGF Technical assistance has increased my access to finance 8.1.1. Strongly Agree 8.1.2. Agree 8.1.3. Neither Agree or Disagree 8.1.4. Strongly Disagree 8.1.5. Refused / Don’t Know (DO NOT READ) 8.2. JLGF Technical Assistance has increased my awareness of how to access finance 8.2.1. Strongly Agree 8.2.2. Agree 8.2.3. Neither Agree or Disagree 8.2.4. Strongly Disagree 8.2.5. Refused / Don’t Know (DO NOT READ) 8.3. JLGF Technical Assistance has supported me to meet banks and OPIC requirements for obtaining a loan 8.3.1. Strongly Agree 8.3.2. Agree 8.3.3. Neither Agree or Disagree 8.3.4. Strongly Disagree 8.3.5. Refused / Don’t Know (DO NOT READ) Final Evaluation Report – Jordan Loan Guarantee Facility 102 9. Is there any specific additional technical assistance that was not provided by the JLGF project that you would like for them to provide? 9.1. Please specify the most important additional technical assistance need: 9.1.1. ____________________________________(RECORD VERBATIM AND POST CODE) 10. Do you have any recommendations on how the technical assistance provided by the JLGF project may be improved? If yes please specify 10.1. ____________________________________(RECORD VERBATIM AND POST CODE) 11. Do you know of any other project, program or institution that offers technical assistance like the JLGF project? (if answer is no, skip to question 16; If yes, please tell us the name of the program) 11.1. Yes Specify the name of the program __________________________________ 11.2. No (skip to question 16) 11.3. Don’t Know (DO NOT READ – skip to question 16) 12. Did you benefit from any other project, program or institution that offers technical assistance like the JLGF project? 12.1. Yes 12.1.1. Please specify the type of assistance received: __________________________________ (RECORD VERBATIM AND POST CODE) 12.2. No (skip to question 16) 12.3. Don’t Know (DO NOT READ – skip to question 16) 13. What is your overall level of satisfaction with similar type of technical assistance received from project/s other than JLGF? You were: 13.1. Very satisfied 13.2. Somewhat satisfied 13.3. Neither satisfied nor dissatisfied 13.4. Somewhat dissatisfied 13.5. Very dissatisfied 13.6. Refused to Answer (DO NOT READ) 13.7. Don’t Know (DO NOT READ) 14. How do you compare the assistance provided by the other project, program or institutions that offer technical assistance like the JLGF project? 14.1. JLGF project assistance is significantly better than other similar projects 14.2. JLGF project assistance is slightly better than other similar projects 14.3. JLGF project assistance is the same as other similar project 14.4. JLGF project assistance is slightly worse than other similar projects 14.5. JLGF project assistance significantly worse than other similar projects. 14.6. Refused to Answer (DO NOT READ) 14.7. Don’t Know (DO NOT READ) 15. Why do you say that? _________________________________ (RECORD VERBATIM AND POST CODE) 16. Do you know of any other project, program or institution that offers Loan Guarantee like the JLGF project? Final Evaluation Report – Jordan Loan Guarantee Facility 103 (If the answer is no, skip to section V; If yes, please tell us the name of the program) 16.1. Yes Specify the name of the program __________________________________ 16.2. No (SKIP TO SECTION V) 16.3. Don’t Know (DO NOT READ – SKIP TO SECTION V) 17. Did you benefit from any other project, program or institution that offers Guarantee loans like the JLGF project? 17.1. Yes 17.1.1. Please specify the type of guarantee received: __________________________________ (RECORD VERBATIM AND POST CODE) 17.2. No (SKIP TO SECTION V) 17.3. Don’t Know (DO NOT READ – SKIP TO SECTION V) 18. What is your overall level of satisfaction of the guarantee received from project/s other than JLGF? You were: 18.1. Very satisfied 18.2. Somewhat satisfied 18.3. Neither satisfied nor dissatisfied 18.4. Somewhat dissatisfied 18.5. Very dissatisfied 18.6. Refused to Answer (DO NOT READ) 18.7. Don’t Know (DO NOT READ) 19. How do you compare the guarantee provided by the other project, program or institutions? 19.1. JLGF -OPIC guarantee is significantly better than other similar projects 19.2. JLGF -OPIC guarantee is slightly better than other similar projects 19.3. JLGF -OPIC guarantee is the same as other similar project 19.4. JLGF -OPIC guarantee is slightly worse than other similar projects 19.5. JLGF -OPIC guarantee significantly worse than other similar projects. 19.6. Refused to Answer (DO NOT READ) 19.7. Don’t Know (DO NOT READ) 19.7.1. Why is better of worst? _________________________________ (RECORD VERBATIM AND POST CODE) III. Section III – Changes in Business Practices and Loan Application (access to capital 1. Has your approach to assessing the feasibility of a business idea changed as a result of the JLGF technical assistance? 1.1. Significantly changed 1.2. Somewhat changed 1.3. Remained the same (SKIP TO Q4) 1.4. Refused to Answer (DO NOT READ – SKIP TO Q4) 1.5. Don’t Know (DO NOT READ – SKIP TO Q4) 2. What is the approach that you changed as a result of the JLGF technical assistance? 2.1. Financial statements 2.2. Improved accounting 2.3. Improved cash-flow projections 2.4. Development of new products and services Final Evaluation Report – Jordan Loan Guarantee Facility 104 2.5. Other (SPECIFY): _____________________________________________________ 3. When it comes to permanence and sustainability, do you expect the changes you have made to your business skills as a result of JLGF technical assistance to be: 3.1. Permanent and effective in the long-run 3.2. Temporary and effective in the short-run 3.3. Don’t Know (DO NOT READ) 4. Did you apply for a loan as a result of the technical assistance provided by JLGF? 4.1. YES 4.1.1. How many, if more than one: ___________________________ 4.2. No 4.2.1. Why not: ___________________________________ (SKIP TO QUESTION 6) 5. Did you succeed in obtaining the loan? (if yes continue if no skip to section 5.2) 5.1. Yes 5.1.1. Purpose of loan/s 5.1.1.1. _____________________ 5.1.1.2. _____________________ 5.1.1.3. _____________________ 5.1.2. Size of loan/s 5.1.2.1. _____________________ 5.1.2.2. _____________________ 5.1.2.3. _____________________ 5.1.3. Status of loan 5.1.3.1. Settled 5.1.3.2. Active 5.1.3.3. Canceled 5.1.3.4. Other _______________________ 5.1.4. Primary benefit of the loan 5.1.4.1. _____________________ 6. Are you planning to apply for a loan over the next year? 6.1. Yes 6.2. No 6.3. Maybe 6.4. Refused to Answer (DO NOT READ) 6.5. Don’t Know (DO NOT READ) (IF ANSWER IS NO, END THE SURVEY) 7. What is the purpose of that loan? 8. Would you use the JLGF for this loan? 8.1. Yes 8.2. No IF THE ANSWER IS NO 8.2.1. Please provide the reason. _______________________________ 8.3. Maybe Final Evaluation Report – Jordan Loan Guarantee Facility 105 8.4. Refused to Answer (DO NOT READ) 8.5. Don’t Know (DO NOT READ) Appendix IV: Roles and Responsibilities Miguel Cabal – Team Leader - Will be responsible for team organization - Lead the technical aspects with USAID and Implementing partner - Lead the preparation of the work plan and evaluation design - Lead the design and writing of the field data instrument - Identify the information gaps and requirements - Conduct interviews with key-informants - Data analysis and integrating the FCRs - Deliver the findings, conclusions, and recommendations to USAID Hisham Yassin - Financial Sector Specialist - Contribute to research design particularly regarding the SMEs financial sector. - Design data collection instruments and developing the questions regarding SMEs and the Banks - Conduct interviews and leading focus groups with bank staff - Summarize the findings - Help in analyzing the results - Support report writing Ragheb Fityan-Monitoring & Evaluation Specialist - Contribute to and support the development of the evaluation design - Support in the design of data collection instruments - Conduct interviews with key–informants - Conduct in-depth interviews - Help in data analysis - Analyze and track the performance indicators and information sources - Support writing the report Mindset - Provide interviewers to collect data, then enter and process the raw data for the beneficiary phone survey. MESP M&E experts - The Evaluation team will report to the Office of Program Management at USAID, Jordan and will work closely with the COR of JLGF. The team will receive guidance to determine plans, and timelines. The team will be managed by the MESP Senior M&E Specialist - The MESP team will identify and bring on board any additional evaluation support needed by the evaluation team to fill any gaps as they may arise through their network of staff evaluation specialists, partner organizations, and consultants. Final Evaluation Report – Jordan Loan Guarantee Facility 106 ANNEX C: REFERENCES AND WORKS CITED “Agricultural Livelihoods and Food Security Impact Assessment and Response Plan for the Syria Crisis in the Neighboring Countries of Egypt, Iraq, Jordan, Lebanon and Turkey. (2013). United Nations Food and Agriculture Organization. Betz, F., and G. Frewer. (2016). Jordan: Neighbourhood SME Financing. European Investment Bank. Fakih, A. and M. Ibrahim. (2016). “The Impact of Syrian Refugees on the Labor Market in Neighboring Countries: Empirical Evidence from Jordan.” Discussion Paper No. 9667. Forschungsinstitut zur Zukunft der Arbeit. Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2012, Quarter 2. Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2015, Quarter 2. Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2016, Quarter 1. Global Communities. Jordan Loan Guarantee Facility Quarterly Report FY2016, Quarter 2. Global Communities. Jordan Loan Guarantee Facility Technical Proposal. Global Communities. Jordan Loan Guarantee Facility Work Plan FY2012. Global Communities. Jordan Loan Guarantee Facility Work Plan FY2013. Global Communities. Jordan Loan Guarantee Facility Work Plan FY2014. Global Communities. Jordan Loan Guarantee Facility Work Plan FY2015. Global Communities. Jordan Loan Guarantee Facility Work Plan FY2016. “Refugees constitute third of Jordan population – World Bank official.” Jordan Times. 19 December 2015. “Seven ‘C’s’ of Credit.” SME Toolkit. International Finance Corporation: World Bank Group. Statistical Database. Central Bank of Jordan. Survey Study on SMEs in Jordan: Analysis of supply-side and demand-side focusing on bank financing. (2016). Association of Banks in Jordan. Taghdisi-Rad, S. (2012). “Macroeconomic policies and employment in Jordan: Tackling the paradox of job-poor growth.” Employment Sector Employment Working Paper No. 118. International Labor Office. USAID. Jordan Country Development and Cooperation Strategy 2013-2017. http://pdf.usaid.gov/pdf_docs/pdacx797.pdf. World Development Indicators. World Bank Group. Final Evaluation Report – Jordan Loan Guarantee Facility 107 ANNEX D: IMPLEMENTATION TIMELINE This timeline highlights important events that impacted, or help explain, the effectiveness of the JLGF activity. Final Evaluation Report – Jordan Loan Guarantee Facility 108 ANNEX E: PERFORMANCE MANGEMENT INDICATORS Indicator Unit FY 2012 FY 2013 FY 2014 FY 2014 FY 2015 FY 2015 FY 2016 FY 2016 Cumulative Actuals Cumulative Targets Actual Actual Actual Target Actual Target Actual Through Quarter 2 Target Goal: Improve and expand sustainable access to finance for SMEs facilitating SME growth and job creation Number of persons receiving new employment or better employment as a result of participation in USG￾funded workforce development programs. Persons receiving new or better employment 66 498 924 2,400 1,046 2,400 346 1,118 2,880 6,482 Objective 1: Mobilize bank lending for SMEs using loan guarantees Number of loans guaranteed by JLGF. (Number of SMEs that successfully accessed bank loans or private equity as a result of USG assistance.) Number of loans 6 41 80 60 112 80 53 150 292 337 Final Evaluation Report – Jordan Loan Guarantee Facility 109 Indicator Unit FY 2012 FY 2013 FY 2014 FY 2014 FY 2015 FY 2015 FY 2016 FY 2016 Cumulative Actuals Cumulative Targets Actual Actual Actual Target Actual Target Actual Through Quarter 2 Target Volume of loans guaranteed by JLGF. US$ $2,985,8 76.00 $8,215,819. 00 $19,560,73 4.00 $30,000, 000.00 $25,468,5 79.00 $19,200, 000.00 $13,753,671.00 $36,600, 000.00 $69,984,67 9.00 $97,001,69 5.00 Number of firms receiving USG assistance to invest in improved technologies. Number of firms 5 22 37 24 18 18 7 20 89 89 Number of woman owned businesses financed. Number of businesses 0 7 7 5 20 7 11 24 45 43 Person hours of training completed in private sector productive capacity supported by USG assistance. Person hours 1,206 2,016 1,992 1,620 898 1,620 364 900 6,476 7,362 Objective 2: Strengthen bank capacity for sustainable SME lending Final Evaluation Report – Jordan Loan Guarantee Facility 110 Indicator Unit FY 2012 FY 2013 FY 2014 FY 2014 FY 2015 FY 2015 FY 2016 FY 2016 Cumulative Actuals Cumulative Targets Actual Actual Actual Target Actual Target Actual Through Quarter 2 Target Number of financial institutions receiving USG assistance in extending services to micro and small businesses. (# Partner banks) Number of institutions 5 7 7 8 7 8 7 8 7 8 Person hours of training completed by financial sector professionals on international standards supported by USG assistance. Person hours 1,456 876 1,145 1,120 1,953 1,120 427 772 5,857 5,344 Final Evaluation Report – Jordan Loan Guarantee Facility 111 ANNEX F: STATISTICAL ANNEX Rejection Database Rejection and Approval Rate by Bank Level of Activity and Year Most Active Middle Activity Level Least Active Total Approval Rate Rejectio n Rate Approved Rejected Approved Rejected Approve d Rejected Approved Rejected 2012 3 3 2 3 2 0 7 6 54% 46% 2013 35 9 9 9 1 0 45 18 71% 29% 2014 60 12 14 3 1 2 75 17 82% 18% 2015 110 19 11 3 0 0 121 22 85% 15% 2016 45 0 8 0 1 1 54 1 98% 2% Total 253 43 44 18 5 3 302 64 83% 17% Approval Rate 85% 71% 63% 83% Rejection Rate 15% 29% 38% 17% Reason for Rejection by Sex of the Owner Reason for Rejection Male Owned or Controlled Women-owned or Controlled Total Loan amount too high 10 2 12 Bad repayment record / Returned checks /Owner has legal problems 11 0 11 Non-eligible activity 9 0 9 Unrealistic business Plan, unclear or uncertain opportunity or purpose 6 0 6 Business not profitable, Assets overpriced 6 0 6 Owner does not meet eligibility criteria, Proprietorship 5 0 5 Financial need not clear /unreliable financial statements 4 0 4 Term higher than 7 yeas / Borrower with current high guarantee credit facility 3 0 3 No business experience 3 0 3 Final Evaluation Report – Jordan Loan Guarantee Facility 112 Reason for Rejection Male Owned or Controlled Women-owned or Controlled Total Other 1 2 3 Conflict of Interest 1 0 1 No compliance with JLGF requirements 1 0 1 Total 60 4 64 Guarantee Loans 252 50 302 Rejection Rate 19% 7% 17% Rejection Rate by Governorate Governorate Loan Guarantee Rejected Approval Rate Rejection Rate Amman 213 33 87% 13% Irbid 33 6 85% 15% Zarqa 21 4 84% 16% Other Governorates 35 21 63% 38% Total 302 64 83% 17% Rejection Rate by Industry Industry Loan Guarantee Rejected Approval Rate Rejection Rate Manufacturing 43 20 68% 32% Trade 172 16 91% 9% Accommodation and Food Services 23 13 64% 36% Transportation and Warehousing 3 4 43% 57% Finance and Insurance 7 3 70% 30% Mining 0 2 0% 100% Professional, Scientific, and Technical Services 10 1 91% 9% Utilities 11 1 92% 8% Construction 11 1 92% 8% Agriculture, Forestry, Fishing and Hunting 2 1 67% 33% Personal and Other Services 20 0 100% 0% N/A 0 2 N/A N/A Total 302 64 83% 17% Final Evaluation Report – Jordan Loan Guarantee Facility 113 Number of Attendants to JLGF Workshops by Partner and Non-Partner Banks by Year Year Partner Bank Not partner bank Total 2012 69 0 69 2013 111 0 111 2014 42 0 42 2015 189 28 217 2016 89 11 100 Total 500 39 539 Number of Attendants to JLGF Workshops by Bank's Activity Level by Year Most Active Middle Level Least Active Total Attendants % of Attendants in Year Attendants % of Attendants in Year Attendants % of Attendants in Year Attendants % of Attendants in Year 2012 29 42.0% 16 23.2% 24 34.8% 69 100.0% 2013 75 67.6% 24 21.6% 12 10.8% 111 100.0% 2014 21 50.0% 17 40.5% 4 9.5% 42 100.0% 2015 105 55.6% 26 13.8% 58 30.7% 189 100.0% 2016 23 25.8% 32 36.0% 34 38.2% 89 100.0% Number of SME Attendants By Sex and Year Female Male Total Attendants % of Attendants in Year Attendants % of Attendants in Year Attendants % of Attendants in Year 2012 47 44.8% 58 55.2% 105 100.0% 2013 6 13.0% 40 87.0% 46 100.0% 2014 40 37.4% 67 62.6% 107 100.0% 2015 20 48.8% 21 51.2% 41 100.0% 2016 94 86.2% 15 13.8% 109 100.0% Total 207 50.7% 201 49.3% 408 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 114 Number of Workshops Attended by SMEs/Star-ups by Sex Number of Workshops Attended Female Male Total Attendants % Attendants % Attendants % 1 188 90.8% 181 90.0% 369 90.4% 2 16 7.7% 17 8.5% 33 8.1% 3 2 1.0% 3 1.5% 5 1.2% 4 1 .5% 0 0.0% 1 0.2% Total 207 100.0% 201 100.0% 408 100.0% SME and Start-up Survey Responses (those that can be combined) [Q1_2_1] 2.1. Governorate: Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Irbid 6 7.1% 3 4.7% 9 6.1% Jerash 0 0.0% 1 1.6% 1 0.7% Mafraq 13 15.5% 4 6.3% 17 11.5% Amman 31 36.9% 21 32.8% 52 35.1% Zarqa 7 8.3% 14 21.9% 21 14.2% Karak 9 10.7% 9 14.1% 18 12.2% Tafilah 11 13.1% 11 17.2% 22 14.9% Ma'an 1 1.2% 0 0.0% 1 0.7% Aqaba 6 7.1% 1 1.6% 7 4.7% Total 84 100.0% 64 100.0% 148 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 115 [Q1_3_3] 3.3. Main Market (Primary Market): Response Number Percent 1. Governorate Level 36 42.9% 2. National Country-wide 28 33.3% 3. Regional and International Level 20 23.8% Total 84 100.00% [Q1_4] 4. Gender of and type of Ownership: Response Number Percent Woman Sole Proprietor 40 47.6% Man Sole Proprietor 15 17.9% Group of Women 4 4.8% Group of Men 11 13.1% Women and Men shared proprietorship 14 16.7% Total 84 100.0% [Q1_5] 5. Is your business in one location or do you have different locations (i.e. branches, point of sale, point of services) Response Number Percent Single Location 71 84.5% Multiple Locations (Number of Locations) 13 15.5% Total 84 100.0% [Q1_8] 8. Do you own or do you rent the space for your main business location? Response Number Percent Own 28 33.3% Rent 36 42.9% Other (specify) 20 23.8% Total 84 100.0% [Q1_9] 9. Separate from your business do you own any property/real-estate? Response Number Percent Yes 41 48.8% No 43 51.2% Final Evaluation Report – Jordan Loan Guarantee Facility 116 Total 84 100.0% [Q1_10] 10. When it comes to sales, does your business have a high or low season (seasonal) or are your sales consistent throughout the year? Response Number Percent Seasonal 57 67.9% Sales are consistent throughout the year 27 32.1% Total 84 100.0% [Q2_1] 1. When it comes to the overall outlook of your sector over the last year, would you say that things are moving in the right direction or in the wrong direction: Response Number Percent Right Direction. What is the main reason? 71 85.5% Wrong Direction. What is the main reason? 11 13.3% Don’t Know(DO NOT READ) 1 1.2% Total 83 100.0% [Q2_2] 2. When it comes to the sales and profits (performance) of your business, over the last five years (ASK SINCE YOUR BUSINESS STARTED IF LESS THAN FIVE YEARS OLD), would you say that your sales and profits have: Response Number Percent Significantly improved 20 23.8% Somewhat improved 37 44.0% Remained the same 12 14.3% Somewhat worsened 10 11.9% Significantly worsened 4 4.8% Don’t Know (DO NOT READ) 1 1.2% Total 84 100.0% [Q2_4] 4. Thinking of your sales and profit over the next year, would you say that your sales and profits will: Response Number Percent Significantly improved 33 39.3% Somewhat improved 33 39.3% Remained the same 5 6.0% Somewhat worsened 6 7.1% Final Evaluation Report – Jordan Loan Guarantee Facility 117 Significantly worsened 2 2.4% Refused to Answer (DO NOT READ) 1 1.2% Don’t Know (DO NOT READ) 4 4.8% Total 84 100.0% [Q2_2_3] 5.2. When it comes to the performance and growth of your business, would you say that access to finance is: Response Number Percent Very Important 42 50.0% Important 36 42.9% Not at all important 3 3.6% Don’t know (DO NOT READ) 3 3.6% Total 84 100.0% [Q2_9] 9. When it comes to two most important banking needs of your business, what is your overall level of satisfaction? Response Number Percent Very satisfied 17 20.2% Somewhat satisfied 37 44.0% Neither satisfied nor dissatisfied 17 20.2% Somewhat dissatisfied 11 13.1% Not satisfied at all 2 2.4% Total 84 100.0% [Q3_1] 1. Are you thinking of implementing or starting any kind of plan or project for your business during the following year? Response Number Percent Yes 64 76.2% No 11 13.1% May be/not sure 9 10.7% Total 84 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 118 [Q3_2_1] 2.1 Are you planning on doing any project or changing your business in any significant way, not mentioned above? Response Number Percent Yes. What are those ways? Please specify: 1 1.4% No 72 98.6% Total 73 100.0% [Q3_3] 3. How much do you think will it cost implementing the above mentioned plan(s)? Response Number Percent JOD 60 82.2% 999. I don’t know 13 17.8% Total 73 100.0% [Q3_3_a] a) Would you consider applying for a loan for financing the above mentioned plan(s)? Response Number Percent Yes 49 67.1% No 13 17.8% May be/not sure 11 15.1% Total 73 100.0% [Q3_3_b] b) How much money would you consider asking for to financial source? Response Number Percent JOD 49 81.7% 999. I don’t know 10 16.7% 888. I don’t want to answer 1 1.7% Total 60 100.0% [Q3_3_c] c) Where do you prefer applying for the loan? Response Number Percent 1. Family and friends 1 1.7% 2. Lenders 1 1.7% 3. Commercial Banks 9 15.0% 4. Islamic banks 15 25.0% Final Evaluation Report – Jordan Loan Guarantee Facility 119 5. Donor funded programs (NGOs) 26 43.3% 6. Micro financing institution 3 5.0% 7. Suppliers 2 3.3% 8. Other financial institution 1 1.7% 9. Other 2 3.3% Total 60 100.0% [Q3_3_d] d) Why do you prefer this source? Response Number Percent 1. Better service 26 43.3% 2. Better interest rate 12 20.0% 4. I have worked with it before 7 11.7% 5. Have better conditions 5 8.3% 6. Have products or services that fit my needs 4 6.7% 8. Better repayment schedule 2 3.3% 9. other 4 6.7% Total 60 100.0% [Q3_5] 5. Are you planning on starting a new business in the near future (next couple of years)? Response Number Percent Yes 43 51.2% No 38 45.2% Don’t Know (DO NOT READ) 3 3.6% Total 84 100.0% [Q3_6] 6. Is this business related to the current business or a completely different business? Response Number Percent Yes 29 67.4% No 14 32.6% Total 43 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 120 [Q4_1] 1. How did you first learn about the JLGF project? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Through a newspaper advertisement 4 4.8% 1 1.6% 5 3.4% Through a business association 9 10.8% 6 9.4% 15 10.2% Through a bank 6 7.2% 3 4.7% 9 6.1% Was contacted directly by JLGF 21 25.3% 7 10.9% 28 19.0% Other (Specify): 43 51.8% 47 73.4% 90 61.2% Total 83 100.0% 64 100.0% 147 100.0% [Q4_3] 3. Of the following, please specify the type of technical assistance you received from the JLGF project? Response Number Percent Training 79 94.0% Loan Guarantee 6 7.1% [Q4_3_1] What type of training? Response Number Percent SME Financing 35 44.3% Financial Analysis 33 41.8% Budgeting 15 19.0% Financial Management and Costing 27 34.2% Other (Specify): 4 5.1% [Q4_3_2] 3.2. What type of loan guarantee? Response Number Percent Single 4 66.7% Multiple 2 33.3% Total 6 100.0% [Q4_3_3] 3.3. When it comes to facilitating access to finance for your business, would you say that the OPIC￾JLGF guarantee was? Response Number Percent Very Important 16 88.9% Final Evaluation Report – Jordan Loan Guarantee Facility 121 Important 1 5.6% Don’t know (DO NOT READ) 1 5.6% Total 18 100.0% [Q4_6] 6. What is your overall level of satisfaction with the services provided by JLGF? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Very satisfied 41 48.8% 40 62.5% 81 54.7% Somewhat satisfied 27 32.1% 18 28.1% 45 30.4% Neither satisfied nor dissatisfied 8 9.5% 3 4.7% 11 7.4% Somewhat dissatisfied 2 2.4% 2 3.1% 4 2.7% Very dissatisfied 5 6.0% 0.0% 5 3.4% Refused to answer (DO NOT READ) 1 1.2% 1 1.6% 2 1.4% Total 84 100.0% 64 100.0% 148 100.0% [Q4_7] 7. When it comes to your specific business needs, to what extent did the JLGF project cover those needs? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Adequately covered my specific business needs 20 23.8% 18 28.6% 38 25.9% Partially covered my specific business needs 34 40.5% 11 17.5% 45 30.6% Did not cover my specific business needs 30 35.7% 34 54.0% 64 43.5% Refused to answer (DO NOT READ) 0 0.0% 1 1.6% 1 0.7% Total 84 100.0% 63 100.0% 147 100.0% [Q4_8_1] 8.1. JLGF Technical assistance has increased my access to finance Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Strongly Agree 22 26.2% 16 25.0% 38 25.7% Somewhat Agree 33 39.3% 24 37.5% 57 38.5% Neither Agree or Disagree 13 15.5% 16 25.0% 29 19.6% Somewhat Disagree 11 13.1% 5 7.8% 16 10.8% Strongly Disagree 3 3.6% 0.0% 3 2.0% Refused / Don’t Know (DO NOT READ) 2 2.4% 3 4.7% 5 3.4% Total 84 100.0% 64 100.0% 148 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 122 [Q4_8_2] 8.2. JLGF Technical Assistance has increased my awareness of how to access finance Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Strongly Agree 29 34.5% 24 37.5% 53 35.8% Somewhat Agree 32 38.1% 28 43.8% 60 40.5% Neither Agree or Disagree 11 13.1% 9 14.1% 20 13.5% Somewhat Disagree 8 9.5% 2 3.1% 10 6.8% Strongly Disagree 3 3.6% 0.0% 3 2.0% Refused / Don’t Know (DO NOT READ) 1 1.2% 1 1.6% 2 1.4% Total 84 100.0% 64 100.0% 148 100.0% [Q4_8_3] 8.3. JLGF Technical Assistance has supported me to meet banks and OPIC requirements for obtaining a loan? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Strongly Agree 19 22.6% 20 31.3% 39 26.4% Agree 33 39.3% 22 34.4% 55 37.2% Neither Agree or Disagree 14 16.7% 17 26.6% 31 20.9% Disagree 12 14.3% 2 3.1% 14 9.5% Strongly Disagree 3 3.6% 0 0.0% 3 2.0% Refused / Don’t Know (DO NOT READ) 3 3.6% 3 4.7% 6 4.1% Total 84 100.0% 64 100.0% 148 100.0% [Q4_9] 9. Is there any specific additional technical assistance that was not provided by the JLGF project that you would like for them to provide? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 28 33.3% 15 23.4% 43 29.1% No 56 66.7% 49 76.6% 105 70.9% Total 84 100.0% 64 100.0% 148 100.0% [Q4_10] 10. Do you have any recommendations on how the technical assistance provided by the JLGF project may be improved? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 29 34.5% 21 32.8% 50 33.8% Final Evaluation Report – Jordan Loan Guarantee Facility 123 No 55 65.5% 43 67.2% 98 66.2% Total 84 100.0% 64 100.0% 148 100.0% [Q4_11] 11. Do you know of any other project, program or institution that offers technical assistance like the JLGF project? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes Specify the name of the program 18 21.4% 19 29.7% 37 25.0% No. 64 76.2% 45 70.3% 109 73.6% Don’t Know (DO NOT READ) 2 2.4% 0 0.0% 2 1.4% Total 84 100.0% 64 100.0% 148 100.0% [Q4_12] 12. Did you benefit from any other project, program or institution that offers technical assistance like the JLGF project? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 13 59.1% 6 31.6% 19 46.3% No 9 40.9% 13 68.4% 22 53.7% Total 22 100.0% 19 100.0% 41 100.0% [Q4_13] 13. What is your overall level of satisfaction with similar type of technical assistance received from project/s other than JLGF? You were: Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Very satisfied 4 28.6% 3 50.0% 7 35.0% Somewhat satisfied 9 64.3% 3 50.0% 12 60.0% Neither satisfied nor dissatisfied 1 7.1% 0 0.0% 1 5.0% Total 14 100.0% 6 100.0% 20 100.0% [Q4_14] 14. How do you compare the services provided by the other project, program or institutions that offer technical assistance like the JLGF project? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage JLGF project assistance is significantly better than other similar projects 4 28.6% 3 50.0% 7 35.0% JLGF project assistance is slightly better than other similar projects 1 7.1% 0.0% 1 5.0% JLGF project assistance is the same as other similar project 3 21.4% 1 16.7% 4 20.0% JLGF project assistance is slightly worse than other similar projects 2 14.3% 1 16.7% 3 15.0% JLGF project assistance significantly worse than other similar projects 2 14.3% 0.0% 2 10.0% Final Evaluation Report – Jordan Loan Guarantee Facility 124 Don’t Know (DO NOT READ) 2 14.3% 1 16.7% 3 15.0% Total 14 100.0% 6 100.0% 20 100.0% [Q4_16] 16. Do you know of any other project, program or institution that offers Loan Guarantee like the JLGF project? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 25 29.8% 21 32.8% 46 31.1% No 55 65.5% 43 67.2% 98 66.2% Don’t Know (DO NOT READ ) 4 4.8% 0 0.0% 4 2.7% Total 84 100.0% 64 100.0% 148 100.0% [Q4_18] 18. What is your overall level of satisfaction of the guarantee received from project/s other than JLGF? You were: Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Somewhat satisfied 7 87.5% 1 100.0% 8 88.9% Somewhat dissatisfied 1 12.5% 0 0.0% 1 11.1% Total 8 100.0% 1 100.0% 9 100.0% [Q4_19] 19. How do you compare the guarantee provided by the other project, program or institutions? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage JLGF -OPIC guarantee is significantly better than other similar projects 3 37.5% 0 0.0% 3 33.3% JLGF -OPIC guarantee is slightly worse than other similar projects 1 12.5% 0 0.0% 1 11.1% JLGF -OPIC guarantee significantly worse than other similar projects 0 0.0% 1 100.0% 1 11.1% Don’t Know (DO NOT READ) 4 50.0% 0 0.0% 4 44.4% Total 8 100.0% 1 100.0% 9 100.0% [Q5_1] 1. Has your approach to business practices changed as a result of the JLGF technical assistance? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Significantly changed 29 34.5% 27 42.2% 56 37.8% Somewhat changed 32 38.1% 24 37.5% 56 37.8% Remained the same 22 26.2% 12 18.8% 34 23.0% Don’t Know (DO NOT READ) 1 1.2% 1 1.6% 2 1.4% Final Evaluation Report – Jordan Loan Guarantee Facility 125 Total 84 100.0% 64 100.0% 148 100.0% [Q5_2] 2. What is the main business practice that you changed as a result of the JLGF technical assistance? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Financial statements 15 24.2% 13 25.5% 28 24.8% Improved accounting 27 43.5% 17 33.3% 44 38.9% Improved cash flow projections 5 8.1% 10 19.6% 15 13.3% Development of new products and services 11 17.7% 5 9.8% 16 14.2% Other (SPECIFY): 4 6.5% 6 11.8% 10 8.8% Total 62 100.0% 51 100.0% 113 100.0% [Q5_3] 3. When it comes to permanence and sustainability, do you expect the changes you have made to your business as a result of JLGF technical assistance to be: Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Permanent and effective in the long-run 52 83.9% 44 86.3% 96 85.0% Temporary and effective in the short-run 8 12.9% 5 9.8% 13 11.5% Don’t Know (DO NOT READ) 2 3.2% 2 3.9% 4 3.5% Total 62 100.0% 51 100.0% 113 100.0% [Q5_4] 4. Did you apply for a loan as a result of the technical assistance provided by JLGF? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes, How many? 13 15.5% 4 6.3% 17 11.5% No. Why not? 71 84.5% 60 93.8% 131 88.5% Total 84 100.0% 64 100.0% 148 100.0% [Q5_5] 5. Did you succeed in obtaining the loan? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 5 33.3% 1 25.0% 6 31.6% No 10 66.7% 3 75.0% 13 68.4% Total 15 100.0% 4 100.0% 19 100.0% Final Evaluation Report – Jordan Loan Guarantee Facility 126 [Q5_6] 6. Are you planning to apply for a loan over the next year? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 37 44.0% 21 32.8% 58 39.2% No 18 21.4% 28 43.8% 46 31.1% Maybe 20 23.8% 14 21.9% 34 23.0% Refused to Answer (DO NOT READ) 1 1.2% 0.0% 1 0.7% Don’t Know (DO NOT READ) 8 9.5% 1 1.6% 9 6.1% Total 84 100.0% 64 100.0% 148 100.0% [Q5_8] 8. Would you use the JLGF for this loan? Response SME Start-Ups Total Number Percent Number Percentage Number Percentage Yes 37 64.9% 25 71.4% 62 67.4% No, Please provide the reason 1 1.8% 0.0% 1 1.1% Maybe 12 21.1% 10 28.6% 22 23.9% Refused to Answer (DO NOT READ) 1 1.8% 0.0% 1 1.1% Don’t Know (DO NOT READ) 6 10.5% 0.0% 6 6.5% Total 57 100.0% 35 100.0% 92 100.0% SME responses (stand alone from start-ups) [Q2_8] 8. What are the two most important banking services for your business? Banking Services (Pre-Coded- DO NOT READ, just chose from the below table the closest option) Response Most important Second most important Number Percentage Number Percentage 1-Chequing account 29 34.5% 2 2.4% 2-Savings Account 10 11.9% 11 13.1% 3-Credit Card 9 10.7% 9 10.7% 4-Business loans 12 14.3% 11 13.1% Final Evaluation Report – Jordan Loan Guarantee Facility 127 5-Fixed term saving certificates - 0.0% 0 0.0% 6-ATM and Debit cards 3 3.6% 10 11.9% 7-Life, accident and other insurance 2 2.4% 5 6.0% 8-Mortgage 1 1.2% 1 1.2% 9-Wire transfer national and international 6 7.1% 8 9.5% 10-Electronic Payment 2 2.4% 6 7.1% 11-Phone transactions 2 2.4% 4 4.8% 12-Services for receiving payments domestically and internationally (help here) 2 2.4% 0 0.0% 13-Savings-Credit products 1 1.2% 2 2.4% 12-Other 5 6.0% 15 17.9% Total 84 100.0% 84 100.0% [Q3_2] 2. Are you planning to do de following actions related to our business in the next 12 months? I will read for you some actions that you may be or may be not considering on doing. Please respond yes, no or not sure as it applies to you Action / Project Yes May be/not sure No Total Number Percent Number Percent Number Percent Number Percent a) Increase the number of workers? 51 69.9% 10 13.7% 12 16.4% 73 100% b) Get machinery or equipment? 46 63.0% 11 15.1% 16 21.9% 73 100% c) Increase the amount of raw material or merchandise you buy? 55 75.3% 7 9.6% 11 15.1% 73 100% d) Buy, remodel or increase the size of the shop or factory? 50 68.5% 8 11.0% 15 20.5% 73 100% e) Change the location or buy an additional shop, warehouse, sales point? 37 50.7% 10 13.7% 26 35.6% 73 100% f) Include new products, services or markets on you production/trade/service line? 54 74.0% 9 12.3% 10 13.7% 73 100% g) Start up a new business? 27 37.0% 13 17.8% 33 45.2% 73 100% h) Improve the promotion campaign? 53 72.6% 8 11.0% 12 16.4% 73 100% i) Adopt new or improved technologies? 56 76.7% 9 12.3% 8 11.0% 73 100% j) Buy e vehicle for the business? 38 52.1% 12 16.4% 23 31.5% 73 100% Final Evaluation Report – Jordan Loan Guarantee Facility 128 [Q5_5_1_5] 5.1.2. Please answer the following questions about the impact of the loan on your business. Variable Yes No Total Number Percent Number Percent Number Percent 5.1.5. Have you experienced growth in employment as a result of the loan? 1 20.0% 4 80.0% 5 100.0% 5.1.6. Has the loan allowed you to increase the salary and/or social security coverage for your employees? 1 20.0% 4 80.0% 5 100.0% 5.1.7. Have your business hours of operation increased as a result of the JLGF intervention? 3 60.0% 2 40.0% 5 100.0% Start-up responses (standalone from established SMEs) [Q1_1] 1. To which of the following age groups do you belong to? Or What is your Age Group? Response Number Percent 20 years old or younger 1 1.6 21-25 years 12 18.8 26 to 35 years 23 35.9 36 to 45 years 17 26.6 46 to 55 8 12.5 56 to 65 3 4.7 Total 64 100 [Q1_3] 3. As far as your current occupation is concerned, would you say you are self-employed, an employee, a manual worker or would you say that you are without a professional activity? Response Number Percent Employee (specify) 39 60.9 Manual worker (specify) 1 1.6 Without a professional activity (specify) 24 37.5 Total 64 100 Final Evaluation Report – Jordan Loan Guarantee Facility 129 [Q1_3code.2] 3. As far as your current occupation is concerned, would you say you are self-employed, an employee, a manual worker or would you say that you are without a professional activity? | Without a professional activity Response Number Percent Unemployed 21 32.8 Student 3 4.7 Total 24 37.5 [Q1_4] 4. If you could choose between different kinds of jobs, would you prefer to be an employee or self￾employed (have your own business)? Response Number Percent An employee 19 29.7 Self-employed 45 70.3 Total 64 100 [Q1_4_1code] 4.1. To those who said "employee": Why would you prefer to be an employee rather than a self￾employed (have your own business)? [Q1_4_1code] 4.1. To those who said "employee": Why would you prefer to be an employee rather than a self-employed (have your own business)? Response Number Percent Regular, steady income (versus irregular variable income) 7 36.8% Job security 4 21.1% Covered by social welfare/insurance 1 5.3% Not enough capital/resources to be self-employed 1 5.3% It is a major step to take/hard to turn back 3 15.8% Less responsibility 2 10.5% Started a business and failed 1 5.3% Total 19 100.0% [Q1_4_2code] 4.2. [Q1_4_2code] 4.2. To those who said "self-employed": Why would you prefer to be self￾employee (have your own business) rather than an employee? Response Number Percent Personal independence/ self-fulfillment 23 51.1% Exploiting a business opportunity 4 8.9% Better income prospects 7 15.6% Freedom to choose place and time of working 6 13.3% Lack of attractive employment opportunities/lack of employment opportunities 2 4.4% Final Evaluation Report – Jordan Loan Guarantee Facility 130 Favorable economic climate 1 2.2% To contribute to society 2 4.4% Total 45 100.0% [Q1_5] 5. Regardless of whether or not you would like to become self-employed (have your own business), would it be feasible for you to be self-employed within the next year? Response Number Percent Very feasible 14 21.9 Fairly feasible 28 43.8 Not very feasible 12 18.8 Not feasible at all 9 14.1 Don’t know (DO NOT READ OUT) 1 1.6 Total 64 100 [Q1_6code] 6. Why would it not be feasible for you to be self-employed within the next year? Response Number Percent No business idea 4 19% Not enough capital/financial resources to be self-employed 8 38% The risk of failure and its legal and social consequences are too big 1 5% It would be difficult to reconcile with my family commitments 1 5% The current economic climate is not good for a start-up 3 14% Wants to pursue studies 1 5% Age restrictions 2 10% Workload pressure 1 5% Total 21 100% [Q1_8] 8. Are you still going to establish the business? [Q1_8] 8.Are you still going to establish the business? Response Number Percent Yes 41 98% No 1 2% Total 42 100% Final Evaluation Report – Jordan Loan Guarantee Facility U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523