DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CAN DONORS CATALYZE VENTURE CAPITAL IN EMERGING MARKETS? Midterm Performance Evaluation Report Middle East North Africa Investment Initiative in Lebanon Performance Management and Support Program for Lebanon (PMSPL II) USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | I CAN DONORS CATALYZE VENTURE CAPITAL IN EMERGING MARKETS? Middle East North Africa Investment Initiative in Lebanon Midterm Performance Evaluation Report Performance Management and Support Program for Lebanon (PMSPL II) for USAID/Lebanon Cover Photo: Training provided by Insure and Match Capital (IMC mentors for portfolio companies (public photo) December 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared by David Apgar, Rana Taher, and Tamara Nassereddine of Social Impact jointly with the USAID Economic Growth team including William Butterfield, Director of Economic Growth, Ms. Rana Helou, AOR and Georges Frenn, AOR from USAID. This document was submitted by Social Impact, Inc., to the United States Agency for International Development under USAID Contract/Order No. AID-268-C-15-00001. Limited redactions have been made to in accordance with the principled exceptions to the presumption in favor of openness established in OMB Bulletin 12-01, “Guidance on Collection of U.S. Foreign Assistance Data. This document is available printed or online. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information is available from: Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com II | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV ABSTRACT The objective of this midterm performance evaluation is to explore the effectiveness of the Middle East and North Africa Investment Initiative (MENA II) strategic model, a 5 years USAID funded investment initiative with a budget of 15 million USD. Specifically, the evaluation will review the rationale, performance to date, and effectiveness of the project components supporting the startup ecosystem in Lebanon. The evaluation addresses four questions: 1. Effectiveness of MENA II’s approach and implementation 2. Efficiency of the project monitoring, evaluation, and learning plan in informing project design 3. Sustainability of combinations of activities or interventions 4. The degree of confidence that MENA II will augment USAID’s program objectives if funding is increased or extended. The evaluation employed a qualitative approach, conducting a desk review, key informant interviews, and focus group discussions with USAID, MENA II staff, portfolio companies, and other actors. The evaluation concluded that MENA II has been effective in meeting its goal of advancing the development of Lebanon’s private sector by increasing the competitiveness and revenue growth of early-stage businesses. Components included venture-capital (VC) investment matching, which mobilized significant investments by other VCs; smaller angel-investment equity guarantees, which had a development impact out of proportion to the relatively small level of investments insured; and ecosystem support for angel investors, the launch of an accelerator, and startup access to advice, all of which strengthened the viability of startups in Lebanon. The main factors behind the effectiveness of MENA II investment matching appear to be the central role of implementing partner IMC in the early-stage investment ecosystem and its robust portfolio of ecosystem-support initiatives. MENA II equity guarantees enabled IMC to build a functioning earliest￾stage investment sector helping fill a gap in the earliest-stage funding. Ecosystem support for angel investors and an accelerator were effective but there is need for improvement in areas like mentoring. The review shows MENA II has achieved roughly three quarters of its life-of-program development goals, ahead of the 50-60% expected at this stage of the program. An extension or expansion of investment matching, equity guarantees, and ecosystem support will advance USAID MENA II development goals. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | III TABLE OF CONTENTS ABSTRACT.............................................................................................................................................................. II TABLE OF CONTENTS...................................................................................................................................... III ACRONYMS...........................................................................................................................................................IV EXECUTIVE SUMMARY......................................................................................................................................VI Evaluation Purpose and Evaluation Questions........................................................................................ vi Project Background ....................................................................................................................................... vi Evaluation Design and Methods, ............................................................................................................... vii Key Findings and Conclusions.................................................................................................................... vii Key Recommendations.................................................................................................................................. x INTRODUCTION................................................................................................................................................. 1 Project Description ........................................................................................................................................ 1 Development Hypothesis.............................................................................................................................. 4 Critical Assumptions...................................................................................................................................... 4 model for mena iI............................................................................................................................................ 5 EVALUATION PURPOSE AND QUESTIONS............................................................................................... 8 METHODS AND LIMITATIONS ....................................................................................................................... 9 Desk Review..................................................................................................................................................... 9 Qualitative Data Collection.......................................................................................................................... 9 Data Analysis..................................................................................................................................................10 Limitations.......................................................................................................................................................11 FINDINGS .............................................................................................................................................................. 12 CONCLUSIONS ..................................................................................................................................................29 RECOMMENDATIONS......................................................................................................................................35 ANNEXES .............................................................................................................................................................. 38 Annex 1: considerations for applying mena ii in other countries......................................................39 Annex 2: Inception Report .........................................................................................................................41 Annex 3: Documents Reviewed ................................................................................................................42 Annex 4: Data Collection Instruments....................................................................................................42 Annex 5: mena ii Intended Results/Scope of Work..............................................................................86 Annex 6: List of MENA II Indicators.........................................................................................................88 TABLE OF FIGURES AND TABLES Figure 1: % of Program Implementation Goals Achieved……………………………………...14 Figure 2: % of PRogram Developemnt Goals Achieved………………………………………..18 Figure 3: Women, % of participants……………………………………………………………21 Table 1: Limitation……………………………………………………………………………...11 Table 2: Targets for outputs and some intermediate results………………………………….14 Table 3: Program targets for development objectives…………………………………………18 IV | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV ACRONYMS ADS Automated Directives System AMEP Activity Monitoring and Evaluation Plan AOR Agreement Officer’s Representative BDL Banque du Liban BLOM Banque du Liban et D’Outre-Mer CDCS Country Development Cooperation Strategy COP Chief of Party CPA certified public accountant CSO civil society organization DO Development Objective DQA Data Quality Assessment EQUI™ Evaluation Quality Use and Impact FGD Focus Group Discussion GE/FE Gender Equality/Female Empowerment GDP Gross Domestic Product GNDR Gender HO Home Office IDCS Institutional Development Calculation Sheet IFC International Finance Corporation IM Investment Matching IMC Insure and Match Capital IP implementing partner IR Intermediate Result IRR Internal Rate of Return ISME Innovation is Small and Medium Enterprise K Thousand KII Key Informant Interview LWAF Lebanese Women Angel Fund M Million M&E Monitoring and Evaluation MBA Master class for Business Angels MEL Monitoring, Evaluation, and Learning MENA Middle East and North Africa MENA II Middle East and North Africa Investment Initiative MIT Massachusetts Institute of Technology MSME Micro, Small and Medium Enterprises USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | V NGO Nongovernmental Organization PACE Partnering to Accelerate Entrepreneurship PMP Performance Management Plan PMPL Performance Management Plan for Lebanon PMSPL Performance Management and Support Program for Lebanon QESI Qualified Early Stage Investor QESB Qualified Early Stage Business R&D Research and Development SI Social Impact SME Small and Medium-size Enterprises SOW Scope of Work TA Technical Assistance UC University of California USAID United States Agency for International Development USG U.S. Government VC Venture Capitalist WAIF Women Angel Investment Fund VI | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The objective of this midterm performance evaluation of the Middle East and North Africa Investment Initiative (MENA II) is to explore the effectiveness of the MENA II strategic model. Specifically, the evaluation will review the rationale, performance to date, and effectiveness of the three MENA II components of investment matching, equity guarantees, and technical assistance supporting the startup ecosystem in Lebanon. The evaluation addresses the following four questions: 1. To what extent is MENA II’s approach and implementation effective? 2. To what extent is the current monitoring, evaluation, and learning (MEL) plan efficient in informing project design? 3. What combinations of activities or interventions have more factors in place for sustainability? 4. Given the above, how confident are we that MENA II will augment USAID’s program objectives if funding is increased or extended? PROJECT BACKGROUND Economic growth in the MENA region has been slow to rebound since the global financial crisis and the beginning of the Arab Spring. This slow recovery has been compounded by large numbers of job seekers entering inefficient labor markets. Many smaller companies and new business startups have more difficulty accessing finance and investment funds. Until 2014, moreover, when the Lebanese central bank (Banque du Liban, or BDL) launched its equity￾investment guarantee program under Circular 331, equity investment in smaller Lebanese companies was scarce, and in Lebanese startups was almost non-existent. BDL approved Circular 331 in August 2013, allowing banks to invest up to 3 percent (later increased to 4 percent) of their equity capital in Lebanese startup companies contributing to the so-called “knowledge economy” and in venture capital funds focusing on these types of companies. The BDL rationale behind this initiative is to create jobs and build up the market for early-stage technology investment. MENA II is a 5-year investment initiative with a budget of 15M (M = million) geared towards improving investor and business access to equity capital, developing the early-stage investment ecosystem, advancing financial system development, and increasing technical and business support directly to early￾stage businesses. Implementing partner Insure and Match Capital (IMC) is a special-purpose for-profit subsidiary of Berytech Foundation, a Lebanese nongovernmental organization (NGO) Berytech is the Lebanese grantee for MENA II. MENA II has three components: investment matching, equity guarantees, and technical assistance (TA) to support the startup ecosystem in Lebanon: • MENA II provides IMC with capital to make matching investments in qualified early-stage investors (QESIs, i.e. venture-capital funds, angel-investor funds, and accelerators) and businesses (QESBs). USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | VII • MENA II also provides IMC with capital to back investment insurance in the form of partial equity guarantees to mitigate the risk of investing in qualified early-stage businesses and develop the Lebanese market for early-stage investment. • Finally, MENA II provides IMC with funds for technical assistance to complement investment matching and equity guarantees by guiding startups in their efforts to grow and attract additional capital. EVALUATION DESIGN AND METHODS, The evaluation employed a qualitative approach, conducting 23 key informant interviews (KIIs) with USAID, MENA II staff, portfolio companies, implementing partners, and BDL, and 3 focus group discussions (FGDs) with the Seeders 2016 pool of angel investors, the LWAF, and IMC mentors. The evaluation also used a desk review of relevant internal and external documents, including monitoring data collected under MENA II. The evaluation approach is gender-sensitive, having ensured that all women angels are interviewed and that males and females are equally targeted in KIIs and FGDs of startups and investors. In the data analysis, findings and recommendations highlight gender gaps. KEY FINDINGS AND CONCLUSIONS The following are the key findings and conclusions for each evaluation question. QUESTION 1: TO WHAT EXTENT IS THE MENA II APPROACH AND IMPLEMENTATION EFFECTIVE? Qualitative findings, along with a review of results just past the program’s midpoint, suggest that MENA II has been effective in meeting its goal of advancing the development of Lebanon’s private sector by increasing the competitiveness and revenue growth of early-stage businesses. MENA II investment matching and equity guarantee activity plays a valuable role in the context of a BDL program using loss sharing to encourage bank investment in Lebanese knowledge-based startups. It shows the feasibility of closing deals beyond the sectors supported by the BDL program and of building an early-stage investment portfolio without the benefit of the central bank’s deep guarantees. The success that IMC has had in sectors not supported by the BDL’s 331 program is especially encouraging because IMC offers its matching investments and equity guarantees at market or near￾market terms. IMC investments thus provide a better indication of the sustainability of early-stage investments in Lebanon than 331 could provide on its own. Investment matching and equity guarantees encourage early-stage investment without simply subsidizing them. By letting them invest in more deals than they could adequately support on their own, investment matching helps VCs diversify their portfolios. Equity investment guarantees appear to have a somewhat different effect. Currently, IMC provides equity guarantees to angel investors—those providing a startup with its very first funding from sources other than friends and family—often in amounts under $150K. Equity guarantees affect not so much the diversification of pools of angel investments as the size of those investments, raising them to a level that will allow some startups to grow. By increasing deal flow, in turn, the angel network that MENA II has launched improves the quality of investment opportunities. VIII | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV The incentives that MENA II provides to IMC also appear to be important in achieving the program’s development goals. Flexibility in the use of the investment-matching and equity-guarantee tools provided by MENA II lets IMC experiment, learn from results, and reprogram as it seeks ways to take advantage of its emerging position in the market for early-stage funding. The main factors behind the effectiveness of MENA II investment matching appear to be the role of implementing partner IMC in the early-stage investment ecosystem and its robust portfolio of ecosystem-support initiatives. Investment matching has mobilized significant investments by other VCs. The requirement to match every investment may, however, have impeded IMC in helping fill the widely perceived gap in earliest-stage funding mentioned above by making small-ticket ($50–150K) investments on its own. Such investments are too small to attract VCs with limited resources for due diligence. MENA II equity guarantees have had a development impact out of proportion to the relatively small level of investment that they have insured. They have enabled IMC to build a functioning earliest-stage investment sector with three pools of angel investors, helping fill the earliest-stage funding gap mentioned above and generating much-needed deal flow for the emerging VC market. Support for these angels and for the launch of an accelerator have also been effective, although there is early evidence that entrepreneur mentoring needs improvement because 50% of mentees drop out. Women’s participation throughout the ecosystem is evident but not yet prevalent. While women own 42% of IMC portfolio startups and comprise all the Lebanese Women Angel Fund (LWAF) pool of angel investors, women’s participation in other aspects of Lebanon’s startup ecosystem hovers around 20%. MENA II has achieved roughly three quarters of its life-of-program development goals, including 63% of its new private investment goal, 76% of its job creation goal, and 89% of its new sales goal. These results are ahead of the 50–60% that one might expect at this stage of the program. QUESTION 2: TO WHAT EXTENT IS THE CURRENT MEL PLAN EFFICIENT IN INFORMING PROJECT DESIGN? The current MEL plan is efficient in informing project design and implementation. It has also been effectively used as a management tool to inform IMC MENA II program and business decisions. It is the clarity of development objectives and simplicity of program outputs that appear to have led IMC to build a strong portfolio of initiatives. For example, the combination of targets for job creation as a development objective, for sales as an intermediate result, and for investment matching as an output gives IMC an incentive to invest in portfolio companies that are trying to grow their sales, which will require increased hiring. At one level, there is no conflict between the commercial goals of other VCs and the development goals set for IMC because IMC does not operate under the commercial constraints of other VCs to achieve high levels of return on investment in their portfolio companies. At a broader level, it is the MENA II focus on early-stage investment that accounts for the low level of conflict between the profit-maximizing goals of private-sector investors in general and the development goal of job creation. Quarterly testing of assumptions about the impact on private-sector investment of the number of active VCs in the Lebanese market, the number of startups in Lebanese angel portfolios, and the number of USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | IX startups in Lebanese accelerators could, over time, improve the accuracy of estimates of the sensitivity of private-sector investment to these factors. And more accurate estimates of the sensitivity of private￾sector investment to these factors would help future programs trying to enhance private-sector investment better allocate resources across support for VCs, angel investors, and accelerators. While it is possible to assess the health of deal flow and the quality of startups within three to five years, it will likely take seven years to assess if exits provide liquidity to initial investors, which will drive the future supply of funds. QUESTION 3: WHAT COMBINATIONS OF ACTIVITIES OR INTERVENTIONS HAVE MORE FACTORS IN PLACE FOR SUSTAINABILITY? Factors enhancing the sustainability of investment matching outweigh threats. IMC has several ways to mitigate the challenge posed by the small size of the domestic market for startups to achieve economies of scale and the challenge of a nascent capital market for exits or sales of initial investor stakes. For example, IMC can continue and even increase its efforts to work with regional partners. And it can strengthen the incentives and support that it provides portfolio companies to work toward an exit that provides liquidity to initial investors and can raise new funds for growth. Equity guarantees are clearly sustainable. IMC could continue to attract angel investment using equity guarantees funded by the fees it charges – as long as losses do not greatly exceed the historical losses in the loan-guarantee portfolio on which IMC based its fees. Continued IMC support for the network in the form of continued angel-investor training and the launch of new pools, moreover, adds to what the early-stage investment market could currently provide without IMC. Equity guarantees supporting angel investors therefore appear to offer both sustainability and additionality. The near-term sustainability of the Speed accelerator is a concern given the important role that accelerators play in generating deal flow for VCs. Speed is a business accelerator partially financed by IMC (through an investment of $875K) to help finance concept-phase start-ups. While the cost of accelerator support may decline, some form of support seems to be essential to the ecosystem. Both angel-investor and startup-ecosystem support appear necessary to strengthen deal flow (or demand for early-stage capital). Without the possibility of angel investments, some startups will not go to the trouble of enrolling in an accelerator; but without the support of accelerators, angel investors would suffer an erosion in the quality of investment opportunities. Increased participation in the early-stage investment market by experienced VCs (the supply side of the market) requires examples of exits, or timely sales of initial investor stakes by responsive startups. IMC could focus portfolio companies on such exits by requiring their business plans to incorporate them and structuring investments to reward progress toward follow-on rounds of funding. The use of equity guarantees to support an angel-investor network and the market for earliest-stage investments appears to have applicability to other markets, and especially to similar projects implemented in other MENA countries. While the application to other countries of the investment matching tool appears equally promising, there is some evidence that the guarantee program under X | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV Circular 331 of the BDL helped ensure there would be enough matching partners in Lebanon to support a robust portfolio of matching investments. Economic growth programs in other countries may want to tailor their interventions to take advantage of changes set in motion by salient local-government initiatives. QUESTION 4: GIVEN THE ABOVE, HOW CONFIDENT ARE WE THAT MENA II WILL AUGUMENT USAID’S PROGRAM OBJECTIVES IF FUNDING IS INCREASED OR EXTENDED? The evidence suggests that an extension or expansion of investment matching will advance USAID MENA II development goals. There is strong agreement on extending and expanding the guarantee and support program for angel investors to the extent markets permit because it has succeeded beyond expectations despite modest outlays. More systematic recruiting of entrepreneurs and startups to apply for angel investments could enhance deal flow and improve the quality of investment opportunities for angel investors to consider. Seen as a portfolio of ongoing experiments, IMC technical assistance has been successful in supporting the startup ecosystem by training investors, launching an accelerator, and providing startups with access to coaching and mentoring. Given the importance of generating deal flow in the ecosystem, several respondents recommended improving the effectiveness of university accelerators and innovation competitions by increasing their business acumen. Family businesses also face a funding gap in Lebanon but are often reluctant to give up equity stakes. Royalty-based capital instruments—which pay investors a share of revenue rather than give them equity stakes—might help fill this gap. KEY RECOMMENDATIONS USAID should extend and expand MENA II within budget limits and continue to provide IMC with clear goals. USAID should continue giving IMC flexibility in the use of MENA II tools to find what approach has the greatest impact. USAID should relax the matching requirement for earliest-stage investments and investigate the ticket size (for example, $100K) below which the development impact of filling the startup funding gap that respondents have identified outweighs the impact of encouraging other investors. IMC should build on progress in integrating women into the startup ecosystem and identify more specific gender gaps within the ecosystem that outreach efforts could be especially effective in filling. USAID programs extending MENA II in Lebanon or elsewhere in the region should include intermediate results for the demand and supply sides of the early-stage investment market, such as the number of startups in accelerators, the number of startups in angel (or earliest-stage) investment portfolios, and the number of independent VC firms investing in them. IMC should encourage sustainable VC participation in Lebanon’s market for early-stage investments by guiding startups to successful exits and continuing to explore regional partnerships. IMC should also USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | XI explore ways to improve the quality of startups for angel investors and VCs to consider (i.e., sustainable deal flow). IMC should investigate ways to help family and other medium-sized businesses exploit growth opportunities, including instruments like royalty-based capital that let investors participate in success by sharing sales revenue rather than equity. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 1 INTRODUCTION PROJECT DESCRIPTION Lebanon’s 15-year civil war ended in 1990; rebuilding occupied the 1990s. State-supported Kafalat, which started in 1999, was the earliest program designed to encourage Lebanese banks to liberalize their lending practices by providing guarantees on bank loans to Lebanese start-ups. Even with Kafalat guarantees, however, bank lenders continued to require excessive collateral. One startup said that a bank required $1 million in real estate as collateral to receive a guaranteed loan by Kafalat of a few hundred thousand dollars. Startups traditionally had great difficulty proving revenue streams to cover loan repayments. Consequently, “young entrepreneurs without personal capital” found it “difficult to access commercial financial services.”1 The global financial crisis of 2008 created a financial shock that resounded across the Middle East and North Africa. 2 According to the ILO, many enterprises in the region stopped hiring and laid off workers “in considerable numbers”.3 The Organization for Economic Cooperation and Development (OECD) similarly reported that the financial crisis “turned into a jobs crisis.”4 Economic growth in the MENA region was slow to rebound, and the large numbers of job seekers entering inefficient labor markets impeded recovery. Uncertainty and lack of stability also affected financial resource flows into the region and constrained private investment. Much of the available capital in the MENA region was directed to low-risk options or to larger companies with direct ties to finance or an ability to draw on their own capital. Smaller companies and new business startups had virtually no access to finance. In response to the global economic slowdown, the Lebanese government looked into measures to support domestic activity by focusing on accelerating public investment projects and providing incentives to the private sector for supporting investments and production. The BDL supported fiscal stimulus by providing subsidized loans and reducing reserve requirements on deposits in Lebanese Pounds.5 From 2010, regional uprisings associated with the Arab Spring exposed Lebanese economic vulnerabilities in liquidity and sovereign creditworthiness. 6 Lebanon failed to attract the capital that flowed out of many of the Arab countries, particularly from Egypt and Syria. From 2011, the Syrian crises further depressed Lebanese exports, tourism, labor markets, and investment. Lebanon’s traditionally strong real estate and tourism sectors subsequently faltered.7 1 International Financial Corporation. Lebanon: A Diagnostic Study on the Demand for Financial Services by Micro and Small Enterprises. 2008. 2 Guita G. Hourani, The Global Financial Crises: Impact on Lebanese expatriates in the Gulf, December 2009. 3 Global Employment Trend: January 2009, International Labor Office, Geneva: ILO, 2009, p. 7 4 OECD, International Migration: Charting a Course through the Crisis, Policy Brief, June 2009, p. 1 5 Horizons Development, Quarterly Publication, The World Bank Middle East Department - Lebanon First/second quarter 2009. 6 Entering a Grey Area: Lebanon’s Economic Challenges in the Arab Spring, The Lebanese Center for Policy Studies, Roundtable reports series, August 2011, p.5 7 Nadim Choukair and Thomas Flynn, Circular 331: 550+ million to create Lebanon knowledge based economy, Fletcher School, 2017 2 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV By 2013, as a result, equity investment in smaller Lebanese companies was scarce, and in Lebanese startups was almost non-existent. BDL approved Circular 331 in August 2013, allowing banks to invest up to 3 percent (later increased to 4 percent) of their equity capital in Lebanese startup companies contributing to the so-called “knowledge economy” and in venture capital funds focusing on these types of companies. BDL guarantees 75 percent of losses on any of these high-risk investments. There are 3 eligibility requirements for bank investment guarantees under Circular 331: the startups must operate almost entirely in Lebanon, they must be innovative, and they must use new technology. BDL estimates that if every bank in the country participated to the maximum limit, the Circular would pump around $650 million into the local entrepreneurship ecosystem. The BDL rationale behind this initiative is to create jobs and build up the market for early-stage technology investment. Since 2014, the support infrastructure for entrepreneurs has expanded with several acceleration programs such as the UK Lebanon Tech Hub, AltCity Bootcamp, Speed at BDD, Smart ESA, and others in the pipeline. Beirut Digital District (BDD) has become a hub for the ecosystem, and 331 has led to the emergence of multiple VCs and matching funds for Lebanese technology entrepreneurs.8 Data on startup proliferation confirms that Lebanon’s tech ecosystem is indeed advancing. According to the World Bank, startups have grown at a 24% compounded rate since 2009. The context for MENA II, launched by USAID in 2014, nevertheless remains that of a nascent and fragile market for early-stage investment. While World Bank figures for Lebanese domestic credit to the private sector were comparable to the average OECD country by 2014, for example, a report produced by Nadim Choukair and Thomas Flynn on Circular 331 indicated that Lebanese banks have continued to be risk averse. The Kafalat respondent says they continue to operate under the “civil war mindset” that prefers short-term to long-term lending and favors financing trade and commerce activities because of their short-term cash flow. Indeed, the World Bank continues to describe the Lebanese startup sector as fragile – a low-density ecosystem with a limited number of clusters. And while 331 has driven the deployment of $200M of investment in the technology sector, startup funding outside this sector remains extremely rare. A final impediment to startup funding in Lebanon is the lack of a robust market for equity in larger companies. Family firms that retain ownership over generations dominate the Lebanese economy. The comparative lack of traded equity investments in the larger Lebanese financial market is important for 3 reasons. First, equity lets investors in portfolios of companies – who are in a position to diversify the risks posed by individual firms – share risk with entrepreneurs who cannot diversify their risks. Second, of course, it provides financing for long-term growth – something banks are reluctant to do even for larger firms because growth plans are more speculative than the kind of inventory and equipment financing needs that bank loans are tailored to meet. And third, equity investors have an incentive to provide expertise, contacts, and customer leads to the startups in which they invest. The limited scope for equity to migrate from the overall financial market in Lebanon to the market for early-stage investment and the limited appetite of domestic investors for higher-risk investments thus heighten the challenges Lebanon faces in meeting its development goals and reducing unemployment.9 8 World Bank Report: Tech Start-Up Ecosystem in Beirut, November 4th 2017 9 This section was extracted largely from the MENA II project document, with some elaboration by the evaluation team. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 3 With this context in mind, USAID launched MENA II as a 5-year investment initiative with a budget of $10-15M (M = million) to improve investor and business access to equity capital, develop the early-stage investment ecosystem, advance financial system development, and encourage increased equity investment in early-stage businesses. It mobilizes early-stage investment through accelerators, which help startups advance from concept to product; so-called angel investors, who provide the earliest-stage funding from sources other than friends and family to advance startups from product to revenue; and VCs, who provide early-stage capital under formal contracts to help startups grow revenue. Implementing partner Insure and Match Capital (IMC) is a special-purpose for-profit subsidiary of Berytech Foundation, the first Lebanese nongovernmental organization (NGO) to help entrepreneurs build startups and small and medium-sized enterprises (SMEs) through incubation, business support, networking, mentoring, funding, and company hosting in a dynamic and vibrant environment. A national standard-setter for supporting entrepreneurs, Berytech is the Lebanese grantee for MENA II. MENA II Lebanon was designed to support the achievement of USAID’s Development Objective 2 (DO2) for Lebanon (Inclusive Economic Growth Enhanced) and Intermediate Result (IR) 2.1 (Increased Private-Sector Competitiveness) and IR 2.2 (Increased Access to Finance), especially for new business startups and women. MENA II Lebanon’s Goal is to “Advance the development of the private sector by increasing competitiveness and revenue growth of Qualified Early Stage Businesses.” To achieve its goals, MENA II provides IMC with 3 tools to support the startup ecosystem: MATCHING INVESTMENT CAPITAL MENA II provides IMC with capital to make matching investments in qualified early-stage investors (i.e. venture-capital funds, angel-investor funds, and accelerators) and businesses. The MENA II investment￾matching tool finances IMC investments in startups matched by other VCs. IMC has deployed nearly $4M in 16 matching investments, according to IMC staff. Investment matching eases fundraising for qualified investors, increases deployment of investment capital, and diversifies investors’ portfolio risks. The resulting reduction of private-sector risk and increased capital available serve to fill a widely recognized funding gap for early-stage businesses. Qualified partners are able to scale up successful investment strategies and increase investment in early-stage businesses. Matching investment capital is provided by IMC under commercial investment terms with combinations of risk and expected return similar or equal to those of its private-sector partners. INVESTMENT INSURANCE MENA II also provides IMC with capital to back partial investment insurance in the form of equity guarantees to mitigate the risk of investing in qualified early-stage businesses and develop the Lebanese market for early-stage investment. IMC has used the MENA II equity-guarantee tool to create a class of angel investors new to Lebanon to fill a gap in startup funding between the levels typically provided by friends and family and those provided by VCs—roughly $50–250K. IMC was able to attract and insure $390K in investment commitments from its Seeders 2016 pool of angel investors without charging a significant premium for its investment guarantee. IMC attracted $690K in investment commitments from its Seeders 2017 angels and $420K in investment commitments from the LWAF pool of angels at roughly market rates for its guarantee and services (where IMC estimated approximate market rates for such insurance from the loss history of a long-standing Lebanese loan-guarantee program). By mitigating 4 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV downside risk, equity guarantees encourage more private-sector investors to provide funds to early￾stage businesses, enhancing the supply of capital to the market for early-stage investment. TECHNICAL ASSISTANCE USAID experience with initiatives like Partnering to Accelerate Entrepreneurship (PACE) has shown that technical assistance and mentoring play an important role in the success of programs that target the early-stage investment market. Many young companies have limited exposure to standard business and governance practices and may lack the requisite skills for attracting investors. The role of technical assistance is to complement matching investments and equity guarantees and help guide companies as they grow and try to attract additional capital. MENA II provides IMC with funds for technical assistance to complement investment matching and equity guarantees by guiding startups in their efforts to grow and attract additional capital. Ecosystem support has trained and supported a new class of angel investors, helped launch an accelerator10, and given startups access to mentoring and coaching. DEVELOPMENT HYPOTHESIS By mobilizing new early-stage equity capital, MENA II Lebanon supports the growth of new businesses and fosters competitiveness in order to create more jobs for Lebanon’s new labor entrants, estimated at 23,000 per year. On the supply side, MENA II Lebanon will prioritize work with private-sector development stakeholders such as accelerators, angel investors, and VCs by: 1) matching new private￾sector investment with early-stage investors or in early-stage businesses to reduce private-sector exposure to investment risk and close the funding gap for early-stage businesses; 2) providing early-stage investors with a partial guarantee on investments in early-stage businesses, thus mitigating some of the risk associated with investing in early-stage businesses and encouraging increased capitalization; and 3) providing a range of assistance for investors, including training related to successful investment models, entrepreneurship and business concepts, angel coordination, and assistance. On the demand side, MENA II strengthens the quality of startups through mentoring and coaching programs, support for ecosystem partners like accelerators, and various competitions and recruiting events to encourage entrepreneurs to apply to accelerators and angel investment pools. CRITICAL ASSUMPTIONS The key underlying assumptions for the successful implementation of MENA II are as follows: VCs will invest more in local startups if matching funds are available that let them diversify their investments by spreading their resources across more portfolio companies; investors will join angel investment pools given high-quality investor training and invest more in the pools if equity-guarantee support is available to limit their capital at risk; ecosystem support for mentors, coaches, accelerators, and potentially other partners is necessary to sustain the effects of investment matching and equity guarantees; and IMC needs to be sustainable beyond 2019 through fees, investment re-flows, and possibly outside investment. 10 Business accelerators provide advice, guidance and various forms of support for businesses in the startup phase. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 5 MODEL FOR MENA II Matching Capital USAID’s capital, provided through MENA II, allowed a private VC, namely Berytech, to create a wholly owned subsidiary that would use this grant money to exploit unproven investment opportunities. The “matching” requirement placed on IMC by USAID performs several functions. The first is to reduce risks to USAID. USAID’s ultimate interest is not the profitability of IMC, but rather the development of the investment/venture capital sector, with the former being a means to the latter. Matching requires the commitment of an additional entity interested in profitability to the deals, providing USAID with an additional degree of assurance that the investment is a sound one. Theoretically, without the matching requirement, there wouldn’t be anything preventing IMC from making large, high-risk bets with USAID money. Between two projects with the same expected value, USAID would generally prefer smaller investments be made in more companies with a lower risk profile. The matching requirement helps USAID achieve this because an additional partner with outside money must approve the deal, thus providing a second layer of opinion, analysis, and approval. Second, the matching requirement reduces administrative costs for IMC, essentially outsourcing due diligence to other entities with their own established hurdle rates. USAID would prefer to spend a higher share of its resources on direct investments, guarantees, and training than on the IMC’s administrative costs. Third, the requirement for a partnership helps build the venture capital ecosystem through increased deal flows for existing VCs. The objective of USAID is not to compete with private-sector VCs for deals, but rather to help build their capacities as well. Risk Guarantees IMC’s offer of a 50% risk guarantee on an investment portfolio (typically consisting of five to seven firms per year) in exchange for 10% equity in that portfolio, while also allowing them to diversify their risk, seems like a great deal for the angel investors. The devil is in the detail of the distribution of expected returns, however. It may help to imagine the example of a portfolio with an expected return of 20% where the expected return on 10% of the portfolio (IMC’s “fee”) just balances the expected payout on 50% loss insurance. This would be the case for a portfolio with a 60% chance of a 40% return, 20% chance of a 0% return, and 20% chance of a -20% return. The expected value of 10% of the portfolio is 2%, while the expected loss payout is -2%. Given the high interest rates that prevail in Lebanon and the associated large cost of capital, at any rate, such a guarantee was needed to attract new investors into the program. While the guarantee serves as a significant incentive to the angel investors, the scheme also offers great benefits to IMC. First, the guarantee attracts diverse sources of new capital into the fund that did not exist. The investors are high–net worth individuals who would like to invest in Lebanese startups but are understandably unwilling to put up large sums of their own money into single investments. The Angel investor network allows these investors to purchase smaller tickets ($17,500) in a larger portfolio of local companies, which is very attractive. Second, the guarantee attracts talented investors into the fund that IMC can utilize to identify and evaluate new investment opportunities. But angel investment requires much more than just money. The angels, despite being successful businessmen and women, do not enter the program with essential 6 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV startup investment knowledge, such as how to identify and measure risks, how to evaluate business plans, how to estimate growth and conservative estimates of return on capital, etc. This is why the guarantee requires the angels to graduate from IMC’s “Seeder” program, which trains the investors on basic standards of investing as well as offering insights into Berytech’s business model. The Seeders course can be thought of as a form of “sweat equity” for the angels and in a sense transforms these high net worth individuals into de facto part time employees of IMC! Third, the angel networks attract attention and support to the program and IMC itself through its outreach activities. The networks are composed of influential people who can open additional doors for the company and appear to be a sustainable development outcome that USAID appreciates (especially the women investors). These factors increase the chances that IMC will receive additional grant funding from USAID and perhaps other donors as well as investments and purchases from other high–net worth individuals. Technical Assistance IMC’s purchase of 33% of Speed as well as its mentorship program and other training activities allows it to establish the missing complementary components needed for the success of its investments. @Speed, a major new business incubator, helps IMC capture another segment of the market, namely concept phase startups. While the matching capital component allowed it to capture higher-risk opportunities that it knew existed at the early-stage level, USAID’s requirement of a technical assistance component reduced Berytech’s risks for investing in such an incubator/accelerator. Berytech now has a closer eye on this potentially highly profitable market segment, giving it a competitive advantage in identifying profitable Series-A investments11. While reviews on the training and mentorship program were mixed among entrepreneurs, there was a clear consensus among the investors on the importance of technical oversight. One major issue is the mismatch of expectations between the investor and entrepreneur that mentorship can help overcome, while training can provide missing technical skills that the entrepreneur might not know that he/she is missing. All three of these components have been successful in allowing Berytech, through IMC, to fill identified gaps in the VC marketplace. These gaps existed in the concept phase and early-stage investment ecosystem that MENA II allowed it to exploit by lowering risk through the grant. The grant capital essentially allowed a private-sector entity to use someone else’s money to try new and risky but potentially profitable investments. Even though IMC uses USAID’s money, it is well incentivized to make market-oriented investments since it retains 50% of the profits. IMC is also incentivized to attract outside capital through the matching requirement and also through the establishment of the angel investor networks. The other major gap that MENA II allowed IMC to exploit was in sectors and 11 A series A round refers to a company's first significant round of venture capital finance. It refers to class of preferred stock sold to investors in exchange for their investment after common stock options are issued to company founders, employees, friends and family and angel investors. (Wikipedia). The main differences between rounds are the maturity levels of the business, the type of investors involved, the purpose of raising capital and how it is ultimately allocated. The funding rounds begging with “seed capital” phase and follow with A, B and then C funding. (https://www.investopedia.com) USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 7 projects not covered by Circular 331,12 namely those outside of the direct “knowledge sector” and those that involve a significant foreign component. MEL Plan Employment generated and investments leveraged are USAID’s top line indicators for MENA II, which are in line with USAID’s stated development objectives in their CDCS. Yet these indicators are directly related to the activities of any VC and are not likely to cause significant distortions to how or where an independent VC would choose to make investments. USAID essentially offered its grantee significant freedom to operate on a commercial basis, without many requirements to meet specific, more development-oriented objectives. For example, USAID could have required IMC to invest in companies from specific regions within Lebanon or in companies that had environmental or social objectives or that were owned by women or hired Syrian refugees. The fact that USAID did not insist on a “double bottom line” could have resulted in outcomes that did not advance development objectives. IMC could have simply become a “fund of funds,” investing USAID money into other VCs. While this might have been a lower transaction cost alternative for IMC that would still make money, a fund of funds might have simply have provided marginal capital to the ecosystem with little value added. The active investor approach of IMC was a preferable approach with greater potential for development impact. USAID’s decision to allow the grantee to operate on a commercial basis with a focus on the profitability of the fund is the key factor in promoting sustainability. A “double bottom line” and increased oversight and direction from USAID over the fund would have threatened the ability of the fund to act almost completely on a commercial basis. As it turns out, IMC’s activities are likely to lead to significant development outcomes, including a more developed venture capital ecosystem with supporting institutions, more diversified investors and sources of capital, more diversified investees, and a greater pipeline of investible startup enterprises, which were the outcomes that USAID sought. One of the main drivers of discipline for IMC to focus not only on profitability but also other aspects of development is the prospect of additional grant funds. This incentive continues to encourage IMC to be creative in its approach and reach out to new partners with the ability to achieve development impact that IMC can point to while still using the resources for profitable investments. USAID uses, and should continue to use, this leverage to encourage if not insist that IMC have a “soft” second bottom line with clear development objectives while maintaining a high degree of commercial orientation for the fund. MENA II impacts should eventually spill over to the poorest segments of the population, which USAID has a mandate to prioritize. Even so, MENA II is not a project that directly works with or directly benefits the poorest segments of the population, and USAID will need to continue to justify this by demonstrating that this program as well as others is achieving, however indirectly, poverty reduction. 12 The BDL issued Circular 331 in 2014, under which it guarantees within certain limits 75% of losses on bank equity investments in startups that meet three conditions: 1) they must be Lebanese, with few activities outside the country; 2) they must be innovative; and 3) they must use new technology. 8 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV EVALUATION PURPOSE AND QUESTIONS The purpose of the MENA II midterm evaluation is to assess the performance of Berytech under MENA II through its subsidiary IMC and to examine and critique the project’s strategic approach. The evaluation intends to determine whether MENA II is likely to augment its program objectives if USAID extends or increases the current level of funding to the MENA II project and recommend whether USAID should further pursue its activities in this field. The evaluation findings will help USAID and project implementing partners (a) understand the immediate results of the project, (b) enhance MENA II interventions in the remaining life of the project, and (c) inform future USAID economic growth programs. The evaluation covers the period from September 2014 to September 2017. The primary audience for the evaluation includes (i) USAID/Lebanon, particularly the Economic Growth Office, and (ii) MENA II implementing partner IMC. Secondary users might include (i) the Middle East North Africa Investment Initiative bureau; (ii) ecosystems partners, including incubators, business accelerators, and others; (iii) other investors, donors, and organizations working on early-stage investment initiatives; and (iv) other USAID missions, regional Bureaus, and access-to-finance programs of the Bureau for Economic Growth, Education, and Environment, as well as the International Finance Corporation (IFC), and the Bureau of Economic and Business Affairs of the U.S. Department of State. In addition to distributing the midterm evaluation to program and technical offices involved with private￾sector development in the Lebanon Mission and Embassy for the purpose of improving the design of related future programs and sharpening private-sector development project monitoring and evaluation, USAID intends to disseminate it to other Missions, regional Bureaus, and Offices of the Bureau for Economic Growth, Education, and Environment interested in the potential for risk-capital donors to strengthen growth of early-stage businesses and markets for early-stage investment. USAID also intends to distribute the mid-term evaluation to interested organizations throughout the donor community. The evaluation addresses the following four evaluation questions: EVALUATION QUESTION 1: EFFECTIVENESS To what extent is [the] MENA II approach and implementation effective in terms of: a. How does the strategic approach shape the incentives of the respective stakeholders? Are the incentives generated likely to lead to the outcomes sought? b. What do we know about how this approach is working to date through MENA II Lebanon implementation? c. Which activities/combinations of activities have been more successful in achieving their objectives and what were the primary factors responsible for their relative success? EVALUATION QUESTION 2: MONITORING AND EVALUATION To what extent is the current monitoring and evaluation (MEL) plan efficient in informing project design and implementation? What factors influenced the ability of the MEL plan to be effective? Is the timeframe of the project sufficient to conduct proper monitoring and evaluation for the project? What are the MEL recommendations for future projects? USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 9 EVALUATION QUESTION 3: SUSTAINABILITY What combinations of activities or interventions have more factors in place to create sustainability and are more likely to create impact over the long run and what were the primary synergies that contributed to that success? EVALUATION QUESTION 4: FUTURE RECOMMENDATIONS Given the above, how confident are we that MENA II will augment USAID’s program objectives if USAID extends/increases the level of funding to the MENA II project? What successes and challenges should be taken into consideration if this project scales up? If this project scales up, should it focus more on one or more components of the project than others? METHODS AND LIMITATIONS The evaluation design consisted of a primarily qualitative approach using KIIs and FGDs. The KII respondents included MENA II project staff, USAID/Lebanon’s Economic Growth Office staff, the office managing Circular 331 at the BDL, matching and angel investors, service providers, and portfolio companies. FGD participants included startup mentors and pools of angel investors, with a separate FGD for women members of the LWAF. No primary quantitative data were collected, though the evaluation used project-monitoring data to triangulate findings where relevant. All people-related data collected was disaggregated by gender to identify any existing gender gaps in the area assessed. So was feedback compiled from women interviewees in the KIIs and the FGDs, used to collect data relevant to the GNDR-2 indicators, which measure gender gaps/achievements in this project. Data collection occurred from November 6 to November 17, 2017. A comprehensive evaluation design matrix with the evaluation questions, data sources, data collection methods, and analysis methods can be found in Annex 1. DESK REVIEW The evaluation team conducted a systematic review of relevant internal and external documents, including the original contract agreement, contract modifications, reports related to BDL Circular 331, MENA II work plans, annual and quarterly reports, USAID’s Country Development Cooperative Strategy (CDCS) for Lebanon, the Mission’s Performance Management Plan (PMP), and all other project documents relevant to the evaluation questions. In addition, the evaluation team reviewed relevant publicly available documents on Circular 331 and Berytech Funds 1 and 2, articles that provided insight into venture capital practices in Lebanon and related World Bank programs, and other reports providing context for assessing the project’s effectiveness and sustainability. Annex 2 contains the list of all documents reviewed. QUALITATIVE DATA COLLECTION KIIs 10 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV The Social Impact evaluation team13 conducted 23 KIIs (43% of which were with women) with USAID key staff members, IMC staff and board directors, a startup coach, portfolio companies, bankers, accelerators, venture-capital matching partners, hosting partners, the BDL, and state-sponsored lender Kafalat. The KIIs focused on project activities and their relevance to the MENA II project’s long-term and short-term needs, factors that contributed toward achievement of project results to date, the sustainability of those results, and implementation challenges. Key informants were purposively selected because of their intimate knowledge of and perspective on different project components. Key informants consisted of a diverse group of stakeholders, including portfolio companies that have been with the project for more than 12 months, venture capital partners who have matching investments in one or more portfolio companies, hosting partners affiliated with the project, angel investors who participated in the three angel pools, the BDL officer responsible for Circular 331, and others within the sector. The interview guides and the list of interviewees can be found in Annexes 3 and 4, respectively. FGDs The evaluation team facilitated three FGDs with angel investors in the Seeders 2016 pool and the LWAF as well as with startup mentors. FGD questions focused on the relevance and effectiveness of both startup and investor training, as well as the effectiveness and sustainability of investment matching, equity guarantees, and ecosystem support. Angel investor interviewees were chosen from two of the three pools as a representative sample with recent involvement in the project. A total of 13 respondents participated in FGDs (43% were women). A two-member team of enumerators that consisted of a moderator and a note-taker conducted each KII and FGD. While the moderator’s job was to make sure all questions included in the data instruments were adequately addressed, the note-takers audiotaped the discussion and prepared transcripts. DATA ANALYSIS The evaluation team began the study with a systematic review of all available program documents to trace the process that MENA II used to implement the project. After the team completed qualitative data collection and received the focus group notes, it reconvened in Beirut to analyze the data and develop findings, conclusions, and recommendations. The team used a structured and systematic approach to analyze the qualitative data and, where possible, triangulated methods and sources to ensure the reliability and validity of results. Key steps included: • Summarizing notes from individual interviews and FGDs • Coding KII and FGD data according to themes relevant to the evaluation questions • Preparing an evaluation data analysis matrix with the themes that emerged in the KIIs and FGDs to identify key study findings as they related to the evaluation questions 13 The evaluation team consisted of six core members—one international consultant who led the team, one SI staff member from headquarters (Washington, D.C.), two local SI staff members, and two local consultants—and an additional SI team member was added as needed. In the interest of time, the team divided into three pairs to conduct the KIIs and note-taking. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 11 • Preparing a detailed outline summarizing key findings based on the data analysis, conclusions for each study question, and overall recommendations Project documents allowed the team to verify claims that emerged from interviews. Quantitative monitoring data offered statistical information (e.g., numbers of beneficiaries and levels of investment) that allowed the evaluation team to determine to what extent the project achieved its targets. LIMITATIONS The team identified five potential limitations associated with its methodology and developed mitigation measures to reduce the likelihood of each occurring (Table 1). Table 1: Limitation TABLE 1 TYPE OF LIMITATION METHOD THAT IT AFFECTED DESCRIPTION MITIGATION MEASURE Selection bias KIIs and FGDs Because qualitative data collection is not intended to produce statistically representative samples, it often relies on purposive sampling, meaning that respondents were selected for their particular roles, responsibilities, and knowledge. As a result, although these respondents typically represent a variety of views on a topic, they may not represent all relevant views, and selection bias can emerge. Evaluation team attempted to reach all or large portions of each kind of stakeholder, e.g., 50% of portfolio companies, 30–40% of VCs, FGDs with two of three extant pools of angel investors. Gender sensitivity and bias KIIs, FGDs, M&E indicators Male participants in decades-old business networks do not always perceive the social obstacles to participation by women. The woman founder of a Beirut incubator, for example, has been the most forceful local advocate for co￾working space as a safe networking opportunity for women. Evaluation team emphasized women in KII, FGD sampling (46% of respondents), arranged women-only FGDs as well as KIIs, analyzed gender disaggregation of indicator results. Time limitations KIIs The evaluation was conducted over a period of four weeks, including just over two weeks of field visits. Schedules and availability inevitably limited the number of KIIs that could be conducted within this period. Evaluation team scheduled FGDs to capture views of more respondents and arranged KIIs to fit busy respondent schedules where possible. Observation bias Site visits Site visits to observe project activities were limited to visiting portfolio companies in their place of work, an accelerator, and an incubator in Beirut. Most Lebanese startup activity currently in Beirut given large population of capital and its accessibility to rest of country. Small population KIIs, FGDs A relatively small number of total participants in Lebanon’s incipient startup ecosystem means early outcomes may not be indicative of later ones as the ecosystem grows. Evaluation team systematically asked respondents to address sustainability and potential evolution of ecosystem. 12 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV FINDINGS QUESTION 1 To what extent is [the] MENA II approach and implementation effective in terms of: a. How does the strategic approach shape the incentives of the respective stakeholders? Are the incentives generated likely to lead to the outcomes sought? Promulgated in 2013, Circular 331 of Lebanon’s central bank (the BDL) sets the context for any assessment of how MENA II, launched a year later, shapes the incentives of participants in the market for funding early-stage businesses. As described in more detail in the Introduction of this report, the BDL guarantees within certain limits 75% of losses on bank equity investments in startups that meet 3 conditions: 1) they must be Lebanese, with few activities outside the country; 2) they must be innovative; and 3) they must use new technology. Participating banks make most of their investments through VCs, or firms making the earliest formal investments in startups for contractually defined stakes and often for seats as board directors, often in amounts from $250K to $2–3M. Circular 331 has had a large impact on early-stage investment in Lebanese knowledge-based startups. Banks have committed $650M since 2014, invested $376M with VCs, and seen $200M of their funds actually deployed, according to a BDL interviewee. The BDL estimates the deployment of funds has provided direct or indirect support for 800 startups and 9,000 jobs. This context makes the incentives provided by MENA II more valuable rather than redundant in achieving development goals. Interview and focus group respondents were nearly unanimous in the view that, while support from 331 is not restricted to technology, its focus on knowledge￾based startups may drain funds from sectors outside information, communications, and technology. The BDL respondent nevertheless sees more disadvantages than advantages to loosening the knowledge-based economy criteria of 331 going forward. Given this context, bank, VC, startup, mentor, and accelerator respondents all viewed the flexibility that MENA II provides IMC to invest in startups in sectors outside those supported by 331 as important. MENA II also provides IMC the ability to invest in startups that, while still based in Lebanon, may have significant operations outside the country. Almost as widespread among interview and focus group respondents was the view that the generosity of support from 331 raises questions about the sustainability of investment it has facilitated beyond the life of the BDL program. In the view of these respondents, the success that IMC has had in sectors not supported by 331 is especially encouraging because IMC offers its matching investments and equity guarantees at market or near-market terms. IMC may benefit from the activity fostered by 331 according to this reasoning, and it provides a better indication of the sustainability of early-stage investments in Lebanon than 331 could provide on its own. Interviews with IMC staff clarified how investment matching and equity guarantees encourage early-stage investment without simply subsidizing them. By letting them invest in more deals than they could “I pay $20K in Lebanon for an engineer who costs $100K in the U.S.” —Entrepreneur USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 13 adequately support on their own, investment matching helps VCs diversify their portfolios. All of our VC interviewees corroborated this finding. As a side-benefit, IMC staff believes, it provides VCs useful experience partnering on deals. Equity investment guarantees appear to have a somewhat different effect. IMC currently provides equity guarantees to angel investors—those providing a startup with its very first funding from sources other than friends and family—often in amounts under $150K. IMC staff believe their guarantees lead angel investors to invest more of their own money, leading to pooled investments by groups of angels large enough to give their portfolio companies a meaningful chance to become viable. Respondents in an angel-investor focus group thought the more important effect may be to induce more investors to join angel investor pools. Either way, respondents believe equity guarantees affect not so much the diversification of pools of angel investments as the size of those investments, raising them to a level that will allow some startups to grow. As for the incentives that MENA II provides to startups, the most important is the simple availability of funds to launch a company, according to half of portfolio-company interviewees (i.e., entrepreneurs who have received funding). By increasing deal flow, as several mentors (or business advisors) and an accelerator respondent pointed out, the angel network that MENA II has launched improves the quality of investment opportunities. The incentives that MENA II provides to IMC also appear to be important in achieving the program’s development goals. As an IMC interviewee indicated, the need for follow-on funds drives it to meet development goals. This is not true of VCs receiving investments from banks supported by 331, as a bank interviewee pointed out, since the BDL program lets VCs charge a 2% management fee for the amount of bank investments in their funds regardless of whether they deploy those funds. Flexibility in the use of the investment-matching and equity-guarantee tools provided by MENA II lets IMC experiment, learn from results, and reprogram as it seeks ways to take advantage of its emerging position in the market for early-stage funding. b. What do we know about how this approach is working to date through MENA II Lebanon implementation? Just past its halfway point, a desk review of Performance Monitoring indicator tables as of September 2017 shows MENA II is on target to achieve most of its life-of-program implementation goals, defined here as targets for outputs and for those intermediate results that affect other intermediate results (Table 2). 14 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV Table 2: targets for outputs and some intermediate results TABLE 2 INDICATOR NAME REFERENCE NUMBER RESULTS LIFE-OF￾PROGRAM TARGET Cash flow from investment matching IM 1.2c $3.1M $6.0M USG funds used for equity guarantees IM 1.2a $0.6M $4.0M USG funds for technical assistance IM 2.1b $0.3M $2.0M Individuals benefiting from training IR 2.1.3a 342 300 MSMEs receiving business development assistance IR 2.1b 56 80 MSMEs benefiting from new linkages IR 2.1.2 8 10 MSMEs funded IR 2.2a 37 90 Figure 1 gives results to date as a percentage of life-of-program targets for outputs and intermediate results that affect other intermediate results. A level over 50-60% for an indicator suggests the program is on track to meet its ultimate target for that indicator. Figure 1: % of Program Implementation Goals Achieved The findings in this subsection put these results in context for each of the three MENA II components of investment matching, equity investment guarantees, and technical assistance (or ecosystem support). The MENA II investment-matching tool finances IMC investments in Lebanese startups matched by other VCs. IMC has deployed nearly $4M in 16 matching investments, according to IMC staff. Appreciation for the simplicity of its investment-matching value proposition and its reliability in 0% 20% 40% 60% 80% 100% 120% MSMEs funded MSME linkages MSME TA Individual TA USG funds for TA USG funds for g'tees Investment matching 41% 80% 70% 114% 15% 15% 52% % of Program Implementation Goals Achieved USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 15 reviewing and executing deals was a recurring theme in interviews with partner VCs. Two of six portfolio-company interviewees expressed some uncertainty about the valuations they received from IMC and its partners. IMC has used the MENA II equity-guarantee tool to create a class of angel investors new to Lebanon that are helping to fill a widely perceived gap in startup funding between the levels typically provided by friends and family and those provided by VCs—roughly $50–250K. IMC recruits these angel investors and groups them into pools. The names of the three pools of angels and angel investments that it has created so far are Seeders 2016, Seeders 2017, and the Lebanese Women Angels Fund (LWAF). As it has deployed the equity-guarantee tool, IMC insures 50% of any losses on pools of angel investments that the angel investors incur within 10 years upon disposition of their investments. Disposition may occur either through the sale of their stakes to the managers of their portfolio companies, to other companies (so-called “trade sales”), to other investors such as VCs, or via a public stock offering. Industry participants refer to these dispositions of investments, or sales of stakes, as exits. A staff interviewee stated IMC was able to attract and insure $390K in investment commitments from its Seeders 2016 pool of angel investors. IMC did not try to charge a premium that could cover the cost of its investment guarantee for this investment pool. Several Seeders 2016 respondents said both the 50% equity guarantee and investor training provided by IMC were necessary for their participation. Surprisingly, given the short time involved, IMC has attracted $690K in investment commitments from its Seeders 2017 angels and $420K in investment commitments from the LWAF pool of angels at what IMC estimates to be market rates for its guarantee and its services, according to IMC staff. (The desk review of monitoring data confirms the total amount raised by angel investors.) By “market” rates, as discussed under Questions 3 below, IMC staff appears to mean rates that can cover the cost of its guarantees, or sustainable rates. In return for its 50% pool guarantee and services on behalf of Seeders 2017 and the LWAF, IMC is taking a 10% equity stake in the startups receiving investments from the insured pools; a onetime setup fee equal to 2.5% of the amounts invested in each pool; and a 5% success fee (defined as 5% of profits on each pool upon exit in excess of a break-even level of profits—or so-called hurdle rate—defined as the level reflecting an 8% per year return on investment). To put this in perspective, a guarantee fee in the form of expected returns on a 10% stake in a pool of angel investments would just cover the expected cost of a 50% loss guarantee on the pool in a variety of scenarios consistent with a moderate expected rate of return on the pool. For example, there are plausible assumptions consistent with a 20% expected return on angel pool investments where a guarantee fee in the form of an ownership stake equal to 10% of pool investments exactly covers expected losses on a 50% guarantee.14 14 Suppose investors have a 60% chance of a 40% return, 20% chance of a 0% return, and 20% chance of a 20% loss. Then the expected return is 20%, and a 10% stake covers 50% of expected losses. “IMC seems to be at the heart of the startup scene.” —Entrepreneur 16 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV The composition and activity of these pools of angel investors belie the small amounts of USG funds set aside to cover the actual guarantees. The typical pool consists of 25 angels each investing $15K. Seeders 2016 made four investments of $80K each, according to an IMC interviewee. Angel investors meet every six weeks to select startups for pitch presentations that they would like to hear about five to six opportunities pre-screened by IMC; to hear three pitches from previously selected startups; and to vote on whether to invest in startups that have presented pitches, according to FGD participants from the Seeders 2016 pool of angel investors. An angel team does due diligence on each startup getting a positive vote; IMC awards certificates to team leaders. Angels conduct a second vote after due diligence to make a final investment decision. Angel pools typically expect to hear pitches from 18 startups, vote to do due diligence on eight of them, and invest in four. Most angel respondents felt the quality of the angel training program was high, although two respondents were unclear how losses were to be shared. Several LWAF respondents were critical of the quality of startups that they had seen in the first two to three months of their pool. Most angel respondents were nevertheless highly enthusiastic about the program and several angel investors from the first pool have joined subsequent pools. Implementation of individual technical assistance activities—those that support the startup ecosystem— was uneven. There is no prescribed way to build a startup ecosystem, however. IMC has therefore had to experiment. Seen as a portfolio of experiments, ecosystem support has accomplished quite a lot and ecosystem participants appear to have learned a great deal from it. • Angel mentoring or training: Seeders 2016 focus group respondents found that the training program IMC set up (called the Master class for Business Angels, or MBA) was effective in building their confidence and qualifying their peers as investors. • Startup mentoring: 18 startups have benefited from an MIT mentoring or advisory program called Confideo, according to mentor focus group respondents. Under this free, MIT-funded program, each business owner sets the agenda for monthly 1.5-hour problem-solving meetings with teams of three trained mentors. There are currently 37 local mentors on the roster who have been through the Confideo training program. However, as a woman portfolio-company founder noted, “[…] all of the mentors are men!” Fifty percent of startups who start the program drop out, according to IMC. Mentors meet every six months to select clients from 10 pitches by startups. Several experienced executives among the mentor respondents expressed frustration that mentees have little knowledge of business basics; educators among the mentor respondents were more comfortable teaching those basics. A recurring theme among mentoring respondents was nevertheless that the selection of startups for mentoring limited the effectiveness of the program. That said, two of six portfolio-company interviewees did not find mentoring useful. • Coaching (not to be confused with mentoring): IMC currently funds and offers free services from an experienced coach for four to five startups in its matching-investment portfolio. Designed by Stanford, this intensive advisory process involves two to three days of support each month solving a problem selected by the coach based on gap analysis and using a transformation plan agreed upon with the coach, according to an IMC interviewee. IMC may charge for this service in the future. While resource-intensive, the process appears to give IMC a leveraged way USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 17 of providing the kind of management support needed for portfolio companies, that otherwise VCs might have to provide through their ambitious, young, and often overstretched principals. • Accelerator: IMC was one of three investors that each put $875K into the Speed accelerator in Beirut, according to interviews with IMC staff and Speed’s founder. Of 100 applications from startups, Speed typically interviews 30 in a first round and 15 of those 30 in a second round before selecting seven for due diligence and offers to participate in one of its three-month programs. These provide a place to work; 15 workshops on product development, marketing, and legal issues; coaching in strategy, technology, and business development; and weekly investor talks and pitch days. They also provide a $30K grant to each participant. Speed has taken up six, seven, five, nine, and seven startups, respectively, in its five semiannual cycles so far. In exchange for the $30K grant and business development services that are worth, it estimates, another $30K, Speed takes a 10% stake in each participating startup. This implies a preliminary $600K valuation of each graduating company—not out of line with practice in more developed markets. The 10% stake is nevertheless controversial in the Lebanese context, according to two portfolio-company interviewees, because the resulting valuation is too expensive for some follow-on investors. While Speed and two VC interviewees disagree on the grounds that startups with lower valuations are unlikely to be worth the time of any VC, Speed agrees that the stake it takes should be lower, perhaps in the range of 5–8%. With more competition and stable funding, Speed may well do so to ensure it ends up with at least 7 startups in every class. Speed also sponsors the two best startups in each class for a two-week Silicon Valley Blackbox immersion program offered with the help of the LebNet North American technology-sector diaspora network. And it sponsors a competition in which five investors select a single startup in each class based on pitches to get $50K for a 5% stake (implying a $1M valuation for the winner). These initiatives set up MENA II to glean a significant amount of knowledge going forward about what aspects of the startup ecosystem are most effective. While the following section on development outcomes will go into more detail on MENA II gender effects and sensitivities, it is worth noting here that 50% of IMC staff are women, including the investment manager, and 25% of IMC’s Advisory Investment Committee are women. Regarding this evaluation, 46% of interview and focus group respondents were women. c. Which activities/combinations of activities have been more successful in achieving their objectives, and what were the primary factors responsible for their relative success? The review of Performance Monitoring indicator tables as of September 2017 shows MENA II has achieved roughly three quarters of most of its life-of-program development goals, defined here as life-of￾program targets for development objectives and for those intermediate results affected by other intermediate results (Table 3).15 15 Please see Conclusions for comments on appropriateness of life-of-program targets. 18 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV Table 3; Program targets for development objectives TABLE 3 INDICATOR REFERENCE NUMBER RESULT LIFE-OF￾PROGRAM TARGET New private-sector investment DO 2a $9.5M $15.0M New jobs DO 2b 151 200 Sales of companies benefiting from program IR 2.1c $2.0M $2.25M MSME financing due to USG assistance IR 2.2.2b $5.9M $10.0M Startups in investment portfolio IM 1.1c 20 25 Startups using new practices due to program IR 2.1a 26 25 Figure 2 gives results to date as a percentage of life-of-program targets for development objectives and for intermediate results affected by other intermediate results. It shows that MENA II results are generally ahead of the 50–60% one might expect at this stage of the program. Figure 2: % of PRogram Developemnt Goals Achieved Coming just past its halfway point, these results indicate MENA II has been effective in meeting its goal of advancing the development of Lebanon’s private sector by increasing the competitiveness and revenue growth of early-stage businesses. The findings in this subsection explore the factors behind the development outcomes of MENA II investment matching and equity investment guarantees, the role of ecosystem support in achieving those outcomes, and the gender effects of the intervention. 0% 20% 40% 60% 80% 100% 120% Startups using new practices Startups in portfolio MSME financing due to MENA II Sales of MSME beneficiaries New jobs New private-sector investment 104% 80% 59% 89% 76% 63% % of Program Development Goals Achieved USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 19 Regarding the investment-matching program, IMC staff estimate there would be at least $13M less total VC investment in the Lebanese market but for IMC’s investment-matching activity ($3.1M so far), and a desk review corroborates this if one includes successful follow-on funding of startups that received their initial funding through the program. The same sources indicate, moreover, that but for MENA II there would be no VC investment outside the technology sector, which comprises roughly half of the IMC portfolio. There are, however, no co￾investors so far who do not enjoy support through the central bank’s 331 program or funding from donors such as the World Bank. IMC’s active portfolio comprises 2 startups with doubtful prospects, 6 for which it is too early to judge, and 2 that already show high potential, in the view of IMC staff. The implied potential 20% success rate would be typical for VCs in more developed markets. Investor and startup interview respondents consistently reported having a very positive experience with IMC. One matching investor who believes MENA II is making a difference in the startup market added, “What USAID is doing is brave.” While most investor respondents believe the investment-matching requirement has encouraged VC investments in startups—especially outside the sectors supported by 331—it appears to have resulted in some missed opportunities in early seed investments, or investments below $100–150K. IMC staff estimate that the inability to find a partner—or the withdrawal of a partner—explains half of the 15 deals worth $4.7M that failed to close even though IMC’s Investment Committee approved them. Many investor and ecosystem-partner interview and focus group respondents argued that IMC is well positioned to help close Lebanon’s gap in early seed funding if it could invest in small deals—those, for example, under $100K—without having to find a partner to match its investment. Regarding the equity-guarantee program, IMC staff and a wide variety of other respondents believe that there would be no angel investment in Lebanon at all but for IMC’s angel-investor guarantees and other support. This would represent $1.6M less than its three pools of angel investors have committed so far. An IMC executive interviewee estimates that the guarantees roughly double the number of participants in its angel pools and notes that an earlier angel initiative lacking such guarantees launched by a local bank was unsuccessful. The combination of guarantees, training, and administrative support strengthens the angel networks that IMC has launched, adds structure to them, and increases their discipline, according to an executive interviewee from state-sponsored small-business lender Kafalat. Interviewees running startups in angel pools also appreciate the broader seed-investment network that IMC is building. “IMC seems to be at the heart of the startup scene,” was a typical comment. Another portfolio-company interviewee noted that Seeders 2016 decisions seemed faster than the six months “But all of the mentors are men!” —Woman entrepreneur IMC’s “organic and sustainable development of the angel sector may be comparable in importance to the much larger -- but possibly unsustainable -- growth of technology-sector VCs under Circular 331.” —State-backed investor 20 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV typically needed by similar groups in France. That said, an ecosystem partner cautioned that some angel investors may need to calibrate their expectations regarding revenue targets, which are ambitious at such an early stage of investment. And several portfolio-company and ecosystem-partner respondents felt that valuations for some startups receiving angel-pool investments seemed harsh. Overall, however, respondents who commented on the subject felt that the effort to establish an angel-investment sector in Lebanon is succeeding, and most suggested or implied that it is succeeding beyond expectations. The factors most often cited behind IMC’s ability to mobilize early-stage investment in Lebanon are its unique investor role and wide-ranging ecosystem support. Investor interviewees and many ecosystem￾partner respondents cited IMC’s role as the only Lebanese VC currently active outside the information, communications, and technology sectors, and one of the few able to invest in Lebanese startups with operations outside the country. Regarding the importance of IMC’s support for the startup ecosystem, these same respondents, joined by several portfolio-company interviewees, cited its role in launching Lebanon’s angel network through equity guarantees and training – and especially the effect it has had in mobilizing concentrated wealth to fill a widely perceived gap in early-stage funding between the levels provided by accelerators ($30–50K) and VCs (over $250K). Most portfolio-company and investor respondents also cited IMC’s flexible investment approval process, startup mentoring, angel-investor education, and contacts provided by its investment committee as playing an important role in startup funding. Many bank, VC, portfolio-company, and of course angel￾investor respondents highlighted the importance of IMC support for women investing in women-run startups. And many investor and portfolio-company respondents cited the contribution of the Speed accelerator to the flow of investable deals, which, according to its founder, has supported 27 startups, 89 jobs, 400 indirect jobs, eight returning diaspora cofounders, two international startups, and seven graduates that have raised $2.25M—all with just three early failures. While women own 40% of IMC portfolio startups and comprise all of the LWAF pool of angel investors, women’s participation in other aspects of Lebanon’s startup ecosystem hovers around 20%. A desk review shows that 6 of 26 Seeders 2016 angel investors, 4 of 23 Seeders 2017 angel investors, and all of the LWAF angel investors have been women. Women run 2 of 11 VCs (both from the same family), comprise 15% of mentors, own 5 of 12 portfolio companies, and run 2 of 7 ecosystem partners (Figure 3). LWAF focus group respondents suggested that enhanced outreach efforts by IMC might make more women aware of startup opportunities throughout the country. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 21 Figure 3: Women, % of participants Even though MENA II is ahead of schedule in achieving its development goals, there is no early evidence of long-term impact. Early funded exits—deals that not only acquired the stakes of early-stage investors in startups but also raised new money for those startups—would provide strong evidence of the sustainability of MENA II outcomes. Specifically, they would indicate that IMC investment mobilization to date reflects more than the effect of the 331 program and pent-up demand given Lebanon’s unique historical context. Of 70 startups funded under IMC’s parent foundation, however, only two have raised significant new money at exit, according to an IMC executive. Of 15 IMC investments, one third have failed; one third have exited, though without raising significant new funds; and one third continue with some hope of eventual breakout growth. This should not be construed as a warning sign, however, as most VC interviewees stated they expect exits to take seven years at a minimum. QUESTION 2 To what extent is the current monitoring and evaluation (MEL) plan efficient in informing project design and implementation? What factors influenced the ability of the MEL plan to be effective? Is the timeframe of the project sufficient to conduct proper monitoring and evaluation for the project? What are the MEL recommendations for future projects? A desk review of the MENA II MEL Plan suggests its development objectives for jobs, private-sector investment, and women’s participation in the startup ecosystem are clear and program outputs for investment matching, equity guarantees, and technical assistance are simple and easy to grasp. See Annex 5 for the results framework for MENA II. MEL Plan indicators measure development objectives adequately and measure most important intermediate results. All are disaggregated by gender but not, given the small size of the country, by geographical location. See Annex 6 for the list of MENA II indicators. The desk review suggested that 2 intermediate results—sales collected by MSMEs as a result of MENA II (IR 2.1c) and number of beneficiaries applying new technologies or management practices as a result of MENA II assistance (IR 2.1a)—are ambiguous about causal relationships and appear to be affected by 0% 20% 40% 60% 80% 100% Ecosystem partners Portfolio companies Mentors VCs LWAF investors Seeders 2017 investors Seeders 2016 investors 29% 42% 15% 18% 100% 17% 23% Women, % of participants 22 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV other intermediate results. And indicators that could provide a richer sense of the health of the supply and demand sides of the market for early-stage funding are lacking. A follow-on project could track the number of active VCs and distinguish between amounts committed and deployed in the market on the supply side and the number of startups in accelerators and angel-investor pools on the demand side. Regarding the adequacy of the project’s timeframe for evaluation and learning, IMC and several VC respondents agreed that 3 years is enough to provide evidence on IMC’s ability to source viable VC investments. IMC and angel-investor respondents felt that 5 years is enough to assess the ongoing ability of pools of angel investors to find credible earlier-stage opportunities. As the BDL and several investor respondents noted, however, 7 years is the minimum time before which VCs expect to see exits. It is also the current limit for BDL guarantees under its 331 program, which runs, unless renewed, until 2020. QUESTION 3 What combinations of activities or interventions have more factors in place to create sustainability and are more likely to create impact over the long run, and what were the primary synergies that contributed to that success? The findings in this section address the sustainability of early-stage investment mobilized by investment matching, the sustainability of Lebanon’s nascent angel network and the equity guarantees supporting it, the sustainability of ecosystem support activities, the sustainability of women’s participation in the startup ecosystem, and the program synergies contributing to sustainability. Many factors support the sustainability of investment activity mobilized by IMC investment-matching: • On balance, participants expect the VC market to continue to grow. A BDL interviewee expects the market to grow modestly by 8% per year with the number of funds remaining constant at about 10—potentially supported by funds from the 14-million-strong Lebanese diaspora (Lebanon’s population is four million) that the central bank hopes to attract by implementing an e-trading platform across all sectors. Several VC interviewees were more pessimistic, concerned that early-stage funds could dry up as the 331 program matures or expires. Even pessimists see investment activity like IMC’s as sustainable, however, as it operates outside the sectors supported by 331. Another VC interviewee believes even a single big exit could galvanize investors and ensure the viability of the market. • The central bank is likely to extend its program under Circular 331. The BDL interviewee believes the governor will wait to see results in the seventh year of the program, when banks are expected to exit their investments, but is ready to extend it if they continue to be favorable. • IMC plays a unique and valuable role contributing to sustainability of the overall market for early-stage investment in three ways. First, it is one of two VCs operating outside the technology sector. Second, it is able to invest in Lebanese startups with significant operations outside the country (unlike 331 investors). As a VC interviewee stated: “There is nothing called “There is nothing called ‘Lebanon’ in our space; our focus is on the region.” —Venture capitalist USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 23 ‘Lebanon’ in our space; our focus is on the region.” Portfolio-company and ecosystem-partner interviewees agreed that startups in sectors requiring scale need to expand beyond Lebanon. And third, IMC has shown the feasibility of overcoming scale limitations on traditional single￾sector VC funds in a small market by building a rich, cross-sector portfolio, including investments in media, technology platforms, biomedical devices, agri-food, fashion design, accelerators, and hardware. • The IMC matching investment pipeline, though young, appears sustainably normal. Management took 32 of 250 candidates to the Investment Committee, which approved 16, according to IMC staff. Of these, eight were led by IMC while its partners led seven. An IMC interviewee believes it could raise $10M while the 331 program continues. • Exits appear feasible. IMC staff and portfolio-company interviewees expect exits from management buyouts, follow-on rounds of investment by other VCs, and trade sales to larger companies. Eighty percent of VC exits in more developed market are trade sales, according to a VC interviewee, and nothing blocks them in Lebanon. Other sources of funds could include universities, insurance firms, and pensions, according to the interview with Kafalat executives. IMC believes outreach to the diaspora could also stimulate follow-on investment. While there are significant threats to the sustainability of investment activity mobilized by IMC investment-matching, respondents suggest IMC is in a position to mitigate most of them: • There could be a crowd at the exit. Offering a worst-case scenario, an IMC executive expressed concern about a potential bottleneck if VCs try to exit 200 startups supported by 331 investments within a short period of time. Possible overvaluation of some of those startups could make follow-on funding more difficult around 2020. In this scenario, it would be doubtful that more than 5% of IMC investments could raise additional funding. The interviewee believes that near-term exits will require regional mergers in any scenario. Mentor and other ecosystem￾partner respondents agreed that exits for investors will pose operating challenges for entrepreneurs. And the Kafalat interviewees warned that follow-on rounds will have to be four times current levels of funding if they follow patterns in other markets. Several VC interviewees pointed out, nevertheless, that this just makes IMC’s independence from BDL guarantees all the more helpful to the development of the market for early-stage funding. • Macroeconomic and political risks pose a challenge in themselves and could complicate the extension of Circular 331. Numerous respondents referred to the recent resignation of the prime minister as an illustration of the volatility of Lebanon’s political environment. Bank, IMC, and Kafalat interviewees also expressed concern about the country’s high 146% debt/GDP ratio and the potential for a tightening in the credit market. Kafalat and bank interviewees said the ratio might complicate an extension of Circular 331 past 2020. • The effect of a failure to extend Circular 331 is uncertain. An IMC interviewee believes that some VC activity would continue without an extension of the program, although the BDL respondent thought there would be none. IMC’s ability to close non-technology deals provides some grounds for optimism; only a few successful exits, however, would prove the sustainability of the market. • The small size of the Lebanese domestic market poses an ongoing challenge. A portfolio￾company interviewee in the agri-food sector described in detail the challenges of operating 24 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV below scale in the local market and thought those challenges would apply to other nonservice businesses. Fortunately, VCs will be able to support regional expansion by portfolio companies with follow-on funds from non-331 investors, as a VC interviewee pointed out, and IMC has started to do this already. • The small scale of IMC is a challenge. Most VC interviewees said that funds with less than $20M of assets under management are unsustainable, with one arguing non-technology VC funds would need at least $50M of assets to be sustainable. IMC interviewees point out that their own experience suggests this may be too pessimistic, however. • Finally, policy infrastructure for the startup ecosystem is lacking. Startup, investor, and ecosystem-partner respondents cited the need for bankruptcy laws, tax reform, and entry of Lebanon into the Patent Cooperation Treaty. The willingness of angel investors to participate in insured pools at market or close-to-market rates provides strong evidence of the sustainability of the angel network and the equity guarantees supporting it. In exchange for a 50% guarantee on losses across a pool of investments, as mentioned above, angels give IMC a 10% stake in portfolio startups, pay a 2.5% setup fee, and commit to a 5% success fee based on pool profits in excess of an 8% per year return on investment. An IMC interviewee explained that it set the 10% stake—effectively the price of a 50% loss guarantee—based on historical losses of 13% on Kafalat loans carrying an 80% loss guarantee and adjusting for the lower level of the IMC guarantee. There was some disagreement over the adequacy of the earliest-stage startup pipeline available to angel investors and accelerators. An ecosystem-partner respondent stated that more deal flow would be needed to sustain the earliest-stage ecosystem than the 100 deals per year now available—and available only with the encouragement of 331 investments. Kafalat interviewees argued, to the contrary, that the Lebanese pipeline was already rich, citing their experience with a grant program offering $15K to startups at the concept stage approved by three-person committees. Committees sponsored by Kafalat’s ISME affiliate have approved 50% of the 130 ideas submitted so far, and grantees have gone on to raise $6M in additional funding. At one point, 9 of 11 mentoring clients had started with ISME grants. The most important, but not all, of IMC’s ecosystem support activities appear sustainable. Kafalat, investor, and ecosystem respondents said that IMC training and support for angel investors is sustainable in itself, for example, and plays an important role in sustaining the market for early-stage investments. Coaching is not currently sustainable, IMC concedes, as it has yet to pass costs to startups. The Speed accelerator, however, faces a near-term sustainability challenge, according to several ecosystem partners. BDL’s Circular 408 restricts board members and other managers of an accelerator using BDL-supported funds from investing in startups graduating from the accelerator. This discourages Speed’s original investors from further rounds of funding, according to a Speed interviewee, because investors like IMC do not want to forego opportunities to invest in its graduates. As a result, the accelerator needs to find alternative funding, possibly through other aid agency or a new BDL program. Speed could raise funds by selling services to VCs or universities, according to the interviewee, or by creating a specialized follow-on fund in sectors such as high technology, gaming, education, and creative or financial services. An ecosystem partner suggested that, regardless of its own sustainability, Speed has encouraged the rise of several unsubsidized incubators that provide support services to startups without taking stakes in them. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 25 Regarding the sustainability of women’s participation in the startup ecosystem, IMC interviewees indicated that the organization is conscious of the need to allocate more resources that target women entrepreneurs for both earliest-stage funding and venture capital investment. To this end, IMC is seeking VC funds such as Foundation Diane that are interested in investing in women-led portfolio companies as a first step in strengthening the incentives for women entrepreneurs who apply for early-stage funding. On the supply side of the market, the LWAF provides an opportunity for women to participate in and shape investment pools that further encourage women entrepreneurs to grow their startups. Interviews and focus groups revealed three principal synergies across MENA II activities relevant to the sustainability of the market for early-stage investment in Lebanon. The first is that guarantees and ecosystem support strengthen deal flow. On the demand side of the market, several ecosystem-partner respondents asserted that deal flow depends strongly on the effectiveness of ecosystem elements such as training and accelerators. Accelerators rely in turn on angel investors to invest in their graduates since few VCs show interest in deals on the order of $50–100K. Angel investors benefit reciprocally from the impact of accelerators on the quality of deals. It is MENA II equity guarantees, finally, that have been decisive in launching Lebanon’s angel network. The second sustaining synergy is that exits and advice encourage VCs. On the supply side of the market, more VCs will invest in Lebanon, according to nearly every VC interviewee, when they see attractive exits—meaning sales of initial VC stakes at multiples of the value of their initial investments. An ecosystem-partner interviewee thought that the biggest obstacle to such exits will be the consistency of startup support after graduation from an accelerator or incubator program. An investor interviewee said there may be an ongoing role for IMC to play as other investors rarely encourage portfolio companies to pay for advice. A bank interviewee argued IMC can support the market by orienting its portfolio companies toward exits. The third sustaining synergy is between investment and talent. The BDL interviewee believes VC investment has already started to attract expatriate talent back to Lebanon. And talent, in turn, will attract investment. A returning expat portfolio-company interviewee added that the country’s cost￾effective engineers have been an enabling factor in raising funds: “I pay $20K in Lebanon for an engineer who costs $100K in the U.S.” That said, the same interviewee lamented Lebanon’s lack of business infrastructure, including the fact that it is not yet part of the Patent Cooperation Treaty. 26 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV QUESTION 4 Given the above, how confident are we that MENA II will augment USAID’s program objectives if USAID extends/increases the level of funding to the MENA II project? What successes and challenges should be taken into consideration if this project scales up? If this project scales up, should it focus more on one or more components of the project than others? The findings in this section restate the main MENA II successes and challenges relevant to a decision about extending the program before summarizing the wide array of stakeholder recommendations made in the course of interviews and focus groups. Successes relevant to a decision about extending or expanding MENA II are covered in the results cited in response to Evaluation Question 1. Those most often cited by diverse respondents are the following: • IMC has met MENA II investment-matching targets and built a portfolio of investments beyond the sectors guaranteed by the central bank’s 331 program. • IMC has launched an angel investment sector through the use of MENA II equity investment guarantees. • IMC is involving women in every stage of the startup cycle, from accelerators providing safe networking opportunities to angel investors, mentors, VCs, and entrepreneurs receiving funding and mentoring. • IMC is building out the ecosystem of accelerators, mentors, and coaches to help startups grow. Key challenges include the following: • Angel-investor and VC respondents emphasized the small size of the local market. • IMC and other VCs emphasized the need for bankruptcy law, tax rebates (though these are not available in some successful markets), and greatly simplified company registration procedures. • Kafalat and bank respondents listed consolidation and stabilization of the nascent VC market, political and economic risks, and reliance on donors like BDL and the World Bank as key challenges. The following distinctive points of view and recommendations from diverse interview and focus group respondents often conflict with one another but tend to be internally consistent. They are useful to review because of the range of assumptions they reflect and, in some cases, their creativity. Asked what they would do if money were no object, IMC staff said they would invest $20–50M in early-stage VC funds or a so-called sidecar fund to match investments by its guaranteed angel investment pools—as well as in later-stage funds specializing in agri-food startups, film production, R&D, and impact investments, where the latter refers to investments targeting social as well as financial outcomes. Interestingly, a regional VC argued that film investments offer Lebanon significant tourism benefits while generating $18 spent for every $1 spent on production. “What USAID is doing is brave.” —Investor USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 27 An ecosystem partner recommended that IMC keep diversifying beyond technology, focusing on SMEs and value-added services. Several portfolio-company interviewees want databases to match startups with markets, talent, and support services. A portfolio company, IMC, and several mentor respondents proposed setting up or supporting university accelerators and innovation competitions to enrich and improve deal flow. An active VC argued that IMC is too shy and should try to do more within its “clear value proposition” of matching promising investments without bureaucracy and without encumbering investment decisions by setting its own target hurdle rate. The respondent urged continued creativity in strengthening ecosystem “ownership, governance, and sustainability.” The VC favors equity guarantees for angel investments while focusing matching investments on useful experiments such as renewable energy, creative services, retail, and design. “We don’t need another VC; we have too many,” was an illuminating comment, albeit one reflecting the institution’s own interests. An IMC executive estimated there are $200M of opportunities for investment in family and other medium-sized businesses, where medium businesses in this context mean established businesses larger than those in VC portfolios. Another VC interviewee also supported a small-business fund for family firms. Seeders 2016 angel respondents urged more entrepreneur recruiting; more startups outside Beirut; matching investments; and a focus on non-software sectors such as hardware, impact investing, and agricultural technology. An interesting comment was that angel investors might be attracted to a technology pool with a lower guarantee. A final wish was for more startup coaching and an effort to focus entrepreneurs on exits. Another VC interviewee suggested a technology focus for earliest-stage investments, a small-to-medium business fund for family firms, and a microfinance fund. An interesting idea was for IMC to work with BDL on co-financing small-business loans with loss sharing to reduce the high levels of collateral that banks require for loans—often as much as the loan itself. Some mentor respondents urged instruction in basic business skills prior to mentoring, with more rigorous screening to identify the most serious management teams. Mentors with educational backgrounds were more comfortable teaching business basics in the course of mentoring. Most resisted radical changes to MIT’s formula for its Confideo program. Kafalat interviewees felt that there was a need for a $20M VC matching fund or a $50M standalone fund willing to operate outside in any sector. Like other VC respondents, they supported continued equity guarantees because Lebanon has no alternatives for the earliest stage of funding ($50–150K). A bank interviewee argued that IMC should take management and technical-assistance fees to be sustainable, strictly separate its investment matching and equity guarantees (hence no matching of angel pools) to avoid conflicts of interest, find a regional partner, and get a success fee to strengthen its 28 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV incentives to exit its investments.16 Indeed, this respondent thought that IMC should exit 25–40% of its portfolio before expanding and set a 15% IRR of its own rather than rely on the investment criteria of its partner investors. The interviewee warned that family businesses are disinclined to give up equity but might be receptive to royalty-based investing alternatives that give investors a chance to earn more if sales rise without taking ownership stakes. An ecosystem partner thinks that IMC should add a $10–15M standalone fund for earliest-stage investments (although a VC interviewee warned that funds consisting of such small investments were hard to sustain). Another interesting idea was sponsoring VC, as opposed to angel, training along the lines of UC Berkeley’s Deal Camp, given its affordable $100K cost for classes of up to 20 investors. Finally, a wide range of respondents suggested working with nascent university accelerators to improve their business focus or offering university departments help with business expertise to encourage them to launch accelerators for their graduate students. 16 Note that IMC already has a success fee for its equity guarantees. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 29 CONCLUSIONS QUESTION 1 To what extent is [the] MENA II approach and implementation effective in terms of: a. How does the strategic approach shape the incentives of the respective stakeholders? Are the incentives generated likely to lead to the outcomes sought? b. What do we know about how this approach is working to date through MENA II Lebanon implementation? c. Which activities/combinations of activities have been more successful in achieving their objectives, and what were the primary factors responsible for their relative success? MENA II has been effective in meeting its goal of advancing the development of Lebanon’s private sector by increasing the competitiveness and revenue growth of early-stage businesses. MENA II investment￾matching and equity-guarantee activity plays an especially valuable role in the context of the BDL program. The 2013 issuance by the Lebanese central bank of Circular 331 supporting early-stage investment in knowledge-based Lebanese startups sets the context for an evaluation of MENA II midterm effectiveness. Guarantees under Circular 331 of 75% of the losses on bank investments in qualifying Lebanese knowledge-based startups distort the market and divert resources from productive non￾technology startup sectors. The volume of investments they have supported, however, is likely to produce at least a few of the kind of exits, or sales of initial investor stakes, that are needed to attract the attention of experienced VCs and sustain the market for early-stage investment. The MENA II strategic approach shapes the incentives of stakeholders by showing the feasibility of closing deals beyond the sectors supported by Circular 331 and by building an early-stage investment portfolio without the benefit of the central bank’s deep guarantees. By increasing confidence in the ability of the market for early-stage investment to nurture competitive businesses, MENA II makes it more likely that investors and entrepreneurs will participate in it. The main factors behind the effectiveness of MENA II investment matching in meeting its development goals appear to be the unique role of implementing partner IMC in the early-stage investment ecosystem and the robust portfolio of ecosystem-support initiatives that it has built. IMC plays a nearly unique role in matching investments outside of the sectors supported by 331, investing in Lebanese startups with significant operations outside the country, and insuring pools of earliest-stage investments (on the order of $50K–150K) by angel investors. IMC ecosystem-support activities under MENA II include structuring angel support programs, launching an accelerator, and facilitating startup access to coaching and mentoring. While investment matching has helped mobilize significant investments by other VCs, the requirement to match every investment may have kept IMC from helping fill a widely perceived gap in earliest-stage funding by making small-ticket ($50K–150K) investments on its own. Such investments are too small to attract VCs with limited resources for due diligence. 30 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV MENA II equity guarantees have been effective in achieving their development objectives to an extent disproportionate to the relatively small level of investment they have insured. They have enabled IMC to build a functioning earliest-stage investment sector with three pools of angel investors from scratch, helping fill the earliest-stage funding gap mentioned above and generating much-needed deal flow for the emerging VC market. Women’s participation throughout the ecosystem is evident but not yet prevalent. Women comprise more than 40% of the entrepreneurs running IMC portfolio companies and an entire pool of angel investors. IMC’s investment manager is a woman, as is the founder of Beirut’s leading incubator. Women run only a small proportion of VC firms, however, and few if any are active mentors. There is early evidence that mentoring needs more experimentation because of the high mentee dropout rate, senior-executive mentor frustration with the lack of business knowledge of many of the entrepreneurs they advise, and some mentee dissatisfaction with the relevance of the advice they receive. Finally, MENA II also appears to have created effective incentives for IMC to achieve its development goals. IMC has an incentive to sustain itself with follow-on funding by meeting those goals, of course. But MENA II also gives IMC the flexibility to test and learn about what works in the startup market. The combination of incentives and flexibility helps explain how IMC has solved a number of complex problems in a short time. For example, IMC learned to apply investment matching to early-round VC investments and guarantees to earliest-stage investments suitable for angel investors—a combination that might not have worked. QUESTION 2 To what extent is the current monitoring and evaluation (MEL) plan efficient in informing project design and implementation? What factors influenced the ability of the MEL plan to be effective? Is the timeframe of the project sufficient to conduct proper monitoring and evaluation for the project? What are the MEL recommendations for future projects? The current MEL plan is efficient in informing project design and implementation in that the clarity of development objectives and simplicity of program outputs have led IMC to build strong portfolio of initiatives. Objective targets for jobs and private-sector investment, according to the MEL Plan, are to be met achieving output targets for matching investment and amounts insured by equity guarantees. As a result, IMC has focused its investment activity on generating jobs and mobilizing other funds. The MEL plan could nevertheless be more effective in informing program design and implementation in other contexts. Several indicators reflecting the health of the demand and supply sides of the early-stage investment market are lacking. More specifically, quarterly testing of assumptions about the impact on private-sector investment of the number of active VCs in the Lebanese market, the number of startups in Lebanese angel portfolios, and the number of startups in Lebanese accelerators could reveal the actual importance of these critical factors over time and thus contribute to future program design. Strong results for sales collected by startups as a result of MENA II (IR 2.1c) and for new jobs due to MENA II assistance (DO2b) suggest life-of-program targets for these indicators could be somewhat higher. Since sales reported after 3 years of a 5-year program are 89% of the current $2.25M life-of- USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 31 program target, it would be reasonable to set a $3M target given similar resources for future programs in Lebanon and other countries. And since new jobs reported after 3 years of a 5-year program are 76% of the current life-of-program target of 200, it would be reasonable to set a target of 250 given similar resources for future programs in Lebanon and other countries. Please note that actual target-setting for program outputs, intermediate results, and development objectives depend strongly on context, so these conclusions are not recommendations. While it is tempting to raise the target for beneficiaries applying new technologies and management practices (IR 2.1a) – currently 4% over its life-of-program target – it may make more sense to define such applications more strictly to see how sensitive results are to the indicator’s definition. It may be similarly tempting to raise the target for individuals benefiting from training (IR 2.1.3a) – now 14% over its life-of-program target – but this does not make sense given recommendations below on modifying the mentoring program, which may result in greater demands placed on mentors and hence a reduction in the number of individuals reached (but improvement in the quality or depth of the interventions). Finally, the low percentage of life-of-program targets achieved so far for spending on equity guarantees (IM 1.2a) and technical assistance (IM 2.1b) – both 15% – might suggest relaxing these output targets in future programs. Such a conclusion would be premature, however. Spending on equity guarantees may well accelerate as Lebanon’s angel sector develops. And IMC may substitute training programs it finances for training services it has obtained free of charge to make them more relevant. While it is possible to assess the health of deal flow and the quality of startups within 3 to 5 years (that is, the demand for funds), it takes 7 years to assess exits that provide liquidity to initial investors (which drive the future supply of funds). QUESTION 3 What combinations of activities or interventions have more factors in place to create sustainability and are more likely to create impact over the long run, and what were the primary synergies that contributed to that success? The factors enhancing the sustainability of investment matching outweigh the threats, given that IMC has several ways to mitigate the challenge posed by the small size of the domestic market for startups to achieve economies of scale and the challenge of a nascent capital market for exits, or sales of initial investor stakes. IMC can continue and even increase its efforts to work with regional partners, for example. And it can try to strengthen the incentives and support that it provides portfolio companies to work toward an exit that provides liquidity to initial investors; it can also raise new funds for growth. The findings on sustainability suggest that IMC equity guarantees and support for Lebanon’s new angel network will be sustainable unless losses significantly exceed expectations. IMC has based its guarantee fees on the only relevant historical information available, and those fees are consistent with assumptions underlying expectations of a range of moderate returns on the angel-investment pools. IMC support for the angel network, furthermore, continues to add to what the early-stage investment market can currently provide on its own, given the unique role that IMC plays in that market and the ecosystem support capabilities that it has developed. Equity guarantees supporting angel investors therefore appear to offer both sustainability and additionality. 32 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV The sustainability over the next year of the Speed accelerator is a concern given the important role that accelerators play in generating deal flow for VCs. If Speed cannot recapitalize itself, the Lebanese ecosystem will lose the investment opportunities that its 12 to 15 annual program graduates create for angel investors and VCs. The loss of an early accelerator pioneer may also discourage the establishment of other accelerators in the future. IMC has an opportunity to increase interest on the part of women investors in supporting women entrepreneurs through its outreach efforts. Not all portfolio-company and VC respondents were aware, for example, of the launch of the all-women angel pool LWAF. Both angel-investor and startup-ecosystem support appear necessary to strengthen deal flow (or demand for early-stage capital). Without the possibility of angel investments, some startups will not go to the trouble of enrolling in an accelerator; but without the support of accelerators, angel investors would suffer an erosion in the quality of investment opportunities. Increased participation in the early-stage investment market by experienced VCs (the supply side of the market) requires examples of exits, or timely sales of initial investor stakes by responsive startups. IMC could focus portfolio companies on such exits by requiring their business plans to incorporate them and structuring investments to reward progress toward follow-on rounds of funding. The use of equity guarantees to support an angel-investor network and the market for earliest-stage investments appears to have applicability to other markets, and especially to similar projects implemented in other MENA countries. Respondents raised nothing about the Lebanese context of the equity-guarantee initiative that its application to other countries would particularly require, except perhaps the existence of loss data on some kind of domestic investment guarantee program like the Kafalat loan-guarantee program in Beirut. While the application to other countries of the investment matching tool appears equally promising, there is some evidence that the guarantee program under Circular 331 of the BDL helped ensure there would be enough matching partners in Lebanon to support a robust portfolio of matching investments. Without such a prior program in place, the success of an application of investment matching to another country may depend on the number of independent VCs already operating in its early-stage investment market and detailed terms affecting the attractiveness of investment matching to potential partners. The factor behind the initial success of investment matching in Lebanon most widely applicable to other countries is probably the agility of USAID/Lebanon in quickly complementing a bold initiative by the central bank. Economic growth programs in other countries may want to tailor their interventions to take advantage of changes set in motion by salient local-government initiatives. QUESTION 4 Given the above, how confident are we that MENA II will augment USAID’s program objectives if USAID extends/increases the level of funding to the MENA II project? What successes and challenges should be taken into consideration if this project scales up? If this project scales up, should it focus more on one or more components of the project than others? The evidence suggests an extension or expansion of investment matching will advance MENA II development goals. IMC’s freedom to experiment and quickly reprogram activities in light of results has USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 33 helped it overcome significant challenges to the development of a startup ecosystem. Moreover, it makes sense not to micromanage its investment activities. Even so, diverse respondents had strong views on four structural issues concerning IMC investment activities going forward: • Matching for all investments: While the matching requirement has helped IMC mobilize significant VC funds for early-stage investments over $250K, it has kept IMC from filling a gap in earliest￾stage deals under $100–150K by investing on its own. • Targets or hurdle rates of return for investments: Investment matching lets IMC be more agile in responding to investment opportunities by relying on partners’ investment criteria and not imposing separate requirements, such as target rates of expected return, of its own. If MENA II relaxes the matching requirement for certain kinds of investments, IMC may wish to set financial requirements for them to ensure the sustainability of its investment activity. • Providing IMC more incentives for exits: While attractive exits that pay early-stage startup investors a multiple of their initial investment will be vital for the development of the market, most respondents believe it is counterproductive to try to rush them. That said, IMC may want to consider ways to provide incentives through investment structure and support through mentoring or coaching that orient portfolio companies toward successful exits. • Pursuing family and other small or medium businesses with an instrument such as royalty-based capital: It is worth considering suggestions for IMC to explore investment in growing family and other medium-sized businesses and investigate the use of royalty-based capital that pays investors a share of revenue. These suggestions reinforce one another given the reluctance of family businesses to give up equity stakes and the ability of some kinds of royalty-based transactions to let investors share in success without taking such stakes. It is true that a desk review of royalty￾based financing instruments shows that most applications in developed markets are too complex for capital markets at Lebanon’s scale and stage of development. Most of the proponents of the idea nevertheless seemed to have in mind a very simple application of the tool that would provide investors only an agreed percent of revenue plus a fixed rate of interest. The addition of a percent of revenue to what otherwise looks like interest on a loan may suffice to provide investors with enough upside to look into the finances of a small company without diluting the family’s ownership of the business, and deserves further consideration. There is strong agreement on extending and expanding the guarantee and support program for angel investors to the extent that markets permit because they have succeeded beyond expectations despite modest outlays. Several respondents recommend systematic startup recruiting. Seen as a portfolio of ongoing experiments, IMC technical assistance supporting the startup ecosystem by training investors, launching an accelerator, and providing startups with access to coaching and mentoring has been successful and appears to be important for continued success. Three ideas to improve and strengthen that support stand out: • Investigating new ways to support accelerator activity—for example, by helping university accelerators and innovation competitions add business acumen to their programs; • Giving startups access to mentors who are more inclined to teach business basics or launching a micro-MBA program for startups on business basics; and 34 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV • Further strengthening investor support, as in the example of sponsoring selected angel investors for programs such as Berkeley’s Deal Camp. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 35 RECOMMENDATIONS QUESTION 1 To what extent is [the] MENA II approach and implementation effective in terms of: a. How does the strategic approach shape the incentives of the respective stakeholders? Are the incentives generated likely to lead to the outcomes sought? b. What do we know about how this approach is working to date through MENA II Lebanon implementation? c. Which activities/combinations of activities have been more successful in achieving their objectives, and what were the primary factors responsible for their relative success? USAID should extend MENA II for five years at the end of the current project and expand it by at least the amount of current funding if the budget permits—while continuing to provide IMC with clear goals and the flexibility to experiment—given the incentives the program has shaped, its creative implementation, and evidence for the effectiveness of the program to date in advancing private-sector development by increasing early-stage business competitiveness and growth. USAID should consider relaxing the requirement for IMC to find a matching partner for each of its earliest-stage investments. To help determine the ticket size—for example, $150K—below which USAID should relax the matching requirement, IMC should assess the range of sizes of investments for which the development impact of filling the startup funding gap that respondents have identified in Lebanon outweighs the impact of encouraging other investors in the local market for early-stage investments. IMC should build on progress integrating women into the startup ecosystem through concerted outreach efforts to women on opportunities in accelerators, angel investing, venture capital, and mentoring. Outreach methods could include social media, events such as pitch days and competitions, and sponsorship of events such as the BDL’s entrepreneurship conferences. IMC should investigate ways to make mentoring more effective. One would be to select mentors more willing to teach basics. Another could be to run a focus group of mentees to determine what kinds of advice most helped them build their startups. A third, in time, might be to recruit successful entrepreneurs to join the mentoring program. USAID should promote MENA II angel networks and the equity guarantees that support them as a model for increasing business startup access to finance in the region. In promoting investment matching as a model for other private-sector development projects in the region, however, the recommendation is slightly different. While nearly as successful as the guarantees and support for an angel network, investment matching in Lebanon has benefited from the unique circumstances of the central bank’s guarantee for technology VC investment. The most important investment-matching success factor relevant to other missions, therefore, may be agility in taking advantage of and supporting these kinds of local government initiatives. 36 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV QUESTION 2 To what extent is the current monitoring and evaluation (MEL) plan efficient in informing project design and implementation? What factors influenced the ability of the MEL plan to be effective? Is the timeframe of the project sufficient to conduct proper monitoring and evaluation for the project? What are the MEL recommendations for future projects? USAID programs extending MENA II in Lebanon or elsewhere in the region should include intermediate-result indicators for the health of the demand and supply sides of the early-stage investment market, such as the number of startups in accelerators, the number of startups in angel (or earliest-stage) investment portfolios, and the number of independent VC firms investing in them. USAID should disaggregate deals by geographic location in MEL plans for similar programs in countries larger than Lebanon. USAID program officers extending MENA II or implementing it elsewhere should estimate the impact on startup sales of its investment and ecosystem-support components by forecasting startup sales and estimating the best- and worst-case sales impact of each component at the beginning of each quarter and then revising them in light of the performance gaps that remain unexplained at the end of each quarter. QUESTION 3 What combinations of activities or interventions have more factors in place to create sustainability and are more likely to create impact over the long run, and what were the primary synergies that contributed to that success? IMC should encourage sustainable VC participation in Lebanon’s market for early-stage investments by guiding startups to successful exits—meaning sales of initial investor stakes that provide liquidity and pay a multiple of the initial investment—and continuing to explore regional partnerships. Given the sustainability of the angel network, USAID should encourage IMC to use its resources to expand it as much as the market allows. IMC already has the resources to insure another $2–$2.5M in angel-pool investments. To expand these investments, IMC would probably need to find a way to recruit more entrepreneurs, which the emergence of new accelerators would help. And IMC would probably need to recruit more angel investors from family businesses, larger corporations, and the diaspora. IMC should identify gender gaps within the startup ecosystem that outreach efforts could be especially effective in filling. A focus group of women portfolio-company owners and angel investors could help provide both the gap assessment and ideas for outreach. USAID and IMC should monitor the sustainability of accelerators in the startup ecosystem and IMC should investigate whether there are more efficient ways to support them than just taking major stakes in them, as in the case of Speed. An example would be the recommendation from a number of respondents to help universities set up accelerators and startup concept competitions by providing them with business advisors to make sure they are sufficiently commercially oriented to contribute to deal flow in the Lebanese market for early-stage investment. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 37 QUESTION 4 Given the above, how confident are we that MENA II will augment USAID’s program objectives if USAID extends/increases the level of funding to the MENA II project? What successes and challenges should be taken into consideration if this project scales up? If this project scales up, should it focus more on one or more components of the project than others? IMC should experiment with a range of target or hurdle rates of return in cases where its partners have no target or, potentially, where there is no partner, including small-ticket investments that it undertakes on its own if USAID decides to relax the matching requirement for them. IMC should explore ways to improve the quality of startups for angel investors and VCs to consider (i.e., deal flow). The two most promising recommendations for doing so are strengthening the business focus of university accelerators (and perhaps prize competitions) and giving startups access to training in business basics as well as advice. The benchmark for effectiveness of these two recommendations— supporting the emergence of accelerators in the ecosystem and adapting mentoring more closely to the needs of entrepreneurs—should thus be improvement in deal flow, as assessed by angel investors and VCs. IMC should investigate ways to help family and other medium-sized businesses exploit growth opportunities. The most promising recommendation for accomplishing this is a fund using instruments such as royalty-based capital that let investors participate in success by sharing sales revenue rather than getting equity. Such instruments would enable family businesses to compensate investors for risk without giving up control of their companies. 38 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEXES Annex 1: Consideration for applying MENA II in other countries Annex 2: Inception Report Annex 3: Documents Reviewed Annex 4: Data Collection Instruments Annex 5: Data Collection Schedule Annex 6: MENA II Intended Results/Scope of Work Annex 7: List of MENA II Indicators USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 39 ANNEX 1: CONSIDERATIONS FOR APPLYING MENA II IN OTHER COUNTRIES Regional, Social, and Economic Context of MENA II While MENA II appears to be achieving very significant successes and creating impact on an economy￾wide level in Lebanon, donors should take caution before attempting to replicate the program in other countries and contexts. Several factors unique to time and place helped enable MENA II’s achievements to date. First, MENA II was particularly well timed and well positioned due to the presence of BDL Circular 331 and the supporting institutions and resources that it provided to the venture capital ecosystem that MENA II was able to exploit. The supply of dedicated funds to the sector is significantly catalyzing its development, and MENA II is benefiting from the associated demand for these resources and the opportunities that it has created. Second, given prevailing high interest rates as well as a number of other macroeconomic factors, there was simply not much demand for venture capital in Lebanon when MENA II entered the market. Therefore, the presence of a donor-funded entity did not have a negatively distorting effect on the market through crowding out or unfairly competing with other VCs. There simply was not much competition in the market to begin with and MENA II is helping to change that through its commercially oriented fund. In other cases, the presence of a donor-funded VC could be market-distorting in a negative way if similar conditions are not met. Third, Berytech was initially well positioned within the nascent ecosystem to take advantage of specific opportunities that were previously known to the grantee. Berytech had a clear competitive advantage to exploit early-stage investment opportunities but was unwilling to take such large risks on its own without the USAID grant that enabled it to reduce its direct exposure and risks. Fourth, Lebanon is unique in that it had a fairly elastic supply of fundable and high–growth potential investments and entrepreneurs with respect to the supply of capital. Lebanon has extensive links with the outside world given its large and diverse diaspora population that typically reimport knowledge and business ideas. The availability of capital created under Circular 331 enabled a significant number of entrepreneurs to choose Lebanon as a headquarters for their operations. Several of the most talented entrepreneurs we met reported that the recent availability of investment capital and other cost advantages unique to Lebanon influenced their decision to set up offices in and around Beirut. According to BDL, the “knowledge economy” sector for startups has been expanding 8% per year since 2013, creating a pipeline of fundable projects. Beyond the uniqueness of Lebanon’s situation, it is still too early to judge the ultimate success of the project or the viability of the larger venture capital ecosystem prior to demonstration through profitable exits. While initial indications are very promising, Circular 331 is a significant market distortion that could lead to a bubble and large investment failures if not managed properly going forward. However, MENA II is helping to build the resilience of this new ecosystem to the negative shocks that will likely appear in the near future. It is doing so by helping develop a diversity of skills, capital, firms, and networks through the MENA II project, which will help protect it from a potential wave of investment failures. 40 | FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION USAID.GOV Risks While the VC ecosystem appears to be off and running thanks to Circular 331 and the support of donors such as USAID through MENA II, there are a number of sources of risk that could lead to the failure of the emerging system. Exits will be the critical factor in determining the sustainability of the new ecosystem and will depend on several key factors. The first is the ability to jump start the Beirut Stock Exchange through the planned privatization and attract firms and active traders to the new digital platform. The second is the ability to attract foreign sources of investment (e.g., international funds) that are willing to purchase Lebanese assets. The issue is that Lebanese assets are generally viewed as risky, so Lebanon will need to improve its general image and demonstrate political stability. There is also talk of the emergence of domestic funds as well, but these are likely to be under-capitalized in the near term. The third is the ability of investees and the management of the young firms to finance buyouts of their own companies through willing local commercial banks. Significant time will be needed to demonstrate a track record of successful exits and this will depend in no small part on the patience of key stakeholders. Another key risk is the over-reliance on two critical sources of funding, namely the BDL and international donors. If BDL decides to discontinue its support for 331 or refuses to extend the guarantees, this could lead to a rush to the door by domestic banks and other investors. While BDL has received considerable praise for 331, it is ultimately an arm of the government of Lebanon and thus subject to political whims. If BDL grows impatient with a slow pace of exits, it might reevaluate its interventions and pull back, threatening the system. Current funds will have exhausted 331 resources in the next year if it is not extended by BDL, and it is unclear if banks will continue to invest in the sector even if 331 is extended. The other major source of capital is donors such as USAID, and an overreliance on donor capital could prove unsustainable. Donors, too, might decide that their investments are becoming increasingly risky or could become increasingly impatient with a slow pace of exits, again threatening the system. MENA II is focused on seed-level capital, but Series-B funding is normally four to five times the investment amount, requiring capital beyond what USAID would ever be willing to commit to the project. As more and more enterprises seek subsequent-round funding, they will have to turn to other commercial sources. Lebanon does not have many institutional investors such as endowment and insurance funds to support this growth, either, so it is unclear if the necessary capital will exist to continue to support the system. Another key risk is domestic and regional politics, which in the Middle East region is well known. A repeat or conflict similar to the one with Israel in 2006 could be enough to scare investors away for an extended period and threaten to collapse the ecosystem. USAID.GOV FINAL EVALUATION REPORT | MENA II MIDTERM PERFORMANCE EVALUATION | 41 ANNEX 2: INCEPTION REPORT I | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION MIDTERM PERFORMANCE EVALUATION MIDDLE EAST NORTH AFRICA INVESTMENT INITATIVE (MENA II) INCEPTION REPORT Performance Management and Support Program for Lebanon (PMSPL II) II | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION MIDDLE EAST NORTH AFRICA INVESTMENT INITIATIVE (MENA II) Midterm Performance Evaluation Performance Management and Support Program for Lebanon (PMSPL II) for USAID/Lebanon This document was submitted by Social Impact, Inc., to the United States Agency for International Development under USAID Contract/Order No. AID-268-C-15-00001. This document is available in printed or online versions. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com III | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION TABLE OF CONTENTS TABLE OF CONTENTS...................................................................................................................................... III List of Figures......................................................................................Error! Bookmark not defined. List of Tables.......................................................................................Error! Bookmark not defined. ACRONYMS...........................................................................................................................................................IV EXECUTIVE SUMMARY........................................................................................................................................ I INTRODUCTION................................................................................................................................................. 2 PROJECT BACKGROUND................................................................................................................................. 2 Purpose/goals of the program...................................................................................................................... 3 Development hypothesis............................................................................................................................... 3 program components..................................................................................................................................... 3 PURPOSE AND INTENDED USE...................................................................................................................... 4 Intended audeince ........................................................................................................................................... 5 EVALUATION QUESTIONS:.............................................................................................................................. 5 EVALUATION DESIGN AND METHODS ..................................................................................................... 6 Data collection................................................................................................................................................. 6 Data analysis methods.................................................................................................................................... 7 Social and gender analysis ............................................................................................................................. 7 Evaluation limitations...................................................................................................................................... 7 Evaluation design matrix................................................................................................................................ 8 EVALUATION MANAGEMENT ......................................................................................................................10 Deliverables, Timeline, and Dissemination Plan.....................................................................................10 Workplan ........................................................................................................................................................12 Team Structure..............................................................................................................................................13 IV | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION ACRONYMS ADS Automated Directives System AMEP Activity Monitoring and Evaluation Plan AOR Agreement Officer’s Representative BALADI CAP Building Alliances for Local Advancement, Development and Investment￾BDL Banque de Liban COP Chief of Party CPA Certified Public Accountant CSO Civil Society Organization DO Development Objective DQA Data Quality Assessment EQUI™ Evaluation Quality Use and Impact GE/FE Gender Equality/Female Empowerment IDCS Institutional Development Calculation Sheet IMC Insure & Match Capital IPs Implementing Partners IR Intermediate Result HO Home Office MENA Middle East and North Africa MENA II Middle East and North Africa Investment Initiative NGO Non-Governmental Organization PMP Performance Management Plan PMPL Performance Management Plan for Lebanon Q4 Fourth Quarter SI Social Impact SME Small Medium Size Enterprise SOW Scope of Work USAID United States Agency for International Development Y1 Year 1 I | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION EXECUTIVE SUMMARY The United States Agency for International Development (USAID) awarded the Middle East North Africa Investment Initiative (MENA II) activity to Berytech Foundation on September 26, 2014. MENA II provides support to early stage businesses in order to create and sustain jobs, advance and develop the investment ecosystem, and encourage increased equity investment in early stage businesses. Lebanon is one of eight Middle East and North Africa (MENA) countries participating in this regional initiative. MENA II Lebanon is a $15 million program for five years, ending on September 2019. The purpose of this mid-term evaluation is to assess the performance of Berytech under MENA II, and to examine closely and critique the effectiveness and sustainability of the underlying methodology of the project’s strategic approach. The evaluation is to recommend whether USAID should further pursue its activities in this field and whether MENA II should augment its program objectives if USAID extends/ increases the level of funding to the MENA II project. The primary audience for the evaluation includes (i) the USAID/Lebanon Mission, particularly the Economic Growth Office, and (ii) MENA II implementing partners. Secondary users might include (i) the Middle East North Africa Investment Initiative bureau, (ii) ecosystems partners including incubators, business accelerators, and others, (iii) other investors, donors, and organizations working on investment initiatives, and (iv) other USAID missions, USAID and International Finance Corporation (IFC) program groups on access to finance, and the Political Economy bureau of the State Department. The evaluation will highlight lessons learned and their implications for the programming of future economic growth projects in Lebanon and elsewhere. A significant focus will be on lessons learned about the effectiveness of the MENA II approach to strengthening private-sector development, job growth, and access to finance, including the access of women entrepreneurs, through a combination of co-investing in early stage businesses, partially insuring investments in early-stage businesses investments, and providing technical support to both entrepreneurs and investors in the startup ecosystem. To support the take up of findings, this evaluation will include a post-evaluation action plan. Aligned with this mission, SI will continue to actively engage USAID, implementing partners (IPs), and other stakeholders in the evaluation process. 2 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION INTRODUCTION The United States Agency for International Development (USAID) awarded the Middle East North Africa Investment Initiative (MENA II) activity to Berytech Foundation on September 26, 2014. MENA II provides support to early stage businesses in order to create and sustain jobs, advance and develop the investment ecosystem, and encourage increased equity investment in early stage businesses. Lebanon is one of eight Middle East and North Africa (MENA) countries participating in this regional initiative. MENA II Lebanon is a $15 million program for five years, ending on September 2019. IMC is a fully-owned subsidiary of Berytech Foundation, which was awarded the local management of the MENAII Lebanon program. IMC provides matching capital to qualified venture-capital investors, equity guarantees against part of the losses of the initial portfolios of new angel-investor groups, and support programs to a broad range of qualified early-stage business and investors in Lebanon, reducing the risks associated with investing in early-stage businesses and encouraging increased capitalization. As MENA II evolves past its halfway point, USAID requested a mid-term performance evaluation to assess MENA II activities for their relevance, effectiveness, and sustainability. USAID and MENA II implementers will be the primary users of the evaluation. The findings will inform future USAID programming in Lebanon. USIAD and IMC will use the findings and recommendations while planning for the project activities in the remaining life of the project. Social Impact, as the prime implementer of the Performance Monitoring Support Project for USAID Lebanon, will be conducting the mid-term performance evaluation. This inception report lays out the plan that the evaluation team will follow to answer the evaluation questions. The report first describes the project’s background, next the evaluation’s purpose and intended use and users, the evaluation questions, and finally the evaluation methodology. The evaluation will be conducted within the context of the Evaluation Quality Use and Impact (EQUI™) approach, processes, and protocols of Social Impact (SI). EQUI processes are designed to achieve a quality evaluation that incorporates learning and utilization. The evaluation will highlight lessons learned and provide recommendations, taking into consideration programming of future economic growth projects in Lebanon. Future efforts will improve early stage businesses investments to create and sustain jobs, advance and develop the investment ecosystem, and encourage increased equity investment in early stage businesses, building upon the successes and lessons learned from USAID’s body of prior and current work in the sector. To support the take up of findings, this evaluation will include a post￾evaluation action plan. Aligned with this mission, SI will continue to actively engage USAID, implementing partners (IPs), and other stakeholders in the evaluation process. PROJECT BACKGROUND Economic growth in the Middle East and North Africa (MENA) region has been slow to rebound since the global financial crisis and the beginning of the Arab Spring. This slow recovery has been compounded by large numbers of job seekers entering inefficient labor markets. Uncertainty and lack of stability have also affected financial resource flows into the region, and constrained private investment. Much of the available capital in the MENA region is directed to low￾risk options or to larger companies with direct ties to finance or an ability to draw on their own capital. Many smaller companies and new business start-ups have more difficulty accessing finance and investment funds. This shortage of investment capital for early-stage enterprises as well as limited appetite for higher-risk investments presents a challenge for MENA countries to meet their development goals and reduce unemployment. 3 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION MENA II is a five-year investment initiative geared to improve investor and business access to equity capital, develop the investment ecosystem, advance financial system development, and encourage increased equity investment in early-stage businesses. It mobilizes investment management companies to work through incubators, angel funds, and venture capital funds. Berytech Foundation is a Lebanese NGO that aims at supporting entrepreneurs creating and developing startups and SMEs, through incubation, business support, networking, mentoring, funding and company hosting in a dynamic and vibrant environment. It is a national reference for the support of entrepreneurs, and is the Lebanon Grantee for MENA II. PURPOSE/GOALS OF THE PROGRAM MENA II Lebanon was designed to support achievement of USAID’s Development Objective 2 (DO2) for Lebanon: Inclusive Economic Growth Enhanced and the PMP IRs, IR 2.1: Increased Private Sector Competitiveness and IR 2.2 Increased access to finance, especially for new business start-ups and women. MENA- II Lebanon’s Goal is to “Advance the development of the private sector by increasing competitiveness and revenue growth of Qualified Early Stage Businesses”. DEVELOPMENT HYPOTHESIS MENA II Lebanon is to stimulate the mobilization of new equity capital, supporting the growth of new businesses and foster competitiveness, in order to create more jobs for Lebanon’s new labor entrants, estimated at 23,000 per year. On the supply side, MENA II Lebanon will prioritize work with private sector development stakeholders such as angel investors, accelerators, and incubators, by (i) matching new private sector investment with early-stage investors or in early-stage businesses or early-stage investors to reduce private sector exposure to risk and close the funding gap in early-stage businesses, by (ii) providing early-stage investors with a partial guarantee on investments in early-stage businesses, thus mitigating some of the risk associated with investing in early-stage businesses and encouraging increased capitalization, and (iii) providing a range of assistance for both investors and entrepreneurs, including training related to successful investment models, entrepreneurship and business concepts, angel coordination and assistance. On the demand side, activities will target innovative ideas for new and early start-ups, and small businesses as well as ways SMEs can add new value to the market. Critical assumptions: MENA II Lebanon is a 20-year program (running till 2034) but the obligated amounts as per the CA are for 5 years (till 2019), so IMC needs to be sustainable beyond 2019 by having re-flows and re-investing in the program activities. The nature of the funding activity is risky as such and hence sustainability is a critical issue for the program and needs to be constantly reassessed for all of its activities. PROGRAM COMPONENTS 4 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION To achieve the stated goals, IM provides the three programs described below: Matching Investment Capital: Matching investment capital will be invested with QESIs (Qualified Early Stage Investors, i.e. funds, accelerators, incubators) and potentially QESBs (Qualified Early Stage Businesses). IMC will match private sector investment capital to the early stage investment ecosystem. In doing so, it will ease fundraising for qualified investors, increase deployment of investment capital and subsequently diversify risk of the investors. The resulting reduction of private sector risk and increased capital available will serve to fill the funding gap for early stage businesses. Qualified partners will be enabled to scale up successful investment strategies and increase investment in early stage businesses. Matching Investment Capital will be provided by IMC under commercial investment terms allowing for mutual/similar risk and return with private sector capital providers. Investment Insurance: This is an equity-guarantee scheme intended to mitigate risk of investing in qualified early stage businesses (QESBs). This component will incentivize private sector investment capital into the early stage investment ecosystem by providing partial investment insurance to QESBs in order to facilitate fundraising. By mitigating downside risk and preserving upside potential, insurance will encourage private sector capital to the early stage investment ecosystem and allow for continued, improved and enhanced investments into qualified early stage businesses. Technical Assistance: Experience has shown that technical assistance and mentoring play an important role in the success of programs targeted at the early stage market. Many young companies have limited exposure to standard business and governance practices and may lack the requisite skills for attracting investors. The role of the technical assistance is to complement the matching equity capital and help guide the companies as they grow and attempt to attract additional capital. This will provide the platform for technical assistance to early stage investors, early stage businesses and to IMC. Matching capital and de-risking investments should attract the largest number of people to the investment scene as well as encourage the creation of angel investment networks by providing them with initial institutional coverage and credibility stamp. A big part should also be also aimed at technical assistance, focusing on the primary stages of enterprise and benefiting both the company as well as the investor. PURPOSE AND INTENDED USE The purpose of this evaluation is to assess the performance of Berytech under MENA II, and to examine closely and critique the underlying methodology of the project’s strategic approach. The evaluation is to recommend whether USAID should further pursue its activities in this field and whether MENA II should augment its program objectives if USAID extends/ increases the level of funding to the MENA II project. 5 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION INTENDED AUDEINCE The primary audience for the evaluation includes (i) the USAID/Lebanon Mission, particularly the Economic Growth Office, and (ii) MENA II implementing partners. Secondary users might include (i) the Middle East North Africa Investment Initiative bureau, (ii) ecosystems partners including incubators, business accelerators, and others, (iii) other investors, donors, and organizations working on investment initiatives, and (iv) other USAID missions, USAID and International Finance Corporation (IFC) program groups on access to finance, and the Political Economy bureau of the State Department. EVALUATION QUESTIONS: The main objective of this evaluation is to explore the effectiveness of the strategical model imbedded in MENA II, and of projects interventions. Specifically, the evaluation will review the rationale, current performance to date, and likely effectiveness of the three respective aspects of the MENA II approach outlined above. For example, the project’s approach to guaranteeing equity capital is implemented for the first time in Lebanon and very little has been written about the theoretical model that underpins this approach. In this respect, the evaluation would review: Effectiveness 1) To what extent is MENA II approach and implementation effective in terms of: a. How does the strategic approach shape incentives? Are they likely to lead to the outcomes sought? b. What is known about how the approach is working to date through MENA II implementation? c. Which activities have been more successful in achieving objectives, what were the primary factors responsible for the relative success? Efficiency 2) To what extent is the current MEL plan efficient in informing project design? a. What factors influenced the ability of the MEL plan to be effective? b. Is the timeframe of the project sufficient to conduct proper monitoring and evaluation of the project? c. What are the MEL recommendations for future projects? Sustainability 3) What combinations of activities or interventions have more factors in place for sustainability? a. What activities are more likely to create impact over the long-run? b. What were the primary synergies contributing to success? Future Programming 4) Given the above, how confident are we that MENA II will augment USAID’s program objectives if funding increased or extended? a. What successes and challenges should be considered if project scales up? b. On what component should project focus if it scales up? 6 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION EVALUATION DESIGN AND METHODS The evaluation team will use a primarily qualitative approach to answer the evaluation questions, conducting interviews and focus group discussions with key project stakeholders. The key informant interview (KII) respondents will include MENA II project staff, USAID/Lebanon’s Economic Growth Office staff, the office managing Circular 331 at Banque Du Liban, matching and angel inventors, service providers, and portfolio companies. The focus group discussion (FGD) participants will consist of the project’s mentor advisors, investment committees, and angel network and women angel investment fund members. This will be supplemented with quantitative data available from project monitoring, but no primary quantitative data will be collected. The evaluation questions as mapped against data sources, data collection methods, and analysis methods are listed in Table 1. DATA COLLECTION Data Collection will begin on November 6th and finish by November 17th, 2017. The evaluation will employ the following data collection methods: 1. Desk Review: The evaluation team will conduct a systematic literature review of relevant internal and external documents. These include the contract agreement, contract modifications, BDL Circular 331 related reports, project work plans, annual and quarterly reports, USAID’s Country Development Cooperative Strategy for Lebanon, USAID PMP, and all other project documents relevant to answer the evaluation questions. In addition, the evaluation team will review relevant external studies of Lebanon’s investment landscape that may lend additional context to the project’s relevance, effectiveness, efficiency and sustainability. 2. Key Informant Interviews (KII): The qualitative interview protocols will focus on exploring the relevance of the MENA II project to the sector long and short-term needs, how effective the project has been so far, and how sustainable the anticipated results will be. The team will conduct KIIs with USAID key staff members, IMC and MENA II staff, angel investors, matching investors, portfolio companies, training, hosting and eco-system partners, World Bank, Kafalat, and the Central Bank office managing BDL Circular 331. The KIIs will provide an in-depth understanding of project achievements to date, factors that influenced the effectiveness of project intervention, and implementation of project interventions. Annex I contains the list of key informants. The list includes the main points of contact and decision-makers from the main project’s stakeholders as identified during desk review, and confirmed with MENA II. The evaluation team (ET) will update this list with any additional key informants identified during the course of initial interviews and a full list of key informants consulted will be included as an annex in the final evaluation report. The ET anticipates conducting up to 30 KIIs. For those who cannot be reached personally, a Skype call will be scheduled to conduct the interview. 3. Focus Group Discussions: The Evaluation team will conduct focus group discussions with selected project stakeholder’s groups: mentors, Seeders 2016 angel network leaders, and Women Anel investment Fund members. Purposive Sampling was used to select the participants of each FGD, based on the following: - For the mentors: There are 39 mentors who provide mentoring to both IMC MENA II portfolio companies and other companies associated with IMC but are not considered portfolio companies. Out of the above list, around nine mentors were recommended for the focus group discussion. The identified mentors provide mentoring to IMC portfolio companies as well as other business ventures that are not considered IMC portfolio companies. A focus on the most active mentors is not expected to bias conclusions about the potential effectiveness of training. 7 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION - Seeders 2016 deal leaders: These are angel investors who were key leaders in the investment portfolio for the Seeders 2016 angel network. This focus group discussion will include all deal-leading members of the network. - Women Angel Investors Funds (WAIF): WAIF is in the early stage of its establishment. It includes a total of 21 women. Based on the recommendation of MENA, a total of 8 names were recommended to cover a wide spectrum of backgrounds including independent investors, key company representatives, professors, bank employees and others working in Lebanon and who are ready to actively participate in a focus group discussion. A focus on the most active women angel investors is not expected to bias conclusions about the gender￾sensitivity of interventions and gender issues regarding the outcomes of enterprise economy development in Lebanon. - Advisory Investment Committee: The AIC committee is responsible for reviewing and approving applications for portfolio investment. This committee consists of three members, one of whom is available in Lebanon. The remaining two are traveling. If possible, a FGD will be organized with all three; otherwise, the evaluation team will conduct KIIs with those available in person or on Skype. DATA ANALYSIS METHODS Once the evaluation team completes data collection and receives all the focus group notes, it will reassemble at the Social Impact Beirut Office for data analysis and development of findings, conclusions and recommendations. The study team will use a structured and systematic approach to analyze the qualitative data and will triangulate across the multiple qualitative methods and data sources available to ensure the reliability and validity of results. The study team will analyze the qualitative data as follows: • Summarize notes from individual interviews and FGDs; • Code KII and FGD data according to themes relevant to the evaluation questions; • Prepare an evaluation data analysis matrix identifying the themes that emerge in the KIIs and FGDs to facilitate systematic and rigorous data analysis aimed at identifying key study findings as they relate to the evaluation questions; and • Prepare a detailed outline summarizing key findings based on all the data analysis for each study question and overall recommendations. SOCIAL AND GENDER ANALYSIS Among other criteria, and to the extent possible, the evaluation team will explore how the allocation of funds/ loans/ business opportunities, and provision of trainings and other capacity building activities may have affected the entrepreneurial potential of male and female beneficiaries differently, with a focus on resource allocation, access and control to resources and assets, participation and decision-making opportunities and development of entrepreneurial skills. This will be based on the data from investment officers and activity managers, as well as FGDs with female entrepreneurs from the women-angels group, themselves. EVALUATION LIMITATIONS Since this is a mid-term evaluation and MENA II closes in September 2019, MENA II partners and USAID will have ample time to benefit from the evaluation findings. However, the timing of the evaluation has implications for data collection. The data for the evaluation will be collected between November 6 and November 17 2017. Considering the profile of the different stakeholders involved in the project, many have heavy work and traveling schedules. In order to maximize turnout, the evaluation team will invite up 8 participants for each FGD (where possible) to ensure at least 4 to 6 people 8 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION show up. It will also arrange the schedule to meet the traveling needs and schedules of participants. For the KIIs, the evaluation team will create a list of 30-32 stakeholders in order to ensure that at least 25 people are interviewed. Where possible, combined interviews for staff and partners will be carried to maximize on the use of time. Six out of the twelve portfolio companies have been with IMC between 12 to 24 months. The remaining range between 3 to 12 months. The evaluation team will meet with around 6 portfolio companies that fall within the first and second category. EVALUATION DESIGN MATRIX Table 1: Evaluation Design Matrix Evaluation Criteria and Question Sub-topic Indicators and Data Sources Data Collection Methods Data Analysis Plan Effectiveness: To what extent is MENA II approach and implementation effective? How does the strategic approach shape incentives? they likely to lead to the outcomes sought? - project documents: CA, Annual Reports, monitoring data. - USAID Staff: EG office, AOR - MENA II Staff - Portfolio companies, Matching Investors, Angel Investors, Service Providers - Outcome indicators: DO 2a (new inv’t), DO 2b (jobs), DO 2c (women), IR 2.1.c (sales) - Output indicators: 1.1.a and 1.1.b (matching inv’t), 1.2.a and 1.2.b (eq g’tees), 2.1.a and 2.1.b (TA) - Intermediate results: 2.1.2 (links), 2.1.3.a (individual TA), 2.1.b (SME TA), 2.2.a (matching inv’t), 2.2.2.b (equity inv’t generated) Desk review Map MENA II workplan to semi-annual and annual reports Interview coding What is known about how approach working to date through MENA II implementation? KIIs Which activities have been more successful in achieving objectives, what were the primary factors responsible for relative success? 9 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Efficiency: To what extent is the current MEL plan efficient in informing project design? - USAID EG office - MENA II AOR - MENA II staff - MENA II MEL plan and monitoring data reports KIIs FGDs Desk Review What factors influenced the ability of the MEL plan to be effective? Interview coding Comparison of plans to implementation Is the timeframe of the project sufficient to conduct proper monitoring and evaluation of the project? What are the MEL recommendations for future projects? Sustainability: What combinations of activities or interventions have more factors in place for sustainability What activities are more likely to create impact over the long￾run? - USAID EG office - Project Stakeholders: portfolio companies matching investors, Angel investors, partners. - Other donors: World bank - BDL - Indicators: IM 1.2.c (reflows), IM 1.1.c (p/f status), IR 2.1.a (new practices) FGDs KIIs Interview coding What were the primary synergies contributing to success? Future Programming: Given the above, how confident are we that MENA II will augment USAID’s program objectives if funding increased or extended? What successes and challenges should be considered if project scales up? - USAID EG office - Project staff - Project Stakeholders: Project Stakeholders: portfolio companies matching investors, Angel investors, partners. - World Bank - BDL FGDs KIIs Interview coding On what component should project focus if it scales up? 10 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION EVALUATION MANAGEMENT DELIVERABLES, TIMELINE, AND DISSEMINATION PLAN As set out in the initial SOW, the ET will deliver the following to USAID: Deliverable Expected Delivery Date Scope of Work: PMSPL II team will prepare the SOW including methodologies, tools, and evaluation work plan and time schedule. The SOW will be submitted to the COR at USAID/Lebanon for approval Completed and approved on September 26, 2017 Inception Report: PMSPL II team and Evaluation team will conduct a systematic literature search and assembling key documents. Both teams will also prepare a draft methodology plan including sampling methodology, beneficiary matrix, primary research questions, interview protocols, focus group questions and a list of proposed individuals to be interviewed. The methodology plan, interview schedules, and interview protocol will be presented to USAID staff prior the start of the evaluation. To be submitted to USAID Nov 6, 2017 for preliminary approval Briefings: The evaluation team will provide a briefing to USAID prior to the start of the evaluation and at the end of the evaluation. The initial briefing will focus on presenting the evaluation team plan for conducting the evaluation. Discussions with the Mission on the proposed methodologies for the evaluation will result in an Inception Report (a deliverable) that states the agreed methodologies. Toward the end of the field work, prior to the development of the Draft Report, the team will carry out a preliminary findings presentation at USAID. In addition to this, a final briefing meeting will be done after the final report is written. Initial in-briefing with USAID conducted on Nov 3rd, 2017 Preliminary Findings presentation scheduled on Nov 21, 2017 Draft Report and outline of the Final Report: A draft report of the findings and recommendations will be submitted to USAID COR clearly describing findings, conclusions, and recommendations, within two weeks of completing the data collection. USAID will provide comments on the draft report within one week of submission. The evaluation team will consider USAID comments and revise the draft report accordingly and as appropriate for an independent evaluation. Where differences of opinion exist these will be presented in a Statement of Differences Annex. Expected delivery Dec 8th with USAID comments to be provided by Dec 15 2017 Final Report: The final evaluation report will be provided to USAID in MS Word and Adobe PDF within 7 calendar days following receipt of final comments from USAID. The report will include all of the components outlined in the “Structure of the Evaluation Report”, but not necessarily in the order specified below. The report will not exceed 25 pages (excluding appendices). Appendices should at a minimum include the scope of work for the evaluation; a list of individuals interviewed; a complete description of the Expected delivery of final report on Dec 22nd subject to receipt of comments from 11 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION methodology used for the evaluation; and any survey or questionnaires used. The report will be submitted in English, electronically in MS Word format and compliant with USAID Graphic Standards. USAID by Dec 15th. All deadlines assume USAID’s ability to give comments on the inception report and draft evaluation report on the timelines discussed during the initial kickoff meeting. These deadlines and other evaluation activities are illustrated in the work plan found on the following two pages. PMPSL will upload the final approved evaluation report to the DEC for public access and distributed to MENA II and other stakeholders at the discretion of USAID/Lebanon. USAID will share the report with other stakeholders at its discretion. The report will follow USAID branding requirements. Comments received after the budgeted end date for this evaluation may be addressed in a Statement of Differences Annex at the discretion of USAID/Lebanon. 12 | INCEPTION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION WORKPLAN 13 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION TEAM STRUCTURE The evaluation team will consist of a Team Leader, USAID EG Director, USAID Senior Economic Growth Specialist, and technical support from PMSPL II evaluation team. SI’s head office will support evaluation management and quality. 1) Evaluation Team Leader, Dr. David Apgar: Dr. Apgar is responsible for coordinating and directing the overall evaluation, including preparation and submission of the draft and final evaluation reports. He also contributes quality assurance to ensure that the evaluation follows SI’s EQUI® process (45 days LOE) 2) USAID EG Director, Dr. Bill Butterfield, has requested participation in formulating questions for the evaluation SOW, in designing key informant interview questions, and, as time permits, participating in interviews with key informants, and in focus group discussions. Dr. Butterfield will work with the team on the data analysis, and articulations the findings of the mid-term evaluation. 3) Senior Economic Growth Specialist, Dr. Georges Frenn, will participate in the evaluation process, as a representative of the Economic Growth Office. 4)PMSPL II evaluation team: PMSPL II local staff assigned to the evaluation supports in preparing the evaluation SOW, Inception report. It also provided logistic and admin support during field work preparation. PMSPL evaluation team will participate in conduction the field work, and the data analysis. PMSPL II resident staff and HO staff participates in drafting, reviewing and editing the draft final report before circulating to the stakeholders for comment and review. Preparation of the evaluation report is the responsibility of the team leader/technical expert. Final editorial responsibility and quality control for the report is with Social Impact. Comments received are to be address by the technical expert supported by the PMSPL II staff. The draft evaluation report is to be reviewed by USAID/Lebanon and, at the request of USAID/Lebanon, a broad range of stakeholders. PMSPL II will certify that there is no conflict of interest or potential conflict of interest with respect to the performance of this evaluation on the part of the contractor and the contractor’s team members. Social Impact will guarantee that substitutions will not be made for individuals proposed as team members without the approval of USAID. 15 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION ANNEX 1: SOW SOW MENA II evaluation - clean co 16 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA 22201 Suite 1000 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socialimpact.com 42 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION ANNEX 3: DOCUMENTS REVIEWED List of Key Documents Reviewed 1 MENA II Contract Agreement 2 Project Modifications Documents 3 Quarterly Report 4 Work Plans 5 MEL Plan ANNEX 4: DATA COLLECTION INSTRUMENTS It should be noted that these protocols are general guides with questions to ask across respondents/groups from each category of stakeholders, such that certain results can be compared across these groups. Depending on the stakeholder, particular questions may be omitted if they are not relevant. Ultimately, protocols will be specifically tailored for each respondent and group as more relevant details become available. The protocols listed here are the ones that were finalized during the inception report development phase. KII Guides Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol IMC Staff Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. 43 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? What is IMCs’ coverage ratio for the guarantees? How was the 10% stake as a price for the guarantee calculated? Explain how the guarantee claims are paid out. Could the USG potentially be on the hook for more than anyone is anticipating? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? What is the current amount of leverage of USAID funds? Would these co-invested funds have been available without USAID participation? What other forms of additionality are associated with USAID funds? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? 44 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 2. To what extent current MEL plan efficient in informing project design, implementation? a. What factors influenced the ability of MEL plan to be effective? b. Is time frame of project sufficient to conduct proper M&E of it? c. What MEL recommendations for future projects? 3.What combinations of activities, interventions have more factors in place for sustainability? 45 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. What factors are in place and what factors are missing to ensure sustainable exits? (This is for startups, matching investors, ecosystem providers, mentors, IMC staff, and the World Bank.) Are there enough independent venture investors in Lebanon to sustain an enterprise economy? If not, what changes are needed to bring diaspora and foreign investors into the market? (This is for the BDL, matching investors, IMC staff, and the World Bank.) Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 46 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol IMC AIC Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? 47 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? What is IMCs’ coverage ratio for the guarantees? How was the 10% stake as a price for the guarantee calculated? Explain how the guarantee claims are paid out. Could the USG potentially be on the hook for more than anyone is anticipating? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? What is the current amount of leverage of USAID funds? Would these co-invested funds have been available without USAID participation? What other forms of additionality are associated with USAID funds? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? 48 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. What factors are in place and what factors are missing to ensure sustainable exits? (This is for startups, matching investors, ecosystem providers, mentors, IMC staff, and the World Bank.) Are there enough independent venture investors in Lebanon to sustain an enterprise economy? If not, what changes are needed to bring diaspora and foreign investors into the market? (This is for the BDL, matching investors, IMC staff, and the World Bank.) Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? 49 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 50 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol IMC BOD Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? 51 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? What is IMCs’ coverage ratio for the guarantees? How was the 10% stake as a price for the guarantee calculated? Explain how the guarantee claims are paid out. Could the USG potentially be on the hook for more than anyone is anticipating? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? What is the current amount of leverage of USAID funds? Would these co-invested funds have been available without USAID participation? What other forms of additionality are associated with USAID funds? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 52 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. What factors are in place and what factors are missing to ensure sustainable exits? (This is for startups, matching investors, ecosystem providers, mentors, IMC staff, and the World Bank.) Are there enough independent venture investors in Lebanon to sustain an enterprise economy? If not, what changes are needed to bring diaspora and foreign investors into the market? (This is for the BDL, matching investors, IMC staff, and the World Bank.) Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? 53 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 54 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Angel Investor Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? 55 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 56 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION 57 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Angel Investor Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? 58 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 59 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Ecosystem Partner Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? 60 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? 3.What combinations of activities, interventions have more factors in place for sustainability? What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 61 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Hosting Partner Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? 62 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 3.What combinations of activities, interventions have more factors in place for sustainability? a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 63 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol USAID Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? Responsibilities 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? 64 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? What is IMCs’ coverage ratio for the guarantees? How was the 10% stake as a price for the guarantee calculated? Explain how the guarantee claims are paid out. Could the USG potentially be on the hook for more than anyone is anticipating? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? What is the current amount of leverage of USAID funds? Would these co-invested funds have been available without USAID participation? What other forms of additionality are associated with USAID funds? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? 65 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 2. To what extent current MEL plan efficient in informing project design, implementation? a. What factors influenced the ability of MEL plan to be effective? b. Is time frame of project sufficient to conduct proper M&E of it? c. What MEL recommendations for future projects? 3.What combinations of activities, interventions have more factors in place for sustainability? What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. a. What activities more likely to create impact over long run? 66 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 67 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Portfolio Company Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? What is the current amount of leverage of USAID funds? Would these co-invested funds have been available without USAID participation? What other forms of additionality are associated with USAID funds? 68 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 3.What combinations of activities, interventions have more factors in place for sustainability? a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 69 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol World Bank Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? 70 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? 71 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. What factors are in place and what factors are missing to ensure sustainable exits? (This is for startups, matching investors, ecosystem providers, mentors, IMC staff, and the World Bank.) Are there enough independent venture investors in Lebanon to sustain an enterprise economy? If not, what changes are needed to bring diaspora and foreign investors into the market? (This is for the BDL, matching investors, IMC staff, and the World Bank.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 72 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Central Bank Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? Are there investments outside circular 331 and donor supported (USAID/IM, World Bank/isme)? And how much does circular 331 related funds represent in the ecosystem? 73 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? b. What is known about how approach working to date through MENA II implementation? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? Is there a lack of liquidity in the market in Lebanon that prevents the private sector from investing in start-ups without matching capital from USAID? Reports say that banks have ample reserves that these reserves were not being put to use so Circular 331 creates no crowding out effect. Is this true, and if so, what were the major constraints to bank lending to VC firms? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What is the current role of incubators/accelerators in the ecosystem? What particular constraints do they face to becoming profitable? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? 74 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What is the current and future need for investment finance in Lebanon? We understand the market for equity was quite limited prior to Circular 331 but now it is growing quite fast. Is this true? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? A few similar equity guarantee programs are generally perceived to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. Are there enough independent venture investors in Lebanon to sustain an enterprise economy? If not, what changes are needed to bring diaspora and foreign investors into the market? (This is for the BDL, matching investors, IMC staff, and the World Bank.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 75 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Training Partner Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? 76 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 3.What combinations of activities, interventions have more factors in place for sustainability? a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 77 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Qualitative Interview Protocol Non Matching Investor Introduction: We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. Your answers will be used for data analysis purposes only and will be kept strictly confidential. No Interview respondents will be identified by name in the report. Before the interview begins, I would like to ask your permission to take notes to capture your responses and to tape record the interview so that we can further enhance our notes for data analysis. Organization: Date: Respondent’s Name: Venue: Official Title: Interviewee: Note-taker: Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? 78 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? Are the valuations (by VC funds, and IM) logical, under or over-valuated? (see Q&A with Endeavor CEO Linda Rottennburg in Executive Magazine issue of August 2017, p.58-59) What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 3.What combinations of activities, interventions have more factors in place for sustainability? How is Circular 331 going? It seems as if it receives mainly positive reviews, but it seems like there is a lot of risk piling up on BDL’s balance sheet. “globally VCs typically look at an average of 1,000 companies to get to a portfolio of 10. “Of these ten, one or two generate virtually all the returns for the fund.” Are there even close to that many fundable startups in Lebanon? All considered, the Yozma program in Israel and other similar equity guarantee programs are generally perceived to be or to have been successful. Almost seems like a free lunch? Why haven’t these schemes been hit with widespread failure in the past? Why do you not expect 331 to run into major problems? I’m thinking of Fannie and Freddie Mac here in the US… that turned out to be a disaster with all that risk covered by US taxpayers. a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 79 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Focus Group Discussion Protocols Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Focus Group Discussion Protocols We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. No respondents will be identified by name in the report; therefore you input will be strictly anonymous It should be noted that these protocols are general guides with questions to ask across respondents/groups from each category of stakeholders, such that certain results can be compared across these groups. Depending on the stakeholder, particular questions may be omitted if they are not relevant. Ultimately, protocols will be specifically tailored for each respondent and group as more relevant details become available. These protocols were finalized during the inception report development phase. Introduction: Please briefly introduce yourselves, name of your school/organization, your role within your school/organization, and the kind of training/coaching/support (name and duration) you received. FGD # __________ Date _______________ FGD Group: _____________________________Total Participants: Male _______Female _______ Location (facility & city/caza): ________________ Moderator: _____________________________Note-taker:_____________________________ Focus group discussions with SEED 2016 DEAL LEADERS Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? 80 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? 81 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Focus Group Discussion Protocols We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. No respondents will be identified by name in the report; therefore you input will be strictly anonymous It should be noted that these protocols are general guides with questions to ask across respondents/groups from each category of stakeholders, such that certain results can be compared across these groups. Depending on the stakeholder, particular questions may be omitted if they are not relevant. Ultimately, protocols will be specifically tailored for each respondent and group as more relevant details become available. These protocols were finalized during the inception report development phase. Introduction: Please briefly introduce yourselves, name of your school/organization, your role within your school/organization, and the kind of training/coaching/support (name and duration) you received. FGD # __________ Date _______________ FGD Group: _____________________________Total Participants: Male _______Female _______ Location (facility & city/caza): ________________ Moderator: ______________________________Note-taker: _____________________________ Focus group discussions with Women Angle Investment Fund (WAIF) b. On what components should project focus if it scales up? Questions Sub-Questions / USAID questions Prompts 82 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? Why just KBE firms? Was this a good decision to limit the scope of the program? What other sectors have high potential for Lebanon? a. How does strategic approach shape incentives? Are they likely to lead to outcomes sought? I was once told that the private sector rarely needs outside capital from the likes of an entity such as USAID. If private investors see a profitable investment opportunity, they will take it for themselves, not share the returns with outside capital. What is the need for direct/matching investments? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important are business angels to the investment finance ecosystem? Can you explain their role in a bit more detail and how IMC is playing a role in the establishment of BA networks? What makes IMC attractive to investees/startups? What is IMC offering that is different from other VC funds on the market? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? Would business angels have invested without guarantee insurance? Would they have invested without technical assistance? Would they have invested without both? (Please note this question is for IMC staff and angel investors.) 3.What combinations of activities, interventions have more factors in place for sustainability? Is the angel network sustainable without subsidized technical assistance? Is it sustainable without subsidized investment insurance? Is it sustainable without both? (This is for angels and IMC staff.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? 83 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Middle East and North Africa Investment Initiative (MENA II) Mid-Term Evaluation Focus Group Discussion Protocols We work for Social Impact, a USAID contractor that provides performance monitoring and evaluation support for various development projects in Lebanon. The purpose of this interview/focus group discussion is to gain further knowledge and insight into the work of the USAID-sponsored Middle East and North Africa Investment Initiative (MENA II). As a knowledgeable person within your organization we value your input to our evaluation. We anticipate that your responses will help us gain further knowledge of MENA II contributions to economic growth sector in Lebanon, understand challenges faced during implementation, and develop recommendations for future programming. Your participation is completely voluntary. No respondents will be identified by name in the report; therefore you input will be strictly anonymous It should be noted that these protocols are general guides with questions to ask across respondents/groups from each category of stakeholders, such that certain results can be compared across these groups. Depending on the stakeholder, particular questions may be omitted if they are not relevant. Ultimately, protocols will be specifically tailored for each respondent and group as more relevant details become available. These protocols were finalized during the inception report development phase. Introduction: Please briefly introduce yourselves, name of your school/organization, your role within your school/organization, and the kind of training/coaching/support (name and duration) you received. FGD # __________ Date _______________ FGD Group: ___________________________Total Participants: Male _______Female _______ Location (facility & city/caza): ________________ Moderator: ______________________________Note-taker: _____________________________ Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 84 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Focus group discussions with MENTORS Questions Sub-Questions / USAID questions Prompts Could you describe your role within your organization and your involvement in MENA II? - -How long have you served in this position? -How long have you been involved with MENA II and in what ways? 1. To what extent MENA II approach, implementation effective? c. Which activities are more successful in achieving objectives, what were the primary factors responsible for relative success? How important is technical assistance to the investees/entrepreneurs? What have been the most effective/least effective methods of providing TA to investees? What are the business enabling environment constraints that most affect the start-up and equity finance ecosystem? Is the project doing anything to alleviate those constraints? 3.What combinations of activities, interventions have more factors in place for sustainability? What factors are in place and what factors are missing to ensure sustainable exits? (This is for startups, matching investors, ecosystem providers, mentors, IMC staff, and the World Bank.) a. What activities more likely to create impact over long run? b. What were primary synergies contributing to success? 4.Given above, how confident that MENA II will augment USAID program objectives if funding increased or extended? Is the positioning of IMC within the ecosystem optimal? What would be the recommendations for IMC positioning within the ecosystem for the coming 5 years? a. What successes, challenges should be considered if project scales up? b. On what components should project focus if it scales up? 85 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION 86 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION ANNEX 5: MENA II INTENDED RESULTS/SCOPE OF WORK USAID/Lebanon Goal (DO2): Inclusive Economic Growth Enhanced pain (DO 2a) Value of new private sector investment leveraged by USAID MENA II assistance (DO 2b) Number of new or improved jobs attributed to USAID MENA II assistance (DO2c): Percentage of female participants in USG-assisted programs designed to increase access to productive economic resources USAID PMP IR 2.2: Increased access to finance, espacially for new business start-ups and women MENA II Purpose A: Increase Access to Finance for Startups USAID PMP IR 2.1: Increased Private Sector Competitiveness MENA II Purpose B : Increase Private Sector Competitiveness (IR2.2a) Number of entreprises that have successfully accessed loans, private equity, or both as a result of USAID assistance IR 2.1 a: Number of beneficiaries who have applied new technologies or management practices as a result of USAID MENA II assistance IR 2.1b: Number of MSMEs receiving business development services from USAID MENA II assistance IR 2.1c: Value of sales collected firm-level for MSMEs as a result of USAID MENA II USAID PMP Sub IR2.2.2: Increase early stage investment financing for new and existing firms USAID PMP Sub IR 2.1.3: Increased workforce development linked to job opportunities MENA II Sub-purpose A.1: Increased investment in Qualified Early Stage Businesses MENA II Sub-purpose B.1: Increased linkages between start-ups, investors and international Sub-IR 2.2.2b: Total Value of debt equity financing generated by entreprises as a result of USG assistance Sub IR 2.1.2: Number of MSMEs and other organizations benefiting from new horizontal and vertical linkages MENA II Sub-purpose B.2: Increased Capacity of different SUb-IR 2.1.3a number of recipents benefiting from worforce development efforts (skills training and/or education) as a result of USG USAID PMP Sub IR 2.1.2: Increased business and trade linkages 87 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Output A.1.1: Lebanese Startups Accelerator initiative supported Output A.1.2: Major Incubators and Business Development Centers supported through matching capital Output A.1.3: Seed Capital Funds supported through co-financing Output A.1.4: Cash Collateral investment Insurance provided to encourage investment in QESBs Output A.1.5: MSMEs (QESBs) supported through co￾financing Output B.1.1: Angel & BDC Network Set Output B.1.2: Access to international linkages and markets facilitated Output B.2.1: Business incubators supported in improving their performance. Output B.2.2: SMEs trained on Innovation Management Output B.2.3: Mentor Network Program Established Equity matching & Equity guarantee & Technical Assistance IM 1.1.b Equity matching pay out rate IM 1.2.b Equity Guarantee pay out rate IM 1.1.a Rate of commitment awarded funds IM 1.1.c Status of the investment portfolio IM1.2.a Value of USG use for investment insurance IM 1.2.c. Investment cash flow including reflows IM 2.1.aTechnical A ssistance pay out rate IM 2 1 b U f f d f TA i l di ib i b i i 88 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION ANNEX 6: LIST OF MENA II INDICATORS 89 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION RF ref Indicator (with disaggregation) Target Unit Reporting Frequency Baseline Year I (Oct 2014 to Sept 2015) Year II (Oct 2015 to Sept 2016) Year III (Oct 2016 to Sept 2017) Achieved Semi￾Annual Achieved Annual Target Annual Achieved Semi￾Annual Achieved Annual Target Annual Achieved Semi￾Annual Achieved Annual Target Annual USAID Lebanon Mission Overall CDCS Goal Objective: Advancing development of the private sector by increasing competiveness and revenue growth of Qualified Early Stage Businesses CDCS Goal: Improved accountability and credibility of public institutions, and broader economic prosperity Project Goal: DO 2: Inclusive economic growth enhanced DO 2a Value of new private sector investment leveraged by USAID MENA II assistance USD Semi-Annual 0 0 $ 1,925,000 $ 3,000,000 $ 3,700,000 $ 4,390,000 $ 3,000,000 $ 1,300,000 $ 3,175,000 $ 3,000,000 DO 2b Number of new or improved jobs attributed to USAID MENA II assistance FTE Semi-Annual 0 0 10 40 31 69 40 39 72 40 Female - - 4 - 7 21 - 9 22 - Male - - 6 - 24 48 - 30 50 - 90 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION DO 2c Percentage of female participants in USAID MENA II assisted programs designed to increase access to productive economic resources % Semi-Annual 0 0 26% N/A N/A 40% N/A 32% 35% N/A Age 10 -29 - - - - - - - - - - >30 years - - - - - - - - - - IR 2.1 a Number of beneficiaries who have applied new technologies or management practices as a result of USAID MENA II assistance (Cumulative) Number Semi-Annual 0 0 3 5 5 9 10 11 14 15 Male - - 3 - 5 9 - 9 9 - Female - - 0 - 0 0 - 2 5 - Joint - - - - - - - - - - IR 2.1 b Number of Number Semi-Annual 0 0 10 5 20 15 14 26 20 91 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION MSMEs receiving business development services from USAID MENA II assistance 10 Size: Micro – 1 to 10 - - - - - - - - - - Size: Small – 11 to 50 - - - - - - - - - - Size: Medium – 50 to 100 - - - - - - - - - - IR 2.1 c Value of sales collected firm￾level for MSMEs as a result of USAID MENA II USD Semi-Annual - - $ 65,000 $ 150,000 $ 443,360 $ 738,000 $ 300,000 $ 788,173 $ 1,212,726 $ 450,000 Local Sales - - - - - - - - - - Exported Sales - - - - - - - - - - IR 2.2: Increased access to finance, especially for new business start-ups and women IR 2.2 a Number of enterprises that have successfully accessed loans, private equity, or both as a result of USAID assistance Number Semi-Annual 0 0 10 10 8 15 20 5 22 20 Beirut/Male - - 8 - 4 12 - 3 17 - 92 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Beirut/Female - - 0 - 2 2 - 2 5 - Beirut/Joint - - 2 - 1 1 - - - Sub IR 2.1.2 Number of MSMEs and other organizations benefiting from new horizontal and vertical linkages Number Semi-Annual 0 0 0 2 2 2 3 6 2 Sub-IR 2.1.3a Number of recipients benefiting from workforce development efforts (skills training and/or education) as a result of USG support Number Semi-Annual 0 0 17 20 126 60 79 199 60 0 Beirut/Male - - 14 - - 65 - 28 136 - Beirut/Female - - 3 - - 61 - 19 63 - Beirut/Joint - - - - - - - - - - Recurring - - 0 - - 0 - 26 26 - New - - 17 - - 126 - 53 173 - Sub-IR Total Value of USD Semi-Annual $ $ $ $ $ $ $ $ 93 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION 2.2.2b equity financing generated by enterprises as a result of USG assistance - - 1,075,000 2,000,000 700,000 1,990,000 2,000,000 2,250,000 2,805,000 2,000,000 Male - - $ 1,075,000 - $ 700,000 $ 1,200,000 - $ 1,250,000 $ 1,565,000 - Female - - - - - $ 400,000 - $ 1,000,000 $ 1,160,000 - Joint - - - - - $ 390,000 - - $ 80,000 - Custom Pay Out Rate % Semi-Annual IM 1.1 b Equity Matching 0% 0% 2.97% 8.00% 4.67% 9.38% 8.00% 2.82% 8.56% 8.00% IM 1.2 b Equity Guarantee 0% 0% 0.00% 0.00% 0.00% 2.60% 6.67% 0.00% 0.00% 6.67% IM 2.1 a Technical Assistance 0% 0% 0.41% 2.67% 0.13% 0.92% 2.67% 0.35% 0.88% 2.67% IM 1.1 a Rate of commitment of awarded funds % Semi-Annual 0% 0% 7.2% 13% N/A 13% 13% 15% 12% 13% IM 1.2 c Investment cashflows including reflows USD Semi-Annual incoming 0 0 $ - $ - $ - $ - $ - 0 outgoing 0 0 $ 445,000 $ 1,200,000 $ 700,000 $ 1,407,000 $ 1,200,000 $ 423,333 94 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION IM 1.1 c Status of the investment portfolio (cumulative) N/A Semi-Annual active 0 3 active 5 5 7 active 10 9 active 10 15 0 0 exited 0 0 0 exited - 0 exited 0 0 IM 1.2 a Value of USG used for investment insurance USD Semi-Annual 0 0 0 0 0 $ 195,000 $ 1,000,000 0 $ 392,500 $ 1,000,000 IM 2.1 b Use of funds for technical assistance including contributions by recipients USD Semi-Annual incoming 0 0 0 0 $ 30,000 $ 62,500 0 $ 40,000 outgoing 0 0 $ 62,000 $ 400,000 $ 20,000 $ 138,675 $ 400,000 $ 52,250 95 | FINAL EVALUATION REPORT USAID/LEBANON MENA II MIDTERM PERFORMANCE EVALUATION Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA 22201 Suite 1000 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socialimpact.com