LEBANON ENTERPRISE DEVELOPMENT (LED) MIDTERM PERFORMANCE EVALUATION FINAL REPORT October 2019 LED BENEFICIARY IN QAB ELIAS. This publication was produced at the request of the United States Agency for International Development. It was prepared by Dr. Michael Ferguson (Team Leader) with the support of Ms. Maya Trad (Local Consultant) and Dr. Rana Taher and Ms. Samar Safar from Social Impact, Inc. - Note: Limited redactions have been made to this version of the report in accordance with the principled exceptions to the presumption in favor of openness established in OMB Bulletin 12-01, “Guidance on Collection of U.S. Foreign Assistance Data.” USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | ii LEBANON ENTERPRISE DEVELOPMENT (LED) MIDTERM PERFORMANCE EVALUATION FINAL REPORT October 2019 Prepared by Social Impact, Inc. for the United States Agency for International Development under USAID Contract/Order No. AID-268-C-15-00001 – Performance Management and Support Program for Lebanon (PMSPL II). This document is not available in print. Documents of this nature are made available to the public through the Development Experience Clearinghouse (DEC) repository at dec.usaid.gov. For additional information, please contact: Social Impact, Inc. Corporate Office 2300 Clarendon Boulevard, Suite 10300 Arlington, VA, 22201 Tel: +703-465-1884 Fax: +703-465-1888 info@socialimpact.com or Social Impact, Inc. Arz Street Librex Bldg. Bloc B—3rd Floor Zalka, Metn, Lebanon Tel: +961-1-879260 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iii | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ABSTRACT The Lebanon Enterprise Development (LED) is a three-year, $14-million activity aimed at increasing employment opportunities for Lebanese citizens by employing a Buyer-Led Approach (BLA) to create jobs with micro, small, and medium enterprises (MSMEs). This evaluation aims at assessing LED’s performance to date and make a recommendation on exercising the extension option in the contract. The evaluation team (ET) carried out a total of 63 key informant interviews (KIIs) and analyzed the project’s monitoring, evaluation, and learning (MEL) database. We found the BLA consistent with the USAID/Lebanon’s Country Development and Cooperation Strategy (CDCS), though with some questions around a range of interventions beyond firm-level TA. There is evidence of strong growth in jobs, sales, and investment, mostly attributable to LED, but job targets are very unlikely to be reached without a longer time horizon. The biggest perceived constraint was the Lebanese economy. The most cost-effective sectors were service industries, and the most cost￾effective technical assistance (TA) was product/design development. The industrial and manufacturing sector, human resources (HR), and ISO certification appear to be the most scalable. There is a limited window on sustainability because of the project’s recent start. Our preliminary recommendations include: (1) revising targets to reflect time lag on jobs, including possibility of post–period of performance (PoP) measurements; (2) loosening criteria for MSME selection and loosening rules during assistance to reach targets in PoP; and (3) conferring and setting deliverables around Objective 2, the business enabling environment (BEE). USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | iv ACKNOWLEDGMENTS We would like to thank PMSPL II’s staff, LED’s staff, and all those that have helped facilitate the work on the midterm evaluation. Evaluation Team Dr. Michael Ferguson [Team Leader] Maya Trad [Local Consultant] Dr. Rana Taher Senior [Senior M&E Specialist, Social Impact] Samar Safar [M&E Specialist, Social Impact] v | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV TABLE OF CONTENTS ABSTRACT III ACKNOWLEDGMENTS IV TABLE OF CONTENTS V LIST OF TABLES AND FIGURES VI ACRONYMS VII EXECUTIVE SUMMARY VIII EVALUATION BACKGROUND 1 PROJECT BACKGROUND 1 EVALUATION PURPOSE 3 EVALUATION QUESTIONS 4 EVALUATION DESIGN, METHODS, AND LIMITATIONS 5 FINDINGS AND CONCLUSIONS 12 EVALUATION QUESTION 1 (RELEVANCE) 12 EVALUATION QUESTION 2.1 (EFFECTIVENESS) 16 EVALUATION QUESTION 2.2 (EFFECTIVENESS) 27 EVALUATION QUESTION 2.3 (EFFECTIVENESS) 32 EVALUATION QUESTION 3 (EFFECTIVENESS) 36 EVALUATION QUESTION 4 (SUSTAINABILITY AND SCALABILITY) 49 EVALUATION QUESTION 5 (GENDER) 54 SUMMARY FINDINGS AND CONCLUSIONS 57 RECOMMENDATIONS 62 ANNEXES 65 ANNEX I: INCEPTION REPORT 65 ANNEX II: EVALUATION DESIGN MATRIX 66 ANNEX III: KII PROTOCOLS 68 ANNEX IV: INFORMATION SOURCES 77 ANNEX V: KIIS BY CATEGORY 80 USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | vi LIST OF TABLES AND FIGURES Table 1. Distribution of KIIs—Clients and Staff..................................................................................................... 7 Table 2. Overview of Committed and Actual Jobs (as of 9/19/2019)..............................................................17 Table 3. Progress on Jobs against Targets............................................................................................................18 Table 4. Progress on Incremental Sales and Leveraged Investment against Targets.....................................18 Table 5. Cost Share Contributions by Type of TA .............................................................................................19 Table 6. Disaggregated Data on Cost-Effectiveness............................................................................................34 Table 7. Progress to Date on Job Targets with Y3 Projections........................................................................37 Table 8. Reasons behind “Lost Deals”...................................................................................................................41 Table 9. Committed Jobs, Pledged Sales Increase, and Pledged Leveraged Investment................................52 Figure 1. Distribution of Interviews by Category .................................................................................................. 8 Figure 2. LED beneficiary in the Beqaa ..................................................................................................................20 Figure 3. LED beneficiary in the South. .................................................................................................................22 Figure 4. Y2 Actuals and Y3 Target for New Jobs ..............................................................................................38 Figure 5. LED beneficiary in Koura. .......................................................................................................................50 vii | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ACRONYMS ADS Automated Directives System ALI Association of Lebanese Industrialists AOR Agreement Officer’s Representative BA Business Advisor BEE Business Enabling Environment BIAT Business Incubator Association of Tripoli BLA Buyer-Led Approach CDCS Country Development Cooperation Strategy CLA Collaboration, Learning, and Adaptation COP Chief of Party COR Contracting Officer’s Representative DO Development Objective ET Evaluation Team (Social Impact) FTE Full-Time Equivalent GAP Good Agricultural Practices IR Intermediate Result KII Key Informant Interview KPI Key Performance Indicator LED Lebanon Enterprise Development LFA Lebanese Franchise Association MEL Monitoring, Evaluation, and Learning MOU Memorandum of Understanding MSMEs Micro, Small, and Medium Enterprises PAD Project Appraisal Document PMP Performance Management Plan PoP Period of Performance PSD Private Sector Development PSMPL II Performance Management and Support Program for Lebanon II SMDC Society of Management and Development Consultants SI Social Impact TA Technical Assistance USAID United States Agency for International Development USAID/L USAID’s Mission to Lebanon WADA Women’s Association of Deir Al Ahmar USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | viii EXECUTIVE SUMMARY PROJECT BACKGROUND The Lebanon Enterprise Development (LED) is a three-year, $14-million activity with a two-year, $10.2- million option period through September 2022. LED is implemented under a contract with Chemonics International (No. AID-268-C-17-00001) and aims at increasing employment opportunities for Lebanese citizens. LED is designed as an activity under the Private Sector Development (PSD) Project. The activity facilitates the provision of business development services and solutions to Lebanese firms, enabling them to expand activities, increase sales, and create jobs. EVALUATION PURPOSE AND QUESTIONS The purpose of this midterm evaluation is to assess LED’s performance to date in meeting the requirements set forth in the contract, reexamine the underlying design hypothesis and critical assumptions and assess their validity, analyze the additionality and sustainability of the activity, and make a recommendation as to whether or not the United States Agency for International Development’s Mission to Lebanon (USAID/L) should exercise the cost and time extension option in the contract. The evaluation looks at relevance, effectiveness, sustainability and scalability, and gender using the following main questions: Relevance: 1. Particularly for Objective 1, how relevant is the LED project with its firm-level focus, “buyer-led,” and generally market-led, “technical assistance through business consultants” and “firm-led” approach: (a) to meet the needs of the private sector in Lebanon? and (b) to meet USAID/L’s CDCS and PSD objectives? Effectiveness: 2. How and to what extent has the project achieved the planned results across its three objectives, particularly Objective 1? 3. How reasonable and achievable are the committed/forecasted numbers, both in terms of jobs to be created, sales increased, and investments to be made at individual enterprise and project-wide goals? Sustainability and Scalability: 4. What is the likelihood that the results LED has achieved so far are sustainable and scalable beyond the life of the project? Gender: 5. What are the outcomes generated as a result of LED’s gender mainstreaming? EVALUATION DESIGN, METHODS, AND LIMITATIONS The ET used primarily a qualitative approach to answer the evaluation questions. A total of 63 key informant interviews (KIIs) were conducted with key project stakeholders including USAID/L management, LED management, LED subcontractor management (Berytech and Business Incubator ix | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Association of Tripoli, BIAT), project-affiliated business advisors (BAs), project-affiliated consultants, and MSME beneficiaries involved in the project. The qualitative inquiries were supplemented with a document/desk review, including an extensive examination of quantitative data available from project monitoring. KIIs were the evaluation’s primary method and most significant source of data. This method was favored here for two primary reasons: (1) its individualistic nature; and (2) its open-ended/inductive character. FINDINGS AND CONCLUSIONS Relevance. With its focus on business interventions and job creation, the BLA is consistent with the mission’s CDCS and push for “Increased private sector competitiveness.” Compared with iterations of the model elsewhere in the world, the LED project has taken a relatively narrow firm-level approach, bypassing complementary aspects like access to finance and generally not addressing sectors or value chains. Effectiveness. After a slow start in the project’s first year (Y1), the project has rebounded strongly on jobs in project’s second year (Y2), with 992 created, though this remains well under target, particularly as measured in terms of full-time equivalents (FTEs). As with job creation, available data suggests that the project has rebounded strongly in Y2 on pledged incremental sales and pledged leveraged investment, though the MEL data on actual sales and investment appears to be incomplete and problematic. There is some indication that the process of improving and expanding business in Lebanon proceeds in a particular order; investment appears to come first, followed by job creation and sales. Beneficiaries reflected positively on what they saw as strong growth in job, sales, and investment; most saw the increases as a direct result of involvement with LED. In terms of additionality, beneficiaries reflected on a wide variety of positive changes in the ways they do business. Although many had existing plans to grow and improve, most said that LED had “accelerated” these by adding strong expertise. Respondents at all levels agreed that strong relationships between staff and firms has promoted success in the project. They agreed that the biggest constraint to success was the economy, which limits achievement at every stage, from firm enrollment to job creation numbers. Project leadership shared a vision of strong project achievement. LED operates with a wide range of criteria for firm participation—rules that cause many firms to be rejected by the project and others to opt out. Some beneficiaries have not responded well to project requirements, from as cost-share, to minimum new-job requirements, to “constant requests” for monitoring data numbers; relatedly, some clients have complained of cash-flow challenges. By its official metric of FTE jobs, LED is well below target, with 476 FTEs near the end of Y2, where the target stands at 1,470. We find a positive sign in the current level of committed jobs, which exceeds the FTE target for the end of Y2; actual hires (not FTEs) stand at about two thirds of the target. Using projections-based key variables for Y3, the actual number of new hires exceeds the target in the more optimistic scenarios. However, in no scenario does the project reach its target on the official metric of FTEs. Success on job targets is strongly mediated by a critical time lag between the provision of TA and creation of the associated jobs and by the 12-month nature of the FTE measurement. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | x Measured in terms of project costs per job, the most cost-efficient sector was “Services,” which encompasses restaurants and hospitality, as well as most of the project’s franchised businesses. In terms of type of TA provided, product/design development emerged as most cost-efficient, followed by HR and ISO/FSSC certification. IT interventions were the most expensive. Large firms were most cost￾efficient, with efficiency decreasing as one moves down the scale of firm size. Sustainability and Scalability. Because of most firms’ relatively recent start in the project, we have a limited window on firm-level sustainability. However, there are no indications, thus far, that improvements and new jobs will not endure. We examined the potential for sustained relationships between consultants and firms beyond LED; both beneficiaries and consultants were optimistic, but only a handful of cases have emerged. We examined the potential for sustained effects on the consulting market generally and found optimism but almost no effects so far. Scalability can be considered in terms of cost-efficiency metrics (see above) or in terms of the relative volume of outputs in key areas of jobs, sales, and investment. On the volume end, two sectors— Industry & Manufacturing and Services—far eclipsed the others in terms of jobs, sales, and investment. (Relatedly, the same two sectors emerged most cost-effective in terms of jobs.) In another parallel to cost-effectiveness, large firms had an advantage over smaller firms on volume of outputs. The top three TA services linked to job creation volume were ISO certification, franchising, and HR. The project continues to struggle with several aspects of Objective 2, improving the BEE, including: (1) the general clarity of its mandate in this area; (2) how to balance Objective 2 with Objective 1; and (3) the perceived low level of resource allocation for Objective 2. Achievements to date on Objective 2 include both general BEE efforts as well as a sector-focused effort to build tourism in Lebanon. Objective 3 is seen as solely to support Objective 1. Efforts to date include a variety of outreach events, a strong social media presence, and the distribution of project materials. Gender. Though the project has shown commitment to gender analysis and training, gender has not manifested strongly in LED’s activities. Gender mainstreaming is limited to the number of females hired for new jobs. To some extent, the project targets firms that are inclined to hire women. Only a handful of women-owned enterprises have been assisted. SUMMARY CONCLUSIONS Question 1: • There is some divergence between the theory/precedent and LED’s practice for understanding and applying the BLA. • The BLA is about creating jobs, and in that sense, it is completely appropriate for a Lebanese context in which job creation can be difficult and elusive. Based on previous aid projects and relevant literature, the ET suggests that a broader sectoral focus might have produced more jobs, but the argument remains speculative. Question 2.1: • Clearly, there has been substantial progress on job creation in Y2; qualitative data on sales and investment also suggests strong progress, though the view is less complete because of challenges in the related MEL data. xi | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV • If we accept the limited amount of reliable data on sales and investment, collectively the progress toward targets suggests that the process of improving and expanding business in Lebanon proceeds in a particular order; investment appears to come first, followed by job creation and sales. • It appears this trend and configuration were not foreseen by LED’s designers, which resulted in strong and uneven discrepancies between the results and targets to date. Question 2.2: • LED has made impressive gains in terms of building relationships and goodwill in the Lebanese business community, which has helped lay a foundation for success. • The Lebanese economy, plus unpopular rules and inconsistencies in how those rules are applied (e.g., variations in cost share), have undermined success. Question 2.3: • The primary drivers of cost-efficiency are scale—that is, bigger businesses are more efficient in creating jobs. • Further disaggregation on cost-efficiency is constrained by the relatively small client population. Question 3: • The LED project will not reach its target on jobs without adaptive management. • At present, the mandate around BEE is poorly understood and functions largely as a distraction from Objective 1. Question 4: • We cannot make meaningful observations on sustainability at present. • There is every indication that firms added jobs and improved businesses with permanent intent. • Like cost-effectiveness, scalability potential is found most with larger firms. Question 5: • Gender is not a major concern as this project struggles to meet its targets around job creation. • Reorienting the project for Automated Directives System (ADS) 205 compliance will require considerable efforts. RECOMMENDATIONS • Consider revising targets to reflect that time lag on jobs has emerged as a clear and indisputable fact in this project. Alternatively, consider allowing actual jobs to substitute for FTE targets. • Take all necessary steps to compile complete and accurate data on actual incremental sales and actual leveraged investment. • Consider post-PoP measurements of key outcomes. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | xii • If USAID’s overriding concern is reaching targets in the project’s PoP, and revising the targets is not an option, consider loosening any number of criteria for MSME selection and the rules applied to MSMEs once part of the program. Meeting current targets will require aggressive push in broadening its reach to new clients. • Consider investing in a costing research expert for a fuller and more meaningful understanding of cost-efficiency across MSME types sectors. • Revisit the distribution of TA provided. Right now, certain types (e.g., franchising and ISO certification) are getting considerably more attention than others. The project should examine this strategy to make a clear determination on whether it is the most deliberate and productive way to go, and then convey those results to project stakeholders at all levels. • Alongside the project’s own TA, investigate and employ linkages to other aid/assistance projects such as UKAid, International Bank for Reconstruction and Development, or others. This may be the only way to heed call for broader approach in areas like access to finance. • LED must balance the need for flexibility (see #3) with the need for consistency. Establish clear and unambiguous rules on cost-sharing, minimum commitment for new hires, and multiple TA services. Generally, make efforts to be and more transparent about rules and benefits for participation in the LED project. • If USAID’s aspirations around Objective 2 (BEE) remain a project priority, USAID and LED must confer and make a reasonable projection of what can be accomplished in the project’s remaining time. A contract modification may be needed for higher budget allocation. • Streamline paperwork and reporting requirements. These are a source of considerable discontent in the project among some of the BAs. Alternatively, consider hiring administrative help to assist BAs with paperwork. Even among the BA who did not express discontent, these moves would help concentrate efforts on the project’s core objective of job creation. • Revisit gender analysis recommendations to determine whether the gender strategy can be deepened in line with those recommendations. 1 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV EVALUATION BACKGROUND PROJECT BACKGROUND The Lebanon Enterprise Development (LED) is a three-year, $14-million activity with a two-year, $10.2- million option period through September 2022. LED is implemented under a contract with Chemonics International (No. AID-268-C-17-00001) and aims at increasing employment opportunities for Lebanese citizens. LED is designed as an activity under the Private Sector Development (PSD) Project. The activity facilitates the provision of business development services and solutions to Lebanese firms, enabling them to expand their activities, increase their sales, and create jobs. ACTIVITY OBJECTIVES As per the contract, LED has three objectives: 1. Create jobs by increasing the competitiveness and enhancing the growth of Lebanese client firms; 2. Improve Lebanon’s business enabling environment (BEE); and 3. Communicate results through outreach and share knowledge with the public and private sectors. LED’s primary objective is to increase job opportunities for Lebanese citizens. In order to achieve this, it will increase sales and foster growth for Lebanese enterprises, thereby allowing them to create jobs. LED’s primary approach is to work with private sector firms to identify and overcome specific constraints to accessing known markets, thereby facilitating new relationships with buyers and boosting sales through the so-called “buyer-led” approach (BLA) to economic development.1 This is in turn will make these firms more competitive and lead to new hiring—with new jobs serving as a primary measure of success across the project. LED’s second and third objectives are subsidiary to the first to the extent that they support the primary objective and the overall activity purpose of job creation. The three objectives are implemented in parallel in a tightly integrated fashion to achieve the purpose of job creation. In its work, LED applies the sector-neutral BLA to the key constraints that stand in the way of Lebanese micro, small, and medium enterprises (MSMEs) consummating sales transactions, growing, and hiring more people. The major part of LED’s work is transactional, which is to say, helping MSMEs solve the business problems that constrain their growth. As a practical matter, such work involves expanding trade linkages and customer bases through the acquisition of necessary trade certifications, new modes of salesforce training, and other targeted actions. In addition, and in certain instances, LED may also facilitate job matching and internships/apprenticeships as well as enterprise-led workforce development. LED collaborates with other development partners, donors, business associations, and USAID’s Mission to Lebanon (USAID/L) programs to build synergies and leverage new opportunities to achieve optimum results. When binding constraints result from the business environment, LED partners with business 1 See James T. Riordan, (2011). “We Do Know How: A Buyer-Led Approach to Creating Jobs for the Poor.” Washington, DC: New Academic Publishing. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 2 associations to play an advocacy role. It assists in trade facilitation, export promotion, or other regulatory or nonregulatory aspects, as it relates to job creation. Above all, the initiatives under LED aim to increase sales that create new jobs, thereby contributing to private sector growth and a more inclusive economy. For analyzing project success, LED’s Monitoring, Evaluation, and Learning (MEL) team applies three criteria: • Veracity: Did the client firm create the number of jobs claimed? • Attribution: Are the jobs related causally to LED actions? • Additionality: Is it true that the increase in jobs would not have happened in the absence of LED support? As an integral part of the learning process, LED’s MEL team has been required to conduct a number of targeted analyses, studies, and assessments. One of these has been a strong focus on cost effectiveness in terms of the “cost per job” attributable to the activity. CROSS-CUTTING PRINCIPLES OF LED AND THE BLA Discipline. Discipline engenders focus, which in turn ensures the efficient use of project resources and enables the achievement of the program’s ultimate goal—job creation. Discipline begins with client selection and a commitment to strict criteria when choosing firms. For example, using a ratio of 5:1— every $1 of LED support leads to an additional $5 in sales—ensures a focus on high-value deals. Agility. The ability to move quickly on opportunities as new potential client firms emerge is critical to secure partnerships that create new jobs. LED’s business advisors (BAs) are dynamic individuals with direct private sector experience across industries. They work from LED’s antenna offices, close to the business communities they serve, and are incentivized to canvass their areas for new opportunities. Trust. A key role of LED is gaining business owners’ trust to overcome aversion to change that will lead to growth. LED’s approach enables BAs to build personal relationships with client firms, buyers, and business-development service providers, serving as the “honest broker” to facilitate deals and services. Accountability. Setting job creation performance targets among clients, in addition to overall program targets, means the entire LED team will work toward one primary goal—reaching or exceeding 17,743 jobs created by the project’s end. A strong and independent MEL team keeps everyone, beginning with the Chief of Party (COP), accountable to this goal. DEVELOPMENT HYPOTHESIS LED’s overarching development hypothesis is that increased private sector employment will contribute to increased and sustained stability in Lebanon. The underlying theory of change is that private sector jobs have not kept pace with the growing numbers of Lebanese looking for work because of specific constraints to growth among private enterprises. Lebanese enterprises will expand their workforce if and when they can address these constraints by identifying and connecting with viable existing markets. 3 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV As per the contract, LED assists enterprises chosen using selection criteria that center on the existence of new, known markets and a concrete plan for growth stemming from new connections to those markets. LED and its BAs work with clients to identify specific buyer/s and understand their requirements; diagnose the key enterprise-level constraints that stand in the way of concluding sales contracts with these buyer/s; and find solutions to these problems using tailored firm-specific solutions delivered principally by Lebanese business service providers and consultants, or (as needed) by international consultants. LED’s work is transactional in the sense of making deals happen because sales growth leads to jobs. Over time, LED prioritizes selected BEE problems and works with the private sector to analyze these issues and deliver solutions. CRITICAL ASSUMPTIONS According to the project documents, key implementation assumptions for the LED activity include: Lebanon’s political, economic, and social stability will remain at its current level or improve. Stability and the prospect of a better future will create an environment for business expansion. Without stability, international buyers and investors will fear doing business with Lebanese firms, local entrepreneurs will delay expansion plans, and access to finance will become tighter. If stability returns, local businesses may look to expand their operations. But if instability remains or gets worse, it will likely damage investor confidence and make expansion unlikely. LED will be able to deploy its resources in a timely manner. The LED contract requires that project staff, consultants, subcontractors, and grantees receive various USAID/L approvals as well as pass robust vetting rules. LED will ensure rapid processing of client enterprises’ support needs, and assumes timely approvals and vetting results from USAID/L. USAID/L is supportive of Collaborating, Learning, and Adapting (CLA) as LED progresses. A necessary criterion for LED’s success is being responsive to evolving private sector needs and Lebanese economic realities. This may require adaptive course corrections during implementation per CLA principles. Adaptive actions that shorten the path to LED goals will be presented to USAID/L for discussion and approval, in the analysis/reporting phase and as available during the fieldwork phase. EVALUATION PURPOSE The purpose of the LED midterm performance evaluation is to assess the relevance, effectiveness, and sustainability of the project and its expected results. The evaluation provides pertinent information about the project’s results thus far, lessons learned, and practical recommendations, including the corrective actions (if any) required for maximizing the project’s benefits. Specifically, the evaluation will assess LED’s performance to date in meeting the requirements set forth in the contract, reexamine the underlying design hypothesis and critical assumptions and assess their validity, analyze the additionality and sustainability of the activity, and make a recommendation as to whether or not USAID/L should exercise the cost and time extension option in the contract. The primary audience for the evaluation includes USAID/L, particularly the Economic Growth Office, and LED’s implementer, Chemonics International. Secondary users include chambers of commerce, industry, and agriculture; and other donors and organizations working on economic growth projects in Lebanon. The evaluation answers the following questions as identified by USAID/L: USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 4 EVALUATION QUESTIONS EVALUATION QUESTION 1 (RELEVANCE) Particularly for Objective 1, how relevant is the LED project with its firm-level focus, “buyer led,” and generally market-led, “technical assistance through business consultants” and “firm-led” approach: (a) to meet the needs of the private sector in Lebanon? and (b) to meet USAID/L’s Country Development and Cooperation Strategy (CDCS) and PSD objectives? This question addresses the following: • What is the overall and high-level relevance of the project to enterprises’ needs? • What are key constraints that stand in the way of Lebanese MSMEs increasing sales transactions, growing, and hiring more people? • What are ways in which LED objectives and performance indicators support USAID/L’s CDCS development objectives (DOs) and PSD objectives? EVALUATION QUESTIONS 2 AND 3 (EFFECTIVENESS) How and to what extent has the project achieved the planned results across the project’s three objectives, but particularly on Objective 1? The answer to this question explores the following subquestions: 2.1 What evidence is there that employment has increased in targeted enterprises, and sales (and investments) were enhanced as a result of USAID/L assistance (veracity and attribution)? To what extent can we say that these gains are the result of program “additionality?” 2.2 What are the main factors that influenced positively or negatively LED’s ability to achieve its purposes? 2.3 How cost-effective has the BLA been in meeting LED’s objectives and key performance indicator (KPI) targets? How reasonable and achievable are the committed/forecasted numbers, both in terms of jobs to be created, sales increased, and investments to be made at individual enterprises and project-wide goals? These questions address the following: • What has worked and how? What has not worked as anticipated and why? • Are there any unintended outcomes or external/internal factors that affected the achievement/nonachievement of the activity results? • What is LED’s enterprise selection process and LED’s effectiveness in meeting enterprises’ needs, and what are LED’s projected and actual KPIs? • Has LED addressed the relevant BEE or created a road map to address it? • Are there any practical recommendations and lessons learned for enhancing the project’s performance? 5 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV EVALUATION QUESTION 4 (SUSTAINABILITY AND SCALABILITY) What is the likelihood that the results LED has achieved so far are sustainable and scalable beyond the life of the project? These questions address the following: • Which LED results are likely to be sustainable beyond USAID/L support, at both the firm and business consultant levels? • Was LED or will LED be able to create a framework to tackle enterprise development (not to be one off support) and address the BEE (or create a road map to address the BEE)? • Was/will the consultancy/technical services delivery market for MSMEs be enhanced, effective, and/or improved going forward? • Are positive results spread across all beneficiaries or limited to just a few, and as such, what potential exists to bring these interventions to scale? • What recommended actions can LED or USAID/L take to enhance prospects for sustainability and scalability for the remaining LED timeframe and for other USAID/L economic growth activities? EVALUATION QUESTION 5 (GENDER) What are the outcomes generated as a result of LED’s gender mainstreaming? EVALUATION DESIGN, METHODS, AND LIMITATIONS The evaluation team (ET) employed a qualitative approach, conducting 63 key informant interviews (KIIs) with key project stakeholders. The KII respondents included USAID/L management, LED management, LED subcontractor management (Berytech and Business Incubator Association of Tripoli, BIAT), project-affiliated BAs, project-affiliated consultants, and MSME beneficiaries involved in the project. We did not collect primary data. The qualitative inquiries were supplemented with a document/desk review, including an extensive examination of quantitative data available from project monitoring. Data collection occurred from September 9 to September 23, 2019. A comprehensive evaluation design matrix with the evaluation questions, data sources, data collection methods, and analysis methods can be found in Annex II. DESK REVIEW The ET conducted a comprehensive literature review of relevant internal and external documents, including the following: ● USAID’s CDCS for Lebanon ● USAID Performance Management Plan (PMP), PSD ● Project RFP ● LED PMP ● LED Project Appraisal Document (PAD) ● LED Contract Agreement ● Select field assessments carried by LED, Berytech, and other subcontractors USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 6 ● LED Work Plans ● LED periodic reports (annual, quarterly, other) ● LED MEL Plan ● LED Gender Analysis ● Select Go-Forms and Memorandums of Understanding (MOUs) between LED and client firms ● Directly relevant external research/theoretical treatises2 MONITORING DATA ANALYSIS The ET received a full download of the MEL database employed by the LED project. The ET then conducted an independent analysis of the database to achieve a current measurement of the project’s major performances indicators—particularly, job creation, incremental sales, and leveraged investment. We compared these measurements against targets and against data previously submitted to USAID as part of monitoring requirements, for a clear indication of the project’s overall and recent progress. In some cases, we widened the definitions of outcome variables for a more nuanced view on progress—for example, we employed the project’s official metric of full-time equivalent (FTE) positions but also considered the sheer number of new jobs created as well the number of new hires to which client firms had committed. In addition, we disaggregated outcome variables by sector, type of technical assistance (TA) provided, firm size, and region—to explore whether subsets of the project’s client firms or particular kinds of interventions were producing better results than others. The team conducted the inquiry under the assumption that up-to-date or nearly up-to-date monitoring data are available for all participating firms, thereby providing quantitative outcome data for the project on a census basis. In addition to the data on enrolled firms, we worked with LED to gather and analyze data on the firms that were profiled but did not enroll in the project (either failing to meet enrollment criteria or choosing not to participate). KIIS The ET conducted 63 KIIs with MSMEs, USAID/L key staff members, Chemonics International’s LED team, LED partners’ management (Berytech and BIAT), business managers, and BAs as shown in Table 1 and Figure 1. 2 James T. Riordan, (2011). “We Do Know How: A Buyer-Led Approach to Creating Jobs for the Poor.” Washington, DC: New Academic Publishing. 7 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Table 1. Distribution of KIIs—Clients and Staff TYPE OF STAKEHOLDER TITLE NO. STAKEHOLDERS BREAKDOWN BY GENDER BREAKDOWN BY REGION USAID COR 1 Interview Male: 1 Beirut: 1 LED Staff COP/Director of Operations and Finance DCOP/Deputy Director Director of MEL Director of Business Promotion 4 Interviews Male: 3 Female: 1 Beirut: 4 LED Consultants Consultants (Service Providers) 15 interviews Male: 11 Female: 4 Beirut: 15 LED Subcontractor Berytech Berytech Deputy General Manager Acting BA Manager 5 BAs 7 Interviews Male: 5 Female: 2 Beirut: 1 Mount Lebanon, South, Beirut, and Beqaa: 6 BIAT Subcontractors Service Providers BIAT Director BA Manager 2 BAs 4 Interviews Male: 4 Tripoli: 4 Beneficiaries Beneficiaries/Firms/ MSMEs 32 interviews Male: 19 Female: 11 Mixed: 2 Mount Lebanon: 11 (Jbeil, Beirut, Choueifet, Ashrafieh, Hamra, Tahouneh) South Lebanon: 12 (Zahle, Beqaa, Anjar, Hemrel) North Lebanon: 9 (Akkar, Koura, Tripoli) Total 63 Interviews USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 8 Figure 1. Distribution of Interviews by Category 5 11 15 32 Interview Distribution USAID & LED Management Berytech & BIAT Consultants Beneficiaries The KIIs were the ET’s primary method and most significant source of data. We favored this method for two primary reasons: (1) its individualistic nature; and (2) its open-ended/inductive character. The individualistic nature of the KIIs with management (USAID/L, LED, etc.) was important simply because all such managers have unique roles on the project, and their experiences needed to be captured to understand the project’s functionality. However, the individualistic aspect also was pertinent at the beneficiary level, including the people who advise the MSMEs (the BAs), those who serve them in the course of the project (the consultants), and the owners of the MSMEs. There is an unusual heterogeneity to this project, in that the businesses tend to be different from each other, but interventions they receive in the form of TA are also unique. Each business requests and receives a different advisory service, based on the variables of supply and demand in each case—that is, there is no single model of intervention being tested. Hence, individual stories become the key and most pertinent evidence. Focus-group interviews, by contrast, bring groups of people together to seek consensus or debate on a shared collection of experiences, which is why they were not used here. The open-ended and inductive nature of KIIs was also highly relevant because we did not enter this evaluation with established theories or hypotheses about what worked and why. Rather, we sought data that reveal the story of the project and search for patterns from observation. It is the view of Social Impact (SI) and many experts in the field3 that these kinds of inductive approaches are a well-advised best practice in most instances of retrospective process evaluations or performance evaluations. The sampling and purpose of the KII in each respondent category are as follows: USAID/L, LED, and LED partners’ management. The ET conducted 7 KIIs with a purposive selection of the principal and most relevant managers at USAID/L (LED COR/AOR); LED (COP, Deputy COP/Director of Operation and Finance, MEL Director, Director of Business Promotion); and the two major subcontractor/partners (Berytech Chairman and CEO, Berytech Deputy General Manager and 3 For example, G. Guest, E. Namey, & M. Mitchell, (2013). Collecting Qualitative Data: A Field Manual for Applied Research. Los Angeles, CA: Sage Publications. 9 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV BIAT project manager). The KIIs provided in-depth understanding of the project’s achievements, the factors that contributed toward achieving the project’s results, and the implementation’s successes and challenges. BAs and consultants/consulting firms. Based on the team’s understanding of their central importance to the project, plus USAID/L’s expressed preference for a focus in this area, we interviewed all of the project’s seven BAs, with the exception of one, who had begun in the position less than a week before the start of the investigation (seven in total). The BAs provided key insights on the successes and challenges in recruiting MSMEs and on the value-proposition and effects of the services LED offered. The project’s direct service providers to MSMEs—the 37 consultants and 20 consulting firms—are more numerous and were sampled (15 consultant interviews in total, including a mix of freelance individuals and firms). The consultant population was stratified by service type/specialization (e.g., franchising, ISO certification, human resources, HR, restructuring) and then sampled at random, with a handful of purposive selections to capture consultants who were particularly active in serving multiple clients. The KIIs provided a direct view on the factors that contributed toward the achievement of project results, outputs, activities, the implementation process, success, and challenges at the firm level. LED beneficiaries. The ET conducted structured KIIs with 32 selected MSMEs (beneficiary firms), about one third of the total client pool at the time of the sample selection. Selection was based on the following sampling procedure drawn from the total client list: • List sorted by region. • Within each region, sorted by completed/ongoing in the program. • Roughly 35 percent of completed firms selected at random. • Roughly 65 percent of ongoing firms stratified by duration in program (newest, middle, most recent) and randomly selected. • No randomization needed/applied for regions with very small numbers overall (Akkar, Nabatieh, South Lebanon). • Handful of purposive substitutions made for logistical reasons (e.g., difficulty to reach). • Handful of female-owned or co-owned MSMEs identified and included in the sample. The ET conducted most of these interviews with the MSME’s owner. We deviated from that rule only when the owner was unavailable and/or specifically requested that the team interview another staff member because that staff member had better knowledge of the firm’s involvement with LED. The total sample of 32 for the pool of client firms is based on the best available science around nonprobability sampling in qualitative research for grounded-theories studies such as this one. Most guidance suggests 20–30 interviews as adequate to exhaust research questions (i.e., achieve “data saturation”);4 we have opted for the high end of that range to account for the relative heterogeneity present in our sample (as discussed above). Like the LED project, our sample included representation of all eight of Lebanon’s governorates. Our sample distribution generally followed LED’s distribution of deals across the regions, which is highly irregular, from lows of two clients per governorate (Akkar, Baalbek-Hermel, El Nabatieh) to as high as 4 G. Guest, et al., (2006). “How Many Interviews Are Enough? An Experiment with Data Saturation and Variability.” Field Methods, 18(1), 59–82. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 10 44 clients per governorate (Beqaa). Refer to Annex V for a summary of KIIs by category. The KIIs were semistructured; the typical length was around 45 minutes, though in certain cases they were longer because clients had much to share on interview questions. The interviews were conducted in-person by the team leader and the local expert consultant, assisted by two notetakers, taking hand-written notes. The only exception was the consultant interviews, which were conducted over the phone by the local expert consultant, due to logistical necessity. We developed a selection of interview guides (Annex III) to reflect the variety of interview subjects. Consistent with best practices in semistructured interviewing and inductive research generally, the interviews covered all major topics in the guide, but the interviewers were free to react and ask additional questions/probes based on the responses received. Prior to each KII, verbal consent was obtained from all those interviewed (see Annex III for consent scripts, attached to each interview guide). All interviews were recorded, and the recordings were used to complete the interview notes. They were also available for further reference for the team to complete their notes and for USAID/L upon request. ETHICAL CONSIDERATIONS All members of the ET received research ethics training from the team leader and were sensitized to the importance of guaranteeing and maintaining the privacy, confidentiality, and protection of the study participants. This study poses minimal risk to participants. The only significant risk was the possibility of a breach in confidentiality of the data collected or the possibility that a participant (particularly the MSME owners) may feel uncomfortable answering questions pertaining to their personal business or finances. All participants were informed about the nature of the questions that they will be asked prior to enrollment and prior to each interview. They were reminded that they may refuse to answer any question to which they feel uncomfortable responding. To mitigate the risk of any breach of confidentiality, strategies were in place to ensure that all participant identifiers are kept confidential and separate from individual data. No name, telephone number, address, or date of birth were collected on the behavioral questionnaire, transcripts, or analyzed data. Those materials were linked to individual identifiers only by way of a PIN, with the key kept in a password￾protected file. Moreover, interviews were conducted in a private setting designated by the participant, and the information they gave is disseminated in the aggregate in most reporting—thus reducing the possibility of identifying any one person or firm. Any identifying information inadvertently mentioned during the interviews or in the behavioral questionnaire were cleaned from the text before analysis. In cases where longer anecdotes or “stories” about individual MSMEs are used in this report, identifying information was altered to preserve confidentiality. All hard copies of study-related materials will be stored in a locked file cabinet at SI’s Beirut office, with access limited to study staff. All electronic data files will be password protected and maintained on the field team’s project computer(s) and/or on the project’s secure OneDrive. Transcripts will be similarly secured at SI’s Beirut office. 11 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Audio recordings will be destroyed upon submission of the report, and all paper-based data will be stored in a secure location for the required time specified by SI’s standards of practice (generally six months or less). DATA ANALYSIS Data Preparation: Interviews were transcribed into summary transcripts on a daily basis (see Ethical Considerations above for procedures regarding the protection of data). Coding: Transcripts were then coded primarily in terms of basic content—meaning responses were categorized and grouped, but generally not interpreted. We employed a specialized software for coding. Based on coding results, the ET worked collaboratively to develop the evaluation themes, findings, and recommendations. Gender Analysis Plan: The team disaggregated data wherever possible, ranging from the number/percentage of the female-owned firms enrolled in the program, to number/percentage of female employees hired. We found that there was limited opportunity to examine major outcomes (staff hired, sales increased, etc.) in terms of female ownership vs. male ownership because so few of the MSMEs in the project are female-owned (see full discussion next in gender section). LIMITATIONS AND BIASES The ET was aware of several noteworthy biases/limitations in this kind of research: Response Bias: Response bias is the risk that key informants may be motivated to provide responses that would be considered socially desirable or influential in obtaining donor support. For example, an owner of a client firm may have provided positive remarks about LED because he/she would like to receive similar support in the future or because he/she wants to please the interviewer. The ET mitigated theses risk by minimizing to the extent possible the connection between the evaluation activity and any future opportunities for support. Selection/Availability Bias: In KIIs, there is a risk that the ET only reaches the most active, responsive, or engaged stakeholders. To mitigate the risk of selection bias, we have relied on random selections wherever possible and taken only minimal input from the implementers and partners in selection. On the monitoring data analysis, there is a comparable risk that the low-performing firms may not have reported full data or may have reported inaccurately. In the end, we did not have reason to believe such a bias was present in the monitoring data. Gender Bias: Gender bias is a risk because most individuals have a subconscious sense of appropriate roles and behavior for women and men. In its training phase, the ET discussed what possible gender preconceptions might come into play during this evaluation and reviewed how to minimize these preconceptions during data collection and analysis. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 12 FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1 (RELEVANCE) Particularly for Objective 1, how relevant is the LED project with its firm-level focus, “buyer-led,” and generally market-led “technical assistance through business consultants” and “firm-led” approach: (a) to meet the needs of the private sector in Lebanon? and (b) to meet USAID/L’s CDCS and PSD objectives? Summary Findings, Question 1: • The BLA is as a model for creating jobs for the poor based on facilitating relationships between buyers and sellers. • With its focus on business interventions and job creation, the BLA is consistent with the mission’s CDCS and push for “Increased private sector competitiveness.” • Compared with iterations of the model elsewhere in the world, the LED project has taken a relatively narrow firm-level approach, bypassing complementary aspects like access to finance and generally not addressing sectors or value chains. • Project leadership articulates a clear focus on a firm-level intervention approach. • Some project leadership questions the fit of this approach in the Lebanese context, suggesting that more jobs might be created by intervening at the business sector level rather than the individual firm level. Conclusions, Question 1: • There is some divergence between the theory/precedent and LED’s practice for understanding and applying the BLA. • The BLA is about creating jobs, and in that sense, it is completely appropriate for a Lebanese context in which job creation can be difficult and elusive. • Based on previous aid projects and relevant literature, the ET suggests that a broader sectoral focus might have produced more jobs, but the argument remains speculative. OVERVIEW OF BLA The BLA is designed by James Riordan and pioneered by Chemonics International, as a model for creating jobs for the poor based on the lack of trust between buyers and sellers in the absence of rule of law as a key impediment for making transactions happen.5 Chemonics claims that at less than $2,000/job, the BLA is seen as a cost-effective approach to job creation because it helps the poor produce what buyers want, adhering to the mantra, produce what you can sell.6 The approach is designed to address the shortcomings of the traditional approach in development projects, based on these principles: (1) the buyer should be recognized as the central figure linking 5 Dr. James Riordan, Development Economist, Expert on Buyer-Led Approach. https://www.youtube.com/watch?v=bzo2AIWUKkc 6 Tracy Shanks, Managing Director, Chemonics International. https://100andchange.foundationcenter.org/profiles/2370/ 13 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV suppliers with market demand; (2) development projects should help people respond to market opportunities; and (3) instead of seeking project success, development programs should emphasize client success. Unlike other approaches, the BLA does not select poor communities to become the target of assistance, but rather works with buyers found largely in commercial urban centers, and then links them back to potential suppliers. Most buyers find small producers to be problematic—poorly prepared, not organized for productive purposes, and so on. In this situation, the BLA can help in addressing the link between producers, buyers, and suppliers by focusing on the producers’ success in carrying out productive activities within the value chains, as opposed to providing subsidized inputs and services.7 The core of the BLA is to facilitate contacts with domestic and foreign buyers of products and help rural families respond to market demand by organizing themselves to become serious suppliers. In the existing literature on the BLA, we can see a number of lessons learned from applying the BLA successfully, including8: • A self-selection process by clients, business enterprises, or families to join the project as opposed to the project selecting the beneficiaries. • Replacing beneficiaries’ cost-share with coinvestment schemes to help get business activities underway. • Match the TA with coinvestment grant funds to be disbursed directly to client groups, with the achievement of each benchmark identified in business plans. • Linking client groups to formal financial institutions to provide value chain financing, including loans using purchase orders as collateral. • Promote local service providers and their abilities to provide services to clients. The BLA is built on the assumption that access to finance is not an obstacle for MSMEs. The banks are expected to be ready to lend if the market is secure, access to technology is found, and good business practice is ensured. The BLA starts by making sure there is a buyer, and works backward with the business to create more jobs based on the motto—produce what you can sell, not sell what you can produce.9 BLA IN RELATION TO CDCS USAID/L’s 2014–2018 CDCS proposes an overarching goal of “improved accountability and credibility of public institutions, and broader economic prosperity.” This goal supports the broader U.S. objective of achieving stability in Lebanon and aligns with various Government of Lebanon strategy pronouncements. The CDCS has two DOs; DO 1: “Improved capacity of the public sector in providing transparent, quality services across Lebanon”; DO 2: “Inclusive economic growth enhanced.” Under CDCS DO2, USAID works to increase the competitiveness of Lebanon’s private sector by strengthening the advocacy role of business associations, improving trade linkages, increasing access to 7 Pattie Preston, (2014, April 24). “Let’s Change Focus—The Buyer-Led Approach to Development.” https://www.devex.com/news/let-s-change-focus-the-buyer-led-approach-to-development-83351 8 Pattie Preston, (2014, May 22). “10 ‘Micro’ Tips for the Buyer-Led Approach to Fighting Poverty.” https://www.devex.com/news/10-micro-tips-for-the-buyer-led-approach-to-fighting-poverty-83535 9 Dr. James Riordan, Development Economist, Expert on Buyer Led Approach. https://www.youtube.com/watch?v=bzo2AIWUKkc USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 14 finance credit, especially in rural areas, addressing labor market issues, and providing for job-focused formal higher education and technical training for vulnerable groups. The rationale for the PSD project flows directly from the goal of the mission’s 2014–18 CDCS: “Improved accountability and credibility of public institutions, and broader economic prosperity.” PSD directly supports DO2, “Inclusive economic growth enhanced.” The PSD project’s purpose is “Increased productive employment in Lebanon through direct support for the private sector.” DO2 has two intermediate results (IRs): IR 2.1, “Increased private sector competitiveness” and IR 2.2, “Increased access to finance, especially for new business start-ups and women entrepreneurs.” Both PSD project IRs feed into the CDCS IR 2.1 and CDCS IR 2.2. According to its PAD, the PSD project will accomplish its aims by supporting the smallest of businesses, microenterprises, and family-run businesses that need technical support and access to finance to improve the livelihoods of the individuals involved in these firms. It will promote entrepreneurship and start-up enterprises, with a focus on women, youth, and innovation, which is essential for overall economic growth. It will support MSMEs with a focus on rural development, particularly through the agriculture, processed food, and tourism sectors as the main source of production and employment in these areas. It will provide direct support to medium and larger-sized businesses with the highest potential to increase sales and therefore grow wage-based employment, the most pressing economic need in Lebanon. It will promote reform at the policy and regulatory level, mainly through advocacy with the private sector. The PSD project states that it will work vertically through value chain–based approaches that understand the integrated roles of different actors involved in the production of targeted goods. It will take a horizontal approach, with a focus on access to internal and export markets and buyers using demand-driven approaches to improve productivity and quality through new technologies. It will take a direct approach, partnering with a private investment organization to take equity positions in some of Lebanon’s most promising start-ups and entrepreneurs while ensuring the investments of others to reduce risks and grow a venture capital ecosystem from almost scratch. PSD will also work with business associations and other nongovernmental organizations to strengthen the role of the private sector and civil society to effectively advocate and contribute to policy, and regulatory and nonregulatory reforms that affect economic growth and job creation. Finally, it will take a cross-cutting approach, integrating environment, gender, and social cohesion into each of its activities because they are essential for the success of the overall program. The LED project, with its BLA, theoretically fits into CDCS DO2 supporting DO2 IR 2.1. It also fulfills the goal, purpose, and objectives of PSD, particularly IR 2.1. The BLA can be used as a strategy/approach for working with MSMEs with a focus on rural development. The approach should help support MSMEs with the highest potential to increase sales and, therefore, grow wage-based employment. It is aligned with PSD’s vision for working horizontally with a focus on access to internal and export markets and buyers using demand-driven approaches to improve productivity and quality through new technologies. It could also potentially fit with the PSD approach for working vertically through value chain–based approaches based on the integrated roles of different actors involved in the production of targeted goods. As with the CDCS and the PSD, the core of the BLA and the LED project is job creation. 15 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV FIDELITY OF THE LED PROJECT TO THE BLA MODEL The LED project employs the BLA in the advancement of these goals and objectives. Yet there appears to be some divergence between the theory and practice for understanding and applying the BLA. Some aspects of the BLA, such as the focus on market demand, focus on the buyer, and matching production capacity to meet buyer demand were applied. However, other requirements or recommendations for ensuring firms’ readiness to meet buyer demand were not addressed within the project. When the BLA was designed and implemented in Latin America, Africa, and elsewhere, it was meant to look at the poor within rural communities, particularly agricultural producers and producer groups. In these other iterations, the BLA project typically worked with producer groups and incorporated a variety of practices that are reflected in the “lessons learned” noted above, including: matching TA with coinvestment grant funds; linking beneficiaries to formal financial institutions to provide value chain financing, such as loans that use purchase orders as collateral; and linking beneficiaries to markets to ensure actual sales transactions. The context for LED in Lebanon has been quite different. By design, most of the areas served under LED are urban or peri-urban. Although there is rich agricultural production in the country, few of these businesses have been targeted by the project because of USAID interest in staying “sector agnostic” with a firm-based approach for identifying and working with MSMEs that can generate jobs regardless of the sector. At the same time, the project has maintained a relatively narrow focus on TA in the form of business consulting to link businesses with new buyers, sidestepping what are seen as complementary components in areas like coinvestment and linkage to formal finance. Moreover, while other BLA projects have kept individual buyers and producers at the heart of their interventions, they have not been wholly “sector agnostic,” in that they take a broader look at the value chains and sector growth of most relevance to their client enterprises (rural and agricultural in most cases). This has not occurred in any systematic way in LED, where the bottom line has been only job creation driven by individual TA interventions with individual firms. In the process, it would appear that some opportunities for impact have been lost. For example, LED worked with and provided TA to a variety of agri-food processors, especially in the Beqaa valley. But LED has not assessed the potential of the agri-food sector as a whole or examined the bottlenecks within this sector as a way to address job creation more broadly. The same applies with other sectors such as dairy, hospitality, food and beverage, and others. LED LEADERSHIP AND PARTNER PERSPECTIVES ON BLA In further reflection of the potential divergence between broader BLA theory and implementation, interviews suggested that LED management and partners do not see the BLA as a development approach, but rather as a market-based approach for working with/and serving clients at the firm level only. When asked directly about their understanding of the BLA: all management answers can be categorized into four general responses: (1) directly serving/going straight to client enterprise; (2) working with clients with identified buyers; (3) working with clients with identified markets; and (4) working with clients to remove barriers to serving buyers. For LED, Berytech, and BIAT staff, the BLA is about what the client, the enterprise, the buyer, and the market need. According to the senior LED manager, the BLA is “understanding the client enterprise and USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 16 what their market is”; identifying a specific buyer and “understanding who they are and what they want to satisfy the buyer.” It is about working with “clients to remove barriers to serve the buyer.” This is mirrored with BIAT and Berytech staff, managers, and BAs who see the BLA all about identifying buyers, expanding resources, and improving technical capacity to meet this demand. According to a BA working with Berytech, the BLA is about having “a specific buyer identified, followed by technical assistance to close the deal.” Regardless of the sector, the LED project sees its mandate as looking at the market demand for certain products and the capacity of the firm to produce, then working with this firm to deliver the product. This is based on the firm’s capacity and ability to secure the market demand and match it with the required need for products. LED provided TA to help make this happen but does not extend itself into other areas of assistance—such as securing the finances to match any needed growth in terms of investment. In interviews, project management did not question the match between the BLA model and their own approach for implementing the project and achieving the required targets. In this sense, the evaluation gleaned little in the way of any rationale or explanation for these choices in design. LED management’s understanding of the project largely concurs with USAID in seeing their activities as focused on individual firm-level interventions, and little else. Some did question LED’s fit within the context of Lebanon—comments that seem related to this divergence in some ways. One partner manager, for example, suggested that an ideal approach for Lebanese business growth would feature “a mixed approach between market, buyer, sector, value chain, and others that would generate the growth and jobs needed.” Another commented that LED’s interventions should be “streamlined across companies working within specific sectors or across value chains.” However, comments such as these were scattered and inconsistent among the LED management, partner management, and staff. They did not amount to a strong alternative vision for the project and provided no basis for a plan to reorient the current project, with its firm-level focus, to a broader intervention that addresses issues at the sector or value chain level. EVALUATION QUESTION 2.1 (EFFECTIVENESS) What evidence is there that employment has increased in targeted enterprises, and sales (and investments) were enhanced as a result of USAID/L assistance (veracity and attribution)? To what extent can we say that these gains are the result of program “additionality?” Summary Findings, Question 2.1: • After a slow start in Y1, the project has rebounded strongly on jobs in Y2, with 992 created, though this remains well under target, particularly as measured in terms of FTEs. • The distribution of jobs is uneven across the project, with a handful of larger firms providing a disproportionate share, and some firms not generating any thus far. • As with job creation, the project has rebounded strongly in Y2 on pledged incremental sales and pledged leveraged investment. • However, actual/reported data on incremental sales and leveraged investment is incomplete and error-prone, limiting the ability to examine progress against targets. 17 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV • Cost share (as a form of leverage investment) stands at 40 percent across the project, with wide variation by the type of TA provided. • Beneficiaries reflected positively on what they saw as strong growth in job, sales, and investment; most saw the increases as a direct result of their involvement with LED. • In terms of additionality, beneficiaries reflected on a wide variety of positive changes in the ways they do business. Though many had existing plans to grow and improve, most said that LED had “accelerated” these by adding strong expertise. • Project leadership shared a vision of strong project achievement, though some noted that it may take longer than expected to reach some targets. • BAs and consultants pointed to the strong relationships with and strong commitments from beneficiaries as the most common drivers of results. Conclusions, Question 2.1: ● Clearly, there has been substantial progress on job creation in Y2; qualitative data on sales and investment also suggests strong progress, though the view is less complete because of challenges in the related MEL data. ● If we accept the limited amount of reliable data on sales and investment, collectively the data suggests that the process of improving and expanding business in Lebanon proceeds in a particular order; investment appears to come first, followed by job creation and sales. ● It appears this trend and configuration were not foreseen by LED’s designers, which resulted in strong and uneven discrepancies between the results and targets to date. ACHIEVEMENTS TO DATE IN NEW HIRES, INCREMENTAL SALES, AND LEVERAGED INVESTMENT Achievements on jobs created to date are summarized in Table 2. Table 2. Overview of Committed and Actual Jobs (as of 9/19/2019) TOTAL ACTUAL JOBS TO DATE 992 Part-time 15 FTE 476 Total committed jobs to date 2,299 Percentage of committed jobs hired 43% Percentage of firms with zero hires 38% Zero hires with kickoff in 2018 6% Zero hires with kickoff in 1st half 2019 36% Zero hires with kickoff in 2nd half 2019 56% Percentage of firms with less than 5 actual hires 29% Share of jobs for top 5 employers 44% USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 18 The figures10 suggest an uneven distribution of actual job creation, with five large employers responsible for nearly half of the jobs created. All five of those are large firms in the food service or food manufacturing sectors. At the same time, nearly 40 percent of enrolled firms have created zero jobs—an observation that can be explained partially but not fully by the length of time these firms have been participating in the project. Table 3. Progress on Jobs against Targets Y1 Y2 TOTAL TO DATE FTE Target 392 1,078 1,470 Total Actual Jobs Created 8 884 992 Committed Jobs 327 1,972 2,299 FTE Jobs Created 2 474 476 The data in Table 3 suggest strong progress on actual hires to date, particularly as compared with Y1. After closing Y1 with less than 1 percent progress in terms of actual FTEs against the target, progress stands at 32 percent of the target. Moreover, if one operates under the assumption that most or all current jobs created will endure long enough to become FTEs, current progress toward targets would stand at 67 percent, or two thirds of the way to the target. (Progress toward future targets, including projections on what may be seen in Y3, are discussed in detail later.) The data available on achievements to date on incremental sales and leveraged investment are summarized in Table 4. Table 4. Progress on Incremental Sales and Leveraged Investment against Targets INCREMENTAL SALES Y1 Y2 TOTAL TO DATE Target $13.72 million $37.73 million $51.45 million Reported (42% of firms only) 0 $15.24 million $15.24 million Pledged n/a $215.19 million $215.19 million LEVERAGED INVESTMENT Y1 Y2 TOTAL TO DATE Target $2.01 million $5.66 million $7.67 million Reported (26% of firms only) 0 $35.04 million $35.04 million Pledged n/a $25.57 million $25.57 million 10 Note that all percentage calculations are based on the 992 actual jobs figures (i.e., without separating the part￾time jobs or calculating by FTE jobs). This analysis decision was made based on two factors: (1) the relatively low percentage of part-time jobs; and (2) the perceived likelihood that the full-time job will endure at least a year, thereby qualifying as a FTE in the future. The implications of actual job count vs. FTE count will be discussed elsewhere in this report in more detail. 19 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Like its progress on job creation, the LED project has rebounded strongly in Y2, as it registered no formal progress against targets in Y1. However, a number of key caveats must be noted here. First, the data reported by clients and included in the MEL database for both incremental sales and leveraged investment is incomplete, with less than half of firms reporting the former and just over one quarter of the firms reporting the latter. This fact alone severely limits the value of the data for comparison against targets. Second, LED management has confirmed that this reported data, particularly for leveraged investment, has been problematic and prone to errors for a variety of reasons, at both the administrative and client levels. This limits the value of the data for comparison against targets even more deeply. Third, although pledged incremental sales and pledged leveraged investment are complete in the sense of capturing all participating firms, LED management characterizes these numbers as the most optimistic outcomes over two years, assuming complete success for all identified market opportunities to which LED support relates (especially the $215 million in incremental sales). Management has stated that they understand that these projections are not fully realistic, as market realities can produce different results from plans. In sum, the MEL data remains rather inconclusive when it comes to both incremental sales and leveraged investment. What we can say with certainty is that the project and its participants maintain high optimism in both areas, as both pledged measures vastly exceed project targets. A final note on achievements from the MEL database—an extension of sorts on leveraged investment to date—is found in the figures on cost share for the provision of TA. Project-wide, the cost share paid by clients stands at 40 percent. LED management has confirmed that these figures have gone up over time, from the early days of the project when cost share was shared more flexibly and even waived in some cases, to current levels where firms are required to pay up to 50 percent. Table 5. Cost Share Contributions by Type of TA TYPE OF TA LED CONTRIBUTION CLIENT CONTRIBUTION COST-SHARE PERCENTAGE Project-Wide $1,226,609 $831,793 68% ISO Certification $282,232 $134,541 48% Franchising $280,175 $164,779 59% HR $156,397 $75,396 48% Branding and Marketing $114,500 $23,250 20% Others $110,203 $100,824 91% Sales Training $83,878 $75,159 90% IT $66,323 $105,824 160% USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 20 Management Training $57,086 $23,060 40% Product Design Development $56,505 $124,090 220% Global GAP $19,310 $4,870 25% Drawing from the LED database, we have broken down client cost-sharing distribution by type of TA in Table 5, from its highest levels in the areas of product design development and IT, to its lowest in branding and marketing and Global Good Agricultural Practices (GAP) certification. There is some temptation to read into these figures—the highest percentages, for example, might represent the area most highly valued by clients, hence their willingness to pay more for them. Equally likely, however, may be that the discrepancies represent variation on when the project offered these services. For example, a higher proportion of branding and marketing TA was offered early in the project, when cost-share requirements were looser, compared with IT, which has come in with greater frequency more recently. Or these variations may be the results of statistical quirks in a relatively small data set. BENEFICIARY PERSPECTIVES ON ACHIEVEMENTS Most beneficiaries reported a solid record of achievements in the key area of job creation, with a majority in the sample citing over 10 jobs already created since the start of their involvement with LED. Most of them viewed these jobs as a direct result of the project. “I wouldn’t have the opportunity to hire people if we were not part of LED,” explained one MSME manager. However, the attribution of new hires to the project was not uniform. A substantial minority of beneficiaries saw the new hires as only an indirect result of LED, or as something that would have Figure 2. LED beneficiary in the Beqaa 21 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV happened even without LED. One manager, for example, stated that he already had planned to open a new branch and hire that number of new employees before LED came along. The sample was more split on the question of new incremental sales increases since the start of the firms’ work with LED. A bit more than half reported sales increases, and nearly all of those were directly related to LED. Said one CEO, “We are doing 20 percent growth whereas in our industry, our competitors are doing 30–40 percent less. The consultant played a huge role in this business growth.” A smaller but sizeable group reported that they had not yet seen any increase in sales since starting with LED. In some cases, this was a question of time—that is, the sales increase would be coming later— while in other cases, it was more related to the nature of the TA. One owner explained, for example, that the assistance he received in the area of HR restructuring was not expected to have a direct or immediate impact on sales. In the area of increased/leveraged investments, the vast majority of respondents reported that internal investments had increased substantially since they started their projects with LED. Nearly all of these investments were internal investments, funded by the owners and their families directly into the businesses. Just a few beneficiaries reported gaining access to new loans from outside sources. However, like the new hires, there was some disagreement in the sample about attributing these new investments to the LED project. The most common response was that the investments were, in fact, directly related to LED. But about half as many beneficiaries viewed their investments as only indirectly related to LED, or not related to the project at all. One manager, for example, saw the investments as an ongoing process that had started before LED. He saw the investments as reinforced by the ISO certification he was applying for under LED, but not immediately related to LED. DISCUSSION OF “ADDITIONALITY” IN PROJECT ACHIEVEMENTS On the question of additionality, or where LED has directly added value to the operation and profitability of these MSMEs, this evaluation took two approaches. First, we looked at the question of actual-to-date improvements to the way the MSMEs do business—that is, how does the firm compare between now and the point it started working with LED? Second, we probed on the more theoretical counterfactual—that is, where exactly would the firm be if LED’s interventions had not taken place? Queries around the first angle revealed a wide variety of reported changes for the better in the way the firms do business. The most common response from beneficiaries was that their managerial processes and operations had improved, generally becoming more systematic and standardized. Said one CEO, “We are much more organized. We have procedures and standards for working, which we didn’t have.” The improved processes in many cases were related to LED’s assistance with the firm’s application of ISO 9001 certification, which centers on international standard quality management principles. Many of these same cases involves the firms’ reported transition from traditional family-owned and family-run business models to more modern business models based on quality management. One manager noted that the ISO process had transformed their more traditional business model by compelling them to consider “very minor details on the ground that we never thought about.” An equally common response on improvement to doing business was that new sales/marketing strategies were being applied. In most cases, these were based on the consultants’ particular focus on USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 22 the sales department in the provision of TA. “We are all improved at the sales level, the way we interact with people is different, and management of sales improved,” said one MSME co-owner. Another firm owner explained that the consultant improved the firm’s sales and distribution market by teaching them how to target hospitals, hotels, and individual buyers. Another quite common response was that the firm was now operating with restructured and improved HR procedures—again, as a result of the consultants’ targeted TA in this area. “Before LED, I used to manage everything, now we are starting to have managerial positions that will take care of the HR, the operational,” explained one owner. “It was a ‘one-man show’ at the beginning, this changed completely.” Other fairly common beneficiary responses on improved ways of doing business included: ● Improved/expanded product lines. ● Expanded facilities and production equipment. ● Work with new franchises, both inside Lebanon and internationally. One general manager added that the mentality of the whole firm and its staff had changed—they were simply thinking bigger now. For additionality, by way of considering the counterfactual (the second angle described above), the ET asked beneficiaries outright where their business would be vis-à-vis the reported improvements if the LED project did not exist or had not assisted them. Here the answers revealed a more varied range of views on the project’s added value. By far, the most common response was that the firms would have seen these changes and improvements, but none of it would have happened this quickly without LED’s assistance. This is to say, many firms already had planned to make improvements in these areas, whether in sales and marketing, Figure 3. LED beneficiary in the South. 23 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV HR, ISO certification, or franchise-based expansion, but say they would have drawn out or postponed the plans without LED, often driven by economic concerns, cash flow at the firm level, or the generally recessed state of the Lebanese economy. Said one owner, “I would [have] postponed the franchise for one year. That was an investment that I was not ready to pay at that time. I was willing to pay, but I was not able to pay.” Another owner/manager reiterated that the changes would have taken more time, adding, “I would do it, but it would not have been easy.” The term “acceleration” or “accelerator” came up multiple times in interviews, suggesting that the LED project had this effect on firms’ improvement/expansion plan. Several owners also described the effect of LED as providing a “push” to move forward on plans. Other respondents added that LED provided both a “push” and raised the quality of the improvement/expansion plan, compared with what they would have done without the assistance. One general manager explained, “I would have been able to find the expertise, but not as well as the LED people.” About half as many respondents (compared with those who described LED as an accelerator) said that the expansions and improvements would not have happened at all without LED. In these cases, the firms generally expressed one of two views: (1) that they had plans but no basis to move forward on them, or (2) that they did not have any concrete or actionable ideas on how to move forward. One general manager whose firm fit in the latter category explained, “I would not have known how to work, where to ask for help, and what to do.” Conversely, a substantial minority of beneficiaries suggested their firms would be in the exact place without LED in terms of all major outcomes and improvements (new hires, sales, investments, etc.). They suggested that LED’s involvement ultimately had not changed anything. For example, one CEO expressed clearly that all the changes he described, he would have done anyway—he just would have found another way to pay for them. As the above discussions suggest, the variation on additionality, especially when considered along the second counterfactual angle, relates closely to the question of how fully formed the firms’ expansion plans were during their engagement with LED. Here we find wide variation and a wide variety of scenarios, including: (1) plans fully formed and initiated before LED came along; (2) plans formed but not initiated before LED came along; and (3) plans in vague state or not formed in any coherent way. The widest segment of the sample would fall in the middle range, but we find a minority of cases on both ends of the spectrum. For example, we can consider the subquestions of relationship with the consultant: we found that around two thirds of the sample relied fully on LED to identify and engage a qualified consultant. But the other one third of the sample had an existing relationship with the consultant and knew what they wanted from the consultant in this project, including several cases where the firm had already hired that consultant and started on what would become the LED activity, before LED “joined in.” In sum, additionality is a relative concept that can be considered in multiple ways. Collectively, the evidence suggests that the vast majority of beneficiary firms are experiencing significant growth and improvement, and LED’s most common role in those changes has been to accelerate change based on ideas the firm already had in mind—as one would expect in a project where identified growth potential is one of the prime criteria for beneficiary selection. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 24 A CASE OF LOW ADDITIONALITY One MSME interviewed is a family-owned, 30-year-old company in the industrial food processing space, based in the Tripoli area. The firm is specialized in cheeses, cold-cut meats, pickles, and prepared foods such as breaded chicken patties. It serves the local market and exports to 12 countries. It currently employs around 140 staff. The firm’s 47-year-old general manager is not a member of the family of owners and was hired in the last two years, primarily to bring more corporate management ideas to the company and move away from family-run traditions. He became aware of the LED project when an advisor from BIAT reached out to the owners. He was interested in working with LED on a project he had already started, which was the systemization of the company’s HR functions. The objectives of the project included: (1) creating an org chart; (2) drafting job descriptions; (3) creating a grading system; and (4) initiating performance management. All of these changes were to be overseen by a newly created HR department. Before LED was in the picture, the general manager had identified and hired a qualified consultant. LED entered several months into the consultancy and subsidized it from that point, contributing about half of the consulting fees, but nothing in the existing plan changed. The firm’s MOU with LED indicated that the firm committed to 40 new hires and $5.2 million in sales, increasing over FY19 and FY20. To date, the HR restructuring has advanced well, despite some resistance to change among the company’s more longstanding employees. However, no jobs have been added, and the general manager expressed doubts about ever reaching the 40 mark because of the economy and other factors. No sales increases have come as a result of the project. Though new internal investments have been made, the general manager does not see these as related to LED. A CASE OF HIGH ADDITONALITY Another MSME in the study is a designer and manufacturer of commercial signs and displays, based in the South Lebanon region. The business is eight years old and serves a wide variety of commercial sectors. The 36-year-old owner/operator became aware of LED through some existing contacts at Berytech. At first, he was uncertain where to focus the TA, but a BA visited the firm and advised him to focus on branding and marketing. An MOU with LED was drawn up, and a consultant was hired. The owner reports that the BA and consultant immediately recognized his unique niche in the market, which is working and printing in multiple media for custom commercial signs and displays. The consultant suggested he drop his work in residential carpentry and focus only on the commercial signs/displays, which he did. He also advised the owner to change the name to highlight his specialty and assisted with new marketing tools including printed brochures and business cards, which he had never used previously. Since completing the consultancy, the owner has expanded his staff from two part-time to six full-time, including three Lebanese, which puts him on track to meet his commitment in the MOU of seven full￾time hires by the end of FY20. As a direct result of the branding, the owner reported sales increases of 25 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV 20–30 percent. He has invested substantial personal funds in a new machine that prints vinyl. He is well below his commitment to LED on leveraged investment but has another year to meet those goals. The owner adds that he would have never thought about hiring a consultant before LED came along, but that he would do it again after seeing these results. LEADERSHIP PERSPECTIVES ON ACHIEVEMENTS The project also considered achievements from the perspective of project leadership on major achievements, particularly in the key outcome areas of new hires, sales, and investment. Their assessment tended to be optimistic about progress thus far and saw no ambiguity in terms of attribution to the project or additionality. As one LED manager summed it up, “We have fully supported these companies to grow.” In discussions on the key indicator of new jobs, management perspectives were quite positive, although they frequently also brought up qualifying considerations, the most common of which was time. This is to say, management felt confident that goals around new jobs would be met, but perhaps not on the original target timeline, and perhaps not by the end of the project. Said one LED manager, “A lot of jobs are going to come in, but definitely after the project period—even in the case of project extension.” A partner manager elaborated by explaining that for employment, the strategy that LED is adopting is long-term, and that LED’s process is “soft,” not providing any equipment, which differentiates LED from other projects. “We have to be patient; we have to wait for the good things to happen. We are changing the culture here,” the partner manager said. This question of goals in relation to the project timeline is a recurring theme that will be discussed further elsewhere in the report. BA PERSPECTIVES ON ACHIEVEMENTS Many of the project’s BAs reported that their major achievements with the LED program included “the strong relationship with MSMEs.” The strong relationship and the trust between BAs and MSMEs were typically built after the second and third meetings, with growing knowledge and experience about the MSMEs’ challenges in the market. The BAs described a transparency between them and MSME representatives. One of them stated, “Good relationships, as they are very comfortable to communicate transparently and share all the gaps.” Some BAs suggested that this relationship enhanced the linkages between companies, leading to more referrals from current MSMEs to new MSMEs to enroll with LED. The BAs elaborated that most MSMEs were strongly committed, cooperative, and highly appreciative of the TA. A number of BAs noted that MSMEs were meeting the cost share, sharing all essential information, and fulfilling the project’s considerable paperwork requirements. Some MSMEs have expressed concerns about the level of monthly paperwork requested by LED, but BAs reported that in the end, most or all of them comply, often after the BAs follow up repeatedly. At the same time, BAs reported that the commitment and collaboration of some MSMEs declined after signing the MOU. As one BA stated, “In general, we do have a good relationship with MSMEs as long as they are serious … But a few of them, after signing the MOU, their level of effort is decreasing, and it is taking from USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 26 us much effort to follow up with them.” They attributed this to the mentality of particular MSME representatives and the level of seriousness toward receiving the TA from LED. The BAs also point to real and tangible gains on jobs and sales among the MSMEs. Several commented on the fact that some employers have already surpassed their commitments on jobs. They also suggested that the exporting strategies developed with LED have played a major role in boosting sales in a short period of time. The minority of BAs also highlighted the following achievements: • Introduced the value of USAID assistance because a significant amount of companies did not know about the role of USAID in the development sector. • Assisted MSMEs for access to finance. • Assisted in the transition of a microenterprise located in the Beqaa region to MSME status. When it comes to fulfilling new job commitments, the BAs tended to be confident, but this confidence was couched in terms of their own work with MSMEs. This is to say, the BAs described a conservative approach in advising the MSME from the beginning and setting job creation goals (i.e., erring on the side of lower rather than higher projections). As one BA explained, “First, the number of jobs added into the go form is very conservative. Second, we always challenge the client to reach his target. Third, before confirming the number of hirings, we do brainstorming with the client as we ask him about each job, impact on the company, and expected hiring time.” At the same time, some of the BAs declared that the job creation is never fully guaranteed, and that completing the hiring is taking considerably more time than expected in some cases. On the subject of the consultants’ work, most BAs agree that the consultants are qualified, are committed to their jobs, and deliver more from what it is stated in the scope of work. They also acknowledged that most MSMEs are satisfied, as very few complaints have seen raised (see the following). The BAs view this as a result of their screening of the consultants before engagement, as well as their intensive monitoring of the consultants once engaged. This monitoring consists of: • Monthly reports from the consultants. • Follow-up calls or visits to the MSME to confirm deliverables and satisfaction. • Follow-up inquiries to confirm that the intervention has had the expected impacts on the company. • Visiting MSMEs randomly and attending meetings while the consultant is working. Overall, the BAs cited just two significantly dissatisfied cases at the level of MSMEs. In the first case, the TA was cancelled because of a mismatch between the MSME and the consultant’s personality. A BA explained the second case, “We faced one case with [a consultant company], while giving franchising services support to [a local restaurant]. At the beginning, the consultant wasn’t serious with the client and was sending standard documents and manuals that weren’t clear for the client. After attending many site meetings and with time, the deliverables improved. [This consulting company] has a good profile and experience in the market.” A few BAs added that MSMEs sometimes become frustrated by the time required for TA deliverables; this is depending on the type of assistance required. For example, some MSMEs have complained that upgrading their facilities to meet ISO requirements has taken a longer time than expected. 27 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV CONSULTANT PERSPECTIVES ON ACHIEVEMENTS The consultants reiterated that strong relationships and strong commitments have developed among the MSMEs enrolled in the project. One consultant stated, “they are committed and more than appreciative of our service. And the cost sharing is affecting this, since they are paying from their own money, which is obliging them to be more dedicated.” The consultant cited a small number of exceptions on commitment, including: (1) MSMEs who enrolled early in the project and paid little or no cost share, thereby reducing their commitment; and (2) MSMEs whose owners have a mentality that is unappreciative of consultant services. In terms of specific achievements, the most common response among consultants was TA at the level of HR. They noted that some MSMEs did not have an HR department at all, while others lacked a proper organogram for the company. The HR-focused TA support has brought new efficiencies to MSMEs, as they are now well-structured and have a standardized system to maximize their HR. Another major achievement cited by the consultants were the incremental sales increases that have resulted from the new deals signed by the firms after receiving sales training and/or ISO certification. Examples include: ● “A food company closed a deal with a trading company in Sweden for private labeling after being ISO certified.” ● “[One MSME] was losing a deal, then due to ISO 22000, he got the deal again.” ● “The new deals signed by the MSME by only receiving an official letter from [a consulting company] that that they will get certified very soon.” The consultants also cited accomplishments in helping MSMEs innovate new products for their markets, improve the quality of their products and system management, and generally increase their productivity. However, a few consultants suggested that MSMEs sometimes do not always follow recommendations fully, which can delay the TA deliverables. They explained these cases are largely because of economic factors; for example: ● “Sometimes they have financial issues (external factors for us and them), which delays a few investments that affect our work improvement.” ● “They have other priorities, especially for surviving in this critical economic situation.” ● “In the food sector, we have requirements for infrastructure, as we do request investment to improve their manufacture, and due to cash flow or some legal documents, the work was delayed.” EVALUATION QUESTION 2.2 (EFFECTIVENESS) What are the main factors that influenced positively or negatively LED’s ability to achieve its purposes? Summary Findings, Question 2.2: • Constituents at all levels agreed that strong relationships between staff and firms have promoted success in the project. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 28 • Constituents at all levels agreed the biggest constraint to success was the economy, which limits achievement at every stage, from firm enrollment to job creation numbers. • Some beneficiaries have not responded well to the project’s requirements, from cost share, to minimum new-job requirements, to “constant requests” for monitoring data numbers. Relatedly, some clients have complained of cash-flow challenges. • LED management feel that the project has not received clear direction around Objective 2 (BEE), and that work draws time and resources away from its job creation efforts. • Other challenges/constraints to success include high turnover among the BAs and lengthy consultant searches in some cases. • LED has demonstrated inconsistency around some of its requirements including cost share, minimum job commitments, and the number of TA services allowed. Conclusions, Question 2.2: ● LED has made impressive gains in terms of building relationships and goodwill in the Lebanese business community, which has helped lay a foundation for success. ● The Lebanese economy, plus unpopular rules and inconsistencies, have undermined success. ENABLING FACTORS TO ACHIEVEMENTS As with many aspects of this inquiry, we will consider enabling factors on multiple levels, based on interviews with LED staff, LED partner management, BAs, consultants, and beneficiaries. Each subpopulation offered its own commentary, but we did find a common theme to much of the discussion of enabling factors, which is widely reflected in the commentary of the previous section on project achievements. That is, most of the comments centered on the relationships with the MSMEs and the MSMEs’ attitudes in those relationships. The project’s management singled out and commended the attitudes of the participating firms and, in particular, their willingness to pay cost share at a difficult time. “They’re paying, when it’s painful to pay, and then they’re investing beyond that,” stated an LED manager. For their part, the BAs singled out the strong relationships that had formed between the project’s service providers and the beneficiaries. One BA explained that he was able to serve beneficiaries well because of this level of relationship and “since I know all their challenges.” The consultants continued this theme of client relationships and commitment. One described the clients as “very committed and very interested and helpful, as we worked all together … to succeed.” On the demand side, the beneficiaries continued in this vein, with the majority of their comments centering on the strong relationships they had developed with both the BAs and the consultants. One operations manager described his BA as “very responsive, responsible, and fast in acting. He tried his best to provide the best support in any way we needed.” Regarding the consultants, most beneficiaries were similarly effusive. The most common response in this area cited the consultants’ commitment and responsiveness. They also mentioned the consultants’ willingness to go the extra mile with the firm, including the provision of additional free services after the project’s level of effort was exhausted. (One caveat to note here, as discussed in the previous section on additionality, is that about one third of the beneficiaries had relationships with the consultants that pre-dated the LED project.) 29 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV CONSTRAINTS TO ACHIEVEMENTS As is natural for these kinds of inquiries, the various levels of respondents were more inclined to elaborate on constraints. The discussions of constraining factors again were diverse but can be broken down into two categories—landscape constraints and demand-side/beneficiary constraints—plus a number of additional items that did not fit into those categories. LANDSCAPE CONSTRAINTS Starting with the management level, multiple respondents suggested that the slow Lebanese economy had hindered the project’s success. An LED manager articulated this constraint in terms of the key indicator of job creation: “Creating a real job is exceptional, because everyone is trying to maintain.” A partner manager added the following, “The economic down-turn of Lebanon limited us to attract more employees, as also the objective of the project is a bit tough to reach during this economic situation.” The Lebanese economy loomed large as the most common response among the BAs. The country’s lack of economic growth has hindered progress on all aspects of the project, beginning with enrollment. “The Lebanese market is affecting the attraction of new clients,” explained one BA. A number of BAs commented on the related concern of lack of access to finance as a constraint to MSME growth, and hence a constraint to participation in LED. Reportedly, banks in Lebanon have been charging interest rates that put loans out of reach for most business owners. Another BA pointed out that operating costs for many businesses, such as electricity bills among manufacturers, have increased substantially and constrained growth, thereby limiting participation in LED. The Lebanese economy was cited with equal emphasis among the beneficiaries. Less than one quarter suggested that the economy was not a major factor for them, but all the rest said it impacted their operations and the work with LED in a major way. The most common effect cited was decreased sales. Explained one MSME cofounder, “We have a bad situation in Lebanon. The market is not stable. Every month we have an issue. People don’t have purchasing power.” Others said the economy had generally slowed their expansion plans. Still others cited recurring problems in collecting payment from Lebanese customers, which has compelled a number of MSMEs to turn to international markets. Said an MSME co￾owner, “In the export market, they pay half upon packaging and half when the container leaves or is shipped. If I don’t have suppliers outside, I cannot survive.” Even among the beneficiaries that had turned more to international markets, there were problems related to the Lebanese economy. Chief among these was the currency devaluation, which greatly affected business that needed to transact internationally in U.S. dollars. As one owner explained, “We have a problem with currency. There are no dollars in the banks. We are forced to exchange in the black market and then pay our suppliers in dollars.” A landscape issue that was cited alongside and sometimes synonymously with the economy was the political landscape. Some beneficiaries saw all economic challenges as rooted in the country’s political instabilities. Said one owner, “We don’t have an economic problem, we have a political problem.” Another general manager elaborated on a specific economic challenge rooted in politics, which is that he was prevented by law from exporting his finished food products to Syria, at the same time that Syria goods were allowed to be imported, undercutting his pricing and sales in many cases. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 30 In addition, LED leadership cited a range of other constraining factors in the Lebanese business landscape with less frequency (cited 1–2 times). These included: ● Lebanon does not have the market to support a BLA. ● Big companies are not interested or difficult to approach. ● Creating jobs in Lebanon is just difficult. DEMAND-SIDE/BENEFICIARY CONSTRAINTS The beneficiaries themselves were seen as presenting a variety of constraints to success, generally in their response to certain requirements of the project. In some cases, these requirements have served as deterrents—that is, MSMEs have opted out of the program because they did not feel they could comply. In other cases, the requirements have created challenges for firms that did choose to participate. Among LED leadership, there was mild consensus around the idea that MSMEs take longer to create jobs than project targets allow. Some of the LED managers cited this concern, but the partner managers were especially mindful of it. As one such manager explained, “If you were expecting to have 3,000 jobs and you have only 200, the big question is whether the 3,000 jobs will happen after (the end of the project).” Another partner cited undue pressure that has developed as result of this time issue, “Chemonics was pushing and saying you are very slow. I showed them that that there is a problem in the concept itself. There is this time-lag issue that is not taken into consideration.” Several LED managers also suggested that the cost-share requirement was a burden and hindrance to success. In addition, LED staff cited a wide range of other constraining factors among the beneficiaries (generally cited 1–2 times). These included: ● MSMEs are constrained by cash-flow issues. ● Paperwork is a major burden to MSMEs. ● Some MSMEs have divergent ideas on what they need. ● The project ultimately has no control on what the MSMEs do. ● Progress on sales is slow to show at the firm level. ● MSMEs want to hire non-Lebanese, and these jobs don’t count toward targets. Among the BAs, the most common response in this area was around cost share—that is, the cost share as a high burden to MSMEs. Explained one BA, “Clients are facing cash-flow problems, and this is a major reason for not accepting the cost sharing, as they are not able to pay.” A response nearly as common was that potential client firms simply did not see value in a project that provided TA and “experts” only as its intervention. Several suggested that Lebanese business owners had a mentality by which they saw little value in the input of experts. In other cases, the project was contrasted with other aid projects that provided grants for facilities equipment—business owners saw value in this form of material aid and were used to receiving it. One BA cited the general “unprofessional” quality of some MSMEs, particularly in the Beqaa region; these firms either chose not to participate or struggled to succeed in the project because of their difficulties in meeting the administrative requirements. This included the firms’ resistance to the constant requests for monitoring data. (The BAs also cited the monitoring data requirements as a burden to themselves.) 31 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV As the direct service providers to the MSMEs, the consultants reiterated several of these challenges. In discussing constraints, one of the two most common responses was that some MSMEs had cash-flow problems that impacted their ability to join and/or succeed in the project. In some cases, this related to the cost share; in other cases, it related to their ability to comply with the recommendations that the consultants made for improving the businesses. The second common response once again related to mentality: some MSMEs were simply resistant to change, especially coming from an outside expert. One consultant said his client “wasn’t accepting an external person imposing new rules and regulation.” Another added that “the mentality of the employees and the owner was hard to deal with.” ADDITIONAL CONSTRAINT 1: CHALLENGES RELATED TO USAID DIRECTIVES There were several key constraints cited that were related to USAID’s directives and organization of the project. Chief among these constraints was Objective 2, seen by many within project leadership as a partially developed push to improve the BEE for MSMEs and job creation. Thus far, the work on Objective 2 has been difficult to balance against the key outcome of firm-level job creation. Said one manager, “The project is regularly challenged to achieve its job markets, and the job targets are significant. To meet those targets, we really have to work hard. It’s hard to find time for Component 2.” These concerns around Objective 2 are discussed more fully later in this report. ADDITIONAL CONSTRAINT 2: STAFFING AND OTHER OPERATIONAL CHALLENGES Among LED leadership, there was considerable discussion of staffing challenges as a constraint to success. The project has faced drawn-out hiring processes and turnover particularly in the area of BA positions. One partner manager noted that vacancies and hiring of BAs has been “a delay factor to reach our monthly target in MSME outreach.” Partner managers generally agree that the hiring of BAs has been based on a strategy that has evolved over time. A partner manager explained that at first, they were looking for “well-connected, typical salesman” but found they were not great with the project’s considerable administrative requirements, and hence did not last. Since 2019, they have been targeting individuals with the following: strong professional networks, proper communication skills, meticulous and committed work styles, administrative skills, and writing skills. Several other operational constraints were cited. One partner manager felt that the project’s slow start hindered its potential to reach targets. One LED manager felt that funding allocations were too low, explaining, “We have to work with a lot of companies and at the end of the day we don’t have a lot of resources for them.” ADDITIONAL CONSTRAINT 3: CONSULTANT SELECTION CHALLENGES In addition to their reiteration of the previous two themes, the BAs mentioned several constraints related to the consultants’ selection process. Most commonly, they cited challenges in finding highly specialized consultants in certain areas. In a few cases, deals were canceled because of these selection challenges. Others cited the time it takes to identify and hire new consultants generally (as opposed to drawing them from the existing pool). Most described a process that takes up to several months and sometimes tests the patience of the MSMEs. Finally, one BA pointed to a case where the MSME owner was resistant to move forward because he was more comfortable working with another consultant than the one LED had chosen for him. In this case, the BA needed to explain the grading system and bidding process that favored the “most competitive bid.” USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 32 CONSISTENCY FACTOR AND POTENTIAL EFFECTS Another factor in the project’s achievements is the level of consistency on its application of certain policies toward MSMEs; in particular, the cost share, minimum hiring commitment, and number of TA services allowed per MSME. This factor is not easily characterized as an “enabler” or “constraint” to success, in that it could be either. On the one hand, it can be argued that flexibility was needed, especially in the beginning, to help meet the needs of a diverse range of MSMEs in a difficult economic time for the country. On the other, the inconsistency has left some MSME owners uncertain about the rules of the project and could become an even greater problem if more MSME participants began speaking with each other and comparing deals. On cost share, according to LED management, all deals are required to have a cost share, except for “a few” at the start of the project. The running average of cost share is around 33 percent, with larger firms and those involved in franchising paying up to 50 percent. However, from its sample of just over 30 MSMEs, this evaluation found five MSMEs that claimed to be paying no cost share and a wide range in the percentage paid by the other firms, in some cases as low as 10 percent. Of the five that claimed to pay none, most but not all were early enrollees in the project, and there was no correlation with other factors such as firm size. On the job commitment question, LED’s stated policy is that it always requires a commitment of at least five jobs over two years from client firms. However, in the evaluation sample, almost one third of beneficiaries interviewed indicated that they were not required to make any commitment on jobs, or they have no recollection of such a commitment (and no intentions to comply with one). On the question of the number of TA services per job, either simultaneously or in multiple rounds of assistance, the LED project indicated that it did not maintain a hard-and-fast rule. However, the project notes that it is expected to achieve its target for assisted enterprises, and repeat assistance does not count toward those results. As a result: “the bar for repeat deals is high, and the expectation is that they will be rare occurrences.” What we can say from our interviews is that we did encounter a handful of cases of MSMEs receiving multiple services. We found many more MSMEs expressing frustration that they were denied multiple services, and that the policy on this was never explained clearly and transparently to them. Given the nonprobabilistic nature of this sample, we would view these findings as exploratory, but clearly pointing to some concerns around consistency. It is possible that some of these cases may be explained away by inaccurate recollections with the MSME representatives we interviewed, though that seems somewhat unlikely. Again, it is difficult to say whether the net effect of such inconsistencies might be an “enabler” or “constraint” to project success. EVALUATION QUESTION 2.3 (EFFECTIVENESS) How cost-effective has the BLA been in meeting LED’s objectives and KPI targets? Summary Findings, Question 2.3: ● The project-wide figure on cost/committed job stood at $534—well below the target of $1,000/job. 33 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ● Most efficient on jobs was the services sector, which encompasses restaurants and hospitality, as well as most of the project’s franchised businesses. ● In TA type, product/design development emerged as most cost-efficient, followed by HR and ISO/FSSC certification. Most expensive were IT interventions. ● Large firms were most cost-efficient, with efficiency decreasing as one moves down the scale of firm size. Conclusions, Question 2.3: ● The primary drivers of cost-efficiency are scale—that is, bigger businesses are more efficient in creating jobs. Further disaggregation on cost-efficiency is constrained by the relatively small client population. COST-EFFECTIVENESS To examine cost-effectiveness, we started with the project’s MEL data and considered two job creation metrics: (1) cost/committed job, whether or not the firm has followed through and created the job (an official metric used by the project); and (2) cost/actual job, meaning the more parsimonious measure based on actual jobs created. We relied more on the former. We also looked at the ratio of LED investment relative to pledged sales increased and pledged leveraged investment, keeping in mind that these are pledged rather than actual totals. We considered these measures as project-wide but also cut or disaggregated in four ways: by firm sector, by TA type, by firm size, and by region. Then we compared these database findings with the testimonies of the various stakeholders we interviewed. The project-wide figure on cost/committed job stood at $534 as of September 19—well below the target of $1,000/job (see Table 6 below for project-wide and disaggregated cost-efficiency figures). However, we must consider that the figure is based on a projection, and to date, we have no proof that firms will follow through adequately to produce a comparable figure based on actual jobs. The current cost/actual stands at $1,237, well over two times the committed figure. In terms of sector disaggregation, by necessity, we employed the same five-sector divisions employed by the LED project in its MEL work, as seen in Table 6. The winner in terms of cost/committed jobs is the services sector, which encompasses hospitality (particularly restaurants) as well as the majority of businesses that are pursuing a franchise model with LED’s assistance (also restaurants/food service). This finding is unsurprising, given the dominance of a handful of restaurants chains/franchises in the project’s productivity on jobs. It is also reflected in the interview testimony, where respondents most-commonly cited franchising as the most cost-effective sector, followed by hospitality. (We recognize that franchising may be misidentified as a sector here, but most of the project’s franchises are in the food￾service sector, thus making franchising and hospitality somewhat synonymous with each and linked to LED’s services category.) USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 34 Table 6. Disaggregated Data on Cost-Effectiveness VARIABLE COMMITTED JOB COST ACTUAL JOB COST SALES/LED INVESTMENT RATIO INVESTMENT/LED INVESTMENT RATIO Project-Wide $534 $1,237 21/1 175/1 Sector Agriculture & Environment $600 $3,121 7/1 294/1 Industry & Manufacturing $553 $1,143 37/1 207/1 Information Technology $609 $5,226 8/1 119/1 Services $475 $953 6/1 145/1 Other $758 $1,907 3/1 132/1 Type of TA Branding & Marketing $702 $1,547 9/1 133/1 Franchising $655 $953 6/1 82/1 ISO/FSSC Certification(s) $452 $1,260 16/1 290/1 Global GAP $623 $4,828 4/1 308/1 HR $425 $1,057 98/1 123/1 Management Training $607 $1,784 29/1 186/1 Sales Training $666 $6,452 2/1 227/1 IT $829 $6,632 2/1 103/1 Product Design/Development $382 $1,449 11/1 238/1 Other $465 $716 6/1 180/1 Size of Firm Micro $681 $2,370 17/1 166/1 Small $622 $2,531 8/1 167/1 Medium $466 $1,035 4/1 166/1 Large $382 $441 74/1 218/1 Region Akkar $472 $865 114/1 534/1 Baalbek-El Hermel $606 $1,717 0.5/1 117/1 Beirut $589 $1,873 4/1 171/1 35 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Bekaa $462 $893 3/1 208/1 El Nabatieh $239 $338 17/1 858/1 Mount Lebanon $570 $1,178 50/1 154/1 North $575 $1,680 7/1 114/1 South $605 $39,336 38/1 184/1 In terms of the type of TA, product/design development emerged as the most cost-effective relative to committed jobs. That was followed by HR training, restructuring TA, and ISO/FSSC certification. Most expensive were IT interventions, followed by branding and marketing. From the interviews, we see some but not complete consistency with these findings. When asked about the most cost-effective intervention type under LED, management pointed to several: ISO certification, product development, HR restructuring, and franchising. Coming into the inquiry, there seemed to be some expectation that franchising would emerge as the most cost-effective form of TA—we found it in the interview data but not the MEL data. However, here we must remind the reader of the relatively small sample (126 firms) and the likelihood that this sample might yield nonrepresentative results when cut 10 ways, as we needed to do in the case of TA type. The clearest trend on cost/job for any of this disaggregation is found in firm size, where we see a clear downward trend, starting with microenterprises and hitting its low point with large firms. This is solid and unsurprising evidence that larger firms created larger numbers of jobs more efficiently. The cuts by region suggest a low outlier in El Nabatieh and the highest costs in Baalbek, but these must be viewed as exploratory findings at best, given the project’s vastly uneven distribution of activities among the governorates. This finding on firm size is reflected in some of our interviews with LED staff, but the views are not entirely consistent. Among the BAs, for example, interviewees suggested that factories with large machinery and production tend to hire more people and are more likely to be cost-effective. Yet other BAs said the opposite, stating that smaller firms are more cost-effective than medium and large companies because of the small TA intervention required. The metrics of LED investment relative to pledged sales increased and pledged leveraged investment are both similarly exploratory, and must be considered in light of the varied circumstances that MSMEs faced in their needs and abilities to expand sales and invest. On expanded sales, HR-focused TA emerged as the most efficient, which is unexpected, given that several interviewees suggested that TA interventions do not immediately affect sales. ISO/FSSC Certification and Global GAP TA emerged as associated most strongly with high leveraged investment. This finding does stand to reason, in that both types of activities required firms to follow through relatively high certification fees that were not covered by the project. THE COST/JOB METRIC The cost over job ratio employed and reported by the LED project, per agreement with USAID, must be recognized as a metric of limited scope and utility. As it is derived simply by dividing LED’s contribution to the consultant fees by the number of jobs created, it fails to factor in a wide variety of USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 36 additional costs on both the demand and supply sides of the project. On the demand side, this includes the MSME’s investments necessary to align with the TA requirements—factory upgrades or ISO annual fees, for instance—as well as the MSME’s cost share contribution to the TA and overhead costs (direct and indirect) incurred by participating in the LED project. On the supply side, this includes overhead costs (direct and indirect) incurred by multiple organizations (Chemonics, BIAT, Berytech) at multiple levels (project offices, branch offices, headquarters, etc.). It also includes idiosyncratic factors such as the value of any time donated by the consultants beyond the scope of their contracts (which we know took place in some instances). USAID continues to have discussions with the LED project about more inclusive and meaningful costing metrics, but to date, this has not been implemented. This level of analysis was also beyond the scope of this evaluation. To be executed correctly, it would likely require the input of a highly experienced costing research expert. It is worth noting that most LED management was highly critical of the metric as it currently stands. “Cost per job is a simplistic metric. Our cost is a push to make them decide to grow, but it is not all that happened,” explained one LED manager. “I think for LED, management, as a management tool, is useful and informative about LED and its efficiency. But it’s not the true cost per job in an economic sense. We would need a wider analysis to capture that.” On firm size, large firms once again emerged as the most cost-effective in terms of linking to increased sales and leveraged investment. This also stands to reason, given that these larger firms tend to have proportionately higher sales potential as well as reserves of cash for investments. Disaggregation by region once again produced a quirky distribution that is likely reflective of a small sample size and uneven distribution, rather than findings or a trend that can be reasonably explained. EVALUATION QUESTION 3 (EFFECTIVENESS) How reasonable and achievable are the committed/forecasted numbers, both in terms of jobs to be created, sales increases, and investments to be made at individual enterprises and project-wide goals? Summary Findings, Question 3: ● By its official metric of FTE jobs, the project is well below target, with 476 FTEs near the end of Y2, where the target stands at 1,470. ● We find a positive sign in the current level of committed jobs, which exceeds the FTE target for the end of Y2; actual hires (not FTEs) stands at about two thirds of the target. ● Using projections based on key variables for Y3, the actual number of new hires exceeds the target in the more optimistic scenarios; however, in no scenario does the project reach its target on the official metric of FTEs. ● Success on job targets is strongly mediated by a critical time lag between the provision of TA and creation of the associated jobs, and by the 12-month nature of the FTE measurement. ● LED operates with a wide range of criteria for firm participation—rules that cause many firms to be rejected by the project and others to opt out; these criteria may need to be revised to support a more aggressive run at the targets. ● The project continues to struggle with several aspects of Objective 2, including: (1) the general clarity of its mandate in this area; (2) how to balance Objective 2 with Objective 1; and (3) the perceived low level of resource allocation for Objective 2. 37 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ● Achievements to date on Objective 2 include both general BEE efforts (events/meetings and a BEE assessment) as well as a sector-focused effort to build tourism in Lebanon (includes attending international tourism events and work with tour operators). ● Objective 3 is seen as solely to support Objective 1; efforts to date include a variety of outreach events, strong social media presence, and distribution of project materials. ● Questions remain about how group-based training (e.g., workforce development) fits with the project mandate, but LED continues to support training initiatives to a limited extent, mostly through its small grants program. Conclusions, Question 3: ● The LED project will not reach its target on jobs unless: (1) targets are lowered; (2) target measures are altered (e.g., actual hires instead of FTEs); and/or (3) USAID allows post–period of performance (PoP) monitoring. ● At present, the mandate around BEE is poorly understand and functions largely as a distraction from Objective 1. PROJECT-WIDE AND FIRM-LEVEL TARGETS To appreciate the project’s outlook for project-wide targets in the base period, we will consider the data from Table 2, with added projections for Y3 (Table 7). Table 7. Progress to Date on Job Targets with Y3 Projections Y1 Total Jobs Created 8 Y1 FTE Jobs Created 2 Y1 Committed Jobs 327 Y1 FTE Target (new) 392 Actual Jobs Created (as of 09/19/2019) 992 Actual FTE Jobs Created (as of 09/19/2019) 476 Actual Committed Jobs (as of 09/19/2019) 2,299 Y2 FTE Target (new & continuing) 1,470 Y3 FTE Target (new & continuing) 3,626 Scenario #1: All committed jobs hired & project doubles job outputs in Y3 (75% conversion active-to-FTE) 4,598 actuals 3,449 FTE Scenario #2: Half committed but currently unfilled jobs are hired & project doubles job outputs in Y3 (75% conversion active-to-FTE) 3,292 actuals 2,469 FTE Scenario #3: All committed job hired & project adds 50 percent job outputs in Y3 (75% conversion active-to-FTE) 3,448 actuals 2,586 FTE Scenario #4: Half committed but currently unfilled jobs are hired & project adds 50 percent job outputs in Y3 (75% conversion active-to-FTE) 2,469 actuals 1,852 FTE USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 38 Currently, by its official metric of FTE jobs, the project is well below target, with 476 FTEs near the end of Y2, whereas the target stands at 1,470. At the same time, we find a positive sign in the current level of committed jobs, which exceeds the FTE target for the end of Y2. This is not suggesting that one can project a one-to-one equivalence between committed jobs, actual jobs, and FTE jobs. Currently, firms have created an average of 47 percent of the new jobs to which they have committed, and only 48 percent of those jobs have endured at a full-time rate long enough to qualify as FTE. But there is every reason to believe both percentages will rise over time, especially considering that most firms are not even close to the end of the two-year window in which the project measures job creation after TA (see additional details on the measurement window next). Beyond the project’s current position, to make some projections, we started with several assumptions. First, we assumed a conversion rate of actual-to-FTE across the board at 75 percent (a substantial increase that takes into account the notion that most of the jobs created by the end of Y2 will have matured into FTEs by the end of Y3). Then we examined the numbers on a full-factorial basis, employing the following varied scenarios: (1) all committed jobs become actual jobs; (2) half of committed but currently unfilled jobs become actual jobs; (3) the project doubles its job outputs in Y3; and (4) the project increases its job outputs by 50 percent total in Y3. The results are seen in Table 7 and graphed in Figure 2. As indicated, in the most optimistic case (scenario 1), the actual number of hires far exceeds the FTE target; the actual FTE count does not meet the target but comes quite close. In all other scenarios, neither actual hires to FTEs rise to the Y3 target. Figure 4. Y2 Actuals and Y3 Target for New Jobs 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Y1 Y2 Y3 (projection) Actual Jobs Created (low & high proj) FTE Jobs Created (low & high proj) Targets 39 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV These scenarios raise the distinct possibility that the project will fail to meet its targets in the case period, measured both in terms of total job creation and in terms of the project’s official metric of FTE. However, there is a key mediating consideration here, which is whether the project will in fact meet or exceed its target of attributable FTEs, just not on the project’s original time. It is clear that LED faces a critical twofold time lag inherent in the project’s activities, as follows: ● It takes time between the provision of TA and creation of the associated jobs, often combined with leveraged or other investments by the firm itself. On this basis, the LED project forecasts impact over a two-year window. The upshot is that this will end before a substantial number of jobs attributable to the project can be counted. For example, in the case of base period–only measurements, a firm that opts into the project midway through Y3 might not be poised to make its new hires for a year or more after LED ends. Like many USAID projects, LED is not designed and funded to allow for post-project monitoring by the implementing partner. ● The nature of FTE measurements involved a substantial time lag, in that the jobs must be running on a full-time basis for 12 months to be counted. Taking again, for example, the case of a firm brought into the project midway through Y3, such a firm might follow through and create all its committed full-time jobs right away, but each will run a maximum of six months before the project ends, meaning that said firm will be recorded for creating half as many jobs (on an FTE basis) as it actually did. SUCCESS STORY ON JOB TARGETS One of LED’s client firms is a well-known restaurant located in Anjar, a tourist destination in the Beqaa region. The restaurant was founded in 1975 by the grandfather and patriarch of the family that still owns and runs it. It began as a small grocery and cafeteria shop and was upgraded to a restaurant in 1985. Since then it started to expand gradually to be one of the biggest restaurants in the area. Currently, it is spread across six different spaces, with a total capacity of 3,700 seats and a playground of 100 square meters. In addition to the restaurant, the family owns and runs an associated hotel and wedding venue, and the business has two franchised fast food shops. Because of the fast growth and expansion it witnessed between 2000 and 2015—adding four snack outlets, a hotel, a resort, and a wedding venue—the restaurant faced difficulties to maintain its quality of service and meeting basic standards (hygiene, communication skills, business and service etiquette, etc.), which led to customer dissatisfaction and decreased sales. These problems were caused by and/or exacerbated by organizational and managerial weaknesses, namely a lack of clear and optimized systems and procedures, and an HR structure that did not clearly assign responsibility and accountability for most of the company’s services and functions. Poor financial management practices also contributed to the restaurant’s problems. And lately, because of labor market changes in Anjar, the company lost some of its workforce. Yet despite these problems and the associated negative impact that they have had on its brand, the company still enjoyed a strong reputation, and the owners were making investments to support growth. Improved management, human resources, and financial systems were needed to match the company’s physical upgrades. LED supported the restaurant to improve management of systems, procedures, structures, and accountability mechanisms to improve the group’s entire operations and human resource base, while focusing on how to improve and maintain the quality of the food and customer experience across the restaurant. The idea was to boost the brand name and provide a solid foundation for growing the company’s revenues and profitability, which in turn would support the company’s plan to increase the quantity and quality of its workforce. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 40 If USAID/L were to exercise its option to extend the project another two years, it seems unlikely that the complication presented by these time lags would diminish. It presumably would allow enough time for all jobs created in the base period to materialize and mature into FTEs. But there would be a whole new set of MSMEs recruited in Y4 and Y5 that would present these complications all over again. CHALLENGES AROUND FIRM SELECTION (INCLUDING ANALYSIS OF “LOST DEALS”) LED operates with a wide range of selection criteria for MSMEs to be included in the project. The criteria prominently cited in the interviews we conducted with LED’s staff included: (1) must have a clear path to growth; (2) must have clarity around market opportunities that will support that growth; (3) must have a capacity for at least five new jobs; (4) must be willing to pay the cost share; (5) must be legally registered as a business; and (6) must pass U.S. Government vetting for sensitive political affiliations. As of September 19, the LED project had enrolled 126 firms, after profiling 590 firms. These numbers stand quite close to Y2 targets of 150 firms enrolled and 750 profiled (84 percent and 79 percent progress, respectively). Undoubtedly, these numbers will rise somewhat in the final days of Y2. This raises the immediate questions of why the project is so close on firm targets yet so distant on the project’s FTE job creation target (476 FTEs created against a target of 1,470, or 32 percent progress). This report has already suggested a number of key factors in that divergence, the most important likely being that the project’s designers failed to take into account the time-lag factors on both initial job creation and maturation into an FTE that can be counted. Of the 590 firms profiled by LED, 314 MSMEs or a little over half did not enroll and are characterized as “lost deals.” From this number, a question may follow: to compensate for the slowness with which jobs and especially FTEs have been created, should the project have a more aggressive position on enrollment—perhaps turning more of those 314 lost deals into assisted firms and vastly exceeding the Y2 target on enrollment? This evaluation takes no retrospective position on this issue but offers some analysis of the lost deals, in order to gauge feasibility going forward, if the project and USAID/L were to decide that a more aggressive stance is advisable. By our analysis, the majority of lost deals (around 200) were MSMEs that were rejected by the project, based on perceived ineligibility for one or more reason. Around 114 deals were cancelled when the potential client opted out. Table 8 shows the most common reasons cited for both varieties of lost deals, according to the LED database. 41 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Table 8. Reasons behind “Lost Deals” Rejected by partners Opted out by the client Start-up & microenterprise companies with no job creation MSME doesn’t have budget for activity Syrian shareholder Deals postponed by the owners Looking only for investment MSME looking for grants only No additionality MSME is downsizing Minimal employment MSME is closing completely Absence of expansion plan No responses from the client Vetting MSMEs involved in other activities Absence of special expert No need for TA now Social enterprise Working with other project with same approach Rejected directly by LED management/partners/USAID (unspecified reason) Client decided not to proceed/is not interested anymore (unspecified reason) No BLA On the beneficiary side, most of the reasons cited suggest that the deal was a nonstarter for LED, because of factors beyond LED’s control. On the other side—cases where LED rejected the project— there may be some room for reasonable change that would allow more deals to take shape. The outright rejection of start-ups and social enterprises, for example, are positions that could be revised based on mutual agreement between the project and USAID. In interviews, partner management and staff had varying points of view when it came to the number of firms selected to join the project. It was clear that BAs were relying heavily on their personal professional networks to recruit firms, though over time they were drawing on more systematic resources such as online directories, outreach events, and referrals from Berytech and BIATs. At the same time, they saw their work as challenged by factors on all sides of the project. For example, some cited what they perceived to be geographic and sectoral restrictions. In the North region, several BAs suggested that it was difficult to find the kinds of innovative, export-focused companies the project was seeking. At the same time, the North offered great potential to recruit firms in the areas of agriculture, education, and health—all of which were not favored by the project, according to several BAs interviewed (though LED management has indicated that there is no policy against working with clients from the education and agriculture sectors). When asked for their views on improving the selection process (presumably resulting in higher numbers of assisted firms), partner staff pointed most commonly to reducing the cost share. As one partner manager said, “Cost share has been a real limiting factor. We are achieving it with some difficulty. There are very few projects in Lebanon that require direct cost share.” Other suggestions on improving the selection process included: USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 42 • Taking a market intelligence approach to build a solid pool of companies based on sectors. • Speeding up the process, from paperwork requirements to consultant selection. • Recruiting and hiring committed BAs. • Diversifying the TA, as currently most is focused on franchising and ISO certification. • Focusing less on job creation and more on income and investment. • Working with more syndicate and associations. • Increasing marketing and outreach events for LED. • Standardizing cost share, instead of the current varied approach. TARGETS/PLANS AND CHALLENGES AROUND OBJECTIVE 2 The project Objective 2, simply stated as “improve Lebanon’s BEE,” has emerged in this evaluation as one of the more contentious practices areas for the project. Although there is universal agreement on the importance of BEE to job creation, the project continues to struggle with several angles of its potential work on Objective 2, including: (1) the general clarity of its mandate in this area; (2) how to balance Objective 2 with the high demands of Objective 1; and (3) the perceived low level of resource allocation toward Objective 2. In terms of clarity for Objective 2, according to the testimony of LED managers, the project continues to face uncertainty around the extent to which they are expected to engage in dedicated BEE activities. One manager suggested that LED has effects on the BEE “in the course of our normal activities [around Objective 1]. What’s not clear is what we should be doing beyond that. If there are any expectations at USAID beyond that, we need clarity.” The common understanding of Objective 2 is that it centers, for the most part, on developing sector competitiveness. To date, the sector the project has focused on developing most is tourism, and LED has launched a number of specific initiatives in this area (see full details of the efforts around tourism described below in this section). At the same time, some LED managers do not feel certain that USAID has bought into the choice to focus on tourism11 At the same time, USAID has indicated its strong preference that LED work directly on building another aspect of the BEE, which is the private market for business consulting. USAID summed up its key questions in this area as follows: Is LED helping to build/support a sustainable, fair, transparent, and efficient consultancy market in Lebanon for MSMEs to allow phasing out of support in a few years or months? Will this market scale up naturally as buyers and sellers increase magnifying the impact and sustainability of USAID assistance? However, like BEE generally, the LED project indicates uncertainty on how they should respond. Stated an LED senior manager: “It’s unclear the extent to which we should be involved in that activity or objective. It’s unclear what we should be doing to build the local consultant market, and how we should do it.” Like BEE generally, project management feel the work they do under Component 1 helps build the consultant market organically “by building confidence in how to use consultants.” But what is unclear are the expectations beyond that. The project clearly is moving forward on some initiatives, such as a small grant to support a new business consulting umbrella group (see section on group-based training and the 11 It should be noted here that USAID’s position is that the current plan was discussed, understood, and approved as part of the LED’s most recent work plan. 43 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV small grants program). The project’s effects in the area of the consulting market are discussed in more detail in the section on sustainability in the consulting market. Beyond the choice of sector focus, the LED project remains uncertain how to “harmonize” the work under Objectives 1 and 2. One manager explained, “It’s unclear to me how engaged we should be in sector￾competitiveness, and what approach we should take, alongside the firm-level assistance mandate, given the tradeoff that exists between those two types of intervention.” The same manager added, “The project is regularly challenged to achieve its job markets, and the job targets are significant. To meet those targets, we really must work hard. It’s hard to find time for Component 2.” Compounding these difficulties in harmonization is the relatively low level of resource allocation to Objective 2—just 10 percent of the budget, according the contract. All managers who commented on this issue felt such an allocation is far too low to achieve meaningful results. One manager added that building real sector competitiveness is both resource-intensive and time-intensive—requiring 10–15 years, far longer than the period of performance for LED, in other contexts. ADDITIONAL CONTEXT FROM LED ON OBJECTIVE 2 In a written statement provided to this evaluation, the LED team elaborated its views on Objective 2. In the team’s view, for approximately the first 18 months of the project, until May 2019, Lebanon was stuck in a pre- and postelection period where public sector engagement and activity was limited. From the private sector side, the uncertainty surrounding the new government and the history of the private sector neglect by past governments, created an environment in which the private sector seemed not ready to engage on any BEE activity. Though LED consistently introduced businesses and private sector organizations to Objective 2 through one-on-one meetings and during outreach events and invited them to partner to advance BEE reforms, these actors showed little interest in this opportunity. Meanwhile, beginning in April 2018, LED and USAID began to discuss how the project could begin to adopt a sector-level approach—as opposed to the firm-level approach envisioned in Objective 1—to broaden LED’s impact. In a revised Year 1 work plan, approved in August 2018, tourism was identified as a potential sector, and an assessment of the sector was planned, approved, and carried out in November 2018. Following the assessment, in the Y2 work plan, approved by USAID in February 2019, LED presented a plan whereby most of LED’s efforts under Objective 2 would focus on addressing sectorial opportunities and challenges within tourism. The LED team suggests that positioning LED’s tourism sector interventions under Objective 2 is justified for several reasons. First, the nature of the proposed sector-level assistance will address systemic challenges related to market access, industry competitiveness, stakeholder cooperation and coordination, and possibly policy reform. These interventions will have industry and economy-level impact on jobs, sales (domestic and export), and investment. Second, as noted, the Lebanese private sector largely has not responded to LED’s invitation to engage in traditional BEE initiatives. However, in the team’s view, USAID appears to still have reservations about allowing LED to put most of Objective 2 resources into a tourism action plan. Despite these concerns and constraints, LED has moved forward on a variety of concrete activities under Objective 2. They can be summed up as follows: USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 44 Traditional BEE—Completed/Ongoing: Despite perceived challenges in the Lebanese landscape, the project has moved forward on some traditional BEE activities, such as: • Through events and meetings inviting private sector stakeholders, including chambers of commerce, associations, and syndicates, to partner with LED to advance BEE reforms, particularly concerning issues that they had already prioritized. • Commissioned a BEE assessment to be conducted by a local organization to uncover potential BEE actions—linked to growth and employment—that LED could pursue. The assessment will be completed in early October 2019. • Released a call for grant applications to local organizations that include a call for BEE reform ideas. Only one applicant, the Lebanese Center for Policy Studies, whose concept was only recently submitted, proposed an idea related to BEE reform. Traditional BEE—Proposed/Planned: • Organize a discussion including the LED team and client about the ideas contained in the BEE assessment to select those where LED has an opportunity to intervene. One idea relates to developing an export guide for the EU market that will help Lebanese companies to better understand opportunities and requirements to export to the EU. • Organize a workshop with LED client enterprises from various sectors or subsectors—likely starting with the food processing sector—to solicit their ideas about necessary and feasible legal, regulatory, administrative, institutional, or policy reforms. With stakeholder input, LED would then prioritize these ideas and then engage relevant stakeholders in a dialogue about how to accomplish the desired reform. This process can begin in Y3 and may uncover some quick-wins that can be tackled during the base period. • Provide a grant to Lebanese Center for Policy Studies, among other actions, to produce evidence￾based information that can be used by the Association of Lebanese Industrialists (ALI) to advocate for a better BEE for exports with decision makers. Tourism Sector—Completed/Ongoing: The project has moved forward on a number of activities to develop sector-competitiveness for tourism, with the stated goals of increasing the number of foreign travelers (tourist arrivals) in Lebanon and increasing their length of stay and in-country spending. Current activities include: • LED sent representatives to attend the Destination Academy with TripAdvisor Middle East event in Jordan, February 23–27, 2019. The event was a regional meeting and lecture series that focused on best practices in digital and content marketing and involving sharing knowledge and experience regionally. Destination Academy with TripAdvisor also provided insights into traveler behavior on their online path to purchase for travel to the region. • LED held several discussions with inbound tour operators about helping them to create a new association to better represent their interests and to facilitate collaboration among them. These discussions culminated in LED’s participation in a founding members’ meeting for a potential inbound 45 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV tourism association in Lebanon on March 4, 2019. Although there was support for the idea, the companies eventually dropped it because of resistance from the Minister of Tourism. • To build private sector capacity to attract visitors from new markets, LED organized a “Driving Growth and Building Resilience” seminar attended by more than 220 people. International tourism experts presented strategies and tactics for how tourism businesses can grow arrivals and capture markets, especially when faced with challenges concerning Lebanon’s image abroad. • To help position Lebanon as a destination of choice in global tourism, LED hosted food and travel blogger Mark Wiens and his crew for 2 weeks in Lebanon. Wiens has produced and released more than 40 videos on his YouTube, Facebook, and Instagram channels. Tourism Sector—Proposed/Planned: • LED will organize a series of “Know Your Market to Grow Your Market” seminars with a focus on geographic market-specific or niche segment-specific seminars, including culinary tourism, meetings, incentives, conferences and events, and experiential tourism. • Build Lebanese tourism trade digital marketing and sales capability. LED will launch a “Taking Lebanon’s Tourism into The Digital Age” initiative to revolutionize marketing and sales approaches and channels, including: − Organize “Trends and Innovations in Online Destination Promotion,” a digital/social media tourism marketing awareness and training seminar in partnership with Lebanese tourism syndicates and digital tourism platforms. − Conduct one-on-one tourism digital and social media promotion clinics for several Lebanese firms to provide customized advice on improving online sales and promotion. − Conduct training for Lebanon’s IT companies and social media firms to provide them with tourism-specific knowledge to equip them to effectively serve Lebanon’s tourism industry. • Develop sales channels in Latin America. Consultations with members of the tourism trade highlighted the rich opportunities for growth available from the Latin American market, especially Brazil, as these countries are home to millions of Lebanese expatriates. LED will: − Organize a “Strategies for Attracting the High-Spending Brazilian Market” seminar on the features and opportunities of the Latin American market, focusing on Brazil. The event will include a dialogue session between Lebanese and Latin American speakers. − Support a sales mission to Brazil with local partners, tour operators, hotels, and airlines organized in the Spring of 2020. This event will build on learning and momentum gained from the above seminar to translate learning into action. A FINAL POINT ON OBJECTIVE 2 AND ACCOMPANYING EFFECTS Finally, as noted above, some LED managers have suggested accompanying effects on the BEE or business landscape simply in the course of carrying out their activities under Objective 1. Perhaps the most notable example that emerged in several interviews with partner leadership were the effects on BIAT and Berytech. Several key staff point out that both are better positioned to serve the Lebanese USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 46 business community since starting with LED. Both have expanded their territories, and at least one has expanded its office/operating facility. Both partners reported that the project has vastly expanded their pool of consultants and deepened the relationship with consultants. This growth is expected to improve their services well beyond the end LED, and they attribute all of these positive changes directly to their participation in LED. TARGETS AND PLANS AROUND OBJECTIVE 3 Objective 3, defined as “Communicate results and share project knowledge with the public and private sectors,” is the project’s dissemination plan. Unlike Objective 2, the team expressed no concerns around Objective 3, since it was viewed primarily as an adjunct or support to Objective 1. Their work in this area can be summed up as follows: COMPLETED/ONGOING ACTIVITIES • Organized 23 outreach events: o 1 project launch in Tripoli, North Lebanon with BIAT, o 1 project launch in Beirut with Berytech, o 2 private sector outreach events for South Lebanon (Sidon), Bekaa (Zahle), o 15 LED Talks with different organizations and in various locations, and o 4 franchising workshops with Lebanese Franchise Association (LFA) and area chambers of commerce. • Participated in five outreach events organized by others: o 1 BIFEX organized by LFA, o 2 Antwork Leadership Programs, o 1 Dar Al Ali Association launch in Tripoli, and o 1 panel discussion entitled “The ABC’s of Funding a Hospitality Project and the Different Financial Sources,” organized by Hodema during HORECA. • Established a social media presence by creating 3 pages on Facebook, LinkedIn, and Instagram; and an online application in Survey Monkey shared on the 3 mentioned pages. • Created or in the process of creating 9 videos covering various events and focusing on several client firms through a series of short videos entitled “Pathways to Prosperity,” showcasing the support given to client firms and the impact on business growth and job creation. • Created 14 visuals to date, where each includes a photo and a quote for different client firms. These new social media templates will help us highlight clients’ success, update the story booklet, and build the USAID photo library. • Produced various informative materials and tools to create awareness about the project. These materials embraced USAID’s corporate identity and remained consistent to the brand while implementing all its outreach activities. Among the materials: o Project profiles in English and Arabic, o Flyers in English and Arabic, 47 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV o Roll-up banners in English and Arabic, o Story booklet updated regularly that includes 156 stories to date, and o Factsheets/infographics updated regularly. PROPOSED/PLANNED ACTIVITIES • During Y3, LED will continue to create content that promotes the project’s objectives and its expected impact at different levels. All assisted firms will be covered either through videos or visuals and testimonials. LED expects to draft at least 100 new stories and will also cover the stories related to the grants’ activities, their implementation, and results. • In addition to the Pathways to Prosperity videos, LED will produce thematic videos and stories focused on a specific type of assistance or sector. • LED will continue to organize LED Talks across the country. If the option period is not granted, LED plans to organize 5 more LED Talks during the first quarter of Y3. If the option period is granted, LED plans to organize a minimum of 11 more LED Talks. TARGETS/PLANS IN GROUP-BASED TRAINING AND THE SMALL GRANTS PROGRAM In addition to its wide range of firm-level training and specialized training for the tourism section, LED has offered group-based training in a variety of ways. Early in the project, several such efforts focused on workforce development and were offered through partner Antwork, a business space and entrepreneurial incubation organization. LED organized an outreach event with Antwork and went on to assist Antwork with a small workforce development activity that involved training and subsequent job placements. LED considered further support for Antwork and other workforce development activities, but ultimately backed off of the workforce development approach. A senior manager explained, “There were/are questions inside LED— and I think maybe within USAID also—about whether workforce development activities were the best intervention/s for LED. The questions relate to whether this type of intervention is the most effective and efficient at creating jobs or whether we are creating jobs at all; or are we simply helping to fill existing vacancies.” In the sense that people were employed following the Antwork initiative, LED was creating jobs (counted toward targets), but these jobs were less “valuable” than jobs created at the enterprise-level because of company growth. There were similar questions raised about whether LED’s involvement in group trainings generally was the most effective and efficient at creating jobs. A senior manager again explained, “In my experience with enterprise development projects, group-based trainings do not typically address the critical constraints that firms face. These constraints seem to be best tackled through tailored firm-level TA. Group trainings are useful for addressing more general or generic topics.” Still, the project continues to support group-based training initiatives to a limited extent, mostly through its small grants program, a $500,000 fund for the base period, with grants that focus largely, though not exclusively, on training programs. To date, two grants have been awarded, and four are in various states of planning, as follows: USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 48 Michel Daher Social Foundation—active and ongoing. In this 10-month grant, the Michel Daher Social Foundation aims to improve the stability and efficiency of companies in Bekaa, leading to the creation of job opportunities by organizing training series’ for employees of 11 local companies in nine key areas—supply chain management, autoCAD, automation programming, motor drive (AC and DC), generator synchronization systems, welding, electrical and mechanical installation, vehicle maintenance, and soft skills. LABORA—active and ongoing. In this 10-month grant, LABORA aims to improve the skills of 400 individuals (200 males and 200 females). LABORA will organize and deliver 70 hours of training covering technical skills (25 hours), soft and life skills (10 hours), Microsoft Office skills (20 hours), and business English (15 hours). Following these trainings, LABORA will place at least 140 individuals in new jobs with private sector employers. Society of Management and Development Consultants (SMDC)—pending approval. This grant will support a new umbrella organization for local consultants. See following box for more details. Women’s Association of Deir Al Ahmar (WADA)—finalizing grant package. WADA plans to scale up its catering operations and to increase its production and sale of traditional food products. Supporting this effort, WADA will organize a series of food-related trainings, upgrade the brand that is used to market its traditional food products, and develop and launch a small marketing campaign to promote these products. In addition, WADA will organize a mosaic tiling training. LFA—application being finalized. LFA plans to establish a Franchise Advisory Promotion and Export Program, where franchisors can turn for professional advice that will help them in managing their franchise more efficiently. LFA will also organize a Franchise Street Festival for Lebanese franchisors to promote their brands, identify potential franchisees, increase awareness about franchising, and increase their customer base. Finally, LFA will conduct a franchise awareness campaign to highlight the benefits of franchising to Lebanese entrepreneurs. THE SMDC INITIATIVE With the view that the business consulting market in Lebanon is expanding, LED has proposed support for an association called “Society of Management and Development Consultants” (SMDC) to be the collective voice of the Lebanese consultants, advocating for the profession. While it is not an area of impact originally envisioned as part of LED, the project responds directly to feedback from USAID that it should also play a role to strengthen the market for local consulting services. LED is aiming to provide a grant to the SMDC, which submitted a grants concept in response to LED’s call for grants. This grant, amounting to $52,200, pending approval, is expected to be awarded in October. It will strengthen the capacity of SMDC; support SMDC to become a fully accredited member of International Council of Management Consulting Institutes Global; and build the capacity of local consultants in Lebanon. In so doing, the grant will improve and develop the market for consulting services by better organizing consultants under the umbrella of SMDC, improving consultants’ capacity through specialized training and certification, and by creating new linkages between the private sector (e.g., businesses and business owners), and local consultants that will create greater opportunities for consultants and also help to accelerate private sector growth and employment. 49 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Lebanese Center for Policy Studies—invited to submit full application. The goal of ESD is twofold: first, to furnish ALI’s board with evidence-based information that can be used to advocate for a better BEE for exports with decision makers; second, to provide ALI members with market information—about markets or products—to promote their exports. EVALUATION QUESTION 4 (SUSTAINABILITY AND SCALABILITY) What is the likelihood that the results LED has achieved so far are sustainable and scalable beyond the life of the project? Summary Findings, Question 4: ● There is no indication thus far that improvements and new jobs will not endure. ● Most firms express very optimistic one-year and five-year outlooks. ● We examined the potential for sustained relationships between consultants and firms beyond LED; both beneficiaries and consultants were optimistic, but only a handful of cases have emerged. ● We examined the potential for sustained effects on the consulting market generally and found optimism but almost no effects so far. ● On scalability, two sectors—industry and manufacturing and services—far eclipsed the others in terms of the volume of jobs, sales, investment; the same two emerged most cost-effective in terms of jobs; relatedly, large firms showed similar links over smaller firms. ● The top three TA services linked to job creation volume were ISO certification, franchising, and HR. Conclusions, Question 4: ● We presently cannot make meaningful observations on sustainability. ● There is every indication that firms added jobs and improved businesses with permanent intent. ● Like cost-effectiveness, scalability potential is found most with larger firms—but this does not speak to the project’s mandate to serve smaller businesses. FIRM-LEVEL SUSTAINABILITY The window on sustainability at the firm level is limited, because most firms kicked off their assistance within the last year. To date, the indications are positive because we heard no stories about the jobs being created and then eliminated. Likewise, we did not hear any stories about firms abandoning their LED-related changes and improvements to the way they do business. The ET did query beneficiaries as to where they expect their businesses to be in one year’s time and five years’ time. As one would expect with that kind of query, the answers were overwhelmingly optimistic and suggested further growth in both domestic and international markets. On the one-year outlook, explained an MSME manager, “There is always a new market. In Lebanon or outside. My new plan is to go to the export market, Kuwait, Dubai, or even Saudi Arabia.” To a lesser extent, most MSMEs expect their sales to grow. “We expect growth of 2 percent to 4 percent on total sales,” stated one MSME manager. Another MSME manager operating in the agriculture sector and benefitting from ISO-related TA under LED, USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 50 declared, “If we acquire the certification, we would be selling to restaurants, and this would more than double our current supply and sales of meat in the market.” On the five-year outlook, most continued to point to growth and expansion, with somewhat more emphasis on the international markets. One fast food MSME owner said he would “open new markets in the Middle East … and look to target Europe” in five years. Some MSME owners were more restrained on the five-year outlook, citing difficulties for businesses in Lebanon to forecast operations in the current challenging economic and political situation. “If I [were] living outside Lebanon, I would have answered this question, but in Lebanon we have surprises,” said a MSME co-owner. EFFECTS AND SUSTAINABILITY IN THE CONSULTING MARKET As per USAID’s expressed interests, we explored actual and potentially sustained effects on the consultant market. Has the LED project helped establish relationships and trends in the provision of business expertise that will have lasting effects on both the employability of consultants and their positive influence on the Lebanese business landscape? We considered this question in two major ways: (1) evidence of sustained relationships between these consultants and the particular firms they worked with under LED; and (2) evidence of increased business for these and possible other consultants outside of the LED project, but based on LED’s influence. To the first point, we asked the beneficiaries whether they would hire their consultant or similar consultants again, should additional TA needs arise, but this time without the support of LED. Around two thirds of the sample stated they were impressed with the consultant to the point that they would indeed hire that individual again, even if they had to pay the full fee. As noted earlier, we must again convey the caveat that about one third of the sample already had a relationship with the consultant, and a handful had hired this particular consultant before LED. Only two beneficiaries indicated that they Figure 5. LED beneficiary in Koura. 51 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV were note sure they would hire again, and two others felt they could not afford to hire the consultant again. The assessment by the consultants themselves was also largely positive, though not quite as uniform. A majority felt that there was some likelihood of being hired again, while a bit less than half felt that clients would not hire them again and could not afford to hire them, or the consultant was unsure. The assessment of this issue at the BA level was similar. Partners management suggested that they will see a positive impact and a sustained relation between clients, and consult in the long run, since MSMEs want to improve their businesses. They noted that most Lebanese companies have lacked this type of interventions in the past, but that we may be seeing a shift in the culture and mentalities of MSME owners. LED management reiterated the optimism: “There’s a likelihood that the firm will come back and use the consultant.” In terms of evidence beyond the theoretical, we found two cases of MSMEs who already received the TA, and after seeing the support, signed an external contract with the consultant. On the broader question of whether these consultants had seen improvement in their non-LED business or in the consulting market generally as a result of LED, the responses were less positive and conclusive. The vast majority of consultants in the sample said they had received no new business related to LED, outside of the project itself. One indicated that he had a related deal pending, and one expressed confidence that new business would be coming. One consultant company noted a related positive effect, that they are working on affordable packages and solutions to be offered for MSMEs so they can hire them against monthly fees. A STRONG CONSULTANT RELATIONSHIP This story begins with a well-known coffee factory located in Jounieh, established since 1966. The firm began with LED in 2018, seeking support in HR and sales. On this basis, LED presented three different consultants from which the beneficiary could choose. The firm made its choice based on a combination of the consultant’s expertise across HR and sales, plus his cost, which the client found to be very reasonable. At the HR level, the consultant started from scratch and built a new department, which in turn has provided a new organizational structure to the company. The work has included intensive training for the HR team on how to recruit and how to deal with employees, plus drafting job descriptions for each position and creating an internal rotation system for employees based on their qualifications. At the sales level, the consultant helped build a new marketing strategy based on the geographical area and relevant value chains, and brought in new techniques for attracting clients. The consultant also provided an intensive one-week sales training for 25 salespeople and agents located in Bekaa and Zgharta. Because of the owner’s high satisfaction with the TA, the relationship with the consultant will endure beyond LED. The company has signed a nine-month contract with the consultant for additional sales support, completely independent of LED and paid in full by the company. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 52 A related discussion that arose among the consultants interviewed was around the effect of the development sector on the consulting business generally. About half of the consultants stated that recent growth in their business was limited to the development sector, and that most all consultancy jobs in general were linked to different programs in the development sector. They saw the development sector as affecting the consulting business both positively and negatively. Some consultants saw the development sector as decreasing the quality of consultant deliverables because of the huge number of projects held by each consultant in parallel and in a limited time. Others saw the development sector as opening new markets for them. Several consultant consults commented on an enduring reluctance among MSMEs to invest in TA. “The problem is the decision to invest money in nontangible service, which is 80 percent of the market mentality,” explained one consultant, who added these relationships still rely heavily on development sector subsidies. Another consultant suggested that when MSMEs owners do unsubsidized consultant support, they tend to favor international consultant firms as more valuable than local consultants. The majority of partner staff was upbeat about potential effects on the consulting market—though obvious they are not as close to the issue as the consultants themselves. They suggested that LED is helping to change the mentality of the business community, with more owners viewing consulting services as a necessity to improve. They also pointed out that LED has allowed experts to reach remote/rural areas that historically have not had access to such expertise, such as the Beqaa region, and that this is helping change the mind-set even further. One LED manager suggested that LED’s work is helping correct past failures in the use/engagement of business consulting in the Lebanese market—that is, businesses hiring consultants, not receiving the needed support, and wondering what they paid for. Respondents at all levels agreed that the state of the Lebanese economy has hindered many positive effects on the consulting market. The priorities of MSMEs remains to invest in machines and raw materials to increase their production and maintain their position in the market, prior to investing in TA to improve their management. SCALABILITY OF PROJECT SUCCESSES We begin on the question of scalability with Table 9, which presents the sheer volume or scale of productivity on jobs, sales, and investment disaggregated again by sector, type of TA, size of firm, and governorate. Table 9. Committed Jobs, Pledged Sales Increase, and Pledged Leveraged Investment TOTAL COMMITTED JOBS TOTAL PLEDGED SALES INCREASE TOTAL PLEDGED LEVERAGED INVESTMENT Project-Wide 2,299 $25,566,140 $215,198,530 Sector Agriculture & Environment 52 $228,000 $9,190,000 Industry & Manufacturing 1,058 $21,659,800 $121,077,530 Information Technology 163 $762,950 $11,842,000 53 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV These MEL data suggest that two sectors—industry and manufacturing and services—far eclipsed the others in terms of the volume of jobs, sales, and investment. We will recall these two also emerged as Services 839 $2,410,140 $57,787,000 Other 83 $206,250 $8,330,000 Type of TA Branding & Marketing 163 $974,250 $15,252,000 Franchising 428 $1,574,000 $22,970,000 ISO/FSSC Certification(s) 624 $4,416,000 $81,979,000 Global GAP 31 $85,000 $5,950,000 HR 368 $15,286,500 $19,297,000 Management Training 94 $1,627,640 $10,630,000 Sales Training 126 $207,500 $19,017,530 IT 80 $163,950 $6,800,000 Product Design/Development 148 $605,800 $13,451,000 Other 237 $625,500 $19,852,000 Size of Firm Micro 341 $3,904,000 $38,644,530 Small 838 $4,312,250 $86,912,000 Medium 424 $705,640 $32,900,000 Large 575 $16,241,250 $47,910,000 Governorate Akkar 44 $2,364,000 $11,085,000 Baalbek-El Hermel 17 $5,000 $1,200,000 Beirut 350 $880,250 $35,317,530 Bekaa 717 $1,117,300 $68,755,000 El Nabatieh 41 $170,000 $8,400,000 Mount Lebanon 612 $17,661,140 $53,573,000 North 453 $1,868,450 $29,628,000 South 65 $1,500,000 $7,240,000 USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 54 the most cost-effective, at least in terms of jobs. These are the project’s larger businesses, from factories to restaurant chains, with over 90 percent of the workforce across the entire LED portfolio. We get the same corresponding finding on outcomes when we disaggregate for “large” on firm size. It would be difficult not to point to these larger businesses and their results as the most expedient way to scale up these kinds of activities. Looking at disaggregation by TA type, the findings are somewhat more ambiguous. The top three services linked to job creation are ISO certification, franchising, and HR. We know from the earlier cost-effectiveness analysis that ISO certification and HR are among the most cost-effective, but franchising is not. Additionally, these three services account for about half of the interventions under LED, but about two thirds of the committed jobs, which does suggest a certain efficiency and perhaps more scalability. HR was the clear winner in terms of linkage to pledged sales, while ISO certification was linked to the highest leveraged investment. Disaggregation by governorate once does not suggest any immediate meaningful patterns, with the picture skewed by the vastly uneven distributions of MSMEs and interventions by region. We do note that the highest volume of jobs came from three regions: Beqaa, North Lebanon, and Mouth Lebanon. We would attribute this to LED’s focus on these regions but also the predominance of the agri-food and hospitality business, which have tended to produce many jobs. In interviews, we asked respondents from all levels for their views on which types of firms and TA services has been most effective in producing results. We found very little consensus at any level on these questions. Both LED staff and partners suggested that agri-foods benefited the most from LED, followed by hospitality, tourism, and other manufacturing. These findings correspond loosely to the data findings above that suggested the “industry and manufacturing” and “services” categories. When asked about what kinds of TA were requested most often by MSMEs and what “worked best” in generating results, the most common responses among project management were franchising and ISO certification—which somewhat links to the data above but overlooks the productivity of HR-focused TA, which was a less common response. For their part, the BAs reflected on the link between TA and job creation in different ways. Some saw franchising as having the biggest impact; others saw it in the improvement of production process; still others pointed to management and HR structuring. EVALUATION QUESTION 5 (GENDER) What are the outcomes generated as a result of LED’s gender mainstreaming? Summary Findings, Question 5: ● Though the project has shown commitment to gender analysis and training, gender has not manifested strongly in LED’s activities. ● Gender mainstreaming is limited to the number of females hired for new jobs; to some extent, the project targets firms that are inclined to hire women. ● Only a handful of women-owned enterprises have been assisted. 55 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Conclusions, Question 5: ● Gender is not a major concern, as this project struggles to meet targets around job creation. ● Reorienting the project for Automated Directives System (ADS) 205 compliance will require considerable efforts. LED has been one of the pioneers in conducting gender analysis at the outstart of the project, their staff have attended the different gender training workshops, and there is a gender focal point assigned to participate and follow up in all the gender training organized by Performance Management and Support Program for Lebanon (PMSPL II). This, however, is not manifested in their activities and work on the ground. In practice, gender mainstreaming is limited to the number of females hired. It does not reflect the work and recommendations carried out in the gender analysis. A senior LED manager explained, “We do try to find companies where we can that are women-owned or women-managed, or where employment within the firm targets women. When we find these opportunities, we give preference to them.” Indeed, about a quarter of the beneficiaries indicated in their interviews that in their lines of work, they already gave preference to female employees before LED came along, for a variety of reasons. At the same time, LED’s messaging to beneficiaries on gender has been inconsistent. A roughly equal number of beneficiaries indicated that LED suggested that hiring greater numbers of women is desired, as compared with those who received no messaging from LED at all on gender. Even so, the net efforts around female hires appear to have yielded positive results to some extent: the percentage of female hires in the project stands at 28 percent, which is above the national average of 23 percent women in the workforce. In terms of targeting women-owned MSMEs, another potential area of gender mainstreaming, LED has been less successful. According to LED management, there have been efforts to find female entrepreneurs, but few results have come. Currently, just three women-owned MSMEs have been enrolled. However, this figure needs to be understood in the context of the Lebanese business landscape, where the fraction of enterprises owned by women is very small. It also should be noted that LED employs a rather stringent definition of women-owner enterprise, which relies on the “Certificate of Incorporation of a Company” that they collect from all client firms. This document lists the firm’s shareholders, and only companies whose shareholders are all women (100 percent) are counted as women owned. “We have to operate in whatever environment we’re given,” explained a LED manager. “There’s a tradeoff between looking for jobs generally and expending resources in looking for opportunities for women. We do that, but at a modest level.” It should be noted that the project’s gender analysis included a wide range of recommendations for gender mainstreaming. At present, almost none of these have been heeded, except for “Targeting women-employed firms” and “Tailoring outreach efforts to women entrepreneurs and job seekers.” More emphasis on gender is needed for the project to be in compliance with ADS 205 on gender integration. This is a serious concern that should be revisited by project leadership, in close coordination with the COR, between now and the project’s end. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 56 SUCCESS STORY FOR A WOMAN ENTREPRENEUR A passionate woman entrepreneur based in Zahle obtained her success in business by packaging and selling grains on the retail market. She started 12 years ago, working against resistance from her husband and her father-in-law, who were generally opposed to women leading in the workplace. Her approach was to repackage grains acquired from her husband’s business (a wholegrain distributor), to achieve higher returns in retail. The first successful deal was to a Sweden company who ordered one container, after being impressed by her package sample. Since then she has grown the business locally and especially abroad, with export licenses to Australia, Canada, Sweden, and the United States. LED provided her with expert sales and marketing training, which she says has improved her business in multiple ways. Recently she attended a sales expo in Dubai, which helped opened a new market for her. She explains that her consultant support was very fruitful throughout this process; they even coached her on how to approach the client and craft emails that impressed the potential client and boosted sales. Overall, because of LED support, the sales have increased from two containers per year to two containers per year, and she recently added further new clients in the Dominican Republic. She summed up her experience with LED: “LED supports me, believes in me, and gives me answers to my questions. I would not have known how to work, where to ask for help, and what to do.” 57 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV SUMMARY FINDINGS AND CONCLUSIONS RELEVANCE Summary Findings, Question 1: • The BLA is a model for creating jobs for the poor based on facilitating relationships between buyers and sellers. • With its focus on business interventions and job creation, the BLA is consistent with the mission’s CDCS and push for “Increased private sector competitiveness.” • Compared with iterations of the model elsewhere in the world, the LED project has taken a relatively narrow firm-level approach, bypassing complementary aspects like access to finance and generally not addressing sectors or value chains. • Project leadership articulates a clear focus on a firm-level intervention approach. • Some project leadership questions the fit of this approach in the Lebanese context, suggesting that more jobs might be created by intervening at the business sector level rather than the individual firm level. Conclusions, Question 1: • There is some divergence between the theory/precedent and LED’s practice for understanding and applying the BLA. • The BLA is about creating jobs, and in that sense, it is completely appropriate for a Lebanese context in which job creation can be difficult and elusive. • Based on previous aid projects and relevant literature, the ET suggests that a broader sectoral focus might have produced more jobs, but the argument remains speculative. EFFECTIVENESS Summary Findings, Question 2.1: • After a slow start in Y1, the project has rebounded strongly on jobs in Y2, with 992 created, though this remains well under target, particularly as measured in terms of FTEs. • The distribution of jobs is uneven across the project, with a handful of larger firms providing a disproportionate share and some firms not generating any yet. • As with job creation, the project has rebounded strongly in Y2 on pledged incremental sales and pledged leveraged investment. • However, actual/reported data on incremental sales and leveraged investment is incomplete and error-prone, limiting the ability to examine progress against targets. • Cost share (as a form of leverage investment) stands at 40 percent across the project, with wide variation by the type of TA provided. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 58 • Beneficiaries reflected positively on what they saw as strong growth in jobs, sales, and investment; most saw the increases as a direct result of their involvement with LED. • In terms of additionality, beneficiaries reflected on a wide variety of positive changes in the ways they do business. Though many had existing plans to grow and improve, most said that LED had “accelerated” these by adding strong expertise. • Project leadership shared a vision of strong project achievement, though some noted that it may take longer than expected to reach some targets. • BAs and consultants pointed to the strong relationships with and strong commitments from beneficiaries as the most common drivers of results. Conclusions, Question 2.1: • Clearly, there has been substantial progress on job creation in Y2; qualitative data on sales and investment also suggests strong progress, though the view is less complete because of challenges in the related MEL data. • If we accept the limited amount of reliable data on sales and investment, collectively the progress toward targets suggests that the process of improving and expanding business in Lebanon proceeds in a particular order; investment appears to come first, followed by job creation and sales. • It appears this trend and configuration were not foreseen by LED’s designers, which resulted in strong and uneven discrepancies between the results and targets to date. Summary Findings, Question 2.2: • Constituents at all levels agreed that strong relationships between staff and firms have promoted success in the project. • Constituents at all levels agreed the biggest constraint to success was the economy, which limits achievement at every stage, from firm enrollment to job creation numbers. • Some beneficiaries have not responded well to the project’s requirements, from cost share, to minimum new-job requirements, to “constant requests” for monitoring data number. Relatedly, some clients have complained of cash-flow challenges. • LED management feels that the project has not received clear direction around Objective 2 (BEE) and that work draws time and resources away from its job creation efforts. • Other challenges/constraints to success include high turnover among the BAs and lengthy consultant searches in some cases. • LED has demonstrated inconsistency around some of its requirements including cost share, minimum job commitments, and the number of TA services allowed. Conclusions, Question 2.2: 59 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV • LED has made impressive gains in terms of building relationships and goodwill in the Lebanese business community, which has helped lay a foundation for success. • The Lebanese economy, plus unpopular rules and inconsistencies have undermined success. Summary Findings, Question 2.3: • The project-wide figure on cost/committed job stood at $534—well below the target of $1,000/job. • The most efficient on jobs was the services sector, which encompasses restaurants and hospitality, as well as most of the project’s franchised businesses. • Under TA type, product/design development emerged as most cost-efficient, followed by HR and ISO/FSSC certification. IT interventions were the most expensive. • Large firms were most cost-efficient, with efficiency decreasing as one moves down the scale of firm size. Conclusions, Question 2.3: • The primary drivers of cost-efficiency are scale—a finding of limited utility in a context where there is a strong mandate to serve smaller enterprises. • Further disaggregation on cost-efficiency is constrained by the relatively small client population. EFFECTIVENESS Summary Findings, Question 3: • By its official metric of FTE jobs, the project is well below target, with 476 FTEs near the end of Y2, where the target stands at 1,470. • We find a positive sign in the current level of committed jobs, which exceeds the FTE target for the end of Y2; actual hires (not FTEs) stands at about two thirds of the target. • Using projections based on key variables for Y3, the actual number of new hires exceeds the target in the more optimistic scenarios; however, in no scenario does the project reach its target on the official metric of FTEs. • Success on job targets is strongly mediated by a critical time lag between the provision of TA and creation of the associated jobs and by the 12-month nature of the FTE measurement. • LED operates with a wide range of criteria for firm participation—rules that cause many firms to be rejected by the project and others to opt out; these criteria may need to be revised to support a more aggressive run at targets. • The project continues to struggle with several aspects of Objective 2, including: (1) the general clarity of its mandate in this area; (2) how to balance Objective 2 with Objective 1; and (3) the perceived low level of resource allocation for Objective 2. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 60 • Achievements to date on Objective 2 include both general BEE efforts (events/meetings and a BEE assessment) as well as a sector-focused effort to build tourism in Lebanon (includes attending international tourism events and work with tour operators). • Objective 3 is seen as solely to support Objective 1; efforts to date include a variety of outreach events, strong social media presence, and distribution of project materials. • Questions remain about how group-based training (e.g., workforce development) fits with the project mandate, but LED continues to support training initiatives to a limited extent, mostly through its small grants program. Conclusions, Question 3: • The LED project will not reach its target on jobs unless: (1) targets are lowered; (2) target measures are altered (e.g., actual hires instead of FTEs); and/or (3) USAID allows post-PoP monitoring. • At present, the mandate around BEE is poorly understand and functions largely as a distraction from Objective 1. SUSTAINABILITY AND SCALABILITY Summary Findings, Question 4: • There is no indication thus far that improvements and new jobs will not endure. • Most firms express very optimistic one-year and five-year outlooks. • We examined the potential for sustained relationships between consultants and firms beyond LED; both beneficiaries and consultants were optimistic but only a handful of cases have emerged. • We examined the potential for sustained effects on the consulting market generally and found optimism but almost no effects so far. • On scalability, two sectors—industry and manufacturing and services—far eclipsed the others in terms of the volume of jobs, sales, and investment; the same two emerged most cost-effective in terms of jobs; relatedly, large firms showed similar links over smaller firms. • The top three TA services linked to job creation volume were ISO certification, franchising, and HR. Conclusions, Question 4: • We cannot presently make meaningful observations on sustainability. • There is every indication that firms added jobs and improved businesses with permanent intent. • Like cost-effectiveness, scalability potential is found most with larger firms. GENDER Summary Findings, Question 5: 61 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV • Though the project has shown commitment to gender analysis and training, gender has not manifested strongly in LED’s activities. • Gender mainstreaming is limited to the number of females hired for new jobs; to some extent, the project targets firms that are inclined to hire women. • Only a handful of women-owned enterprises have been assisted. Conclusions, Question 5: • Gender is not a major concern because this project struggles to meet the targets. • Reorienting the project for ADS 205 compliance will require considerable effort. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 62 RECOMMENDATIONS OPTION TO EXTEND LED TO FIVE-YEAR PROJECT • If USAID holds to its original vision for this project, as stipulated in the contract, then LED should not receive an extension. Chemonics clearly set out to win this contract but put little thought into the achievability of its targets—all the more inexcusable when one considers their experience with this exact BLA model. The project almost certainly will fail in its core mandate of FTE job creation. In grappling with those job numbers, LED’s managerial conduct has been inconsistent and, at times, inefficacious. The project is unlikely to make any broadly meaningful or measurable impact on the general BEE or the business consulting market. • If USAID is willing to look beyond the contractual vision and accept that the economic ground has shifted in many ways, then the project may deserve a five-year life. Creating over 900 jobs in less than two years, in one of the worst economic climates in recent memory, is a remarkable achievement. LED has made most of its gains in the project’s second year, suggesting that it may be just now hitting its stride, with momentum that could build over five years. There is every indication that most or all job gains will be sustained. Moreover, the project continues to build new bridges and relationships between USAID and the Lebanese business community that can only benefit future economic growth programming. TARGETS/FIRM SELECTION • Consider revising targets to reflect the time lag on jobs, which has emerged as a clear and indisputable fact in this project. Alternatively, consider allowing actual jobs to substitute for the FTE target. • Consider post-PoP measurements of key outcomes. • If USAID’s overriding concern is reaching targets in the project’s PoP, and revising the targets is not an option, consider loosening any number of criteria for MSME selection and the rules applied to MSMEs once part of the program. Meeting the current target will require an aggressive push for new firms that create jobs quickly in the project’s remaining time. • Take all necessary steps to compile complete and accurate data on actual incremental sales and actual leveraged investment; the lack of such data at present is a serious shortcoming in terms of gauging progress against targets. • Consider investing in a costing research expert for a fuller and more meaningful understanding of cost-efficiency across MSME types and sectors. • Reexamine firm selection process, with more marketing as well as a more centralized and coordinated approach, with less reliance on BA and partner personal and professional networks. LED could play a stronger role in directing the selection strategy, with centralizing training for BAs and others, and a more direct hand in the coordination and distribution of MSME referrals. • Revisit and establish a clear rule around engagement with certain sectors. Some rules (e.g., education, agriculture) are not well-understood at the ground level. In line with the previous third bullet, more flexible rules will mean higher numbers. 63 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV TECHNICAL ASSISTANCE • Revisit the distribution of TA provided. Certain types (e.g., franchising and ISO certification) are getting considerably more attention than others—largely because these are perceived to offer the greatest potential for job creation. Is this strategy the most deliberate and productive way to go (especially considering that the job creation numbers do not always support these priorities)? Constituents need a clear understanding of what happened around TA and why. • Investigate whether TA could be streamlined and packaged at more competitive prices for greater numbers. • Alongside the project’s own TA, investigate and employ linkages to other aid/assistance projects. This may be the only way to heed call for a broader BLA in areas like access to finance. CONSISTENCY AND TRANSPARENCY • LED must balance the need for flexibility (see third bullet) with the need for consistency: − Establish clear and unambiguous rules on cost sharing (i.e., a clearly defined range). One approach would be to create a formula based on the number of jobs that will be created, size of the company, cash flow of the company, leveraged investment, and so on, and apply it in all cases. − Apply consistent rule around minimum commitment for new hires. The project says the minimum is five now, but interviews suggest inconsistency. − Establish and apply a fully consistent rule around multiple TA services. Multiple services probably should be eliminated completely, since they are deemed unavailable to most beneficiaries. • Generally, make efforts to be clearer, more consistent, and more transparent about rule and benefits for participation in the LED project. OBJECTIVE 2 • If Objective 2 (BEE) remains a project priority, several actions are recommended: − USAID and LED must confer and make a reasonable projection of what can be accomplished in the project’s remaining time. − A contract modification will be needed to allocate the proper funding for Objective 2 (almost certainly more than 10 percent). − If aspirations around building the consulting market in Lebanon remain a priority, USAID and LED must agree how deeply a project of this scale can affect the market and set specific goals. GENDER • Revisit gender analysis recommendations to determine whether the gender strategy can be deepened in line with those recommendations (beyond the number of females occupying new jobs). MEL • See previous bullet on actual incremental sales and actual leveraged investment. USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 64 • Sector categories should be better defined and broken down into more meaningful and consistent categories (e.g., hospitality, food and beverage, etc.). • Standardize the list of TA into a much smaller number of categories (e.g., franchising, ISO certification, etc.) • Standardize definitions of MSMEs and communicate it clearly within the LED team. Designations seem to be mixed up to some extent. • To bring reporting in line with the MEL plan, track related external investments in businesses (outside including loans, grants, etc.) in addition to leverage internal investment. • Discuss with MSMEs whether there are more meaningful ways to parse increased sales and investment figures for stronger attribution to the project. Clearly the currently reported results have varying levels of attribution. OTHER • Streamline paperwork and reporting requirements. These are a source of considerable discontent in the project. Alternatively, consider hiring administrative help to assist BAs with paperwork. 65 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ANNEXES ANNEX I: INCEPTION REPORT [REDACTED] USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 66 ANNEX II: EVALUATION DESIGN MATRIX EVALUATION QUESTION EVALUATION SUB￾QUESTION DATA COLLECTION METHODS AND SOURCES INFORMATION TO BE COLLECTED ANALYSIS METHODS Relevance: 1) How relevant is the LED project with its “buyer led”, and generally market led, “technical assistance through business consultants” and “firm led” approach: a) to meet the needs of the private sector in Lebanon? b) to meet USAID/Lebanon CDCS and PSD objectives? Desk review KII USAID EG office LED AOR LED staff (COP/DCOP/MEL Director) USAID CDCS USAID PMP PSD PAD PSD RF LED RFA LED Proj Document LED MEL Plan Desk Review Secondary Data Data Coding Effectiveness 2) How and to what extent has the project achieved planned results? 2.1 What evidence is there that employment has increased in targeted enterprises and sales were enhanced as a result of USAID/L assistance (veracity and attribution)? To what extent can we say that these gains are the result of program “additionality?” Desk review KII USAID EG office LED AOR LED staff (COP/DCOP/MEL Director/Director of Business Promotion Berytech Staff BIAT Staff LED Project Document LED MEL Plan LED Workplan LED Quarterly/Annual Report LED List of Stakeholders LED List of Beneficiaries LED Database KII Notes Desk Review Secondary Data Data Coding 2.2 What are the main factors that influenced positively or negatively LED’s ability to achieve its purposes? 2.3 How cost-effective has been the BLA in meeting LED objectives and key performance indicator (KPI) targets? 67 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV 3) How reasonable and achievable are the committed/forecasted numbers, both in terms of individual enterprises and project￾wide goals? Desk review KII Berytech Staff BIAT Staff Consultants (12-15) Beneficiaries (32-39) LED IPTT KII Notes Phone Interview Desk Review Secondary Data Data Coding Sustainability & Scalability 4)What is the likelihood that the results LED has achieved so far are sustainable and scalable beyond the life of the project? Desk review KII Berytech Staff BIAT Staff Consultants (12-15) Beneficiaries (32-39) KII Notes Phone Interview Data Coding Gender 5)What are the outcomes generated as a result of LED’s gender mainstreaming? Desk review KII USAID EG office LED AOR LED staff (COP/DCOP/MEL Director/Director of Business Promotion Berytech Staff BIAT Staff Beneficiaries (32-39) LED Documents KII Notes Desk Review Secondary Data Data Coding USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 68 ANNEX III: KII PROTOCOLS KEY INFORMANT INTERVIEW FOR LED BENEFICIARIES (SMES) RESEARCHER NAME: NOTE TAKER NAME: INTERVIEW DATE: RESPONDENT ID: RESPONDENT POSITION: YEARS IN CURRENT BUSINESS: RESPONDENT GENDER: RESPONDENT AGE: SECTOR/VALUE CHAIN: RESPONDENT REGION: START TIME: END TIME: HELLO, I AM _______________ AND I WORK WITH SOCIAL IMPACT, AN AMERICAN COMPANY CONDUCTING RESEARCH FOR THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) LEBANON OFFICE. WE ARE CONDUCTING A PERFORMANCE EVALUATION OF THE LEBANON ENTERPRISE DEVELOPMENT (LED) ACTIVITY. WE WANT TO UNDERSTAND HOW SUCCESSFUL LED HAS BEEN IN MEETING ITS OBJECTIVES AND HOW ITS WORK OVER THE PAST TWO YEARS CAN INFORM FUTURE USAID/LEBANON WORK. WE INTEND TO INTERVIEW APPROXIMATELY 60 INDIVIDUALS, AND YOU HAVE BEEN SELECTED TO PARTICIPATE IN THIS RESEARCH BECAUSE WE BELIEVE YOU CAN PROVIDE A UNIQUE PERSPECTIVE ON LED AND ITS POTENTIAL CONTRIBUTIONS IN THE LEBANESE BUSINESS ENVIRONMENT. YOUR PARTICIPATION IN THIS INTERVIEW IS COMPLETELY VOLUNTARY AND YOU ARE FREE TO DECLINE TO RESPOND TO ANY QUESTION FOR ANY REASON AND TO END THE INTERVIEW AT ANY TIME. YOU WILL NOT BE PROVIDED WITH ANY PAYMENT FOR YOUR PARTICIPATION IN THIS INTERVIEW. YOUR HONEST RESPONSES WILL HELP USAID INFORM AND IMPROVE ITS FUTURE PROGRAMMING IN LEBANON. YOUR NAME AND THE NAME OF YOUR BUSINESS WILL NOT BE CONNECTED TO ANY INFORMATION YOU PROVIDE IN THIS INTERVIEW. THE DATA FROM THIS INTERVIEW (EXCLUDING YOUR PERSONALLY IDENTIFIABLE INFORMATION) WILL BE USED BY THE SOCIAL IMPACT TEAM FOR ANALYSIS PURPOSES ONLY AND YOUR ANSWERS WILL ONLY BE REPORTED TO USAID AS PART OF AN AGGREGATED DATA SET. TO ENSURE ALL YOUR RESPONSES ARE CAPTURED ACCURATELY AND CAN BE REVIEWED DURING DATA ANALYSIS AND REPORT WRITING, WE WOULD LIKE TO RECORD THE INTERVIEW. THE RECORDINGS WILL BE KEPT IN A SECURE DATA STORAGE AND WILL BE DESTROYED AFTER SUBMISSION OF THE REPORT. THE REPORT MAY BE MADE PUBLIC UPON FINALIZATION. THIS INTERVIEW WILL LAST APPROXIMATELY ONE HOUR. DO YOU UNDERSTAND THAT YOUR PARTICIPATION IN THIS INTERVIEW IS VOLUNTARY, UNPAID, AND THAT YOU CAN END THE INTERVIEW AT ANY POINT? IF YOU HAVE ANY FOLLOW UP QUESTIONS, PLEASE CONTACT HARVEY HERR HHERR@SOCIALIMPACT.COM. ARE YOU WILLING TO PARTICIPATE? YES___ (CHECK) NO___ (CHECK) DO YOU ALLOW VOICE RECORDING OF THE INTERVIEW? YES___ (CHECK) NO___ (CHECK) CAN YOU GIVE ME A BRIEF INTRODUCTION TO YOUR BUSINESS? HOW AND WHEN DID YOU GET STARTED? INVOLVEMENT WITH LED TELL ME ABOUT HOW YOU BECAME INVOLVED IN THE LED PROJECT. HOW DID YOU FIRST FIND OUT ABOUT IT? TO DATE, WHAT KIND OF ASSISTANCE HAVE YOU RECEIVED FROM LED? PLEASE DESCRIBE EACH COMPONENT OF THE ASSISTANCE YOU HAVE RECEIVED. IS THE ASSISTANCE CONTINUING? IF SO, WHEN IS IT EXPECTED TO END? WAS THERE ANY ASSISTANCE YOU REQUESTED THAT LED WAS UNABLE TO PROVIDE? DID YOU WORK WITH ONE PARTICULAR BUSINESS ADVISOR WITH LED? TELL ME ABOUT YOUR INTERACTIONS WITH THE BA(S). IS THERE ANYTHING THE BA(S) COULD HAVE DONE MORE TO SUPPORT YOUR WORK? DID THE BA(S) CONNECT YOU WITH ONE OR MORE BUSINESS CONSULTANT? TELL ME ABOUT YOUR INTERACTIONS WITH THE CONSULTANT(S). IS THERE ANYTHING THE CONSULTANT(S) COULD HAVE DONE MORE TO SUPPORT YOUR WORK? LET’S TALK ABOUT HOW YOUR BUSINESS HAS CHANGED SINCE YOU STARTED WORKING WITH LED. ARE YOU DOING BUSINESS IN A DIFFERENT WAY NOW? IF SO, PLEASE DESCRIBE. 69 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV [PROBE: BETTER WORKER CAPACITY, BETTER BOOKKEEPING, ETC.?] HAVE YOU BEEN ABLE TO ADD ANY NEW EMPLOYEES? [NOTE: WE WILL ONLY ASK ABOUT TRENDS HERE. SPECIFIC NUMBERS DRAWN FROM M&E DATA] WHAT KINDS OF JOBS WERE CREATED? DID YOU MAKE ANY PARTICULAR EFFORT TO HIRE FEMALE STAFF? CAN YOU SAY WITH CERTAINTY THAT THESE EMPLOYEES WERE HIRED AS A RESULT OF THE ASSISTANCE FROM LED? OR IS POSSIBLE THAT YOU MIGHT HAVE HIRED THEM EVEN WITHOUT LED? WHAT IS THE TOTAL NUMBER OF EMPLOYEES THAT YOU HAVE COMMITTED TO HIRE? HOW CONFIDENT ARE YOU THAT YOU WILL BE ABLE TO HIRE ALL OF THEM AS PLANNED? HAVE YOUR SALES INCREASED SINCE YOU STARTED WITH LED? [NOTE: WE WILL ONLY ASK ABOUT TRENDS HERE. SPECIFIC NUMBERS DRAWN FROM M&E DATA] ARE YOU CERTAIN THESE CHANGES IN SALES ARE A DIRECT RESULT OF YOUR INVOLVEMENT WITH LED? WHAT ABOUT NEW INVESTMENT IN THE BUSINESS AS A RESULT OF INVOLVEMENT WITH LED? [NOTE: WE WILL ONLY ASK ABOUT TRENDS HERE. SPECIFIC NUMBERS DRAWN FROM M&E DATA] HAVE YOU INVESTED YOUR OWN FUNDS IN THE BUSINESS? HOW YOU SECURED NEW LOANS? HAVE RECEIVED ANY PRIVATE INVESTMENT IN YOUR BUSINESS? HAVE YOU RECEIVED ANY INVESTMENTS IN YOUR BUSINESS FROM OTHER DEVELOPMENT PROGRAMS/NGOS? ARE YOU CERTAIN THAT THESE INVESTMENTS ARE A DIRECT RESULT OF YOUR INVOLVEMENT WITH LED? OVER THE PERIOD YOU HAVE BEEN INVOLVED WITH LED, HAVE THERE BEEN ANY DEVELOPMENTS IN LEBANON THAT AFFECTED THE BENEFITS OF THE PROGRAM? FOR EXAMPLE: HOW HAS THE LEBANESE ECONOMY AS A WHOLE FARED OVER THIS PERIOD? HAS THERE BEEN ANY POLITICAL INSTABILITY THAT AFFECTED YOUR BUSINESS? [PROBE FOR ANYTHING ELSE] GENERALLY, WHERE DO YOU THINK YOUR BUSINESS WOULD BE RIGHT NOW IF LED HAD NOT COME ALONG? IN A SIMILAR POSITION, OR A DIFFERENT POSITION? PLEASE DESCRIBE. WHERE DO YOU THINK YOUR BUSINESS WILL BE A YEAR FROM NOW? PLEASE EXPLAIN. WHERE DO YOU THINK YOUR BUSINESS WILL BE FIVE YEARS FROM NOW? PLEASE EXPLAIN. WILL YOU CONSIDER HIRING THE SAME CONSULTANT OR OTHER BUSINESS CONSULTANTS ON YOUR OWN, WITHOUT THE ASSISTANCE OF A PROJECT LIKE LED? CLOSING LOOKING BACK ON YOUR INVOLVEMENT WITH LED, IS THERE ANYTHING YOU WISH THE PROJECT COULD HAVE DONE DIFFERENTLY? DO YOU HAVE ANY QUESTIONS FOR US BEFORE WE END? USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 70 LED INTERVIEW FOR PARTNERS & USAID (LED/BIAT/BERYTECH/ USAID) RESEARCHER NAME: NOTE TAKER NAME: INTERVIEW DATE: RESPONDENT ID: RESPONDENT POSITION: YEARS IN CURRENT BUSINESS: RESPONDENT GENDER: RESPONDENT PARTNER: START TIME: END TIME: HELLO, I AM _______________ AND I WORK WITH SOCIAL IMPACT, AN AMERICAN COMPANY CONDUCTING RESEARCH FOR THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) LEBANON OFFICE. WE ARE CONDUCTING A PERFORMANCE EVALUATION OF THE LEBANON ENTERPRISE DEVELOPMENT (LED) ACTIVITY. WE WANT TO UNDERSTAND HOW SUCCESSFUL LED HAS BEEN IN MEETING ITS OBJECTIVES AND HOW ITS WORK OVER THE PAST TWO YEARS CAN INFORM FUTURE USAID/LEBANON WORK. WE INTEND TO INTERVIEW APPROXIMATELY 60 INDIVIDUALS, AND YOU HAVE BEEN SELECTED TO PARTICIPATE IN THIS RESEARCH BECAUSE WE BELIEVE YOU CAN PROVIDE A UNIQUE PERSPECTIVE ON LED AND ITS POTENTIAL CONTRIBUTIONS IN THE LEBANESE BUSINESS ENVIRONMENT. YOUR PARTICIPATION IN THIS INTERVIEW IS COMPLETELY VOLUNTARY AND YOU ARE FREE TO DECLINE TO RESPOND TO ANY QUESTION FOR ANY REASON AND TO END THE INTERVIEW AT ANY TIME. YOU WILL NOT BE PROVIDED WITH ANY PAYMENT FOR YOUR PARTICIPATION IN THIS INTERVIEW. YOUR HONEST RESPONSES WILL HELP USAID INFORM AND IMPROVE ITS FUTURE PROGRAMMING IN LEBANON. YOUR NAME WILL NOT BE CONNECTED TO ANY INFORMATION YOU PROVIDE IN THIS INTERVIEW. THE DATA FROM THIS INTERVIEW (EXCLUDING YOUR PERSONALLY IDENTIFIABLE INFORMATION) WILL BE USED BY THE SOCIAL IMPACT TEAM FOR ANALYSIS PURPOSES ONLY AND YOUR ANSWERS WILL ONLY BE REPORTED TO USAID AS PART OF AN AGGREGATED DATA SET. TO ENSURE ALL YOUR RESPONSES ARE CAPTURED ACCURATELY AND CAN BE REVIEWED DURING DATA ANALYSIS AND REPORT WRITING, WE WOULD LIKE TO RECORD THE INTERVIEW. THE RECORDINGS WILL BE KEPT IN A SECURE DATA STORAGE AND WILL BE DESTROYED AFTER SUBMISSION OF THE REPORT. THE REPORT MAY BE MADE PUBLIC UPON FINALIZATION. THIS INTERVIEW WILL LAST APPROXIMATELY ONE HOUR. DO YOU UNDERSTAND THAT YOUR PARTICIPATION IN THIS INTERVIEW IS VOLUNTARY, UNPAID, AND THAT YOU CAN END THE INTERVIEW AT ANY POINT? IF YOU HAVE ANY FOLLOW UP QUESTIONS, PLEASE CONTACT HARVEY HERR HHERR@SOCIALIMPACT.COM. ARE YOU WILLING TO PARTICIPATE? YES___ (CHECK) NO___ (CHECK) DO YOU ALLOW VOICE RECORDING OF THE INTERVIEW? YES___ (CHECK) NO___ (CHECK) 71 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV INTRODUCTION/WARM-UP (ADDRESSED TO ALL PARTNERS/USAID) WHEN DID YOU JOIN YOUR ORGANIZATION? WHAT IS YOUR MAJOR ROLE WITH LED PROJECT? INVOLVEMENT WITH LED PROJECT (ADDRESSED TO ALL PARTNERS/USAID) WHAT IS YOUR UNDERSTANDING OF THE BUYER-LED OR MARKET-LED APPROACH AT THE CENTER OF THE LED PROJECT? HOW RELEVANT IS THIS APPROACH TO WHAT YOUR ORGANIZATION/AGENCY DOES? DO YOU FEEL THIS APPROACH MEET THE NEEDS OF THE PRIVATE SECTOR IN LEBANON? WHAT IS YOUR POINT OF VIEW ON THE LED’S FIRM SELECTION PROCESS? DOES IT ALLOW IN TOO MANY FIRMS? TOO FEW FIRMS? HOW COULD IT THE SELECTION PROCESS BE IMPROVED? HOW AND TO WHAT EXTENT HAS THE PROJECT ACHIEVED PLANNED RESULTS? [PROBES: EMPLOYMENT INCREASED? SALES INCREASED? INVESTMENT INCREASED? WHAT ARE THE MAIN FACTORS THAT INFLUENCED POSITIVELY OR NEGATIVELY LED’S ABILITY TO ACHIEVE THESE RESULTS? DO YOU THINK THE BENEFICIARIES WILL BE ABLE TO HIRE ALL OF THE NEW EMPLOYEES TO WHICH THEY HAVE COMMITTED? PLEASE EXPLAIN. WHAT ARE THE MOST JOBS CREATED WITH LED PROJECT? JOB TYPE? JOB LEVEL? (JUNIOR, SENIOR, MANAGERIAL) AVERAGE SALARY PER JOB? IS THERE ANY SECTOR THAT HAS BENEFITTED FROM THE LED PROGRAM MORE THAN OTHERS? WHY IS THAT? [PROBES: FRANCHISING] IS THERE A TYPE OF SERVICE PROVIDED IN THE LED PROGRAM THAT HAS PRODUCED MORE RESULTS THAN OTHER? WHY IS THAT? [PROBES: FRANCHISING] IS THERE A SECTOR THAT HAS SHOWN TO BE MORE COST-EFFECTIVE THAN OTHERS? WHY IS THAT? [PROBE: FRANCHISING? LOW COST PER JOB?] IS THERE A TYPE OF SERVICE THAT HAS SHOWN TO BE MORE COST-EFFECTIVE THAN OTHERS? WHY IS THAT? [PROBE: FRANCHISING? LOW COST PER JOB?] [SKIP WITH USAID] TELL ME ABOUT YOUR EXPERIENCE IN FINDING AND HIRING CONSULTANTS. WHAT HAVE BEEN THE MAJOR SUCCESSES AND CHALLENGES THERE? DO YOU THINK THE BUYER-LED APPROACH WILL CREATE A SUSTAINABLE RELATION BETWEEN CONSULTANTS AND BENEFICIARIES BEYOND THE LIFE OF THE PROJECT? IN OTHER WORDS, WILL THE SMES CONTINUE TO PAY FOR SUCH SERVICES AFTER LED’S CONTRIBUTION IS GONE? WHY OR WHY NOT? IS THERE ANY EVIDENCE THAT THE LED HAS INCREASED DEMAND FOR THESE KINDS OF BUSINESS CONSULTING SERVICES? IN OTHER WORDS, HAS THE MARKET FOR PRIVATE-SECTOR BUSINESS CONSULTING IMPROVED? HAVE YOU EXPERIENCED CHALLENGES IN THE STAFFING OF THE PROJECT? PLEASE EXPLAIN. HOW WAS GENDER A FACTOR IN THE DESIGN OF THE LED PROJECT, BEYOND THE INITIAL TARGET OF 25% OF NEW HIRES BEING FEMALE? WHAT WOULD YOU SAY HAS BEEN THE BIGGEST SUCCESS UNDER LED TO DATE? WHAT WOULD YOU SAY HAS BEEN THE BIGGEST CONSTRAINT TO SUCCESS UNDER LED? INVOLVEMENT WITH LED PROJECT (ADDRESSED TO LED) HOW ARE NEW JOBS COUNTED? (FULL TIME EQUIVALENT, PART-TIME…?) HOW DO YOU ENSURE THAT THESE ARE A DIRECT RESULT OF INVOLVEMENT WITH LED? HOW ARE INCREASED SALES COUNTED? HOW DO YOU ENSURE THAT THESE ARE A DIRECT RESULT OF INVOLVEMENT WITH LED? USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 72 HOW ARE NEW INVESTMENTS COUNTED? EXACTLY WHAT KINDS OF INVESTMENTS ARE TRACKED? HOW DO YOU ENSURE THAT THESE ARE A DIRECT RESULT OF INVOLVEMENT WITH LED? [PROBE: GRANTS, PRIVATE EQUITY, LOANS] TELL ME ABOUT YOUR ACTIVITIES AROUND OBJECTIVE 2, “ENABLING ENVIRONMENT” AND OBJECTIVE 3, “COMMUNICATIONS AND OUTREACH” HAVE THIS BEEN A PRIORITY FOR LED? WHY OR WHY NOT? HAVE YOU HAD ANY ISSUES WITH THE SUBMISSION OF MONITORING DATA RECEIVED FROM YOUR PARTNERS AND BENEFICIARIES? PLEASE EXPLAIN. [PROBE: HIRING DATA, SALES DATA, ENGAGEMENT AT PARTNER LEVEL] INVOLVEMENT WITH LED PROJECT: (ADDRESSED TO USAID) ARE YOU PLEASED WITH THE LED PROJECT AND ITS ACHIEVEMENTS TO DATE? PLEASE EXPLAIN. 73 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV LED INTERVIEW FOR BUSINESS ADVISOR RESEARCHER NAME: NOTE TAKER NAME: INTERVIEW DATE: RESPONDENT ID: RESPONDENT POSITION: YEARS IN CURRENT BUSINESS: RESPONDENT GENDER: RESPONDENT AGE: RESPONDENT REGION: START TIME: END TIME: HELLO, I AM _______________ AND I WORK WITH SOCIAL IMPACT, AN AMERICAN COMPANY CONDUCTING RESEARCH FOR THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAITI OFFICE. WE ARE CONDUCTING A PERFORMANCE EVALUATION OF THE LEBANON ENTERPRISE DEVELOPMENT (LED) ACTIVITY. WE WANT TO UNDERSTAND HOW SUCCESSFUL LED HAS BEEN IN MEETING ITS OBJECTIVES AND HOW ITS WORK OVER THE PAST TWO YEARS CAN INFORM FUTURE USAID/LEBANON WORK. WE INTEND TO INTERVIEW APPROXIMATELY 60 INDIVIDUALS, AND YOU HAVE BEEN SELECTED TO PARTICIPATE IN THIS RESEARCH BECAUSE WE BELIEVE YOU CAN PROVIDE A UNIQUE PERSPECTIVE ON LED AND ITS POTENTIAL CONTRIBUTIONS IN THE LEBANESE BUSINESS ENVIRONMENT. YOUR PARTICIPATION IN THIS INTERVIEW IS COMPLETELY VOLUNTARY AND YOU ARE FREE TO DECLINE TO RESPOND TO ANY QUESTION FOR ANY REASON AND TO END THE INTERVIEW AT ANY TIME. YOU WILL NOT BE PROVIDED WITH ANY PAYMENT FOR YOUR PARTICIPATION IN THIS INTERVIEW. YOUR HONEST RESPONSES WILL HELP USAID INFORM AND IMPROVE ITS FUTURE PROGRAMMING IN LEBANON. YOUR NAME WILL NOT BE CONNECTED TO ANY INFORMATION YOU PROVIDE IN THIS INTERVIEW. THE DATA FROM THIS INTERVIEW (EXCLUDING YOUR PERSONALLY IDENTIFIABLE INFORMATION) WILL BE USED BY THE SOCIAL IMPACT TEAM FOR ANALYSIS PURPOSES ONLY AND YOUR ANSWERS WILL ONLY BE REPORTED TO USAID AS PART OF AN AGGREGATED DATA SET. TO ENSURE ALL YOUR RESPONSES ARE CAPTURED ACCURATELY AND CAN BE REVIEWED DURING DATA ANALYSIS AND REPORT WRITING, WE WOULD LIKE TO RECORD THE INTERVIEW. THE RECORDINGS WILL BE KEPT IN A SECURE DATA STORAGE AND WILL BE DESTROYED AFTER SUBMISSION OF THE REPORT. THE REPORT MAY BE MADE PUBLIC UPON FINALIZATION. THIS INTERVIEW WILL LAST APPROXIMATELY ONE HOUR. DO YOU UNDERSTAND THAT YOUR PARTICIPATION IN THIS INTERVIEW IS VOLUNTARY, UNPAID, AND THAT YOU CAN END THE INTERVIEW AT ANY POINT? IF YOU HAVE ANY FOLLOW UP QUESTIONS, PLEASE CONTACT HARVEY HERR HHERR@SOCIALIMPACT.COM. ARE YOU WILLING TO PARTICIPATE? YES___ (CHECK) NO___ (CHECK) DO YOU ALLOW VOICE RECORDING OF THE INTERVIEW? YES___ (CHECK) NO___ (CHECK) USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 74 INTRODUCTION/WARM-UP WHEN DID YOU START WORKING AS A BUSINESS ADVISOR WITH BIAT/OR BERYTECH? WHAT IS YOUR RELEVANT EXPERIENCE/BACKGROUND? INVOLVEMENT WITH BENEFICIARIES WHAT ROLE DO YOU PLAY IN LED’S FIRM SELECTION PROCESS? DO YOU THINK THE PROCESS ALLOWS IN TOO MANY FIRMS? TOO FEW FIRMS? HOW COULD IT THE SELECTION PROCESS BE IMPROVED? TELL ME ABOUT YOUR RELATIONSHIP WITH YOUR SME CLIENTS. DO YOU FEEL YOU GET TO KNOW THESE FIRMS WELL? WHAT KIND OF TECHNICAL ASSISTANCE IS MOST REQUESTED BY BENEFICIARIES? IN YOUR EXPERIENCE, WHAT KIND OF TECHNICAL ASSISTANCE IS MOST LIKELY TO RESULT IN NEW HIRES? INCREASE SALES? HOW CONFIDENT ARE YOU THAT ALL CLIENT FIRMS WILL BE ABLE TO HIRE ALL OF THE NEW EMPLOYEES TO WHICH THEY HAVE COMMITTED? PLEASE EXPLAIN. IS THERE ANY PARTICULAR KIND OF FIRM THAT HAS BENEFITTED FROM THE LED PROGRAM MORE THAN OTHERS? WHY IS THAT? IS THERE ANY PARTICULAR KIND OF FIRM THAT HAS SHOWN TO BE MORE COST-EFFECTIVE THAN OTHERS? FOR EXAMPLE, ANY TYPE OF FIRM THAT TENDS TO ACHIEVE A LOW INVESTMENT/NEW JOB RATIO, OR GENERALLY DOES WELL IN TERMS OF NEW HIRES, INCREASED SALES, OR NEW INVESTMENTS? WHY IS THAT? GENERALLY, HOW WOULD YOU DESCRIBE SME COMMITMENT AND COLLABORATION? WHAT HAS BEEN YOUR BIGGEST SUCCESS IN WORKING WITH BENEFICIARIES? WHAT HAS BEEN YOUR BIGGEST CONSTRAINT TO SUCCESS IN WORKING WITH BENEFICIARIES? INVOLVEMENT WITH CONSULTANTS WHAT IS YOUR INVOLVEMENT IN MATCHING CONSULTANTS TO BENEFICIARIES? WOULD YOU SAY THAT FINDING AND MATCHING CONSULTANTS HAS BEEN DIFFICULT? WHAT IS THE TYPICAL TIMEFRAME OF CONSULTANT RECRUITMENT PROCESS? HAVE DELAYS IN THE RECRUITMENT PROCESS AFFECTED BENEFICIARIES IN THE PROGRAM? HOW SO? DOES THE CONSULTANT ALWAYS SUPPLY THE REQUIRED TA AS OUTLINED IN THE SCOPE OF WORK? PLEASE EXPLAIN. IS THE QUALITY OF THE CONSULTANT SERVICE ALWAYS ADEQUATE? HOW DO YOU MONITOR THAT QUALITY? IN MOST CASES, WOULD YOU SAY THAT THE CONSULTANT SERVICE HAD A DIRECT EFFECT ON JOB CREATION/SALES FOR THE BENEFICIARY? DO YOU THINK MOST BENEFICIARIES ARE SATISFIED WITH THEIR CONSULTANT SERVICE? PLEASE EXPLAIN. DO YOU THINK THE LED PROJECT WILL CREATE A SUSTAINABLE RELATION BETWEEN CONSULTANTS AND BENEFICIARIES BEYOND THE LIFE OF THE PROJECT? IN OTHER WORDS, WILL THE SMES CONTINUE TO PAY FOR SUCH SERVICES AFTER LED’S CONTRIBUTION IS GONE? WHY OR WHY NOT? IS THERE ANY EVIDENCE THAT THE LED HAS INCREASED DEMAND FOR THESE KINDS OF BUSINESS CONSULTING SERVICES? IN OTHER WORDS, ANY EVIDENCE THAT MORE NON-PARTICIPANT FIRMS ARE USING SUCH SERVICES OR WILL DO SO AFTER LED IS GONE? CLOSING WHAT CAN THIS PROJECT DO BETTER TO ACHIEVE ITS AIMS AROUND JOB CREATION? DO YOU HAVE ANY QUESTION FOR US BEFORE WE END? 75 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV LED INTERVIEW FOR CONSULTANT PHONE CALL INTERVIEW RESEARCHER NAME: INTERVIEW DATE: RESPONDENT ID: START TIME: RESPONDENT AGE: RESPONDENT GENDER: END TIME: HELLO, I AM _______________ AND I WORK WITH SOCIAL IMPACT, AN AMERICAN COMPANY CONDUCTING RESEARCH FOR THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID) HAITI OFFICE. WE ARE CONDUCTING A PERFORMANCE EVALUATION OF THE LEBANON ENTERPRISE DEVELOPMENT (LED) ACTIVITY. WE WANT TO UNDERSTAND HOW SUCCESSFUL LED HAS BEEN IN MEETING ITS OBJECTIVES AND HOW ITS WORK OVER THE PAST TWO YEARS CAN INFORM FUTURE USAID/LEBANON WORK. WE INTEND TO INTERVIEW APPROXIMATELY 60 INDIVIDUALS, AND YOU HAVE BEEN SELECTED TO PARTICIPATE IN THIS RESEARCH BECAUSE WE BELIEVE YOU CAN PROVIDE A UNIQUE PERSPECTIVE ON LED AND ITS POTENTIAL CONTRIBUTIONS IN THE LEBANESE BUSINESS ENVIRONMENT. YOUR PARTICIPATION IN THIS INTERVIEW IS COMPLETELY VOLUNTARY AND YOU ARE FREE TO DECLINE TO RESPOND TO ANY QUESTION FOR ANY REASON AND TO END THE INTERVIEW AT ANY TIME. YOU WILL NOT BE PROVIDED WITH ANY PAYMENT FOR YOUR PARTICIPATION IN THIS INTERVIEW. YOUR HONEST RESPONSES WILL HELP USAID INFORM AND IMPROVE ITS FUTURE PROGRAMMING IN LEBANON. YOUR NAME WILL NOT BE CONNECTED TO ANY INFORMATION YOU PROVIDE IN THIS INTERVIEW. THE DATA FROM THIS INTERVIEW (EXCLUDING YOUR PERSONALLY IDENTIFIABLE INFORMATION) WILL BE USED BY THE SOCIAL IMPACT TEAM FOR ANALYSIS PURPOSES ONLY AND YOUR ANSWERS WILL ONLY BE REPORTED TO USAID AS PART OF AN AGGREGATED DATA SET. THE REPORT MAY BE MADE PUBLIC UPON FINALIZATION. THIS INTERVIEW WILL LAST APPROXIMATELY 30 MINUTES. DO YOU UNDERSTAND THAT YOUR PARTICIPATION IN THIS INTERVIEW IS VOLUNTARY, UNPAID, AND THAT YOU CAN END THE INTERVIEW AT ANY POINT? IF YOU HAVE ANY FOLLOW UP QUESTIONS, PLEASE CONTACT HARVEY HERR HHERR@SOCIALIMPACT.COM. ARE YOU WILLING TO PARTICIPATE? YES___ (CHECK) NO___ (CHECK) DO YOU ALLOW VOICE RECORDING OF THE INTERVIEW? YES___ (CHECK) NO___ (CHECK) USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 76 INTRODUCTION/WARM-UP WHEN DID YOU START WORKING AS A CONSULTANT? DO YOU WORK UNDER A CONSULTANT FIRM? OR AS AN INDIVIDUAL CONSULTANT? WHAT TYPE OF TECHNICAL SERVICES DO YOU PROVIDE? WHICH OF THOSE SERVICES HAVE YOU PROVIDED UNDER LED? INVOLVEMENT WITH LED PROJECT WHEN DID YOU BECOME A TECHNICAL EXPERT UNDER LED PROJECT? HOW DID YOU BECOME AWARE OF THE OPPORTUNITY? HAVE YOU WORKED WITH ONE BA IN ALL CASES? TELL ME ABOUT YOUR RELATIONSHIP WITH THE BA(S). ARE YOU CONTRACTED DIRECTLY WITH LED? SO FAR, HOW MANY SMES HAVE YOU COVERED? WHICH REGIONS? WHICH SECTORS? IN ALL CASES, DO YOU FEEL THAT YOU WERE CONNECTED WITH AN SME THAT MATCHES YOUR TECHNICAL EXPERTISE? PLEASE EXPLAIN. WHAT IS THE LOE (LEVEL OF EFFORT) PER SME PROVIDED? HOW MANY DAYS? HAS IS VARIED IN EACH CASE? WAS THE TIMEFRAME ALWAYS REALISTIC IN ALLOWING YOU TO COMPLETE YOUR OBJECTIVES WITH EACH SME? PLEASE EXPLAIN. [PROBE: WHO DEFINED TIMEFRAME, YOU OR LED?] INVOLVEMENT WITH SME GENERALLY, HOW WOULD YOU DESCRIBE SME COMMITMENT AND COLLABORATION? DO THE SMES ALWAYS DO WHAT YOU ADVISE THEM TO DO? WHAT HAS BEEN YOUR BIGGEST SUCCESS IN WORKING WITH BENEFICIARIES? WHAT HAS BEEN YOUR BIGGEST CONSTRAINT TO SUCCESS IN WORKING WITH BENEFICIARIES? HAVE YOU DISCOVER NEW CHALLENGES FACED BY THE SMES THAT WEREN’T STATED OR SPECIFIED BY THE BA FROM THE BEGINNING? DID YOU THINK THAT THE SME MIGHT HIRE YOU FOR ANY ADDITIONAL TECHNICAL ASSISTANCE IN THE FUTURE WHEN LED IS NO LONGER INVOLVED? HAS YOUR INVOLVEMENT WITH LED HELPED INCREASE YOUR BUSINESS AS A CONSULTANT, EVEN AMONG NON-PARTICIPANT CLIENTS? HAS THE FOR-PROFIT BUSINESS CONSULTING MARKET IN LEBANON IMPROVED? CLOSING WHAT CAN THIS PROJECT DO BETTER TO ACHIEVE ITS AIMS AROUND JOB CREATION? DO YOU HAVE ANY QUESTION FOR US BEFORE WE END? 77 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV ANNEX IV: INFORMATION SOURCES TYPE OF DOCUMENT 1. USAID documents USAID’s Country Development Cooperative Strategy for Lebanon (CDCS 2014-2019) USAID Performance Management Plan (PMP) Private Sector Development Project (PSD) Project RFP LED Performance Management Plan (PMP) LED Project Appraisal Document (PAD) 2. LED Contract Statement of Work (Section C of the Contract) 3. LED MEL Plan & Work Plans LED MEL Plan LED 1st Annual Work Plan FY18 (Oct. 17 – Sept. 18) LED Revised 1st Annual Work Plan FY18 (Oct. 17 – Sept. 18) LED 2nd Annual Work Plan FY19 (Dec 18 – Sept 19) 4. LED Progress Reports LED Quarterly Report (Oct. – Dec. 2017) USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 78 LED Quarterly Report (Jan. – March 2018) LED Quarterly Report (April – June 2018) LED Annual Report FY18 (Oct. 2017 – Sept. 2018) LED Quarterly Report (Oct. – Dec 2018) LED Quarterly Report (Jan – March 2019) 5. Other LED Documents LED MEL Database LED Gender Analysis Assisted Firms Contact List Consultant List (August 2019) Personnel Table (August 2019) 6. Related sources Dr. James, Riordan, Development Economist, Expert on Buyer Led Approach https://www.youtube.com/watch?v=bzo2AIWUKkc Tracy Shanks, Managing Director, Chemonics https://100andchange.foundationcenter.org/profiles/2370/ 79 | LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT USAID.GOV Pattie, Preston, Let’s change focus-the buyer led approach to development, 24 April 2014, https://www.devex.com/news/let-s-change￾focus-the-buyer-led-approach-to-development-83351 Pattie, Preston, 10 'micro' tips for the buyer-led approach to fighting poverty, 22 May 2014, https://www.devex.com/news/10-micro￾tips-for-the-buyer-led-approach-to-fighting-poverty-83535 USAID.GOV LEBANON ENTERPRISE DEVELOPMENT: MIDTERM EVALUATION REPORT | 80 ANNEX V: KIIS BY CATEGORY TYPE OF STAKEHOLDER NO. STAKEHOLDERS GOVERNORATE USAID 1 Interview Phone interview (1) Chemonics Staff 4 Interviews Mount Lebanon (4) Sub-contractors (Berytech and BIAT) 4 Interviews Beirut (2) North Lebanon (2) Business Advisors 7 Interviews Beirut (5) North Lebanon (2) Beneficiaries 32 interviews Aakkar (1) Beirut (3) Beqaa (9) Mount Lebanon (8) Nabatieh (1) North Lebanon (8) South Lebanon (2) Consultants 15 interviews Phone interviews (15) Total 63 Interviews Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA 22201 Suite 1000 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socialimpact.com