LIVELIHOODS AND INCLUSIVE FINANCE EXPANSION (LIFE) MIDTERM PERFORMANCE EVALUATION FINAL REPORT March 2020 This publication was produced by Social Impact, Inc. at the request of the United States Agency for International Development. It was prepared by Dr. Michael Ferguson with the support of local consultants Ziad Antonios, Elias Samia, and Dr. Lara Macaron and Social Impact’s Senior M&E Specialists, Dr. Tamara Nassereddine and Dr. Rana Taher. Note: Limited redactions have been made to this version of the report in accordance with the principled exceptions to the presumption in favor of openness established in OMB Bulletin 12-01, “Guidance on Collection of U.S. Foreign Assistance Data.” Photo Courtesy of the LIFE Project LIVELIHOODS AND INCLUSIVE FINANCE EXPANSION (LIFE) MIDTERM PERFORMANCE EVALUATION DRAFT REPORT March 2020 Prepared by Social Impact, Inc. for the United States Agency for International Development under USAID Contract/Order No. AID-268-C-15-00001 – Performance Management and Support Program for Lebanon (PMSPL II). This document is not available in print. Documents of this nature are made available to the public through the Development Experience Clearinghouse (DEC) repository at dec.usaid.gov. For additional information, please contact: Social Impact, Inc. Office 2300 Clarendon Boulevard, Suite 10300 Arlington, VA, 22201 Tel: +703-465-1884 Fax: +703-465-1888 info@socialimpact.com or Social Impact, Inc. Arz Street Librex Bldg. Bloc B—3rd Floor Zalka, Metn, Lebanon Tel: +961-1-879260 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ABSTRACT The United States Agency for International Development (USAID)’s Livelihoods and Inclusive Finance Expansion (LIFE) project is a five-year (2016 -2021) $20.3 million activity that aims at improving livelihoods, supporting microfinance, and expanding inclusive finance in Lebanon. This evaluation aims to assess the performance of LIFE in terms of relevance to the current economic context, effectiveness in terms of objectives and impact, efficiency in terms of monitoring and evaluation, sustainability, and success in gender mainstreaming. The evaluation team carried out a total of 47 key informant/group interviews and 25 focus group discussions, plus an independent analysis of the project’s monitoring, evaluation, and learning database. The project has encompassed an unusually diverse range of activities, as its priorities have shifted repeatedly over its history. Over this time, LIFE has demonstrated a remarkable ability to connect with and remain responsive to stakeholders at all levels. Noteworthy and enduring achievements include livelihoods work (grant/training packages) for end-user beneficiaries and capacity building for partners. Other promising work, including partnerships with commercial banks, has been undermined almost completely by the financial crisis. Although all questions of sustainability will be heavily conditioned by the current crisis, the impacts of training/capacity building are likely to be sustained at all levels. LIFE’s inclusion of women has been consistent and effective in every component of its work. Recommendations include full and continued USAID support for the project, but with a new and more focused theory of change to support a narrower range of activities with interconnected, well-defined ends, mindful of the current economic crisis. ii | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ACKNOWLEDGMENTS The evaluation team would like to thank PMSPL II’s staff, LIFE’s staff, and all those that have helped facilitate the work on this midterm evaluation. Evaluation Team Dr. Michael Ferguson [Team Leader] Ziad Antonios [Local Consultant] Elias Samia [Local Consultant] Dr. Lara Macaron [Local Consultant] Tamara Nassereddine, DBA [Senior M&E Specialist, Social Impact] Dr. Rana Taher, Ph.D. [Senior M&E Specialist, Social Impact] iii | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 TABLE OF CONTENTS ABSTRACT I ACKNOWLEDGMENTS II ACRONYMS IV EXECUTIVE SUMMARY V EVALUATION BACKGROUND 1 Project Background 1 Evaluation Purpose 3 Evaluation Questions 4 FINDINGS & CONCLUSIONS 5 Evaluation Question 1 (Relevance) 5 Evaluation Questions 2 & 3 (Effectiveness) 7 Evaluation Question 4 (Efficiency) 15 Evaluation Questions 5 & 6 (Sustainability) 16 Evaluation Question 7 (Gender) 19 SUMMARY: FINDINGS & CONCLUSIONS 23 RECOMMENDATIONS 26 Future of the LIFE Project 26 Other Recommendations 27 ANNEXES 29 Annex I: LIFE Grants 29 Annex 2: Details on evaluation design, methods, sample, limitations 33 Annex 3: DFC Supplement 40 Annex 4: Evaluation Design Matrix 42 Annex 5: Interview Guides 42 Annex 6: Inception Report 42 TABLES AND FIGURES Tables Table 1: Distribution of KIIs—Clients and Staff.......................................................................................................33 Table 2: Group interviews with MFI and bank employees. ...................................................................................36 Table 3: FGDs with end-user beneficiaries................................................................................................................37 Figures Figure 1: LIFE beneficiary at work in Bekaa.................................................................................................................7 Figure 2: Mixed-gender beneficiary focus group in the North. ............................................................................18 Figure 3: Female store owner and LIFE beneficiary......................................Error! Bookmark not defined. iv | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ACRONYMS ADS Automated Directives System AOR Agreement Officer’s Representative BDS Business Development Services BdL Banque du Liban CBO Community-Based Organization CDCS Country Development Cooperation Strategy COP Chief of Party DFC Development Finance Corporation DO Development Objective Fintech Financial Technology IT Information Technology KII Key Informant Interview LIFE Livelihoods and Inclusive Finance Expansion LIM Lebanon Investment in Microfinance LMFA Lebanese Microfinance Association LUPD Lebanese Union for People with Physical Disabilities M&E Monitoring & Evaluation MEL Monitoring, Evaluation, and Learning MFI Microfinance Institution MSMEs Micro, Small, and Medium Enterprises NGO Non-Governmental Organization PMP Performance Management Plan PSD Private Sector Development PSMPL II Performance Management and Support Program for Lebanon II PWD Persons with Disabilities RCT Randomized Control Trials RF Results Framework RFI Request for Information RFP Request for Proposal TA Technical Assistance TOC Theory of Change USAID United States Agency for International Development USAID/Lebanon USAID’s Mission to Lebanon v | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 EXECUTIVE SUMMARY PROJECT BACKGROUND Livelihoods and Inclusive Finance Expansion (LIFE) is a five-year (2016 -2021) $20.3 million activity that aims at improving livelihoods, supporting microfinance, and expanding inclusive finance in Lebanon. The purpose of the LIFE activity is to improve economic opportunities and the livelihoods of low-income individuals and microenterprises by expanding and strengthening microfinance associations’ and member institutions’ services and enhancing entrepreneurs’ business skills. LIFE is designed as an activity under USAID/Lebanon’s Private Sector Development (PSD) project. Since its inception, the project underwent two modifications that increased the obligated amounts and proposed significant changes in the outputs based on USAID’s request. EVALUATION PURPOSE AND QUESTIONS The objective of this mid-term evaluation is to assess the performance of LIFE in terms of relevance to the economic and political context, effectiveness in terms of components and activities in achieving objectives and impact, efficiency in terms of monitoring and evaluation for design/implementation purposes, sustainability, and success in gender mainstreaming. At the request of LIFE’s Agreement Officer’s Representative (AOR), the evaluation also scouted for feedback on a possible Development Finance Corporation (DFC) intervention for the microfinance sector within Lebanon. Evaluation questions (EQs) include: EQ1: Relevance - In what ways has the economic and political context created new challenges for the LIFE project and its objectives? In what ways has the project pivoted in order to remain relevant? EQ2: Effectiveness - How are the three project components working to date? To what extent is the activity achieving its objectives, including projected impacts at the beneficiary level? EQ3: Effectiveness - What combinations of activities and components have been most successful in achieving their objectives? Why? EQ4: Efficiency - How and to what extent is the current monitoring and evaluation plan efficient in informing project design and implementation? EQ5: Sustainability - To the extent visible at the midpoint of a project, what activities or interventions appear to be more sustainable than others? EQ6: Sustainability - What should be the nature of USAID’s support for the Lebanese Microfinance Association (LMFA) going forward? EQ7: Gender - To what extent were the recommendations generated by the Gender Analysis integrated in LIFE’s implementation? What are the outcomes generated as a result of gender mainstreaming? As noted above, the evaluation team added the question: What is the stakeholder perspective on the potential value of establishing a DFC fund with local banks to guarantee microfinance loans and promote lending to MSMEs? It was not viewed as one of the principal evaluation questions and is reported to USAID/Lebanon in Annex 3 only. vi | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 EVALUATION DESIGN & METHODS A total of 47 key informant interviews (KIIs)/group interviews were conducted with key project stakeholders including USAID/Lebanon management, LIFE management, LIFE partners management and trained staff, Bank du Liban (BdL), the Lebanese Microfinance Association (LMFA), microfinance institution (MFI) partners, non-governmental organizations (NGOs) partners, and commercial bank partners. In addition, the evaluation included 25 focus group discussions (FGDs), covering 178 micro and small businesses, with people who benefited from training and/or training interventions provided by LIFE’s MFI and NGO grantees. The evaluation team supplemented the qualitative inquiries with a document/desk review, including an independent examination of quantitative data available from LIFE’s monitoring, evaluation and learning (MEL) database. Data collection occurred from January 22, 2020 to February 8, 2020. FINDINGS EQ1: Relevance. The trajectory of the LIFE project since its start in Q1 FY2017 has been one of uneven progress and shifting priorities brought about by both internal and external forces. LIFE began slowly with marginal progress over its first five quarters then witnessed a change in leadership and a remarkable rebound in FY2018 in areas including business development services (BDS) training for beneficiaries, capacity building for MFIs/NGOs, and collaboration with the LMFA. After a major modification in Q1 FY2019 shifted focus to collaborations with commercial banks, the project built relationships and plans over the next year, but progress was slowed by bureaucratic delays at the banks. Then in Q1 FY2020, the October 17, 2019 revolution struck, upending the work with the banks and slowing much of the work with the MFIs/NGOs. LIFE is actively exploring responses to the crisis but has yet to settle on a coherent new direction. Its short-term responses include subsidies for loan officer incentives, to help sustain lending through the crisis, and crisis-management training. Partners and beneficiaries continue to call on the project to help “restore confidence in the financial sector,” but offer few specifics on what that means. Beneficiaries continue to call on the project to provide more direct aid and subsidies. EQs 2 and 3: Effectiveness. Reflective of the project trajectory described above, LIFE missed its key targets in FY2017, then witnessed a rebound in FY0218, followed by another dip in FY2019. The main reasons for the most recent dip are delays in outputs linked to commercial-bank activities and an unrealistic escalation of targets from FY2018. Over the life of the project, LIFE’s management cited relationship-building with the banks, capacity￾building with the LMFA, and livelihoods work with end-users as its major accomplishments. The same managers saw the clearest evidence of impact in the financial technology (Fintech) upgrades among the partners and the changed attitudes among the banks (i.e. new interest in financial inclusion). LIFE’s partners pointed to livelihoods gains as the biggest success among their beneficiaries. Within the partner organizations themselves, Fintech upgrades and membership in regional “Hubs” were seen as the most important advances. The partners saw the biggest impacts in terms of the livelihoods gains among the beneficiaries, with Fintech upgrades and advances in the regional hubs as the most impactful to their own institutions. Both LIFE’s management and partners saw the disruptive effects of the current economic crisis as the project’s biggest challenge. vii | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 LIFE’s beneficiaries saw deep and lasting impacts as results of the training and grants packages they received, on both business and household levels. At the same time, there were two significant sources of discontent among beneficiaries: 1) some trainees felt it unfair that they did not receive a grant; and 2) some beneficiaries did not appreciate the requirement to take a loan, seeing the loan as a “burden.” Compared to other donor-funded projects, LIFE’s partners appreciated the project’s local staff/local grounding, its relatively intense support and follow-up, and its commitment to customizing interventions to partner and beneficiary needs. When asked about the most effective intervention combinations, there was mild consensus that the simple combination of grants plus BDS training was the most effective. EQs 4: Efficiency. The LIFE project did not begin with a vision for monitoring and evaluation (M&E) as informing design and implementation. Its primary purpose on the project was viewed in largely traditional terms—i.e. the retrospective measurement of achievement only. More recently, and in response to USAID’s growing interest in adaptive management, LIFE’s management has attempted to institute a broader and more prospective use of M&E data for implementation purposes. Most notably, this has included the direct involvement of M&E staff and data in the grant-selection process. Partners (i.e. the sources of most M&E) conveyed no shared understanding of the design/implementation function of M&E—suggesting that more buy-in is needed at the source level for this vision of M&E. EQs 5 and 6: Sustainability. LIFE’s management and partners felt Fintech upgrades, partner capacity￾building, and regional Hubs will have the most sustained effects. The beneficiaries spoke more generally about sustainability and saw sustained effects of the training they received at both the business and household levels. Both male and female beneficiaries saw particularly sustained effects among the project’s female beneficiaries. Many beneficiaries felt any window to sustainability was obscured by the current crisis—i.e. it was difficult to say at the present time what would be sustained. Among the member organizations, there are decidedly mixed views on the value of the LMFA. Most commonly, LIFE’s partners pointed to advocacy for the microfinance sector among donors and BdL as the LMFA’s most important function. In light of USAID’s planned cuts to support, the future of the LMFA is highly uncertain. Most respondents agreed that member dues are unlikely to sustain the organization in its current form. There were calls for new external support but no actionable specifics. The most feasible suggestion at present appears to be vastly downsizing the LMFA to reduce costs. EQ 7: Gender. The LIFE project has done exemplary work in its outreach to women and gender mainstreaming. Stakeholders at all levels agree that the project has been highly inclusive to women and responsive to the needs of female beneficiaries. Many stakeholders agree that LIFE is likely having a deeper overall impact on its female beneficiaries. At the same time, by management’s own admission, the project has largely ignored the Gender Analysis in charting its course on gender mainstreaming. Given the Automated Directives System (ADS) requirements around adhering to the recommendations of GA, this trajectory may be problematic. viii | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 LIFE’s outreach efforts have also extended effectively to youth and persons with disabilities (PWD). Over 25 percent of LIFE’s beneficiaries are below the age of 30, and the project has established a highly productive collaboration with the Lebanese Union for People with Physical Disabilities. CONCLUSIONS EQ 1: The project has encompassed an unusually diverse range of activities as its priorities have shifted repeatedly over its history. The economic crisis continues to demand a strategic response. EQ 2 & 3: The project has demonstrated a remarkable ability to connect with, inspire, and remain responsive to stakeholders at all levels. LIFE’s most noteworthy and enduring achievements have been in the realm of livelihoods work (grant/training packages) and capacity building for partners. Other promising work, especially with the banks, has been undermined almost completely by the financial crisis. EQ 4: LIFE’s M&E systems and their use remain relatively basic, despite a growing awareness of their value in implementation/adaptive management and some technological upgrades. LIFE’s M&E system is beset with unrealistic targets and does not capture the value of LIFE’s work at the partner and community levels. EQs 5 & 6: The effects of training/capacity-building are likely to be sustained at all levels. All questions of sustainability will be heavily conditioned by the financial crisis. Major downsizing and cost-cutting is the LMFA’s most feasibile path to sustainability. EQ 7: Regardless of the Gender Analysis, LIFE’s inclusion of women has been consistent and inspirational in every component of its work. The LIFE project exhibits a broad and highly effective approach to the inclusion of vulnerable populations such youth and PWD. RECOMMENDATIONS The evaluation team recommends that USAID pursue all options to allow the LIFE project to continue its work to the fullest extent possible. Our basis for this recommendation, first and foremost, is the project’s highly competent team and exceptional leadership. The range of activities that the project has supported certainly shows enough promise to merit USAID’s continuing support. At the partner and beneficiary levels, it is clear that the bulk of these activities are both wanted and needed. At the same time, we advise USAID to continue its support with key conditions. We propose that the project has gone in too many directions and spread itself too thin. The diffuse approach has been enabled by a lack of a stable and coherent theory of change (TOC) since the start. For the remaining period of performance, we highly recommend that the LIFE team and USAID/Lebanon work together to create a prospective TOC that will support a narrower range of activities serving well-defined, inter￾connected ends. Our recommendation would be to focus a tight package of livelihoods interventions and capacity-building among the MFIs/NGOs, with parallel de-emphasis of the banks. ix | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Other recommendations: • Consider retooling the grant programs so that all those trained get grants. • Extend coaching/mentoring elements of the training programs. • Anticipate and plan for bureaucracy/delays in any future work with banks, in case the project continues substantive activities with them. • Revisit the Gender Analysis; review and incorporate more of its specific recommendations (e.g. gender awareness sessions in all training offered). • At least short-term: consider coupling in-kind grants with working capital (purchasing goods and paying salaries) to help mediate the effects of bank cash-withdrawal restrictions. • Include more crisis-management training. • Retool the LMFA to decrease its operational costs substantially. • Institute protocols for post-post-training follow-up. • In general, seek newer and better ways to capture LIFE’s story and its ability to inspire stakeholders. 1 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 EVALUATION BACKGROUND PROJECT BACKGROUND The Livelihoods and Inclusive Finance Expansion (LIFE) project is a five-year (2016 -2021) $20.3 million activity that aims at improving livelihoods, supporting microfinance, and expanding inclusive finance in Lebanon. ACTIVITY OBJECTIVES The purpose of the LIFE activity is to improve economic opportunities and the livelihoods of low￾income individuals and microenterprises by expanding and strengthening microfinance associations’ and member institutions’ services and enhancing entrepreneurs’ business skills. The activity consolidates the gains made under the previous LIM project, expanding inclusive finance to the unbanked and higher-risk, micro-business clientele and bringing the Lebanese microfinance system to the next level of service provision and financial sustainability. LIFE is designed as an activity under USAID’s PSD project. In terms of the Mission’s Country Development Cooperative Strategy (CDCS), the activity is a critical component of USAID/Lebanon’s Development Objective 2 (DO2): Inclusive Economic Growth Enhanced, and therefore both directly and indirectly supports other ongoing economic-growth projects in areas like value-chain and small￾business development. Consistent with USAID/Lebanon’s CDCS, the Central Bank of Lebanon (Banque du Liban, or BdL) continues to stress the need for financial inclusion, especially at the level of low-income people. Over half of Lebanon’s population lacks access to formal financial services, and communities without access to banking services tend to be poor and rural. Rural populations experience higher-than-average rates of unemployment and host large numbers of Syrian refugees. Women, youth, and PWD are disproportionately excluded from the financial system. Without access to finance and other business services, entrepreneurs and small, family-run businesses in these communities struggle to grow their incomes. With more than 95 percent of businesses in Lebanon considered medium, small and micro enterprises (MSMEs), the LIFE activity’s targeting of microfinance for MSMEs is expected to increase local job opportunities and also encourage the entry of vulnerable groups such as women and youth into the formal and already extensive (but weak) MSME network. LIFE therefore has taken steps to make financial and non-financial services more readily available to the country’s marginalized populations, helping MFIs to develop services that address the problems faced by MSMEs and vulnerable populations. The LIFE activity supports an integrated and balanced approach to further advance the microfinance sector in general and microcredit in particular. It assists with institutionalizing the provision of support that borrowers require to manage their finances and apply the skills for growing their enterprises. While Lebanon is a well-reputed commercial banking center, its financial-inclusion policies and microfinance ecosystem have not kept pace with international best practices in terms of market penetration, innovation, and institutional sustainability. USAID saw a tremendous opportunity to expand inclusive finance and build both the supply and demand for financial services that can propel economic growth, investment, and job creation in Lebanese communities that need it most. As part of this effort, 2 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 USAID/Lebanon asked in 2019 for a strategic amendment of LIFE’s scope, shifting one the project’s goals from access to credit to microenterprises to financial inclusion at the level of banks, which meant that the project had to seek partnerships between MFIs and banks. LIFE has to-date delivered what may be its final assistance to LMFA (which was established under LIM) and is encouraging it to seek its own sources of funding in order to ensure its sustainability. Following LIM, USAID/Lebanon believes that it has successfully graduated the microfinance sector from assistance and should now focus its assistance towards the livelihoods of end-beneficiaries. ACTIVITY COMPONENTS The LIFE project has three components: 1) Inclusive finance policies enacted through empowered local organizations: With a focus on elevating the discourse on access to finance and banking services to the national level, the project works through the LMFA, commercial banks, and other stakeholders to develop policies that promote a vibrant, sustainable financial sector that economically empowers the unbanked, low-income, and underserved clientele, particularly in rural areas. 2) Expanding the range of microfinance products available to the poorest: The project is improving the ability of commercial banks and MFIs to increase their range of products and services and expand their reach of clients and borrowers to include microenterprises and the poorest, with an emphasis on higher-risk, conflict-affected areas (areas where economic pressures and social tensions have increased significantly as a result of the high influx of Syrian refugees). This requires an emphasis on youth, people currently without a bank account, the disabled, and low￾income households headed by women. 3) Livelihoods and business support: The project provides customized business-management training, technical know-how, mentoring, and basic business-development skills to micro, mostly family-owned enterprises to give them the skills they need to grow their sales and income. Often, the objective is to enable these small businesses to position themselves to gain access to commercial finance for the first time, a critical ingredient for business growth. The project also provides small grants that generally support technology adoption for improved productivity. LIFE has enacted these components primarily through its grant-making activities to MFIs, NGOs, and commercial banks. See Annex 1 for a complete list of grants and their program areas. DEVELOPMENT HYPOTHESIS The project does not have a stated development hypothesis. CRITICAL ASSUMPTIONS The attainment of the LIFE activity’s outputs, outcomes, impacts, and goals are conditional upon certain external factors remaining unchanged or changes occurring only as anticipated. The critical assumptions for the LIFE project to achieve its targets and objectives, is as follows: • The recent election and appointment of a prime minister provides a platform for a stable government and functioning governing context in coming years with policy makers open to 3 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 policy changes supporting improved access to finance, especially continued government support for the development of the LMFA. • The Syrian crisis and the political and macroeconomic stress that it places on Lebanon’s public infrastructure, services, and economy does not significantly worsen, leading to internal conflict, political strife, or economic crisis. • Despite macroeconomic strains and rising public debt, the overall economy remains stable with continued low but positive economic growth, moderate inflation, sufficient reserves to maintain the exchange rate peg, and relatively stable interest rates. (At the time of publication, this status was in doubt after the recent decision to default on Lebanon’s sovereign debt payment and further devaluation of the Lira.) • MFIs do not witness substantial change in their risk appetites and remain willing to expand business despite macroeconomic uncertainty. • Consumers’ and microenterprises’ risk appetites also remain unchanged, and demand for financing remains stable. Risks and mitigation measures will be monitored throughout the implementation of the project. In addition to monitoring the indicators presented in the Activity Monitoring & Evaluation Plan, the M&E staff will conduct supplementary observational and contextual data collection to track unanticipated consequences and unexpected risks. This supplementary data will be collected via publicly available information and through sharing and collaborating with stakeholders and other projects, specifically, as follows: • Political stability and openness to policy reform will be monitored via observation of government communications and official reports. • Monitoring the Syrian crisis and its effects on the Lebanese political and economic environment will primarily be through news reports and international crisis monitoring. • The Lebanese macro economy will be monitored via reports from the central bank, the ministry of finance, international organizations such as the International Monetary Fund, and Bloomberg as well as other credit rating agencies in the private sector likely to reflect the marketplace view of Lebanon’s economy. • MFIs’ willingness to continue expanding business will be monitored via qualitative data collections and assessments that are part of ongoing project activities. • Consumer demand for finance will be monitored through qualitative data collection, such as focus groups, and tracking the number of loan applications. EVALUATION PURPOSE The objective of this mid-term evaluation is to assess the performance of LIFE in terms of relevance to the economic and political context, effectiveness in terms of components and activities in achieving objectives and impact, efficiency in terms of monitoring and evaluation for design/implementation, sustainability, and finally LIFE’s success in gender mainstreaming. The audience for the evaluation is the USAID/L Mission, plus other relevant stakeholders as agreed upon with the Mission. In addition to the above, and upon the request of LIFE’s AOR, the evaluation will scout for feedback on a possible DFC intervention for the microfinance sector within Lebanon (results reported only to USAID/L in Annex 3 of this report). 4 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Initially, the evaluation was scheduled to take place in October 2019, but with the outbreak of the revolution on October 17 and the closure of the country for three weeks, PMSPL II postponed it until January 2020. Since October, the economic and financial situation in the country has deteriorated seriously, marked by the closure of banks for three weeks, the devaluation of the Lebanese Pound within the market, the institution of capital control on depositors, the regulation of USD cash withdrawals, stopping the Letter of Credit and other international transfers, and so on. This affected all sectors within the Lebanese economy including MFIs and microenterprises. These recent developments pose important questions for LIFE, particularly after its shift toward financial inclusion and engagement with formal finance in 2019. Today, LIFE is at a crossroads as the whole strategy of financial inclusion faces major challenges and obstacles in light of a shaky banking and financial sector. This mid-term evaluation will take into consideration the status of the Lebanese economy in drawing conclusions about the project’s successes and challenges. At the same time, it will be primarily retrospective in nature and will not aim to provide speculative answers as to what will happen six months down the line on Lebanon’s social and political landscape. EVALUATION QUESTIONS EVALUATION QUESTION (EQ) 1 - RELEVANCE In what ways has the economic and political context created new challenges for the LIFE project and its objectives? In what ways has the project pivoted in order to remain relevant? EVALUATION QUESTIONS 2 AND 3 - EFFECTIVENESS EQ2: How are the three project components working to date? To what extent is the activity achieving its objectives, including projected impacts at the beneficiary level? EQ3: What combinations of activities and components have been most successful in achieving their objectives? Why? EVALUATION QUESTION 4 -EFFICIENCY How and to what extent is the current monitoring and evaluation plan efficient in informing project design and implementation? EVALUATION QUESTIONS 5 AND 6 (SUSTAINABILITY) EQ 5: To the extent visible at the midpoint of a project, what activities or interventions appear to be more sustainable than others? EQ6: What should be the nature of USAID’s support for the LMFA going forward? EVALUATION QUESTION 7 (GENDER) To what extent were the recommendations generated by the Gender Analysis integrated in LIFE’s implementation? What are the outcomes generated as a result of gender mainstreaming? SUPPLEMENTAL QUESTION As noted above, LIFE’s AOR has requested that this evaluation add an exploratory inquiry around a possible DFC intervention for the microfinance sector within Lebanon. The results are reported in Annex I1. We have articulated the relevant question as follows: What is the stakeholder perspective on 5 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 the potential value of establishing a DFC fund with local banks to guarantee microfinance loans and promote lending to MSMEs? EVALUATION DESIGN, METHODS, AND LIMITATIONS The evaluation team used primarily a qualitative approach to answer the evaluation questions, conducting KIIs, group interviews with partner trainees, and FGDs with beneficiaries. We conducted a total of 36 KIIs with key project stakeholders including USAID/Lebanon management, LIFE management, the BdL, the LMFA, MFI partners, NGO partners, and bank partners. We held 11 group interviews with banks and MFIs employees who received training and capacity building under LIFE. In addition, we conducted 25 FGDs, covering 178 micro and small businesses, with people who benefited from training and/or training interventions provided by LIFE’s MFI and NGO grantees. The KIIs were conducted by the evaluation team, while the FGDs were facilitated by Social Impact’s subcontractor, InfoPro. Average size of the FGDs was seven participants, with roughly equal numbers of female-only groups, male-only, and mixed. The gender distribution across the entire sample (including interviews and FGDs) was 58 percent female. We supplemented the qualitative inquiries with a document/desk review, including an extensive examination of quantitative data available from project monitoring, evaluation, and learning (MEL) database. A comprehensive evaluation design matrix with the evaluation questions, data sources, data collection methods, and analysis methods can be found in Annex 4. NOTE: For full details on evaluation design, methods, sample, and limitations, see Annex 2. FINDINGS & CONCLUSIONS EVALUATION QUESTION 1 (RELEVANCE) In what ways has the economic and political context created new challenges for the LIFE project and its objectives? In what ways has the project pivoted in order to remain relevant? A HISTORY OF CHANGE/FLUX The trajectory of the LIFE project since its start in Q1 FY2017 is one of flux and shifting priorities, brought about by both internal and external forces. According to those closest to project (e.g. LIFE Management, USAID officials), for its first five quarters, the project was characterized by a lack of consensus on direction and consequent lack of progress of activities and targets. In Q2 FY2018, the project’s Chief of Party (COP) was replaced, and for the next year there was considerable progress on major activities including BDS training for beneficiaries, capacity-building for MFIs and NGOs, and productive collaboration with the LMFA. In Q1 FY2019, USAID/Lebanon redirected the project with a major modification that shifted its focus toward a more explicit focus on financial inclusion via partnerships with commercial banks, including the Pathway to Savings initiative linking MFIs and banks, and the banking “truck” initiative, whereby BLOM 6 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Bank created and launched a vehicle that brought banking services to rural areas on a mobile basis. LIFE’s engagement with NGOs shifted to focus on regional “Revitalization Hubs,” regional networks that served to distribute grants and encourage collaboration among the member organizations. Meanwhile, LIFE began supporting the work of Nudge Lebanon (see text box below in report), which aimed to train commercial banks’ staff on the use of behavioral economics techniques and test the results of that training with rigorous research. By the close of FY2019, some of these activities had progressed well, while others remained in their infancy. In particular, the collaborations with the banks had moved slowly, largely because of bureaucratic delays in decision-making within the banks. Then the revolution of October 17, 2019 hit the country, and the majority of these activities came to a halt. Nearly all work with the banks was stopped, though their grants and relationship with LIFE remain. The Nudge project was halted and will soon be canceled officially. The MFI and NGOs have continued to push forward on some of their work, but they have vastly slowed the pace of lending activity and any collaborations involving the banks. LIFE’S EVOLVING STRATEGY SINCE OCTOBER 17, 2019 As of February 2020, with the post-October 17, 2019 economic crisis still in full swing, LIFE remained deep in discussion with all of its partners on how best to respond and remain relevant in the current landscape. The project issued a formal request for information (RFI) to MFI partners in December 2019, seeking ideas for new programming directions; results from that are still under review. Meanwhile, the project is fielding proposals from bank partners on how best to reallocate their existing grant money. Much of the discussions with partners—particularly the banks and any MFIs/NGOs involved in lending—has centered on the concept of “restoring confidence in the financial sector.” As one LIFE manager articulated it: “After October 17th, most people have pulled out their funds from the banking system and put their money under the mattress, that’s why we can’t still promote financial inclusion. What we have to do is to restore the confidence in the banking system because business can’t survive without passing through banks.” As of yet, there is no clear consensus on the activities that will help work toward that end. LIFE already has deployed what it sees as a number of short-term solutions. Chief among these is an idea developed in collaboration with the MFIs to subsidize loan officer incentives for new loans. The initiative provides financial assistance to these institutions and thereby helps ensure that lending continues at the time when it could easily contract or stop. In some cases, LIFE added limited operational/salary support to these packages as well. Another important short-term response has been the addition of crisis management training (e.g. how to manage discontent among customers) to the training packages for partner institutions. HOW STAKEHOLDERS SUGGEST THAT LIFE RESPONDS We asked partner and beneficiary stakeholders for their suggestions as to what LIFE can do to remain relevant in the current crisis. The answers were largely broad and non-specific. Like LIFE management, the partners often spoke of “restoring confidence in financial sector” but did not offer specifics as to what that would entail. The training sessions on crisis management were 7 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 recognized and praised as highly relevant; some partners also suggested more intensive financial literacy training, potentially for partner staff as well as beneficiaries. Finally, there were general and predictable calls for new external funding: new loan funds, new loan guarantees, and new grants for beneficiaries. For their part, beneficiaries with loans called for modified terms: reduced interest, longer terms, etc. Others called for more grant funding, including aid to start-up enterprises, or just more unrestricted aid distribution. At the same time, a sizeable minority of beneficiary respondents took the pessimistic view that LIFE could do nothing to improve the situation of beneficiaries at the present time. EVALUATION QUESTIONS 2 & 3 (EFFECTIVENESS) How are the three project components working to date? To what extent is the activity achieving its objectives, including projected impacts at the beneficiary level? What combinations of activities and components have been most successful in achieving their objectives? Why? M&E INSIGHTS ON EFFECTIVENESS? Like other aspects of this evaluation, the MEL database analysis revealed a project storyline characterized by change and flux. At the onset of the project in Q1 FY2017, the performance indicator plan included 22 indicators distributed over goals, purposes, and sub-purposes. In the early years of the Figure 1: LIFE beneficiary at work in Bekaa. Photo Courtesy of the LIFE Project 8 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 project, a total of four indicators were dropped off, presumably due to irrelevance as activities began to unfold. In Q1 FY2019, USAID/Lebanon launched the new PSD strategy with a modification that shifted LIFE’s focus to partnerships involving the commercial banks. On the MEL side, LIFE introduced further indicators and dropped others. By Q4 FY2019, the total number of indicators stood at 21, heavily revised since the start of the project. In terms of target achievement, as the LIFE project and all stakeholders freely admit, very little was accomplished in the project’s first five quarters. With the coming new project leadership in Q2 FY2018, we see a remarkable rebound in which 13 of 21 target indicators were met or exceeded. In FY2019, we see the success rate dip again, with only six of 21 targets met. Tracing the project’s successes and challenges over 2018-19, we see that in FY2018, LIFE exceeded its goals in areas including the number of grants, the value of financial support to beneficiaries, the number of beneficiaries benefiting from loans and savings accounts, and the number of beneficiaries receiving BDS training. In FY2019, success was sustained in terms of end-beneficiaries receiving BDS. The dip in 2019 related largely to basic outreach among beneficiaries, which is to say, five of the non￾achieved targets are extracted from “number of grant beneficiaries.” Relatedly, the project did not make its goals on the number and value of loans, grants disbursed, and savings accounts opened. The evaluation team sees two major reasons behind these target challenges. First, starting in FY2019, by necessity after the Q1 FY2019 modification, the project began to rely on the output of commercial banks in terms of reaching beneficiaries. Progress with the banks over that year was unfortunately slow, with most agreements reaching the implementation phase only at the end of that fiscal year. It appears that the main reasons for the slow progress were bureaucracy and delays in decision-making on the part of the banks. The second reason for the FY2019 challenges was the unrealistic escalation of targets from the previous year. Many performance indicators jumped by several order of magnitude between 2018 and 2019—in retrospect, an ill-advised projection that dates to the project’s proposal phase. Finally, it is worth noting that the targets related to job creation were never reached or even approximated in any phase of the project’s operation. Again, we believe that the job creation goals were poorly conceived. This is to say, due to the nature of its interventions, LIFE was never well-placed to create many jobs. It is an indicator that dates to the project’s proposal phase, with little relevance to on-the-ground reality. MANAGEMENT ON PROGRAMMATIC SUCCESSES/CHALLENGES In reference to LIFE’s Component 1, management pointed to its relationship-building with BdL and commercial banks. They cited getting both interested in financial inclusion in ways they had not before. As a USAID official put it, “We pushed banks to have a different eye for Financial inclusion; this topic was dismissed by them but now they accept it.” The relationship with the BdL in this process was seen as key. “The way we cultivated the relationship with BdL was important,” explained a LIFE manager. “Because we can’t do financial inclusion without the umbrella of BdL policies.” A secondary success prominently mentioned was the advocacy and technical assistance (TA) conducted with the LMFA. The work with the LFMA (e.g. assisting the association in the development of its communication plan) was seen as a lynchpin to the project’s work and advances in the 9 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 microfinance sector. There was no consensus on the challenges seen with Component 1. The economic crisis was cited, as was some banks’ continuing lack of understanding of the microfinance sector. A lack of unity among the diverse membership of the LMFA was also cited as a recurring hindrance. For Component 2, the most common success cited was once again getting banks interested in microfinance (with those activities cutting across Components 1 and 2). A senior LIFE manager specifically cited the accomplishment of getting five banks to sign and to cost-share 2:1 on programs to promote inclusivity. Another manager noted, “This is first time that USAID has supported commercial banks in Lebanon.” BLOM Bank’s mobile banking truck project, intended to spread savings products to unbanked Lebanese in rural areas, was mentioned as a budding success, unfortunately cut short by the financial crisis. Another frequently-mentioned success was the work in building the capacity of partner MFIs. Explained a LIFE manager, “We were tailoring the content of the workshop for each MFI alone based on their needs in order to come up with the best return on investment.” Challenges cited for Component 2 largely mirrored Component 1: the disruptive effects of the financial crisis and stubborn attitudes on microfinance among certain bank partners. For Component 3, the most commonly-cited successes within the Livelihood work were, logically, the training sessions and grants offered to beneficiaries. One LIFE manager described the professional fulfillment felt when seeing “the reaction of people when they receive an in-kind grant, especially in Tripoli.” Others cited the outreach to women and the project’s influential Women Rising social media campaign in particular as notable successes, while several other cited success in outreach to PWD and youth. Only a few challenges were mentioned—e.g. difficulties working with certain MFIs on livelihoods—and none were mentioned more than once. PARTNERS ON PROGRAMMATIC SUCCESSES/CHALLENGES Speaking on their varied livelihoods combinations of BDS training, vocational training, one-on-one coaching, in-kind grants, loans, and MFI linkage, partners across the board spoke in positive terms about their results. All called the livelihoods programming a success or a success to come, the latter in cases when the programming had just started. Said one NGO staff, “The grants achieved their intended purpose, since the borrower improved his way in doing business and the proof is that most of our clients were satisfied from the results.” The most common reason cited for success was the project’s and the partner’s willingness to customize training and other components to the particular needs of the beneficiaries. Said one NGO manager, “The beneficiaries are happy with the topics since they match their needs.” 10 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 The assessments of the Fintech grants and the capacity-building for partner staff were similarly positive. All such efforts were described as successful in achieving their purpose. Activities ranged from introduction of the new Tahaki M&E software, to the digital integration of MFI systems, financial literacy training, customer care training, supervisory training, and stress-management training. Once again, the overwhelming reason cited for success was the project’s participatory approach and its efforts to customize Fintech/training to staff and agency needs. Explained an MFI manager, “The yearly performance appraisal for all the staff identify the training needs for each employee, and LIFE was responding positively to the requests.” Said an NGO manager, “[The trainers] customized the training and delivered different trainings delivered to different groups. The training was delivered according to design and needs, and we went over the issues with them to ensure success.” The only significant challenge cited was the fact that the economic crisis had hindered the use of some of these upgrades and trainings, as some partner business had fallen off. Other activities singled out as successful included the establishment of the so-called “revitalization Hubs,” where groups of NGOs combined and coordinated their efforts to distribute grants, serve particular regions, and share knowledge. As one partner summed it up, “We have a geographic division around Badawi. Each member in the HUB covers a geographic area and chooses a project tailored to their strengths. We selected areas of interventions and met many times to connect our work together. We did mutual visits to learn from each other: capacity building, lessons learned. We try to complement each other and all the knowledge given to the beneficiaries by providing the needed support to improve their work and provide access to market. We meet on regular basis to coordinate among each other.” At the same time, some activities were cut short more dramatically by the revolution and financial crisis, almost as soon as they began. Among these were a variety of savings initiatives, involving collaborations between the MFIs and commercial banks to bring unbanked and underserved customers to formal finance. Explained a manager at MFI Emkan, “Our initiative was designed by Emkan and adopted and supported by LIFE, which permits the microcredit beneficiaries of Emkan to open saving account with Bank MED, partner of Emkan on this initiative. All the fees associated with the debit Nudge Initiative Nudge Lebanon is a nongovernmental institute that specializes in applying behavioral science to a variety of public policy settings. The basic premise is to employ positive reinforcement and indirect suggestions as ways to influence the behavior and decision making of groups or individuals—for example, the systematic use of SMS to remind or “nudge” individuals to behave in certain ways. Nudge received a grant from LIFE to provide training for banks on using behavioral science techniques to encourage savings activity among rural women—either opening accounts or spurring deposits among dormant accounts. Over FY2019, Nudge trained BLOM Bank, Audi Bank, and BLC Bank accordingly, and launched what became three separate Randomized Control Trials (RCTs) to test the effects. The RCTs were lean and effective designs that employed customer data collected by the banks, rather than costly independent data collection. By the close of FY2019, Nudge had completed the training and collected baseline data for the RCTs. Early incremental observations suggested positive behavioral change among the treatment groups. At the same time, the banks were utilizing behavioral science techniques in their own work, above and beyond the research. They experimented with ways to motivate both customers and their own staff. After October 17, the banks cancelled all the research. They continue to experiment with some of the techniques but in a greatly diminished manner. Nudge’s grant with LIFE will close in early 2020. 11 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 card issued by the bank for the opened accounts will be covered by the project. It is a first-time initiative within the microfinance sector to create a tool for saving for microfinance beneficiaries.” In another case, LIFE supported BLOM Bank to create the afore-mentioned mobile banking services delivered by truck to unbanked people in Lebanon’s rural areas. The program ran for barely two months before the revolution and economic crisis caused it to be shut down. Another casualty of the economic crisis was the Nudge initiative, which endeavored to bring training rooted in behavioral science principles and associated research to the activities of several commercial banks. MANAGEMENT ON IMPACTS When asked to comment on project impact at each level of stakeholder, the MFI partners gave three predominant responses. First, they mentioned the Fintech grants, bringing “solutions & digitizing the process that will increase efficiency and reduce cost on MFI.” Second, referencing both the MFIs and the LMFA, they cited the broad improvement in institutional capacities and human resources that resulted from the training sessions offered to staff and the assistance in strategic planning. Finally, management respondents cited the increased outreach and enhanced loan portfolios that resulted from these improvements among many MFIs. The responses regarding impact among the NGOs largely followed suit, with prominent mention of the Fintech grants as impactful. They also mentioned the role of the Hubs as regional centers of the collaboration and professional resources. Among the bank partners, the impact was largely seen in terms of the changing attitudes on financial inclusion. “We supported them going to the unbanked population,” said one LIFE Manager. “We steered their focus toward financial inclusion and accelerating some of the existing policies.” Among the beneficiaries, management pointed to a wide variety of impacts, with no strong consensus on any one area. There were several mentions of the strong inclusion of and benefit to women, and in parallel, youth and PWD. “For people with disability, LIFE was effective especially in Beqaa and North,” explained a USAID manager. “Many trainings were done to improve their income.” Elsewhere, they pointed to the impacts of the skill building in BDS, financial literacy, and “how they are using social media for marketing purposes.” What Distinguishes LIFE’s Approach to Training? In order to understand what distinguished LIFE’s approach to training beneficiaries, the evaluation team spoke with the principal staff from the three firms contracted to deliver training to partner organizations. All were veterans of donor￾funded training programs and deeply￾experienced training professions. All pointed out that LIFE connected the trainer and partner staff for collaborative design sessions before the training—this was highly unusual in their experience. In this way, the training was customized to respond to the needs. Prior to each training, they secured approval/buy-in for the content and approach from the management of the partners involved. The trainers also pointed to LIFE’s general presence and follow-up in all phases of the training. They described close and supportive relationships with LIFE staff from the beginning to the end of their work. “The whole environment is friendly,” one trained added. The trainers added that nearly all of their training sesions were filled to capacity, across the board. Again, this was unusual in their experience. 12 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 PARTNERS ON IMPACTS When discussing the impacts of their livelihoods activities, LIFE’s partners zeroed in on the effects on beneficiaries’ business, with the improved business skills/grant combinations feeding advancements directly into the enterprises, including increased productivity, elevated incomes, and, in some cases, new job creation. Explained an MFI manager, “The specific effect on beneficiaries was mainly the increase in their production and sales, decrease in their expenses, better bookkeeping records.” Added an NGO manager, “The in-kind grants helped them to empower their businesses and having a competitive advantage over their competitors by having new machines that can accelerate their production and enhance its quality.” Several respondents suggested that these impacts were particularly acute among their women and PWD beneficiaries. In terms of the impact of the Fintech and staff capacity-building activities, the respondents pointed to new information technology (IT) efficiencies and a wide variety of other improvements in organizational capacities, including customer service, supervision, time/stress management, and communications. For example: “Directly after the stress management training, they produced a timesheet for each field employee where they would organize their time and the HR would know how much time is put on each activity, which resulted in achieving the target earlier and improving their future targets.” The regional Hubs received high praise for their impact in strengthening professional networks, building institutional capacities, and, in some cases, expanding the client base of partners. Even among activities cut short by the financial crisis, positive impacts were reported. Several banks participating in the Nudge initiative, for example, reported that they were still using those behavioral science-based techniques to positive effect, even after the relationships with Nudge was discontinued. One bank staff member related this anecdote, “In the example of the call center, we would get 2 customers for 100 calls. It was around 2%. We looked at the script and used the technique of loss aversion. When we introduced Nudge techniques, modified the script by focusing loss aversion, the results jumped from 2% to 9%. This was possible with Nudge.” BENEFICIARIES ON IMPACTS The beneficiaries commented directly on the impact the grants and/or training packages in a variety of ways. Most of them concurred that the in-kind grants they received achieved their intended purpose—which is to say, the project purchased and provided productive assets (or in some cases, raw materials) that were used to launch, expand, or sustain businesses (Small amounts of cash for business purposes were awarded in a minority of cases as well). The only scenarios when grants did not achieve their intended purpose were related to the economic crisis—e.g. equipment was awarded but business expansions failed because a customer base melted away. Some beneficiaries also expressed their discontent that they were not awarded a grant that they sought or applied for; in some cases, they cited what they felt to be an unfair or biased awarding process. In terms of more immediate impacts of the grants, the most common response was that increased productivity was achieved. Said one enterprise owner, “I’ve always been dreaming of having such advanced machines and now my dream came true since my work became 10 times easier and quicker than before.” Several beneficiaries added that the equipment awarded had relieved some of the physical demand of running their business—e.g. they were granted a mixing machine that eliminated the strenuous hand-mixing of bread dough. The malcontent among those who did not receive a grant 13 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 continued here as well, with a handful of respondents suggesting that the application process drew away time and resources, and that their livelihoods suffered as a result. Most recipients of training—whether as a stand-alone activity or coupled with a grant or loan program—were similarly upbeat about the intervention. Consistent with LIFE’s approach of customized training packages, based on partner and beneficiary needs, the range of training sessions offered to beneficiaries was exceptionally wide. BDS training was the most common, but offerings included: legal training, communication/negotiation skills, crisis management, social media/online marketing, various forms of vocational instruction, and life skills. The vast majority of beneficiaries agreed that the training achieved its intended purpose of imparting actionable knowledge on their given subject. Most beneficiaries credited the exceptional skills of the trainers in achieving these results. “The trainer was very open-minded, and didn’t mind answering my questions,” explained one beneficiary. “He used to give everyone a chance and made sure everyone was learning efficiently.” Another beneficiary added that he “previously got used to chaotic behavior but that wasn’t the case during the two-three hours of consecutive training where the atmosphere emphasized on discipline and focus.” There was a significant minority of beneficiaries who felt the training sessions did not achieve their purpose. The most common reason was that they were unable to apply any insight in the current economic landscape. However, others felt the sessions were too theoretical or shallow or general—i.e. not tailored to the particular business sectors that they represented. A significant contingent commented that the training sessions alone were not enough to make a difference, without material aid such as in-kind grants. One enterprise owned explained it as follows, “We benefited from the training; however, it was as if someone gave you a car without gas. Actually, it is as if they taught you how to drive without getting you a car.” In terms of shorter-term impacts of the training, the beneficiaries most commonly responded that they had acquired and started to apply better business skills. For example: “The training helped us a lot in calculating expenses. Before, we used to go out on a whim and calculate it randomly. Whenever I would receive an order, I wouldn’t consider the expenses of gas and other appliances.” Others added that they were applying more specific aspects of the training, such as improved communication skills or new use of social media for marketing their businesses. Still others added that the effects were being seen in their personal/household lives, noting that they were living and working with more self-confidence or a new sense of hope. Others saw distinct effects spilling over into their households. Said one beneficiary, “These trainings helped me and my family know how manage our financials a bit, especially in the current situation we are in.” Another added, “I taught my daughter all skills and knowledge I acquired during these trainings and by that we started introducing educational conversations and discussions at home and we produced somehow a family job.” Around one-quarter of the random sample had received a loan from an MFI as a component (typically required) of the grant and training package. Based on the testimony received, this was a more controversial aspect of the LIFE program—though a retrospective issue only, since the LIFE project has already phased out the pairing of required loans with other program components. The responses were split roughly down the middle between beneficiaries who appreciated the loan and put it to good use, and those who did not. Those in the former group tended to view the loan as important business capital that complemented the equipment upgraded received as in-kind grant. One 14 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 participant explained that “the grant provided the needed tools whereas the loan helped him/her buy the products.” At the other end of the spectrum were beneficiaries who wanted the grant but did not want or need a loan. Some felt the loan requirement was coercive on the part of the MFI partners, and they had concerns about their ability to pay it back. “The loan is a burden on our shoulders now,” said one. A significant minority of loan recipients felt that the MFIs had been less-than-transparent or even deceptive about the loan requirement. Said one enterprise owner, “[The MFI partner] was not honest and clear, and they were told that they will receive support depending on the business of each. However, they were not told about the obligatory loan prior to the sponsor and that was very disappointing.” LIFE COMPARED TO OTHER DONOR-FUNDED PROJECTS Most partners came to the LIFE project experienced in collaborations with other external projects, funded by USAID or other international donors. When asked, they contrasted these experiences with the LIFE project in a variety of ways and often linked these differences to the impacts of LIFE’s work. There was no strong consensus on what was different about LIFE’s approach, but at least one theme emerged. Several respondents commented on the local character of the LIFE team (staffed almost entirely by Lebanese professionals) and their physical proximity to the activities, operating from a local office—as opposed to other donor-funded projects that were managed remotely by foreign professionals. An MFI manager explained, “We believe that LIFE grant was the most important to us as they are located in Lebanon, close to us, we can share thoughts and ideas better and conduct several meetings.” Added an NGO manager, “The area managers selected for the project are from the area…He knows the area and real situation and experience. He knows the area and networks.” Relatedly, a number of partners commented on LIFE’s unusually strong follow-up and supportive presence during all phases of implementation. “Our experience with LIFE was different because we were not used to the daily follow-up from the donors,” said an NGO manager. “We felt the non-trust at the beginning because they wanted to be present at every activity. We felt that they don’t have confidence in us. But it is important to mention that they were very supportive and at the mid-time of the project we became more friendly and we worked together as family.” Another NGO manager continued on this theme: “Compared to another partnership, we meet regularly. We do it as a group. We work with the partners to develop the tools, budgets and other. LIFE follows closely and consistently.” Other differences cited by the partners included LIFE’s approach (unique in their experience) of customizing training sessions to the needs of individual organizations. As stated by an NGO manager, “They connect the training providers and the beneficiaries before the training. It is not common with other projects. This way, the training is customized to respond to the needs.” Still others commented that the LIFE project was more professional, more transparent, and more focused on sustainability than other donor-funded projects. The only significant complaint that emerged in these comparisons was that LIFE and USAID’s paperwork/documentation requirements were comparatively burdensome. This was an unpopular aspect of the project that emerged in partner responses to a variety of inquiries. 15 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 COMBINATIONS OF ACTIVITIES & COMPONENTS MOST SUCCESSFUL The evaluation team concedes that this was not an especially fruitful line of inquiry. LIFE’s partners employed many different combinations of activities and interventions, and generally they responded that whatever combination they were using was the most successful. In many ways, these responses stand to reason, given the LIFE project’s deliberate effort to customize programs to the explicit needs of partner organizations and their beneficiaries—i.e. stakeholders tended to view their own combinations as best because it was exactly what they asked for. We saw a mild consensus among LIFE’s management and partners that a particular and relatively simple combination of grant plus training was most commonly effective. As one LIFE manager articulated it, “BDS and grant is the best synergy. Mix of theory and practice.” As reflected in the previous section, there was a mix of views on loan + in-kind grant combination. Some felt it was a productive mix, but about an equal number saw the loan requirement as problematic, due to a lack of transparency and some coercion in compelling beneficiaries to take loans that they did not necessarily want. LIFE’s management also added the point that some MFIs misused this aspect of the program for the sake of reaching monthly internal loan targets. EVALUATION QUESTION 4 (EFFICIENCY) How and to what extent is the current monitoring and evaluation plan efficient in informing project design and implementation? USE OF MEL IN DESIGN & IMPLEMENTATION MEL is part of a larger monitoring and evaluation hierarchy aimed at ensuring that USAID/Lebanon achieves intended results. Uppermost in the hierarchy is the CDCS that sets the overall high-level goals and strategy the mission has set for the next five years. This is followed by the Results Framework (RF) for the Economic Growth Project under which the economic activities such as LIFE falls. The monitoring and evaluation plan is designed by the LIFE project to fit within the overall strategy and the higher-level indicators. Based on the inclusion of this evaluation question, it is clear that USAID/Lebanon maintains a strong interest in employing M&E for project design, implementation, and ongoing adaptive management. However, it is not clear whether those priorities were in place at the start of the LIFE project, and as such, LIFE did not begin with a MEL plan focused on those purposes. Its initial vision for MEL was more traditionally retrospective, set up to determine whether targets were met and goals were achieved. Complicating the basic functions of the MEL team, much of the project’s early years were spent adjusting MEL indicators to keep pace with the project’s shifting priorities. Nevertheless, over time, the LIFE team has adapted to the current vision of the Mission for utilizing MEL data in a prospective manner. Most notably, LIFE’s M&E team is now involved in a grant evaluation committee to assess grant proposals—thus employing MEL data to weigh proposal quality and shape the next generation of programming. They will also have a direct hand in formulating the selection criteria for current and future grant-making. As one LIFE staff member summed up, “M&E now has a say in the design and can guide us to focus at the indicators.” 16 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 PARTNERS ON MEL PURPOSE As a complement to our inquiries on this topic, we questioned LIFE’s partners on what they saw as the purpose of the MEL data that the project requested of them. The intent was to determine whether the partners—i.e. those responsible for providing MEL data—had an understanding or shared vision of the purpose as informing design and adaptive management on an ongoing basis. The answers were clear and unambiguous—none of those interviewed saw a prospective purpose to MEL data. They saw it as retrospective only, to determine whether past activities had met their goals. This is a significant finding because it demonstrates that there is no buy-in at the source level for MEL data to serve the ends that USAID/Lebanon seeks. In order to achieve this vision going forward, such disconnects must be addressed. TAHAKI & OTHER COMMENTS ON MEL SYSTEMS In the year leading up to the evaluation, LIFE adopted Tahaki Pro as a MEL/data management tool. The project provided the software as well as training to most of the partners. The new Tahaki system is used to manage and centralize a beneficiary database to avoid data clutter, manage data and link them to beneficiaries, enhance field efficiency through data-collection mobile apps, improve decision making and planning through customized dashboards that allow data analysis, and automate reporting and sharing with different stakeholders in real-time. It was universally agreed that bringing in Tahaki was a very substantial project achievement and a vast improvement on the previous system, which was a relatively basic and error-prone system of Excel-based tracking described by one LIFE manager as “very hectic.” The only concerns raised about Tahaki among the partners was a need for more user accounts within each agency in addition to some difficulties around data migration from the previous/existing systems. Beyond the Tahaki upgrades, most partners’ commentary reflects a common view that USAID’s MEL requirements are unusually extensive and burdensome. The quantity and frequency of data requested posed challenges in terms of staff time and also beneficiary confidentiality, particularly around vetting requirements. Explained one partner manager, “One big challenge we faced during the project is when we asked our participants to give us their IDs to be able to photocopy it, some of them refused and didn’t participate in the training.” In addition, there were a number of specific complaints about photos and other documentation required as proof that beneficiaries had received and used their support as intended. The requirements were highly unpopular among partner staff and sometimes raised beneficiary suspicions about the project and USAID’s true intent. EVALUATION QUESTIONS 5 & 6 (SUSTAINABILITY) To the extent visible at the midpoint of a project, what activities or interventions appear to be more sustainable than others? What should be the nature of USAID’s support for the LMFA going forward? 17 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 MANAGEMENT & PARTNERS ON SUSTAINABILITY Most of LIFE’s management and partners concurred that there was a limited view on sustainability at this point in the project—especially since some of the activities in question were merely months old. When asked to speculate on what they thought would be sustained, they tended to speak in terms of three particular interventions that they felt would have sustained effects at partner agency level. These are: 1. Revitalization Hubs. As described earlier, in FY2019, LIFE began providing grants to groups of NGOs and community-based organizations (CBOs) that shared joint social missions and regional foci. Management saw strong sustainability potential in these Hubs in terms of strengthened professional networks, enhanced institutional capacities, and expanded client base. They described how the relationships between the regional NGOs had evolved: “Before this hub, the relationship between NGOs was a competitive one. Now, it is more collaboration and support," explained one partner manager. Most saw these new relationships as sustainable because they required little funding to maintain themselves over time—only the commitment of the partners involved. 2. Capacity building to staff. The training/capacity building provided to partner staff was cited as having a significant and sustainable impact. Often these discussion centered on the fact that the training topics were designed based on the staff needs assessed by the partners and therefore highly relevant: "The acquired knowledge from the training is gained by the staff and can be used on the personal level, each trainee took what they want and felt that each topic could be beneficial.” Most partners saw the training a permanent investment in their staff: “We have a very high retention rate among our employees, and we invest heavily in our employees, so we believe the training aspect of LIFE support is the most important to us and most sustainable.” 3. Fintech Improvements. Both management and partners point to the Fintech grants as having broad, sustained effects among the partner organizations. The new Tahaki software, digitized loan applications, and digital payment systems were all cited prominently in terms of optimizing costs and enhancing the productivity of staff. Explained one partner manager, "The Fintech grant is the most sustainable for us since in the world nowadays you need to automate your operations and improve efficiency to serve your clients better otherwise you will be left behind.” In addition, at the beneficiary level, LIFE’s management and partners saw the BDS training sessions as permanent improvements to the lives of their beneficiaries—more so than the in-kind grants. "In￾kind grants are good, but the knowledge is more sustainable and will affect the income at the end," one LIFE manager opined. BENEFICIARIES ON SUSTAINABILITY The beneficiaries tended not to speak in terms of the sustained effects of particular programming areas but rather in terms of the sustained effects in their lives of the complete intervention package to which they were exposed. Many beneficiaries commented that the knowledge gained through the training sessions was used in their daily life, whether at the household level or business level. As one beneficiary explained, “I now believe everyone should own a daily, monthly, and yearly notebook to enlist all the crucial details concerning the high and low seasons.” Others pointed to the noticeable economic impacts on business that translated into sustained household improvements. "I was able to afford my son’s college tuition, as well as my other children’s school tuition,” said one enterprise owner. Still other beneficiaries 18 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 focused on sustained personal improvements including enhanced self-confidence, independence, and active role in the community. A substantial number of beneficiaries—both men and women—felt that some of the project’s most sustained impacts would be among the female beneficiaries. They spoke generally of how women were more empowered than before and were equipped with the necessary knowledge and skills to run a business. These views were borne out in a wide variety of inspirational anecdotes from female beneficiaries. “My husband is dead,” related one female participant. “I don't receive any income, and I used to sew and fix clothes using traditional machines until this association knew about me and provided me with plenty of advanced equipment and supplements. As a result of that, I feel so independent and brave now.” In these same conversations about sustainability, however, positive views were commonly balanced by a more sobering view of the economic landscape since the events of October 17. There was fairly strong consensus that improvements had been set back, at least temporarily, because enterprises were seriously impacted or failing. "The situation is terrible now, my business has almost nearly stopped, and my son's salary has been cut in half. People used to come to visit me for an aesthetic session every week, and now I am barely getting clients in a month,” said one beneficiary. Compounding the situation were road blockages, bank restriction on cash withdrawals, high inflation, and fluctuation of the Lebanese Pound exchange rate. VIEWS & SUSTAINABILITY OF THE LMFA Most LIFE’s partners viewed the LMFA as an entity to facilitate coordination and communication among MFIs and to give credibility to the sector. They saw the organization as playing a role in advocacy for MFIs and a good platform for regular meetings and sharing thoughts about the sector. Said one partner Figure 2: Mixed-gender beneficiary focus group in the North. Photo Courtesy of InfoPro 19 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 manager, “[The LMFA] provides a common voice towards regulators and common strategy for the industry as a whole.” Others saw the organization as particularly crucial as the microfinance industry in the current economic landscape, “It is one association that can carry the interests and challenges of all the MFIs especially under this crisis.” At the same time, however, views on the LMFA’s efficacy were mixed, with partners offering more negative than positive comments overall. For example, one MFI manager said, “We paid our annual membership, but we don’t get any support or benefit as counterpart.” Many saw the inefficacy as rooted in the lack unity among the LMFA members, which include both for-profit MFIs and social mission-driven non-profit NGOs, as well as microlenders of all types and scales. “Each MFI has different vision and mission - as what is beneficial for some MFIs might not be for others and vice versa,” one partner manager explained. With regard to the LMFA’s future, most partner members were well-aware that the LIFE project has supplied most of the organization’s operating funds, and that this support is unlikely to last. Most felt there were two options for the organization to survive. First, the organization could seek new external support; views on the feasibility of this option were mixed. Second, the organization could downsize and reduce costs substantially—for example, reducing its staff, or letting go of its main office and using MFIs’ facilities to do meetings. A few individuals thought increasing member dues might be a solution, but others felt this idea was untenable because members would not pay more. To elaborate on the vision of the LMFA’s future, we asked partners what they felt the organization could do to serve its members better. Here, the responses were diverse and widely-varied, with no real consensus, beyond possibly an enhanced role in lobbying/advocacy for the industry. Suggestions included, in no particular order: • Creating a credit bureau to share data between MFIs about blacklisted clients. • Bringing new donors on board for the MFI industry. • Providing financial grants to support MFIs. • Providing MFIs with tools to cover forex fluctuations in the presence of a parallel market. • Providing MFIs with solutions on how to pay back foreign investors since banks are putting restrictions on transfers due to capital control. • Conducting needed research on the MFI sector. • Facilitating the strategic planning of the microfinance sector. • Facilitating MFI entrance into regulated services such as micro-insurance and savings. • Promoting transparency between MFIs so they can share data that could be beneficial to the entire industry. EVALUATION QUESTION 7 (GENDER) To what extent were the recommendations generated by the Gender Analysis integrated in LIFE’s implementation? What are the outcomes generated as a result of gender mainstreaming? LIFE’S TARGETING OF WOMEN LIFE’s management described a deep and central commitment to serving women at multiple levels in the project, from the percentage of female partner staff trained to the percentage of female beneficiaries. 20 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 They noted that the project has exceeded the outreach target for women (35 percent) in both areas. (According to Q4 FY2019 MEL data available, the percentage of female beneficiaries and female staff trained stands at 47 percent each.) As further evidence of its commitment, management also pointed to a number of women-focused initiatives it had undertaken, such as the successful Women Rising social media campaign. Most of the partners strongly agreed that the LIFE project clearly conveyed a message on the need to support and empower women, especially in rural areas. It was a mandate from the beginning of their involvement with the project, though most agreed that the message did not come with specific percentage or quota of women they needed to reach. A majority of partners added that this message resonated well with their own existing efforts to reach women; most of the MFIs and NGOs had pre￾existing targets for female participation, between 30 and 75 percent. Most beneficiaries agreed that LIFE’s programs did not target women specifically but targeted all vulnerable populations including women. A majority of respondents suggested that women outnumbered men in the training sessions (though, again, the MEL data indicates that men slightly outnumbered women overall). A number of beneficiaries noted that training included sessions specific to female-associated vocations such as sewing, baking, hairdressing, knitting, and interior design. DIFFERENTIAL IMPACT MEN VS. WOMEN When asked whether men and women were impacted equally by the project, a slight majority of partner respondents felt that women were impacted more (with nearly all the rest suggesting that men and women were impacted equally). As one partner manager explained, “Men are originally more business oriented or having their existing businesses, however, women are used to be housewives. So, by providing them the necessary equipment to start or to improve their businesses, it was obviously seen that they were more impacted than men.” Another explained, “The impact on women was bigger because women became more independent and self-confident.” A number of partner respondents added that women were more engaged, committed, motivated and passionate than man, and gained more confidence in the business world. Said one partner staff, “In our experience, the biggest impact that we’ve seen is the perception of doing business in our areas; business now is for man and women.” Among beneficiaries, the vast majority agreed that men and women were impacted equally. However, a significant minority felt that women saw greater benefit than men. Nearly everyone agreed that women were profoundly impacted, even if it the effects were equal among men, and the testimony is filled with inspirational stories from female beneficiaries. Said one female participant, “We’ve lived with the understanding that women should get married at an early age and spend all their lives by the sink serving their families. We’re not familiar with being educated and professional. However, I’m now working and proving myself in different regions in the South.” A number of beneficiaries suggested that impact on women was enhanced because they were more committed. They saw men as more distracted and dismissive of the content than women: “[We] dislike the presence of male participation in the same training because they would keep smoking and filling up coffee. They weren’t paying attention as much as we were.” Said another, “Trainings were definitely targeting women 21 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 because all men during the sessions were laughing and making fun; that’s why women benefited more because they were more excited to learn and more dedicated to the project.” INTEGRATION OF GENDER ANALYSIS INTO PROJECT Although gender mainstreaming appears to be the central focus of LIFE at all levels, it is clear that the project’s Gender Analysis has not been the guiding principle behind these initiatives. Most of the management conceded that they had not read the analysis and/or were unaware of it. Indeed, while some of the recommendation of the Gender Analysis are reflected in the project’s gender mainstreaming, many others are overlooked. We note the below as just a few examples of these recommendations: Component 1 • Support LMFA to include a dedicated gender section in the LMFA web portal. • Support identification and training of male champions within managerial and decision-making roles within the government and financial institutions to champion efforts, policy decisions, and activities that support inclusion of women, youth, and disabled. Component 2 • Support development of loan products targeting women are linked with complementary social services and consider group lending as well as cooperatives • Encourage MFIs to provide financial incentives for the recruitment of female loan officers • Train MFIs in data management collection of sex disaggregated data on guarantor profile for male and females, industry for loan use for male and female as well as performance and non￾performance rates per sex. Component 3 • Identify new market niches related to IT and mobile technologies in profitable sectors where women can participate (rather than the traditional sectors). • Hold grant conferences to explain gender-specific clauses in contracts to bidders. Ensure a gender equality empowered grant process in evaluation, sourcing and contract management Photo Courtesy of the LIFE Project Figure 3: Female store owner and LIFE beneficiary. 22 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 It must be noted that the Gender Analysis contains many recommendations, and it is the position of the evaluation team that LIFE has adhered to the spirit if not the letter of these recommendations in its work around gender mainstreaming. It is up to USAID/Lebanon to determine the implications of the project’s divergence from the Gender Analysis (and hence divergence from the letter of the ADS) in its work. IMPACT ON OTHER VULNERABLE POPULATIONS: YOUTH/DISABLED Discussions of outreach to female populations often segued into discussions of outreach to other vulnerable populations—particularly youth and PWD. Even though there were no specific targets attached, the LIFE project included a specific mandate to reach both, and there was strong consensus among all levels of stakeholders that the project has done that. According the most recent MIS available, 26 percent of project beneficiaries were under the age of 30. This squares with multiple comments among management and partner staff regarding the project’s dedicated efforts to reach youth through various partners. As one partner manager explained, “LIFE mission aligns with [our] objectives because we target that same group of people especially women and youth. [We] also focus on women and youth, similar to LIFE project.” At least one partner staff offered the view that the project’s biggest impacts have been among its youth clientele. Around half of the beneficiary focus groups included explicit comments about the project’s inclusion of and impact on youth participants. Said one beneficiary, “The majority of the youth were unemployed and because of the situation none of them were working so these trainings helped them learn how to be more productive and how to manage their businesses in the future.” Another added, “I had my young daughters with me; I felt that they benefitted from the trainers and were constantly asking questions.” The project’s precise outreach to PWD is more difficult to track, because it is not recorded systematically in the MEL systems. But it is the clear that the commitment has been there. One of the project partners grantees is the Lebanese Union for People with Physical Disabilities (LUPD). The purpose of the grant was specifically to help PWD in rural areas of Beqaa and South to improve their quality of life by supporting them to develop businesses or find employment; it has reached 182 PWD beneficiaries, according to the most recent available MEL data. As a LUPD manager commented, “[LIFE’s Chief of Party] has a long experience and a vision regarding disabled people and their rights. That’s why he contributed a lot to make sure that the implementation is effective and by designing the project accordingly.” The inclusion of PWD was noted and reflected on among beneficiaries as well, both those associated with LUPD and others. Said one LUPD-affiliated beneficiary, “We are neglected by the government, so because we have this fighting spirit and we want to persevere and work, this project was a very nice opportunity … because they highlighted our cases and gave us a push.” Another beneficiary from a non-LUPD FGD added that she “came across two disabled women attending the trainings and learning from them which was beautiful to see and witness.” 23 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 SUMMARY: FINDINGS & CONCLUSIONS Question 1 Findings: • LIFE began slowly, followed by a remarkable rebound in FY2018. • The project made slow but significant progress with the commercial banks in FY2019. • The October 17 revolution and fiscal crisis have upended major areas of its work, setting some of it back to square one. • The project is actively exploring a response to the crisis but has yet to settle on a coherent new direction. • Its short-term responses to the crisis appear to be effective this far. Conclusions: • The project has encompassed an unusually diverse range of activities, as its priorities have shifted repeatedly over its history. • The economic crisis continues to demand a strategic response. Questions 2 & 3 Findings: • Key targets were missed in FY2017, followed by a rebound in FY0218, followed by another dip in FY2019. • The main reasons for the most recent dip are delays in outputs linked to commercial bank activities and the unrealistic escalation of targets from FY2018. • Management said relationship-building with the banks, capacity-building with the LMFA, and livelihoods work with end-users were its major accomplishments. • Management saw the clearest evidence of impact in the Fintech upgrades among the partners and the changed attitudes (i.e. new interest in financial inclusion) among the banks. • Partners pointed to livelihoods gains as the biggest success among their beneficiaries; among partner organizations themselves, Fintech upgrades and membership in regional “Hubs” were seen as the most important advances. • Partners saw the biggest impacts in terms of livelihoods gains among the beneficiaries, with the Fintech upgrades and the advances in the regional hubs as most impactful to their own institutions. • Management and partners saw the disruptive effects of the current economic crisis as the project’s biggest challenge. • Beneficiaries saw deep and lasting impacts as results of the training and grants packages they received. • There were two significant sources of discontent among beneficiaries: 1) some trainees were upset that they did not receive a grant as well; and 2) some beneficiaries did not appreciate the requirement to take a loan along with their grant. • Compared to other donor-funded projects, partners appreciated LIFE’s local staff/local grounding, its relatively intense support and follow-up, and its commitment to customize interventions to partner and beneficiary needs. • There was mild consensus that the simple combination of grants plus BDS training was the most effective one. 24 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Conclusions: • The project has demonstrated a remarkable ability to connect with, inspire, and remain responsive to stakeholders at all levels. • LIFE’s most noteworthy and enduring achievements have been in the realm of livelihoods work (grant/training packages) and capacity-building for partners. • Other promising work, especially with the banks, has been undermined almost completely by the financial crisis. Question 4 Findings: • The LIFE project did not begin with a vision for M&E as informing design and implementation. • Its primary purpose has been the retrospective measurement of achievement only. • More recently, LIFE’s management has attempted to institute a broader and more prospective use of M&E data; this includes the direct involvement of M&E staff in the grant selection process. • Partners (i.e. the sources of most M&E) have no shared understanding of the design/implementation function of M&E. Conclusions: • LIFE’s M&E systems and their use remain relatively basic, despite a growing awareness of their value in implementation/adaptive management and some technological upgrades. • LIFE’s M&E system is beset with unrealistic targets and does not capture the value of LIFE’s work at the partner and community levels. Questions 5 & 6 Findings: • LIFE’s management and partners felt Fintech upgrades, partner capacity-building, and regional Hubs would have the most sustained effects. • Beneficiaries saw sustained effects of the training they received at both the business and household levels. • Male and female beneficiaries saw a potential for particularly sustained effects among the project’s female beneficiaries. • Many beneficiaries felt any window to sustainability was obscured by the current crisis. • Among members, there are mixed views on the value of the LMFA. • Advocacy for the MFI sector among donors and BdL is seen as its most important function. • In light of USAID’s planned cuts to support, the future of the LMFA is highly uncertain. Conclusions: • The effects of training/capacity-building are likely to be sustained at all levels. • All questions of sustainability will be heavily conditioned by the financial crisis. • Major downsizing and cost-cutting is the LMFA’s most feasibility path to sustainability. Question 7 Findings: • The LIFE project has done exemplary work in its outreach to women and gender mainstreaming. • The project is likely having a deeper overall impact on its female beneficiaries. 25 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 • The project has largely ignored the Gender Analysis in charting its course on gender mainstreaming, and implications of that are undermined. • LIFE’s outreach efforts have also extended effectively among youth and PWD to a large extent. Conclusions: • Regardless of the Gender Analysis, LIFE’s inclusion of women has been consistent and inspirational in every component of its work. • The LIFE project exhibits a broad and highly-effective approach to the inclusion of vulnerable populations, such as youth and PWD. 26 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 RECOMMENDATIONS FUTURE OF THE LIFE PROJECT The evaluation team recommends that USAID pursue all options to allow the LIFE project to continue its work to the fullest extent possible. Our basis for this recommendation, first and foremost, is the project’s highly competent team and exceptional leadership. We saw the quality of the team thanks to a multitude of conversations with stakeholders at all levels. The project, for example, drew high praise for its emphasis on local staffing and intensive face-to-face support—which we knew to be a conscious strategic decision, after speaking with LIFE’s management. It distinguished the project from other donor-funded efforts and yielded key dividends in inspiring stakeholders to success. The range of activities that the project has supported certainly shows enough promise to merit USAID’s continuing support. At the partner and beneficiary levels, it is clear that the bulk of these activities are both wanted and needed, especially in the current economic climate. At the same time, we advise USAID to continue its support with key conditions. We propose that the LIFE project has gone in too many directions and spread itself too thin. The shifts in direction and expanding range of tangential activities have arisen from a variety of sources/causes, from the project’s own leadership to shifting priorities at USAID/Lebanon, to the October 17 revolution. While all activities have had their own merit, they do not add up to a coherent whole. To invoke one example: the Nudge research was well-designed and fascinating, but unfortunately it was cut short by forces beyond the control of the project. However, even if the research had reached its conclusion, it is unclear how it would have contributed to anything else LIFE is doing, since none of the other interventions are based on behavioral science. The evaluation team feels that the diffuse approach has been enabled by a lack of a stable and coherent TOC since the start. For the remaining period of performance, we highly recommend that the LIFE team and USAID/Lebanon work together to create a prospective TOC that will support a narrower range of activities serving well-defined, inter-connected ends—mindful of current crisis and what can reasonable be accomplished in the project’s remaining life (and remaining budget). There is no reason to attempt to retrofit a TOC and what has already taken place; rather, it should guide activities going forward. Our general inclination on a prospective TOC would be to a tight package of livelihoods interventions and capacity-building among the MFIs/NGOs, with parallel de-emphasis on the banks. The reasons for this are numerous and varied, ranging from clear evidence of what has worked so far, to near-certainty that further bureaucratic delays will limit any deep achievements with the banks before the project ends. It is unclear whether USAID/Lebanon is ready to back such a plan, but whatever is decided, our recommendation is they create the TOC around the new plan and keep the focus tight—i.e. no more voluntary shift, pivots, or tangents in the time that remains. 27 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 OTHER RECOMMENDATIONS GENERAL PROGRAMMING NOTE: these are directed at management of the LIFE, though some will require substantial collaboration with USAID, as indicated • Consider retooling grant programs so that all those trained get grants. The complaints from those who did not receive grants were more than “sour grapes.” The current format, in which beneficiaries who receive grants sit side-by-side with those who do not, is a recipe for alienation. The level of anger witnessed among the non-recipients suggests that many of them may be lost forever to the NGOs/MFIs involved, which is a high price. • Extend coaching/mentoring elements of training programs. These elements were found in only a minority of training programs but were highly popular whenever we encountered them. • Anticipate and plan for bureaucracy/delays in any future work with banks, in case the project continues substantive activities with them. The unanticipated delays were a major hindrance with meeting targets in FY2019. Given that current plans have been largely abandoned, the project should view itself at square one once again with the banks—meaning any new plans will require new rounds of lengthy approvals and bureaucracy. • Revisit the Gender Analysis; review and incorporate its specific recommendations (e.g. gender awareness sessions in all training). This is not to suggest that the LIFE project has neglected the gender aspect of the project. However, by its own admission, it has not used the Gender Analysis as its guide, which, according to the ADS, it is required to do. It is not reasonable that the project would implement all of the Gender Analysis’s many recommendations, but it should consider doing more and make its intentions clear to USAID/L, lest the project be accused of non-adherence to the ADS in the future. CRISIS-RELATED • At least short-term, consider coupling in-kind grants with working capital (purchasing goods and paying salaries) to help mediate the effects of bank cash withdrawal restrictions. • Offer more crisis-management training. The crisis-management training has been well-received in all instances this far and is likely to become a vital part of the LIFE’s ongoing efforts to confront the effects of the economic crisis. • Create a revised plan for the LMFA with substantially decreased operational costs. This is likely the organization’s only path to survival in the context of the crisis, or perhaps anytime. The mixed views among members suggests that many will not agree with elevated member fees, and sustained external support seems unlikely, especially when the current crisis presents so many competing priorities. MEL-RELATED • Institute protocols for post-post-training follow-up. Although LIFE was widely praised for its post-training follow-up, one area where it is lacking is in post-post-training follow-up (e.g. six months after training is concluded). These observations are key to determine what knowledge has been retained longer-term and how it is being used. It could help inform vital tune-up training and will help flesh out the story of impact from the LIFE project itself. These processes can be initiated now by the project’s M&E team and perhaps carried on by an external evaluator (with a minimal investment) at the project’s close. 28 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 • In general, seek newer and better ways to capture LIFE’s story and its relationships with stakeholders. This goes beyond routine MEL data collection and beyond the traditional beneficiary-focused success stories that the project collects. Much of LIFE’s success is reflected elsewhere—for example, in its ability to discern the needs of partner organizations and customize technical assistance programs, thereby inspiring success at the beneficiary level. Again, these processes can be initiated now by the project’s M&E team and perhaps carried on by an external evaluator (with a minimal investment) at the project’s close. 29 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ANNEXES ANNEX I: LIFE GRANTS Grantee Start Date End Date Status Period of Performance Activities # of Beneficiaries Lebanese Micro Finance Association (LMFA) 01- Jun￾17 31- Jan￾19 Closed out Y1-Y3 Strengthening of the institution covering overhead expenses and conference participation 2 staff and 9 members of LMFA; 43 trainees on Introduction to microfinance by LMFA Lebanese Development Cooperative (LDC) 01- Jun￾17 30- Nov￾19 Closed out Y1-Y3 Access to Micro credit In-kind Grant Support loans and in-kind grants Association d'Entraide Professionnelle (AEP) 01- Sep￾17 31- Aug￾18 Closed out Y1-Y2 Access to Micro credit In-kind Grant Support 163 beneficiaries received loans and in￾kind grants Al Hadatha Association 18- Dec￾17 28- Feb￾19 Closed out Y1-Y3 Access to BDS, Micro-Credit In-kind Grant Support 208 received basic business skills trainings and part received VT; 42 received in-kind grants and microloans SHEILD Social, Humanitarian, Economical Intervention for Local Development 21- May￾18 17- May￾19 Closed out Y2-Y3 Access to BDS In-kind Grant Support 101 received basic business skills trainings 95 received in-kind grants Entrepreneurial Development Fund (EDF) 02- Jul-18 01- Jul-19 Closed out Y2-Y3 Access to BDS In-kind Grant Support 358 received basic business skills training 115 received in-kind grants; 1 beneficiary was able to access a $30,000 loan SHIFT- Social Innovation Hub 04- Jul-18 03- Jun￾19 Closed out Y2-Y3 Access to BDS In-kind Grant Support 239 beneficiaries received at least 1 out of 8 sessions of business development services, 181 of them received at least 4 out of 8 sessions and 94 received in-kind grants. 30 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Lebanese Union For People With Physical Disabilities (LUPD) 16- Jul-18 15- Jul-19 Closed out Y2-Y3 Access to BDS In-kind Grant Support 182 people with disabilities received guidance sessions and/or basic business development trainings; 23 received in-kind grant support related to a business; 1 was able to access micro-credit; 20 of them benefited from internships and 3 were able to find jobs (2 full time jobs and 1 part time). The Lebanese Association for Development-Al Majmoua 15- Aug￾18 31- Oct￾19 Active Y2-Y4 Access to BDS, Micro-Credit In-kind Grant Support 132 micro and small businesses Nudge Lebanon 01- Feb￾19 31- Jan￾20 Active Y3-Y4 Strengthening Women’s Financial Inclusion Through Behaviorally Informed Interventions and Policymaking reporting on data will start as of Q4Y3 Lebanese Micro Finance Association (LMFA) 21- Feb￾19 21- Apr￾20 Active Y3-Y4 Strengthening of the institution covering overhead expenses and conference participation 2 staff and 9 members of LMFA Emkan Finance S.A.L 25- Feb￾19 24- Apr￾20 Active Y3-Y4 Financial Inclusion of Emkan Finance Clients with BankMed (Pathway to Saving) reporting on data will start as of Q4Y3 The Lebanese Association for Development-Al Majmoua 08- Apr￾19 08- Jun￾20 Active Y3-Y4 Developing an online Digital Learning Management System Development (E￾Learning) reporting on data will start in Y4 Professional Mutual Aid Association (AEP) 15- Apr￾19 14- Oct￾19 Completed Y3-Y4 Introducing a new Management Information System (MIS) to support AEP digitalization process 15 Direct beneficiaries (AEP staff) benefited from the MIS system 31 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 IBDAA Microfinance SAL 18- Apr￾19 17- Apr￾20 Active Y3-Y4 IBDAA Financial Inclusion through Digital Financial Services (IBDAA Fi DFS) reporting on data will start in Y4 Lebanese Development Cooperative (LDC) 24- Apr￾19 23- Apr￾20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support 25 beneficiaries (businesses) to date received 2 days training, in-kind grants and 8 accessed loans Al Hadatha Association - Reduced Economic Vulnerablity 2 23- May￾19 22- May￾20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start as of Q4Y3 VITAS SAL 08- Jul-19 07- Jul-20 Active Y3-Y4 Customer Centric Digitization software reporting on data will start in Y4 Rene Mouawad Foundation (RMF) 15- Jul-19 14- Jul-20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start as of Q4Y3 DAR Al Zahraa￾North Hub 15- Jul-19 14- Jul-20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start as of Q4Y3 Akkarouna Association - North Hub 15- Jul-19 14- Jul-20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start as of Q4Y3 BLOM Bank 07- Aug￾19 06- May￾20 Active Y3-Y4 Increase access to savings for unbanked populations through promoting the Start Saving Account and visiting rural areas in the BLOM truck reporting on data will start in Y4 SHEILD Social, Humanitarian, Economical Intervention for Local Development - follow on grant 14- Aug￾19 13- Aug￾20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start as of Q4Y3 32 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Banque Libano Francaise SAL (BLF) 12- Sep￾19 11- Mar￾21 Active Y3-Y5 Increase access to savings for unbanked populations through creating a special saving account, mobile ATMs in rural areas, etc. reporting on data will start in Y4 Development for People and Nature Association (DPNA) 24- Sep￾19 23- Sep￾20 Active Y3-Y4 Access to BDS, Micro-Credit In-kind Grant Support reporting on data will start in Y4 33 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ANNEX 2: DETAILS ON EVALUATION DESIGN, METHODS, SAMPLE, LIMITATIONS DESK REVIEW The evaluation team conducted a comprehensive literature review of relevant internal and external documents. These included: ● USAID/Lebanon’s CDCS. ● USAID/Lebanon’s Performance Management Plan (PMP) PSD. ● LIFE’s request for proposal (RFP). ● LIFE’s contract agreement. ● LIFE’s contract modifications. ● Sector assessments carried out by LIFE ● LIFE’s annual workplans. ● LIFE’s periodic reports (annual, quarterly, other). ● LIFE’s MEL plan. ● LIFE’s Gender Analysis. ● LIM’s Mid-term Evaluation. ● BdL publications related to financial inclusion. ● USAID/Lebanon’s Economic Growth Assessment. ● Other project documents relevant to the evaluation questions. ● External studies of Lebanon’s financial sector KEY INFORMANT INTERVIEWS The evaluation team conducted 36 KIIs with USAID/Lebanon’s key staff members, Palladium group’s LIFE team, LIFE’s partners (MFIs, NGOs, and Banks), business managers, and other stakeholders (LMFA, BdL) as shown in Table 1 below. Table 1: Distribution of KIIs—Clients and Staff TYPE OF STAKEHOLDER TITLE NO. KEY INTERVIEWS BREAKDOWN BY GENDER BREAKDOWN BY REGION USAID AOR 1 Interview Female: 1 Beirut: 1 USAID EG Director 1 interview Male: 1 Beirut: 1 LIFE Staff All senior management 7 Interviews Male: 7 Female: 2 Beirut: 7 LIFE Partners MFIs Ibdaa Idaa Emkan Majmoua ADR LDC EDF AEP 10 Interviews Male: 8 Female: 8 Beirut: 6 Tyr: 1 Mount Lebanon: 3 34 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 TYPE OF STAKEHOLDER TITLE NO. KEY INTERVIEWS BREAKDOWN BY GENDER BREAKDOWN BY REGION VITAS Makhzoumi LIFE Partners NGOs Nudge Akkarouna RMF DAR SHIFT Al Hadatha DPNA LUDP Shield ISWA LITD 11 Interviews Male: 14 Female: 9 Beirut: 2 Tripoli: 4 Akkar: 1 Saida: 3 Tyr: 1 LIFE Partners Banks BLOM Bank BLF Saradar Bank 3 Interviews Male: 3 Female:4 Beirut: 3 Other Stakeholders BDL (2) LMFA 3 interviews Male: 2 Female: 1 Beirut: 3 Total 36 Interviews The KIIs were our primary method and most significant source of data. We favored this method for two primary reasons: (1) its individualistic nature; and (2) its open-ended/inductive character. The individualistic nature of the KIIs with management (USAID/Lebanon, LIFE team, Partners etc.) was important simply because all such managers have unique roles on the project, and their experiences needed to be captured to understand the project’s functionality. The open-ended and inductive nature of KIIs was also highly relevant because we did not enter this evaluation with established theories or hypotheses about what worked and why. Rather, we sought data that reveal the story of the project and search for patterns from observation. It is the view of Social Impact and many experts in the field1 that these kinds of inductive approaches are a well-advised best practice in most instances of retrospective process evaluations or performance evaluations. The sampling and purpose of the KIIs in each respondent category are as follows: 1 For example, G. Guest, E. Namey, & M. Mitchell, (2013). Collecting Qualitative Data: A Field Manual for Applied Research. Los Angeles, CA: Sage Publications. 35 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 USAID/Lebanon, and LIFE team: The evaluation team conducted nine KIIs with a purposive selection of the principal and most relevant managers at USAID/Lebanon and LIFE. The KIIs provided in￾depth understanding of the project’s achievements, the factors that contributed toward achieving the project’s results, and the implementation’s successes and challenges. LIFE Partners: LIFE has provided 25 grants to MFIs, NGOs, and banks to offer capacity building and in-kind contributions (livelihood grants) and technology (fintech grants) to those organizations and their beneficiaries. The evaluation team conducted KIIs with the key personnel in all of these organizations— i.e. a census approach rather than a sampled approach, given the manageable number of organizations involved. In addition, we conducted KIIs with staff responsible for financial-inclusion initiatives at the Central Bank and LMFA. The KIIs provided a direct view of the factors that contributed towards achieving the project’s results, the project’s outputs and activities, and implementation successes and challenges at the organization level. We developed a selection of interview guides (Annex 5) to reflect the variety of interview subjects. Consistent with best practices in semi structured qualitative interviewing and inductive research generally, the interviews covered all major topics in the guide, but the interviewers were free to react and ask additional questions/probes based on the responses received. Prior to each KII, verbal consent was obtained from all those interviewed (see Annex 5 for consent scripts, attached to each interview guide). All interviews were recorded, and the recordings were used to complete the interview notes. They were also available for further reference for the team to complete their notes. GROUP INTERVIEWS LIFE provided capacity-building and training sessions to the staff of MFIs through two primary channels: (1) recruiting trainers to conduct training sessions; and 2) issuing grants to the LMFA to provide training to MFI staff. To date, this intervention has targeted nine MFIs. Capacity-building sessions also targeted bank employees through the grant issued to the Lebanese Nudge association. Under this grant, Nudge delivered training to employees of three different banks. The evaluation team conducted 11 group interviews with MFIs and bank employees who benefited from this capacity building to assess the effectiveness, efficiency, sustainability, and challenges of this intervention. We opted for the group interview format here for the sake of efficiency and the fact that individual experiences will be less relevant than organization-wide experiences. Group interviews, described in table 2 below, were selected as follows: • MFIs: We conducted seven group interviews with groups of MFI staff at each MFI’s premises. We selected the MFIs based mostly on practical considerations—we chose all those that have a significant number of employee beneficiaries (five and above) to make it more feasible to gather 4-5 people in the same location for the group meeting. • Banks: We conducted group meetings with the employees of three banks with active grants. To facilitate bank approval process, we allowed bank management to select the beneficiaries who attended the group meetings, with instructions to select the employees who bring the most relevant knowledge among the various branches involved. 36 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 • Trainers: The evaluation team also conducted one group interview with the trainers who delivered the training. Table 2: Group interviews with MFI and bank employees. TYPE OF STAKEHOLDER STAKEHOLDER NO. GROUP INTERVIEWS BREAKDOWN BY GENDER BREAKDOWN BY REGION MFIs Emkan Employees Majmoua Employees AEP Employees LDC Employees Makhzoumi Employees IDBAA Employees VITAS Employees 11 group interviews Male:23 Female: 36 Beirut: 8 Mount Lebanon: 3 BANKS Bank Audi Employees BLC Bank Employees BLOM Bank Employees Training Service Providers Training Service Providers FOCUS GROUP DISCUSSIONS The evaluation’s primary view on end-user beneficiaries and any emergent impacts at that level came through a series of 25 FGDs facilitated by InfoPro. The FGD format is chosen for its capacity to identify consensus and/or key points of debate among a fairly wide client base. Our main purpose in selecting FGD participants was to have a sample in which the beneficiaries who received a mix of services (such as in-kind grant, training, or both) have an equal probability of being chosen. For each pool of beneficiaries under each grant for the assisted organizations, we randomized the beneficiaries and chose the first 20 names from the list, if the main pool had 100 beneficiaries or less. If the main pool had 200 beneficiaries or less, we chose the first 40 names, in order to have two FGDs, and so on. In some rural areas, where cultural norms make it difficult to have mixed-gender groups, we spread the FGDs between females and males after the randomization. We adjusted the samples in a few cases to avoid an unreasonable geographic spread and excessive logistical burden in bringing groups together. The below table provides the specifics of the FGDs in term of stakeholder, geographic distribution, number, and gender of participants. 37 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Table 3: FGDs with end-user beneficiaries. GEOGRAPHIC DISTRIBUTION STAKEHOLDER FGD DESCRIPTION NO ATTENDEES B BREAKDOWN BY GENDER Tripoli LDC Mix FGD 6 Female: 1 Male: 5 RMF Mix FGD 7 Female: 7 SHIFT Female FGD Female FGD 8 10 Female: 18 Akkarouna Mix FGD Female FGD 4 4 Female: 7 Male: 1 DAR Mix FGD 10 Female: 5 Male: 5 AEP Male FGD 4 Male: 4 Akkar LDC Male FGD 4 Male: 4 Al Hadatha Male FGD-Coop Male FGD-Municipality Male FGD-other Female FGD 9 11 10 7 Male 30 Female: 7 Bekaa LUDP Female FGD Male FGD 9 9 Female: 9 Male: 9 EDF Female FGD Male FGD Mix FGD 8 7 8 Female: 14 Male: 9 Baalback Al Majmoua Mix FGD 5 Female: 3 Male: 2 Emkan Mix FGD 4 Female: 3 Male: 1 South Al Majmoua Mix FGD 9 Female: 7 Male: 2 EDF Female FGD Mix FGD 5 4 Female: 5 Male: 4 Shield Female FGD Female FGD 9 8 Female: 17 Total 25 FGD Attendees: 179 Female: 103 Male: 76 38 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 MONITORING DATA ANALYSIS The evaluation team received a full download of the MEL database employed by the LIFE project. We then conducted an independent analysis of this database to achieve a current measurement of the project’s major performances indicators. We compared these measurements against targets and against data previously submitted to USAID as part of monitoring requirements, for a clear indication of the project’s overall and recent progress. We then analyzed the changes in indicators and targets against consecutive project modifications. The team conducted the inquiry under the assumption that up-to-date or nearly up-to-date monitoring data are available for all participating firms, thereby providing quantitative outcome data for the project on a census basis. ETHICAL CONSIDERATIONS All members of the evaluation team received research ethics training from the team leader and were sensitized to the importance of guaranteeing and maintaining the privacy, confidentiality, and protection of the study participants. This study poses minimal risk to participants. The only significant risk was the possibility of a breach in confidentiality of the data collected or the possibility that a participant (particularly end beneficiaries) may feel uncomfortable answering questions pertaining to their personal business or finances. All participants were informed about the nature of the questions that they will be asked prior to enrollment and prior to each interview. They were reminded that they may refuse to answer any question to which they feel uncomfortable responding. To mitigate the risk of any breach of confidentiality, strategies were put in place to ensure that all participant identifiers were kept confidential and separate from individual data. No name, telephone number, address, or date of birth were collected during the FGDs, in the transcripts, or analyzed data. Those materials were linked to individual identifiers only by way of a PIN, with the key kept in a password-protected file. Moreover, interviews were conducted in a private setting designated by the participant, and the information they provided is aggregated in most reporting—thus reducing the possibility of identifying any one person or firm. Any identifying information inadvertently mentioned during the interviews or in the behavioral questionnaire were cleaned from the text before analysis. In cases where longer anecdotes or “stories” about individual partners or beneficiaries are used in this report, identifying information was altered to preserve confidentiality. All hard copies of study-related materials were stored in a locked file cabinet at Social Impact’s Beirut office, with access limited to study staff. All electronic data files are password-protected and maintained on the field team’s project computer(s) and/or on the project’s secure OneDrive. Transcripts are similarly secured at Social Impact’s Beirut office. Audio recordings will be destroyed upon submission of the report, and all paper-based data will be stored in a secure location for the required time specified by Social Impact’s standards of practice (generally six months or less). 39 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 DATA ANALYSIS Data Preparation: The evaluation team transcribed the interviews into summary reports on a daily basis (see Ethical Considerations above for procedures regarding the protection of data). The FGDs were transcribed into detailed and summary reports by the InfoPro team and transmitted to the evaluation team on a rolling basis. Coding: We then coded the transcripts primarily in terms of basic content—meaning responses were categorized and grouped, but generally not interpreted. We employed Dedoose software for coding. Based on the coding results, we worked collaboratively to develop the evaluation’s themes, findings, and conclusions. LIMITATIONS AND BIASES The evaluation team noted several potential biases/limitations in this kind of research: Response Bias: Response bias is the risk that key informants may be motivated to provide responses that would be considered socially desirable or influential in obtaining donor support. For example, an owner of a client firm may provide positive remarks about LIFE because he/she would like to receive similar support in the future or because he/she wants to please the interviewer. The team mitigated these risks by minimizing, to the extent possible, the connection between the evaluation activity and any future opportunities for support. Recall bias: It is a common evaluation problem where beneficiaries may respond to questions posed by the evaluation with answers that blend their experiences into a composite memory. Respondents who may have participated previously in similar activities may not distinguish their experience with LIFE. Additionally, depending on when beneficiaries participated in LIFE activities, their perceptions of events may change over time and their ability to remember specific details may fade. The team mitigated this risk by conducting as many KIIs and FGDs as possible within the available evaluation period to triangulate responses and increase the validity of the findings. Selection/Availability Bias: In KIIs, there is a risk that the team might only reach the most active, responsive, or engaged stakeholders. To mitigate the risk of selection bias, we relied on random selections in the case of beneficiaries and taken only minimal input from the implementers and partners in selection. On the monitoring data analysis, there is a comparable risk that the low-performing firms may not have reported full data or may have reported data inaccurately. We discussed such scenarios with the implementers and did not find any significant concerns. Gender Bias: Gender bias is a risk because most individuals have a subconscious sense of appropriate roles and behavior for women and men. In its training phase, the evaluation team discussed what gender preconceptions might come into play during this evaluation and reviewed how to minimize these preconceptions during data collection and analysis. Limitations Created by Social Unrest: Entering the project, the ongoing social unrest in Lebanon was a major concern in terms of its potential to disrupt the fieldwork schedule. The team mitigated this risk primarily by focusing on the morning hours for fieldwork, when protest activities tended to be minimal. In the end, there were no significant disruptions to the plan. 40 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ANNEX 3: DFC SUPPLEMENT LIFE’s AOR has requested that we add an exploratory inquiry around a possible DFC intervention for the microfinance sector within Lebanon. We articulated the relevant question as follows: What is the stakeholder perspective on the potential value of establishing a DFC fund with local banks to guarantee microfinance loans and promote lending to MSMEs? We summarize the answers received from both LIFE partners and beneficiaries in the below table: Question Majority of Partners Majority of Beneficiaries Will you be interested in new loan funding/newly available loans? Yes • MFIs: resounding yes • Banks: more cautious • BDL: strongly encouraged Split roughly as follows: • 2/3 no • 1/3 yes (No discernible pattern by gender or region) Preferred Currency for New Loans LBP LBP Which industries will benefit the most All, with emphasis on agriculture Not asked Use of loan proceeds Not Asked • Business needs (working capital, asset purchase) • Minority: HH needs Preferred Loan Size Need further internal assessment USD 1 - 15k (EQV in LBP) Will SME importers benefit from such a facility? Yes Not Asked Difficulty in repaying the loan Yes – will require careful pricing Yes - will require generous terms Partners on DFC Based on feedback received from partners, it is some but not uniform support for DFC involvement in loan funding. Any such initiative would require further assessment on the relevant industries, product structure, size, and pricing. 41 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 Among the banks, AUDI’s reply was the only definite no. As for BLF, their stance is unsure given the current market conditions; they would prefer to wait and see what happens with the new government and with the Eurobond repayment that the government have to meet in 2020. BLOM and BLC believe such an initiative is highly needed in the current market condition. Saradar was interested specifically in either LBP with low interest or USD as fresh money. From a regulator perspective, the Central Bank of Lebanon spoke encouragingly about an initiative, and even said they are willing to consider increasing the loan- to-deposit ratio for banks from 25 to 30 percent to encourage such an initiative and stimulate lending among banks. From an MFI perspective, the big four Al Majmouaa, Vitas, Emkan, and Ibdaa stated that they would be interested in such a grant subject to either it being a grant, or a subsidized guaranteed loan with low interest rates, so that the clients would be able to repay back the loan in current economic conditions. They emphasized that the design and structure of the product/loan would be of critical importance. The LMFA summed up its view by noting that there exists a gap of approximately USD 200 million in loans from what the market needs and what is on offer from MFIs. Among the NGO’s, organizations including AEP, Shift, LDC, and ADR were all in favor of such as initiative and would prefer either a grant or a loan with subsidized low interest rates. We received a clear no from NGOs working in Tripoli, as loans are perceived to be against the principles of Islamic Shariaa and their clients are mainly Muslims. For its part Makhzoumi was not keen on the idea of new loans in the current conditions, preferring to wait and see what happens through 2020. Beneficiaries on DFC In terms of beneficiaries’ interest in the DFC Fund, the replies were split, with the “no” outweighing “yes” by a roughly 2:1 ratio. As one respondent phrased it: “In all seriousness, in this situation even if someone offered me 200000 L.L and told me I could repay it whenever I wanted, I wouldn’t take it.” Even those expressed interest in new loans conceded that it would very difficult to repay back the loan in the current economic crisis in the country. The majority of beneficiaries stated that they would use such a loan to expand and/or purchase assets for their businesses, with a minority of respondents indicated that they would use it for household needs In terms of the most suitable loan size, for all respondents it ranged between USD 1k and 15k equivalent in LBP. Nearly all saw difficulties in paying back a loan in USD, given the current dual exchange rate (official/black market). Summing up the feedback from beneficiaries, overall there was interest, but mixed with a strong dose of skepticism and restraint. Our non-probabilistic sample allow for very little disaggregation of these responses by beneficiary type. There were no discernible patterns to the responses by gender by region. 42 | LIFE – MIDTERM PERFORMANCE EVALUATION USAID.GOV MARCH 2020 ANNEX 4: EVALUATION DESIGN MATRIX Evaluation Design Matrix.docx ANNEX 5: INTERVIEW GUIDES KII Protocols - LIFE Evaluation.docx ANNEX 6: INCEPTION REPORT LIFE - Inception Report - FINAL 01.23.2020.pdf Social Impact 2300 Clarendon Boulevard, Suite 1000 Arlington, VA 22201 USA