EVALUATION REPORT Final Performance Evaluation of the Lebanon Community Resilience Initiative This publication was produced for review by the United States Agency for International Development by Social Impact, Inc. Photo credit: Janane Malass, SI Evaluation Team, May 2017 i ii FINAL PERFORMANCE EVALUATION OF THE LEBANON COMMUNITY RESILIENCE INITIATIVE USAID/Office of Transition Initiatives [TO 10, Activity 13; AID-OAA-I-13-00006] August 4, 2017 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Adam Reisman and Nathaniel Rosenblatt on behalf of Social Impact, Inc. Contact : Social Impact, Inc. 2300 Clarendon Blvd., Suite 1000 Arlington, VA 22201 Gabrielle Plotkin gplotkin@socialimpact.com iii DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iv TABLE OF CONTENTS Table of Contents.......................................................................................................................... v Acronyms...................................................................................................................................... vi I. Executive Summary.............................................................................................................vii II. Introduction......................................................................................................................... 10 III. Methodology ........................................................................................................................ 13 IV. Findings ................................................................................................................................ 16 V. Synthesis and Conclusions.................................................................................................. 31 VI. Recommendations and Lessons Learned.......................................................................... 34 Annexes........................................................................................................................................ 40 ANNEXES Annex I: Evaluation Statement of Work Annex II: Data Collection Protocols Annex III: Documents Reviewed Annex IV: Persons Interviewed Annex V: Deep Dive Assessments Annex VI: Disclosure of Any Conflicts Of Interest Annex VII: Annex VII: OTI Statement of Difference: Lebanon Community Resilience Initiative Final Evaluation Annex VIII: MSI’s Statement of Difference to the Final Evaluation of the Lebanon Community Resilience Initiative Annex IX: SI Response to the Letters of Difference Issued by USAID and MSI Regarding LCRI Final Evaluation v ACRONYMS COP Chief of Party CVE Countering Violent Extremism ET Evaluation Team FER Final Evaluation Report FGD Focus Group Discussion IP Implementing Partner KII Key Informant Interview LCRI Lebanon Community Resilience Initiative LCSI Lebanon Civil Society Initiative M&E Monitoring & Evaluation MSI Management Systems International OTI Office of Transition Initiatives PPR Program Performance Review PDO Project Development Officer PVE Preventing Violent Extremism SI Social Impact, Inc. SME Subject Matter Expert SOW Scope of Work TOC Theory of Change USAID United States Agency for International Development USG United States Government WBDB OTI’s Web-based Database vi I. EXECUTIVE SUMMARY INTRODUCTION Lebanon continues to be challenged by an ongoing, complex crisis that threatens to destabilize its fragile security and worsen the country’s political, social, and economic conditions. The ongoing conflict in neighboring Syria has brought roughly two million refugees into Lebanon, which has strained already limited resources and services, exacerbated Lebanese sectarian and host community-refugee tensions, and encouraged the spread of militant and extremist groups seeking to exploit such tensions. PROGRAM DESCRIPTION The Lebanon Community Resilience Initiative (LCRI) is a program implemented on behalf of the United States Agency for International Development’s (USAID) Office of Transition Initiatives (OTI) by Management Systems International (MSI) from August 2014-September 2017. LCRI’s stated goal has been to “strengthen the resiliency of Lebanon's most vulnerable communities in the face of two critical destabilizing factors: sectarian and host community-refugee tensions and susceptibility to violent extremism.” In support of this goal, the program has the following, mutually reinforcing objectives and sub-objectives: 1. Objective 1: Mitigate Rising Sectarian and Host Community-Refugee Tensions Sub-Objectives: 1. Promote peaceful alternatives to violence; 2. Reduce marginalization and isolation of community groups; and 3. Strengthen moderate actors. 2. Objective 2: Counter the Influence of Violent Extremist Groups Sub-Objectives: 1. Counter extremist messaging; 2. Strengthen moderate actors and increase moderate space; and 3. Strengthen youth empowerment and civic participation. PURPOSE OF THE EVALUATION USAID/OTI contracted Social Impact (SI) to conduct an independent final performance evaluation of LCRI. The evaluation assesses the extent to which USAID/OTI’s program achieved its intended results and identifies where the opportunities are greatest for United States Government (USG)actors to continue building on OTI’s program strategy and approach. EVALUATION METHODOLOGY This evaluation employed a mixed-methods design consisting of a document and OTI database review, as well as key informant interview (KIIs), focus group discussions (FGDs), beneficiary and MSI/awardee surveys, and “deep dive” activity assessments, which included additional KIIs, FGDs and site visits with a sample of 18 awardees and their beneficiaries. The SI Evaluation Team (ET) conducted KIIs with 47 stakeholders, including staff from OTI, LCRI and LCRI awardees, subject matter experts (SMEs), and USAID and US Embassy personnel. It also conducted 13 FGDs – five with 24 awardees and 8 with 65 project beneficiaries. The ET also undertook eight site visits as part of its activity assessments. The survey was administered both online and in person to 90 activity beneficiaries and 18 MSI and awardee staff members. The ET undertook its fieldwork in Lebanon in April and May 2017. vii FINDINGS For evaluation findings by question, please see pages 6-18 in the report. SYNTHESIS AND CONCLUSIONS Evaluation Question #1 - Did LCRI utilize an appropriate strategy to achieve USAID/OTI’s program goal in the country? The LCRI program’s strategy was largely appropriate to achieving OTI’s program goal in Lebanon. At the objective level, the strategy was better suited to achieving Objective I (mitigating sectarian and host community-refugee tensions) than Objective 2 (countering violent extremism). The program struggled to identify new organizations for countering violent extremism (CVE) programming, and awardees had difficulties identifying key targeted groups, e.g., “moderate” leaders, due the lack of strategic guidance in these areas. However, the strategy was well-designed from the perspective of enabling the program to respond quickly and flexibly to both programmatic opportunities and constraints when they emerged in the field. LCRI’s CVE strategy more closely resembled a preventing violent extremism strategy because it improved the resilience of communities who were more predisposed to opposing radical recruitment by extremist groups in the first place rather than addressing these groups’ influence directly. The ET found that LCRI had been effective in identifying the geographic areas most relevant to addressing the program’s objectives, but that there were no strategic documents describing how awardees were to be targeted, nor sufficient guidance on defining targeted beneficiary groups, e.g., what constituted “moderate” leaders, “vulnerable” youth, etc. However, LCRI did conduct work on better defining criteria for such groups later in the life of the program. The ET also found that the LCRI strategy provided space and processes for regular assessment of and responsiveness to opportunities and constraints, and that LCRI demonstrated, through activities such as those in Qaa, Tripoli, and Aarsal, the use of this strategy in responding to shifting conflict dynamics. Evaluation Question #2: Did LCRI effectively implement its strategy to achieve the intended results? LCRI was generally effective in implementing its strategy under Objective 1 (mitigating tensions), which it demonstrated through its activities in Qaa and Aarsal, but less so under Objective 2 (countering violent extremism), although the ET also found some examples of effective implementation under Objective 2 – e.g., in Akkar, Ras Baalbeck, and Aarsal. The ET found that LCRI was challenged by a lack of standardized or consistent approaches related to articulating and providing consistent implementation guidance, such as in communicating program objectives to partner organizations, particularly in CVE; selecting partner organizations appropriate to carry out the program’s objectives; and targeting and outreach to beneficiaries, particularly youth. The ET also found that LCRI staff acknowledged that CVE understanding could have been stronger among program staff, and was unable to find sufficiently-detailed CVE-specific activity guidelines within the provided program documents to support LCRI CVE activities. There was evidence that activities that combined “tangible” deliverables and ”soft” activities, e.g., LCRI208, which involved both furnishing a sports center and youth sports training, were more likely to improve program effectiveness than those activities without the soft components. Relatedly, a number of awardees stated that tangibles were not necessary precursors to building confidence with awardee organizations in new communities. The ET also identified a number of management issues in discussions with awardees, field and OTI staff, which the ET views as having negatively influenced the program’s implementation and effectiveness in the achievement of LCRI results. Evaluation Question #3: To what extent did LCRI contribute to the achievement of program goals and objectives? viii The ET found that within specific communities (e.g., Qaa and Aarsal) and among specific awardees, LCRI could point to some successes in contributing to the achievement of the program’s goals and objectives. Activities like those implemented in Qaa and Aarsal succeeded within their communities in no small part because LCRI identified and engaged with established and willing local partners who could mitigate intra- and inter-community tensions and weather crises when attempting to build bridges among traditionally antagonistic groups. Such activities were able to help LCRI achieve progress on objective 1, especially sub-objectives 1 (promote peaceful alternatives to violence) and 2 (reduce marginalization and isolation of community groups). However, the ET also found that several factors, detailed under the ET’s responses under evaluation questions 1 and 2, reduced the program’s ability to effectively implement activities under objective 2. These factors included the program’s less strategic focus on countering (versus preventing) violent extremism (Finding 1.2), limited guidance for program staff on CVE activity design and awardee and beneficiary selection (Finding 2.2), and related challenges for LCRI in communicating the program’s CVE strategy to awardees (Finding 2.4). RECOMMENDATIONS AND LESSONS LEARNED 1. OTI should develop clearer standards for how the program defines and utilizes key concepts related to awardee and beneficiary targeting (i.e., “vulnerability” and “moderate actor”). 2. OTI should explore the possibility of supporting additional activities conducted by LCRI that show strategic promise. The fact that LCRI was able to implement at least one activity with Palestinian camps, prisons, religious leaders, and former fighters suggests a basis for more strategic follow-up work 3. OTI should support or encourage training for implementing partner (IP) staff on how to socialize sensitive concepts with awardees, such as program objectives related to CVE and how and when to communicate those objectives with beneficiaries. Relatedly, consider more training by CVE experts for staff and more expert involvement in: 1) designing CVE-specific activities; 2) selecting and approaching suitable partners; and 3) identifying and targeting vulnerable/at-risk populations. 4. To improve CVE results, IPs should target organizations that are established and locally connected, as the absence of local partners with these characteristics raises the cost of implementing sensitive programming. However, while bringing in an outside partner is likely to lower the chances that your activity will achieve a successful outcome, the strategic importance of an area – i.e., it is considered critical that programming be conducted here – then the added risk of failure is acceptable 5. OTI and IPs should avoid the use of programs that only involve tangibles without pairing them with “soft” activities that directly relate to the program objectives. 6. OTI should continue to emphasize rapid responsiveness of program activities in high priority areas for program objectives. 7. To help address program management issues, future OTI programming in Lebanon (or similar country environments) should establish a more robust oversight plan and regular schedule of visits to awardees and project sites for all field and other relevant staff. 8. OTI and IPs should design and enforce a more consistent and tangible monitoring and evaluation (M&E) system that would provide more feedback on what worked and what did not, especially to encourage active learning in the emerging CVE field. ix II. INTRODUCTION CONTEXT Terrorism and extremism in Lebanon arise from a complex array of domestic, geopolitical, and socioeconomic issues. Although these conditions have been present for decades, ongoing conflict in neighboring Syria has catalyzed confessional and host community-refugee tensions, raising the risk of renewed violence in Lebanon, often perpetrated by violent extremists. Lebanon borders Syria, the world’s most violent conflict and perhaps most significant ungoverned space. The cross-border threat of terrorist organizations, though significant since the beginning of the Syrian war, grew dramatically when the Shiite militia and political party Hezbollah1 became fully involved in providing military support to the Assad regime in 2014. between 7,000 to 10,000 members of Hezbollah’s militants reportedly operate in Syria, though most Lebanese Sunnis and Palestinian refugees in Lebanon support the Syrian rebels and between 800 and 1,000 of them have gone to fight in Syria.2 The spread of this conflict’s influence into Lebanon exacerbates the rift between the country’s Sunni and Shia populations.3 While The rise in extremist Sunni political movements in Lebanon can be characterized as an expression of the spillover of the Syrian conflict, two other crucial factors have influenced this trend: Hezbollah’s control of state institutions and its “veto power” over Lebanese politics,4 and the exploitation of Sunni frustration by Salafist factions as a result of weakened Sunni political leadership.5 The Future Movement, a secular party long popular with many Sunnis, declined in early 2014 when it entered a coalition government with Hezbollah and supported a crackdown on the Sunni militias active in Tripoli that it had partially sponsored until then. The growing sense of Sunni marginalization from the centers of power has prompted a rise in support for more radical, Salafi groups, who play on the desire to create a direct Sunni opposition to Hezbollah (among other more radical political views) rather than toeing the line of the traditional Sunni political bloc. The decision of many former supporters to opt for more radical and violent ways to express political frustration further undermines stability in Lebanon by fragmenting a major power bloc within Lebanon’s delicate balance of political power. The decision to postpone parliamentary elections in November 2014 and the prolonged absence of a president from May 2014 until Michel Aoun’s election in October 2016 indicate that the current Lebanese government has limited room to undertake effective reforms to combat this instability.6 Lack of socioeconomic resources exacerbates sectarian and host community-refugee tensions and contributes to the rise of extremism. Even before the Syrian war, Lebanon has been the place of habitual residence refugees, including Palestinians and their descendants since 1948. Many of them live in refugee camps that were meant to be temporary, though they became permanent in practice and have since been a source of instability and a hotbed for extremist group recruitment. 1 “Party of God,” a Shia Islamist militant group and political party based in Lebanon. Hezbollah was established after the Israeli invasion of Lebanon in 1982 by Iran. It is based principally in Beirut and effectively controls Lebanon’s Shiite￾dominated south and the Beqaa Valley, allowing terrorists to move around these regions with relative impunity. It is also an effective political party in Lebanon and holds twenty-three of the 128 seats in the Lebanese parliament. It has been classified as a terrorist organization by the US, France, the Arab League, the Gulf Cooperation Council, Israel, etc. The EU, UK, New Zealand, and Australia have proscribed Hezbollah’s military organization, while making a distinction with its political wing. Horowitz, 2013 and Council on Foreign Relations, 2007. 2 Alami, 2015. 3 Lefévre, “Tackling Sunni Radicalization in Lebanon.” 2014. 4 Interview with Subject Matter Expert, 2 May 2017. 5 Ibid. 6 Lefévre, “Tackling Sunni Radicalization in Lebanon.” 2014. 10 The Syrian war aggravated Lebanon’s resource issues. Lebanon, a country of roughly 4.5 million people, has taken in roughly two million Syrian refugees, making it the country with the most refugees per capita in the world.7 This results in two concerns: a huge population of refugees entering from Syria who live in dire poverty, and a fierce contest between Syrians and Lebanese over access to jobs and services. Within Sunni and Syrian refugee populations, the tensions between these communities provides fertile ground for extremist factions to recruit for jihadist operations. This primarily includes fighting Hezbollah, as some of the Salafist groups consider jihad against Hezbollah to be a religious duty that all Sunnis must perform to lift the injustice they are suffering in Lebanon and Syria.8 The regrouping of the Al-Nusra Front9 and ISIS in the Qalamoun region bordering Lebanon is an example of this threat. From the Lebanese and Shia side, many view the refugee issue as a “threat to internal security,” resulting in waves of arrests of refugees who entered Lebanon illegally.10 Several hotspots of terrorism and extremism have developed in Lebanon as a result of these contextual changes. Since 2013, a string of suicide car bombings has occurred in the Southern suburbs of Beirut and the Beqaa Valley, partially enabled by the relatively permissive environment in these areas described above. The majority of the Lebanese Sunni community lives in socioeconomically marginalized areas, which has further enabled extremism to take root in these areas. The northern city of Tripoli has emerged as a symbol of these trends; the city has witnessed sustained violence, particularly centered around the Sunni neighborhood Bab al￾Tabeneh. Prisons and Palestinian camps are also hotbeds for extremist recruitment, as is the southern Lebanese city of Saida. PROJECT BACKGROUND This evaluation focuses on the Lebanon Community Resilience Initiative (LCRI)—a program implemented on behalf of the United States Agency for International Development’s (USAID) Office of Transition Initiatives (OTI) by Management Systems International (MSI) from August 2014 to September 2017. LCRI’s stated goal is to “strengthen the resiliency of Lebanon's most vulnerable communities in the face of two critical destabilizing factors: sectarian and host community-refugee tensions and susceptibility to violent extremism.” In support of this goal, the program has the following, mutually reinforcing objectives and three sub-objectives: 1. Mitigate Rising Sectarian and Host Community-Refugee Tensions o Promote peaceful alternatives to violence; o Reduce marginalization and isolation of community groups; and o Strengthen moderate actors. 2. Counter the Influence of Violent Extremist Groups o Counter extremist messaging; o Strengthen moderate actors and increase moderate space; and 7 UNHCR has registered roughly 1 million Syrian refugees in Lebanon, but many acknowledge these estimates as low, due to barriers to and inaccuracies within the registration process; Schmitt, 2016. 8 Ibid 9 Also described as al-Qaeda in Syria or al-Qaeda in the Levant, a Salafist jihadist terrorist organization fighting against Syrian government forces in the Syrian Civil War with the aim of establishing an Islamic state in the country. 10 “Terrorism continues to shake Lebanon’s security.” Al-Monitor, 2013. Figure 1: Map of Lebanon Indicating LCRI Implementation Areas 11 o Strengthen youth empowerment and civic participation. LCRI has sought to achieve these objectives and sub-objectives through the provision of over 250 grant awards to Lebanese community organizations operating in areas that are susceptible to destabilization from the Syrian crisis. LCRI grants support a wide range of activities, from infrastructure to capacity building. The grants— organized as “clusters” of activities—seek to strengthen host communities by promoting alternatives to violence and reducing marginalization of people vulnerable to radicalization.11 In particular, the program targeted moderate community members, women, and youth between the ages of 15 to 29. The program operated in Northern Lebanon (Tripoli and Akkar), the Beqaa Valley, and briefly in Beirut. LCRI builds upon the OTI-funded Lebanon Civic Support Initiative (LCSI)—OTI’s first engagement in the country—which was implemented from 2007 to 2014. EVALUATION PURPOSE AND AUDIENCE The purpose of this evaluation is to determine the extent to which LCRI met programmatic objectives and achieved both intended and unintended impacts. In particular, evaluation learning and recommendations will help United States Government (USG) actors in Lebanon to build on approaches and partnerships in the geographic areas of implementation. Recommendations from the evaluation will inform future programming and policy making, as appropriate. The key users of the evaluation will be: 1) The USAID Mission and US Embassy in Lebanon, and 2) USAID/OTI in Washington. EVALUATION QUESTIONS This evaluation answers three primary evaluation questions as defined in the evaluation Scope of Work (SOW) contained in Annex I. These primary questions are listed below: I. RELEVANCE: Did LCRI utilize an appropriate strategy to achieve USAID/OTl’s program goal in the country? II. EFFECTIVENESS: Did LCRI effectively implement its strategy to achieve the intended results? III. IMPACT: To what extent did LCRI contribute to the achievement of program goals and objectives? 11 “Clusters” refers to a series of grants that OTI provides in particular geographic areas, each of which contributes to a certain program sub-objective. 12 III. METHODOLOGY OVERVIEW A five-person team from Social Impact, Inc. (SI) conducted this summative, mixed-methods evaluation over three phases from March to June of 2017. In the first phase, the evaluation team (ET) members reviewed program documents and OTI’s Web-Based Database (WBDB) in order to inform the evaluation design, which it refined in consultation with OTI-Washington and field-based staff. In phase two, the team deployed for fieldwork in Lebanon over the course of four weeks. During this time, the team conducted Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), and unstructured, direct observations of LCRI projects in Beirut, Tripoli, Akkar, and the Beqaa Valley. It conducted a survey of staff, awardees, and beneficiaries, which was completed either via an online platform or in person. In the third and final phase, the team collated and analyzed the data collected during its fieldwork and drafted the evaluation report. For further descriptions of the methods employed by the team, see below. Data collection instruments for each method used during the evaluation are included in Annex II of this report. EVALUATION DESIGN Sampling Strategy In consultation with OTI and field staff, the ET designed a two-tiered sampling strategy: first, the ET randomly selected five groups of awardees (with ten awardees in each group) to each participate in an FGD (one FGD in Beirut, two each in Beqaa and the North).12 From this list of 50 awardees (10 in Beirut, 20 for Beqaa, and 20 for the North), the ET conducted a purposive sample of 18 projects that were selected for deep dive activity assessments. The purposive sample selected activities in order to achieve a diversity of awardees, location, and program objectives, and selected projects with the largest budgets based on amount disbursed. For a final list of the activities selected for “deep dives,” see Annex V. As the ET encountered significant difficulty making contact with awardees and confirming meetings with them throughout the evaluation, particularly in Tripoli, the random sampling for FGDs and selections for activity assessments were expanded to a “contingency sample” in order to ensure that the evaluation met its targets. However, this raised the issue of response bias, as noted in the Limitations section. For a list of documents reviewed, see Annex III. DATA COLLECTION METHODS Document Review The ET read project-relevant documentation provided by USAID/OTI and subject-matter specific reports covering the period of implementation from LCRI start-up in September 2014 to March 2017, the period leading up to close-out of LCRI. This included OTI’s WBDB, which the ET primarily consulted to develop a sampling strategy of awardees. Key Informant Interviews The ET conducted KIIs with respondents familiar with the LCRI program objectives, implementation, activities, and results. While the ET conducted mostly individual KIIs, group discussions with key informants were arranged where appropriate. Respondents included Washington- and Beirut-based USG personnel, Subject 12 At the request of USAID and MSI, the ET excluded seven of the grantees selected in the random sample due to the nature of their activities, which were deemed to be unrepresentative of the larger LCRI portfolio. 13 Matter Experts (SMEs), and implementing partner (IP) staff. They also included awardee organizations’ leadership during activity assessments. KIIs were semi-structured interviews that lasted approximately one hour. For a list of key informants, see Annex IV. The ET conducted a total of 47 KIIs—21 of which were women and 26 of which were men. Focus Group Discussions The ET conducted FGDs with randomly selected awardees as noted above, and beneficiaries from selected LCRI activities where possible. The ET conducted five awardee FGDs with 27 representatives from 24 awardees. The ET also conducted 12 FGDs with beneficiaries from 11 activities. These FGDs included 92 participants, 60 of which were male and 32 of which were female. Activity Assessments The ET also conducted deep dive activity assessments, which included additional KIIs, FGDs, and site visits with a sample of 18 purposively-selected awardees and their beneficiaries. Mini-Surveys The ET distributed online and in-person mini-surveys, primarily with closed-ended questions, to provide quantitative data on LCRI activities in order to supplement qualitative data from FGDs. The brief survey was distributed to in-country LCRI staff (OTI and IP), awardees, and beneficiaries. The survey for OTI/IP in￾country staff was distributed online and has 18 respondents, which were all IP and awardee points of contact designated by the IP. The survey for awardees and beneficiaries was translated into Arabic and completed in person by 90 respondents. This group consisted of 31 awardees and 59 beneficiaries (geographically, 51 of these respondents were from Beqaa, 19 from Tripoli, and 10 each from Akkar and Beirut).13 As noted below, the IP designated the points of contact for the awardees and, likewise, awardees identified the FGD participants, which likely introduced some selection bias. Site Visits If the activity assessment involved an activity with a tangible output (e.g., solar lights and a soccer field), the ET conducted a site visit and recorded their observations. LIMITATIONS The LCRI evaluation has several notable limitations: 1. Lack of Baseline: Because OTI’s approach is based on flexible and rapid implementation of small projects, baselines are not required of the awardees nor would a reliable and consistent system of documenting baselines across 250 activities be expected, making measurement of impact difficult. 2. Lack of Counterfactual: Similarly, in such fluid and volatile operating environments, construction of a reliable counterfactual is very difficult and would require varied approaches in multiple locations. This limitation also complicates measurement of impact. 3. Sampling Limitations: The LCRI program was implemented through more than 250 activities implemented by over 100 awardees. Given the limited amount of time available for data collection, it was not possible for the ET to study every awardee nor every activity in-depth. To limit potential 13 The ET acknowledges that this group is heavily weighted toward respondents from Beqaa, and had relatively few respondents from Akkar and Beirut – this was a product of locations the Team was able to visit (in the case of in-person surveys) and who they received responses from (in the case of the online surveys). 14 biases, the ET selected awardees at random, thereby ensuring as representative of a sample as possible. And, to ensure that the selected activities represent a variety of potential experiences and demographics, a purposive sample was proposed that takes into account key criteria. Despite these mitigation efforts, however, it remains possible that some views and experiences were not revealed through the evaluation. 4. Selection Bias: Reliance on awardees for coordination of FGD outreach to participants introduced bias in which the awardee has more control over who does or does not participate. This means that the awardees may have selected participants that had exceptionally positive or negative experiences with the program and thus may not have been representative of the entire beneficiary population. 5. Response Bias: Although the ET sought the most representative sample possible, it encountered challenges with the implementation of its final sampling strategy. Due in part to some sensitivities with the program discussed in the body of the report, particularly in the North region, many awardee organizations did not respond to the ET’s requests for interviews, responded only after multiple attempts at contact, and did not agree to meet with the evaluators. The issue was particularly acute in Tripoli, where response rates were only around 50 percent. To make up for this, the ET utilized its contingency sample. The ET developed this additional random sample of activities in light of the low response rates it faced in its implementation of the original random sample. In five cases where the sampled organizations did not reply to the ET, it selected an alternate from the contingency sample to examine. 6. Logistical/Security Challenges: Lack of accessibility due to security concerns prevented the ET from selecting locations that were of high priority and high volume of activity implementation (i.e., Arsaal and Wadi Khalid). Though travel to these areas was not advisable, the ET received input through FGDs from awardees operating in these areas. 15 IV. FINDINGS QUESTION #1 - DID LCRI UTILIZE AN APPROPRIATE STRATEGY TO ACHIEVE USAID/OTI’S PROGRAM GOAL IN THE COUNTRY? Mixed Interpretation of Sub-Objectives: Activity database analysis and KIIs with senior field and monitoring and evaluation (M&E) staff show that LCRI categorized similar activities across sub-objectives and objectives, with the main difference among them often being the adverse effects of conflict drivers articulated for a particular location (i.e., sectarian tensions or CVE, as explained in the activity notes). The strategy also did not provide guidance on applying key terminology, e.g., the criteria for determining youth “vulnerability” or what constituted a “moderate” actor. This led to different interpretations of the strategy among IP staff and issues related to beneficiary targeting, identifying awardee partners, and socializing key program concepts with these partners, although it should be noted that LCRI did actively discuss – e.g., in program yellow light meetings – and document – e.g., in activity final evaluation reports (FERs) – how to more consistently interpret and apply this terminology to these program tasks and particular activities. Such efforts resulted in more consistent definitions toward the middle and later phases of the program, such as in the Cluster Review of activities under the “Increase Moderate Space and Increase Moderate Actors” sub-objective in December 2015, although internalization and utilization of the new language for “moderate actors” was not common among LCRI staff, according to interviews with several staff members. Strategy Focused More on Preventing than Countering Violent Extremism: Based on an analysis of program documents, KIIs with senior LCRI field staff, awardee leadership involved in countering violent extremism (CVE) projects, and FGDs with beneficiaries, the ET found that LCRI targeted community members who were mostly predisposed to opposing radical recruitment in the first place. This finding relies both on an assessment of groups targeted by the activities as well as direct statements from relevant staff and implementers. For example, in its activity assessment of LCRI 116 (Using the Arts to Voice Youth Concerns in Majdel Anjar), the ET’s consensus after speaking with youth beneficiaries was that they participated in the arts/photography activity because they were interested in these areas and enjoyed volunteering, but not because they appeared prone to extremist recruitment. LCRI’s approach, therefore, more closely resembles a preventing violent extremism (PVE) strategy rather than a countering violent extremism strategy, as it improved community resilience to the proliferation of these groups, rather than addressing their influence directly.14 In contrast, a CVE strategy would have more directly identified and protected extremist “entry points” against infiltration by these groups (see Figure 2). A CVE-focused strategy, for example, could have included more nuanced beneficiary targeting or more direct socialization of program objectives with strong awardee partners. In their KII comments, three LCRI staff supported the view that the program had a PVE focus, or at least a prevention element. One noted that “mitigating and countering are heavily overlapping,” and that “by the end of the first year, there was an acknowledgement of this.” 14 The Team utilized the Department of State & USAID Joint Strategy on Countering Violent Extremism’s definition of CVE to develop this finding. The definition states that CVE consists of “proactive actions to counter efforts by violent extremists to radicalize, recruit, and mobilize followers to violence and to address specific factors that facilitate violent extremist recruitment and radicalization to violence.” Figure 2: Preventing Versus Countering Violent Extremism 16 KIIs with OTI leadership indicated the LCRI intention was to, in part, pivot from a resiliency focus under the previous programs to efforts that would more directly confront the influence of violent extremist organizations. However, the majority of awardees, as well as USAID Mission staff, indicated in interviews that LCRI focused on resilience or stabilization, and continued to develop activities and target beneficiaries based on this “PVE” focus. As part of its “deep dive” assessments of select activities, the ET studied 11 activities under Objective 2, including their targeted beneficiaries, locations, and alignment with the related objective. Of these 11 activities, five were considered strongly supportive of a CVE objective and three were considered mildly supportive of a CVE objective – 72 percent off the 11 selected activities. The three remaining activities were considered only weakly aligned with the CVE objective (see table in Annex V). Two examples of these ratings and their justifications are: • LCRI189: Relevance to CVE objective: Strong. The IP not only worked in an area of Beirut that is at risk of radical group recruitment, but the organization also has a detailed beneficiary targeting strategy that uses local informants to carefully identify and support the most at-risk individuals in the area. • LCRI128: Relevance to CVE objective: Weak. While extremist groups recruit followers in Akkar, this one-off project that provided office supplies to support the Civil Registration Office in Aabde did not develop outcomes in any meaningful way related to the program objectives. Interviews with the awardee found little to no awareness of the program’s CVE objective. Geographic Targeting: In KIIs, FGDs, and site visits with the awardees selected as part of the ET’s “deep dive” activity assessment methodology, 10 of 18 awardees felt that LCRI had been effective in identifying the geographic areas most relevant to addressing the program’s objectives. Five interviews with LCRI and OTI staff and two FGDs with awardees on the subject of geographic targeting found that previous programs struggled to implement activities in Hezbollah-controlled South Lebanon. LCRI, therefore, chose not to initiate programming there based on security and other factors. It focused instead on those areas in which the program could make the greatest progress toward its two primary objectives and sub-objectives. LCRI decided to close down its programming in Beirut, based on a lack of funds it felt would have been necessary to generate traction and scalable impact in the city, given the costs of operating there. Based on a database review of programs and KIIs with IP staff involved in Beirut programs, the ET found that recent activities were demonstrating progress toward the program’s objectives. For example, LCR189 (“Engaging Tariq Jdideh Youth in Skills Training and Neighborhood Revitalization”) enabled awardee Rescue Me to train 30 youth on basic rehabilitation skills and provide psycho-social support in a neighborhood vulnerable to extremist recruitment. Among its other benefits, the activity resulted in a significant increase in the young participants’ sense of self-worth, fueled by their new, sustainable skills, e.g., in plastering and woodworking; tangible benefits for their families, e.g., the participants’ rehabilitated homes; and a lasting sense of accomplishment. Given activities such as these, several field staff felt the Beirut programming should have continued. “We were starting to [have a] good impact in Tariq Jdideh, and I don’t agree with the justification for shutting down [in] Beirut,” said one of the staff members. “There is not really anyone left doing this kind of work.” Organizational Targeting: There were no strategic documents describing how awardees were targeted, and three KIIs with IP field staff and OTI staff, as well as a review of program documents (e.g., the Program Performance Review [PPR]) found that the program struggled to identify new organizations for CVE programming. The ET also found through its deep dive activity assessments that there were two key organizational characteristics driving success in achieving program objectives: trust and capacity. To this end, organizations with an operational base and established presence (which reflect on both trust and capacity) in their respective communities were generally more successful in implementing their activities. For example, the ET determined that the Sharing and Giving Association’s work in Chmostar (LCRI162) met program objectives, in part, due to its community presence. However, programs such as “Empowering Youth as Road Safety Activists in Northern Beqaa” (LCRI220) that were implemented by other groups from outside the area faced more challenges in accomplishing activity goals. Not all communities have ideal organizational partners, which may force implementers into a difficult choice of either working with weaker partners or not working in a 17 community at all. If pursuing the former, the challenges and a higher likelihood that the effort will not be successful in achieving intended outcomes must be recognized. Targeting of Moderate Leaders: The ET found a minimal definition of what constituted “moderate” actors in the strategic documents for the program,15 although LCRI did expand its definition as part of its Cluster Review of activities under the “Increase Moderate Space and Increase Moderate Actors” sub-objective in December 2015.16 However, KIIs with two LCRI IP field and M&E staff indicated that usage of this terminology wasn’t widespread among staff, and that they still struggled to identify where and when to apply the “moderate” sub-objectives. As one field staff member noted, anyone that the field staff wanted to work with could be justified as support for a moderate actor, as there was no standard definition against which to measure it. Youth Targeting: As with LCRI’s difficulties in defining what constituted a moderate actor, two program staff noted that they also faced challenges in consistently communicating to awardee organizations who were the “vulnerable” youth that should be engaged by the program. The issue was the lack of a standard definition that was both widely internalized and institutionalized by LCRI staff, although the program did work to develop more specific language later in its life – e.g., during the May 2016 Strategy Review Session. “Our definition of ‘vulnerability’ varied from one place to another,” stated one OTI senior staff member. “In one, it was unemployed people, in another, it was drop-outs, and in another, it was based on people living in a particular neighborhood.” Program Responsiveness: Activity assessments in Qaa (for activities LCRI237, LCRI172), Tripoli (activity LCRI052), and three FGDs and KIIs with awardee organizations in Tripoli and Aarsal all yielded evidence of the program’s responsiveness to conflict dynamics. Program responsiveness comprises four key factors: having a team that knows the local dynamics well, autonomy for the Project Development Officers (PDOs) in making programming decisions, fast-moving activity cycles, and a willingness to work where other donors would not. Examples of this included, in Qaa, constructing solar lights in response to the infiltration of the community by multiple suicide bombers at night; in Tripoli, the reconstruction of markets destroyed by sectarian clashes; and in Aarsal, the construction of a public park after residents were barred from leaving the town to visit the nearby park in Baalbeck. Support within the larger USG assistance portfolio: KIIs with the USAID Mission and Embassy staff in Beirut and USAID/OTI staff in Washington unanimously praised the program for helping shed light on dynamics and working in parts of Lebanon that no other part of the USG could access. In addition, USAID Mission staff noted that a core component of their strategy was to help Lebanon manage the influx of refugees from Syria by ensuring stability and resiliency in its program areas. Strategic Areas for Future Lebanon CVE Programming: Four KIIs with SMEs, awardee organizations, and background reading on the Lebanese CVE context indicate that there were opportunities for CVE activities that LCRI could have exploited more fully than it did. These opportunities included sustaining activity in the Palestinian camps (e.g., Baddawi), in prisons (e.g., Roumieh), with religious leaders, and with former fighters (e.g., returning from Syria). The ET acknowledges that a lack of resources, U.S. regulations and legal constraints significantly limited the degree to which LCRI could work in these areas. However, these opportunities are worth noting because the program was able to implement activities in each of these areas, suggesting there may be ways to focus on these key drivers in future programming despite the substantial challenges to 15 Defined as nonviolent, tolerant and neutral” (OTI Program Document Lebanon) 16 “Moderate actors” can be defined as individuals, groups and organizations that are non-partisan and espouse civic values. Often, these actors have the potential to act as change agents in their communities” (OTI Cluster Review: Increase Moderate Space and Increase Moderate Actors, December 2015). 18 implementation. Specifically, they merit consideration for future programming by USAID/Lebanon, and the ET provides suggestions on such programming in the aforementioned four areas in the “Recommendations and Lessons Learned” section.17 For example, LCRI was prohibited by the U.S. Embassy from working with Palestinian refugees within the camps until the fall of 2015, but even then, they received little clarity on USG policy regarding what type of engagement was permitted, according to an interview with key former staff. This, coupled with reluctance on the part of LCRI PDOs 18 to work with Palestinians within the camps and the program’s very tight budget, minimized LCRI activities involving these individuals in camps. Nevertheless, LCRI was able to engage refugee-based Palestinian youth through LCRI119 (Bringing Lebanese and Refugee Youth Together Through the Arts), which effectively increased positive interactions between these two groups in an area with escalating tensions, and LCRI213 (Engaging Lebanese and Refugee Youth in Positive Messaging through Arts), which succeeded in giving youth a positive and visible role in their community resulting in increased feelings of empowerment. QUESTION #2 - DID LCRI EFFECTIVELY IMPLEMENT ITS STRATEGY TO ACHIEVE THE INTENDED RESULTS? Activity Completion Versus Program Objectives: Based on its KIIs, site visits, and document review, the ET found that there was greater project focus on tangible outputs in LCRI grants in specific regions19 and much less emphasis on the transformative potential of activities. This focus on tangibles frequently manifested itself in the uncertainty expressed by awardee and LCRI staff on how to move beyond tangible activity results, i.e., to identify how the activities actually linked to—and could be leveraged to work toward—LCRI’s objectives. In the North, for example, the ET noted this uncertainty in Tripoli among the groups that had participated in repairing the vegetable market, the clothing market, and the “stairs” areas between Jabal Mohsen and Tebbaneh.20 These findings were supported by a database analysis of LCRI projects, which identified 63 out of 203 activities (31 percent) that had theories of change (TOCs) with indirect connections to program objectives (see Box 1 below). Additionally, 35 of these 63 activities (56 percent) were solely focused on tangible deliverables. Two examples of common issues with these less directly linked TOCs are as follows: 1. LCRI invested in developing committees, comprised often of youth or “moderate actors,” under the assumption that the program’s investment would empower these committees to mitigate tensions or reduce extremist influence. However, activity assessments of these types of projects during the evaluation (e.g., LCRI185: “Lebanese-Syrian Cooperation to Improve Road Lighting in Ghazze”) found that program efforts focused on implementing the activity itself (e.g., building solar lights), rather than ensuring that its objective was met (e.g., increasing the influence of that committee to resolve disputes in the community). Other examples include supporting governorate offices through tangibles (e.g., computers, office furniture), with the rationale that these improvements would help their capacity as a moderate actor. However, activity assessments of these activities (e.g., LCRI238) found limited effect in terms of advancing program objectives. 2. LCRI invested in building moderate spaces or public spaces assuming that the community would protect the space either because a) extremist control would limit their access or b) the community 17 In “Recommendations and Lessons Learned” section, see “Table 2. Recommended Areas for Future CVE Programming in Lebanon,” for suggestions on future programming in these four areas. 18 The exception was an LCRI staff member’s identification of legitimate Palestinian actors in the Beqaa region and integration of Palestinian refugee youth into the program’s LCRI119 and LCRI213 activities. 19 Such activities focused solely on providing material items, such as street lights [LCRI185] or office furniture [LCRI238]. 20 The PPR of the LCRI program identified this issue as well. 19 would benefit from the space being moderate. However, activity assessments of these projects (e.g., LCRI153, LCRI180) often found limited advancement of program objectives by awardees and significantly limited beneficiary knowledge of the program objectives. Comments on Box 1: More CVE Guidance & Support Needed: The ET was unable to find sufficiently detailed CVE-specific activity guidelines within the provided program documents to support these CVE activities, and three IP staff admitted in KIIs that this subject matter knowledge was lacking within LCRI. More comprehensive guidelines would build on many of the issues noted in this section, including: awardee and beneficiary targeting, clarity around program objectives, measuring psychosocial behavior change, and an understanding of the interconnected nature of CVE drivers. The ET does note that LCRI did utilize strategic review sessions and rolling assessments later in the life of the program to review and refine its CVE approach. Knowledge of Local Drivers: All activity assessments found that interviewed awardees and beneficiaries reflected a similar assessment of local drivers of conflict and tension as those described in the program documents. The findings suggest that IP staff were of high caliber, with a strong ability to quickly engage with key community stakeholders, and that this was a strength. In regard to research produced through the project, there were differing perceptions of its utility. Though field staff did not find it as helpful, it was more helpful for OTI staff. Tangibles Stronger When Combined with ‘Soft’ Activities: By comparing activity assessments during the evaluation, the ET found that activities involving tangibles were more successful in meeting program objectives when they incorporated “soft” activities than when they did not. Activities involving both types included LCRI208 ()furnishing a sports center coupled with youth sports training, and LCRI116 (implementing a community project following civic activism training). Conversely, the ET found that awardees that received one-off tangibles displayed the least awareness of LCRI objectives of any organizations. Awardees illustrating this lack of familiarity included those implementing LCRI238 (equipment support to the governorate office in Baalbeck-Hermel); LCRI128 (equipment support to the civil registration office in Akkar); and LCRI34 (equipment support to the Majdel Cooperative). Box 1: Database Analysis - Tangibles Based on a database analysis of LCRI projects, 53 out of 203 projects (26 percent) were considered tangibles. Of these 53 projects, 18 (34 percent) were considered one-offs, either because a) there was no follow up activity in the location, b) this was the only grant awarded to the local partner, or c) the partner only implemented tangible activities. The ET’s activity assessments found that the one-off tangible activities in its sample - LCRI153, “Establish a Public Park and Youth Center in Bire,” or LCRI180, “Establishing a Public Park in Fnaideq” - did not advance program objectives. However, the ET also acknowledges more broadly that one-offs under LCRI may be a product of OTI’s “action learning” model, in which testing a new approach, working with a new partner, working in a new location, or some combination of these may be worth the risk of an activity “failing.” However, the remaining “tangibles” were not considered one-offs, i.e., they were layered activities – i.e., a sequence of activities that are designed and implemented over time to achieve a cumulative effect in a specific area. involving awardee organizations or places that received both tangible and ‘soft’ activities. These combined tangible-soft activity grants were considered more effective. Examples included work in Aarsal (e.g., LCRI054, building linkages between youth from Aarsal and the Shia town of Laboueh and LCRI132 building solar lights on the road between the two villages) or in Tripoli, where soft activities (e.g., LCRI075, Dardachat café) combined with tangibles (e.g., LCRI120, rebuilding nearby souks). 20 A related finding was that, although two IP staff in KIIs argued that tangibles were a prerequisite to working in new communities, three awardee organizations and three other IP staff disagreed that this was a necessary component of the program. This latter group of interviewees cited three reasons for this: 1. Tangibles without clear justification for program objectives distract awardee organizations from the goals of the program. 2. Awardees come to expect tangibles as an ‘entry fee’ for the program in new areas. 3. Most multiple project partners in the program had already worked with OTI under previous programs (e.g., LCSI). Tangibles were not necessary for these partners. Program Approach: In KIIs, neither OTI nor IP staff could articulate a communication strategy for awardees or beneficiaries. This was evident in the varied responses to program objectives and beneficiary targeting strategies discovered in the activity assessments and awardee FGDs. This lack of clarity inhibits the ability to achieve objectives because actors may be focusing their efforts on very different things. Five awardees noted in their KIIs that LCRI could have been more explicit in describing its objectives. “Future programming [on CVE] should communicate its objectives more explicitly,” said the leader of one LCRI awardee. “The sensitivities are there, but you have to work with the strong organizations [in the community] and communicate [the objectives] with them bluntly.” For veteran awardees, this challenge often manifested in continuing focus on objectives they pursued earlier under LCRI or under previous OTI Lebanon programs. For example, because an awardee interviewed by the ET who had worked under LCSI had focused its previous activities on conflict resolution, they decided this was again an appropriate objective for their LCRI activity, and program staff did not tell them otherwise. The awardee’s staff told the ET they were never clear about the purpose of LCRI. “It was not described by the program,” said one staff member. It was still thought of “as being conflict resolution.” OTI senior staff noted that they could have communicated the program’s objectives more effectively to awardees in the field. The PPR conducted on the program stated that “partners were not clear on what the [program’s] objectives were,” according to one KII with senior staff. The staff member added that “we did not make explicit the idea of what to say to your partner.” Layering More Effective with Established Organizations: The program practiced “layering” of successive grants to key partners with the aim of following up and building on the results generated by early grants. The ET found through four awardee KIIs and activity assessments that established organizations such as those based in Aarsal (see Aarsal case study below) and the Youth Initiative Association in Majdal Anjar were generally more capable of executing program objectives over an iterative series of layered programs than nascent organizations. Although it is less common that a new organization can be as effective, the activity assessment of the Qoubbe Youth Academy found it was also successfully implementing a layered series of Box 2: Aarsal – A Case Study in LCRI Program Success Due to security concerns, the ET did not travel to Aarsal. However, based on KIIs with OTI and IP staff, document review, the involvement of Aarsal-based awardee organizations in FGDs, and a KII with one awardee organization over the phone, the ET believes that the case of LCRI’s work in Aarsal was a unique success for the program’s objectives. Aarsal, a majority Sunni enclave along the Syrian border in a predominantly Shia and Christian area of North Beqaa on the Syrian border, met the program’s strategic rationale in terms of both sectarian tensions and radical recruitment. By 2014, with a Lebanese armed forces incursion against violent extremist organizations based in the area, Aarsal was completely isolated from the rest of Lebanon. LCRI assessed the issues in the community, and worked with established local partners on a diverse array of programs that provided tangible benefits to reduce community insecurity (e.g., solar lights near military checkpoints) and increased engagement with residents of different religious backgrounds in nearby communities (e.g., Qaa and Laboueh). Awardee organizations praised LCRI staff for being constantly engaged in their activities despite accessibility restrictions given security conditions, and for being the only donor organization committed to programming in the area. 21 activities. Comments on Box 2: LCRI Program Management – Issues & Improvements: In KIIs with the ET, five awardee organizations, five IP staff, and a majority of the interviewed OTI staff in Lebanon and Washington noted issues with several areas of LCRI’s management of the program, which they felt played a role during its early years in hampering its efficiency and damaging staff morale. However, OTI identified and LCRI addressed many of these issues during 2016, which led to smoother, more efficient implementation during the program’s final year. Key issues, and the steps LCRI implemented to resolve them, consisted of the following: Transition to OTI Model: OTI staff referenced MSI’s “growing pains” in implementing LCRI, which was their first OTI project as an implementer. This resulted in “a slow transition to new partners,” and new systems, as well as issues with project oversight, said one senior OTI staff member. A senior LCRI staff member noted, for example, that it took “two years to get a sole source approval for certain programs from their home office,” and that they were not able to obtain the authority to approve sole sourcing at MSI Beirut until roughly spring 2016. The LCRI program conducted a review of its procedures in the third quarter of 2016 and with the help of a new director of operations, restructured several procedures in areas such as operations, oversight and compliance. Four of the awardee organizations that met with the ET noted improvements in these areas in late 2016 and 2017. In fairness, the ET also acknowledges that this was MSI’s first program with OTI as an IP, and that they had a significantly smaller budget to work with (one third the size) than LCSI. Awardee Engagement Issues: Early in the life of the program, multiple staff members (three from LCRI and a senior OTI staff member) also noted that key staff, including the Chief of Party (COP) and Deputy COP, “rarely” travelled North to visit awardees, and that grant and procurement staff were too “overstretched” or busy with procurement paperwork to make it into the field. In 2016, OTI identified this problem, and LCRI established a plan to address it. Field visits picked up in the last quarter of 2016, and then dramatically increased once LCRI’s leadership changed in 2017. High Turnover Rate: A majority of OTI staff and five awardees felt that the program’s high staff turnover was an issue as well. in the North and in the Beqaa, for example, there were three PDOs in each area over the life of the program. This turnover also meant constantly having to rebuild relationships with LCRI, according to another awardee staff member. "Every grant it seemed like we were working with someone new in the MSI staff," the staff member told the ET. Resignations also created disruptions in managerial continuity, institutional knowledge, and inefficiencies as the program sought to replace other departed staff members. For example, the FER for LCRI’s “Establishing the First Youth Center in Aarsal” activity noted that changes in procurement specialists and the departure of the operational manager caused several delays and challenges involving procurement. These included delays in submitting the appropriate documentation for approval, and appropriate procedures – e.g., in gathering quotations on items, submitting purchasing orders – were not followed. LCRI addressed the staff turnover issue in part by making several notable personnel changes beginning after the middle of 2016, including bringing on a new country representative (in September) and director of operations. It also made several operational modifications that strengthened procurement and other key procedures (see “Transition to OTI Model” bullet above). Monitoring Improved During Latter Phase of LCRI: Based on conversations with three LCRI M&E and field staff, as well as in KIIs with four awardee organizations from different regions, the ET found that the program’s monitoring regime was weak during the program’s early years but improved during 2016. Particularly in the North, the ET found that LCRI staff visits were infrequent, and checks on organizational activity reporting were inconsistent and cursory. LCRI M&E staff also agreed that this was the case, adding that the situation was exacerbated by staff turnover, especially the resignation of the program’s original M&E officer in January 2016. The promotion of one of the remaining M&E officers to the senior M&E officer role after this and 22 the COP taking over the management of the M&E Team, enabled greater stability and consistency in the program’s M&E portfolio. Staff visits to awardee sites also increased. QUESTION #3 - TO WHAT EXTENT DID LCRI CONTRIBUTE TO THE ACHIEVEMENT OF PROGRAM GOALS AND OBJECTIVES? Objective 1: Mitigate Rising Sectarian and Host Community-Refugee Tensions According to eight interviews with awardee organizations and four beneficiary FGDs, sectarian and host community-refugee tensions seem to have subsided in the latter half of the program. Although most attribute this to external factors, such as national political agreements, eight awardees (through KIIs) and four groups of beneficiaries (through FGDs) also cited the work of LCRI. Examples of referenced LCRI work under this objective follow below: • Activities such as LCRI172 (Promoting Positive Interactions through the Arts in Qaa) demonstrated how the program could build relations and trust among youth from diverse backgrounds despite ongoing security threats linked to a series of suicide bombings in Qaa in June 2016. Despite the bombings, LCRI and awardee “Centre de Lecture et d'Animation Culturelle” (CLAC) were able to continue with the activity, in which Christian Lebanese, Sunni Lebanese, and Syrian youth came together, wrote and produced a set of audience-participatory theater pieces emphasizing tolerance, acceptance, and similar themes. “There are Syrians in the area, but we were not in contact with them before the activity,” said one program beneficiary. “Now we are friends and we are always meeting with them.” The bombings actually prompted these youths to rewrite the pieces to focus specifically on these violent incidents, address stereotypes, and highlight their joint Syrian-Lebanese collaboration. This activity built a foundation for long-term ties among youth from antagonistic backgrounds because they were able to use the creation of these pieces as a vehicle to collectively work through post-bombing tensions, and to build understanding and empathy between them. Additionally, the commitment of Qaa’s Mayor to continue the activity and CLAC’s ramping up of actions after the bombings – e.g., providing opportunities for parents to partake in some of the training sessions in team building and arts that their children were experiencing, helping Syrian parents ensure legal documentation for their children to enable their ongoing participation – helped keep Syrians engaged in the activities, and improved relations between Lebanese and Syrians locally. • The resulting interactive theater pieces, which were performed ten times across the Beqaa Valley and in Beirut. demonstrated that such ties can be created despite (and be informed by) devastating security incidents. and that such negative incidents can be turned into a demonstration of unity. It also highlighted the ability of LCRI to rapidly respond to changing conflict dynamics by moving swiftly after the bombings to respond to community concerns (sub-objective 1: Promote Peaceful Alternatives to Violence). Such activities demonstrate a means for USAID to continue engagement with a key vulnerable group in Lebanon – youth – through an activity type that can incrementally build both understanding and trust among rival community – arts-based activities – and adapt to shifting conflict dynamics on the ground when the right ingredients are in place – e.g., a committed, local partner willing to respond to these shifting dynamics, a supportive, pro-active local government, engagement and trust-building with parents of youth, and a safe environment for the activity to build – and rebuild – confidence. • While it did not travel to the area due to security risks, the ET believes that LCRI was able to build an effective set of projects in and around Aarsal based on views expressed in a majority of KIIs with OTI and IP staff, the ET’s document review, the involvement of Aarsal-based awardee organizations in FGDs and a phone-based KII with one awardee. This effectiveness was the result of the strategic layering of activities that enabled the establishment of relationships and incremental trust building among Sunni, Shia, and Christian communities in Aarsal and the surrounding area. This mix of both tangible and soft activities in Aarsal and nearby towns such as Laboueh, Fakehe/Ain, and Ras Baalbeck facilitated increased empathy and reduced tensions between largely Sunni-majority Aarsal, its Shiite-majority neighbors, and the influx of largely-Sunni Syrian refugees. 23 • The program was also able to build up Aarsali beneficiary skills and then transition to strengthening relationships with the partners themselves, such as the Aarsal Civil Society Coalition (ACSC). For example, LCRI’s capacity-building support to 20 ACSC representatives helped the coalition enhance its outreach and visibility as a moderate actor in Aarsal, which in turn enabled it to strengthen its capabilities to counter negative stereotypes about Aarsal by increasingly serving as a reliable civil society source in Aarsal with the media. Other ACSC-supported efforts, such as the creation by a youth-led media team of the alternative news source Orsal Lens, provided additional resources to amplify the voice of moderate actors and coverage of LCRI activities, such as “Establishing Neighborhood Youth Committees in Aarsal” (LCRI227). This supported program sub-objectives 1.1: Promote peaceful alternatives to violence; sub-objective; and 1.2: Reduce marginalization and isolation of community groups. • LCRI’s Aarsal work additionally demonstrates progress at the community level toward several program sub-objectives under Objective 1 through activities such as sports and scouting between youth in Aarsal and Laboueh. For example, these activities helped dispel negative stereotypes each community had about the other, create local scouting organizations (e.g., Education Scouts of Aarsal) to continue engagement of youth in these communities, and expand this trust-building to incorporate additional northern Beqaa villages. This also supported sub-objective 1.1 (Promote peaceful alternatives to violence) and sub-objective 1.2 (Reduce marginalization and isolation of community groups). Awardees and beneficiaries responding to the survey for the most part supported the view that progress had been made toward mitigating host community-refugee or sectarian tensions in their areas, and that LCRI deserved at least some of the credit for this. Improved Community Resistance: A majority (52 percent) of respondents felt that that communities were now more resistant to host community-refugee, or sectarian tensions since the program commenced in 2014 (compared to 25 percent who thought their resistance was about the same, and 23 percent who felt it had decreased). Of the surveyed regions (Beirut, Beqaa, Akkar, and Tripoli), a majority of respondents thought that communities were more resistant to these tensions except those in Beirut,21 where 50 percent thought such communities had about the same level of resistance (versus 20 percent who thought resistance had increased, and 30 percent who believed it decreased).22 See Figure 3 below. Figure 3: In your opinion, how would you describe these communities with regard to mobilization toward (i.e., ability to resist) tensions in the community (host-community/refugee or sectarian tensions)? 21 This and other results involving Beirut respondents may have been influenced by the fact that LCRI operated in Beirut for a relatively short period of time. 24 Decreased Tensions Noted: Awardees and beneficiaries from three of the four surveyed regions viewed resilience to sectarian tensions as having increased since LCRI commenced. But these respondents may not see LCRI activities as having played a role in improving that resiliency: 61 percent of survey respondents disagreed or strongly disagreed with the statement “LCRI helped to reduce host community and refugee tension,” while nearly 60 percent disagreed or strongly disagreed when asked whether “LCRI helped to promote peaceful alternatives to violence.” Regionally, respondents from Beqaa and Beirut were the most negative: more than two-thirds of the roughly 5023 individuals from Beqaa, and all ten from Beirut, either disagreed or strongly disagreed with these statements. Respondents from Akkar and Tripoli were more favorably inclined, with 90 percent agreeing or strongly agreeing that LCRI had helped promote peaceful alternatives to violence, and 74 percent agreeing or strongly agreeing that the program had helped to strengthen people who promote nonviolent means for resolving disputes. See Table 1 below. Table 1: Q3 - Please indicate if you Strongly Agree, Agree, Disagree or Strongly Disagree Strongly Agree Agree Disagree Strongly Disagree 23 The number of responses to this survey question (#3) varied among the 10 offered statements; the lowest response rate was 47 individuals, while the highest was 51. [PERCENT AGE] (2) [PERCENT AGE] (5) [PERCENT AGE] (3) Beirut More resistant Same level of resistance Less resistant [PERCENT AGE] (23) [PERCENT AGE] (11) [PERCENT AGE] (13) Beqaa More resistant Same level of resistance Less resistant [PERCENT AGE] (6) [PERCENT AGE] (2) [PERCENT AGE] (2) Akkar More resistant Same level of resistance Less resistant [PERCENT AGE] (13) [PERCENT AGE] (3) [PERCENT AGE] (1) Tripoli More resistant Same level of resistance Less resistant 25 with the following statements: In this(ese) region(s), LCRI helped to promote peaceful alternatives to violence. 21% 20% 39% 20% In this(ese) region(s), LCRI helped to reduce isolation of marginalized groups. 18% 26% 34% 22% In this(ese) region(s), LCRI helped to strengthen people who promote nonviolent means for resolving disputes. 19% 23% 38% 20% In this(ese) region(s), LCRI helped to reduce the influence of extremist messages that encourages violence. 14% 28% 34% 23% In this(ese) region(s), LCRI helped to empower young people to be more active in civic activities. 25% 17% 18% 40% In this(ese) region(s), LCRI helped to reduce sectarian￾based tension. 22% 24% 25% 29% In this(ese) region(s), LCRI helped to reduce host community and refugee tension. 20% 20% 33% 28% In this(ese) region(s), LCRI helped to reduce the influence of violent groups. 11% 31% 46% 11% In this(ese) region(s), LCRI helped to improve activity beneficiaries’ understanding of people from different religions. 27% 19% 26% 27% In this(ese) region(s), LCRI helped to improve activity beneficiaries’ understanding of people from different countries. 10% 27% 45% 19% Approach Toward Marginalized Groups Credited with Reduction in Violence: Awardees and beneficiaries were more positive about the program’s efforts directed toward reducing the isolation of marginalized groups (e.g., poor communities, refugee communities). Nearly one-third of survey respondents stated that the program’s strategy to support marginalized groups was the most significant for reducing host community and refugee-based violence. LCRI staff agreed, ranking it the program’s most important strategy for reducing host-community and refugee-based violence (33 percent), and its second-most important when applied to reducing sectarian violence (28 percent). Notably, LCRI staff also perceived a different option— supporting “moderate” voices who promote nonviolent dispute resolution—as another strong contributor to reducing sectarian violence (33 percent) and reducing host-community and refugee-based violence (28 percent), but awardees and beneficiaries did not agree, with only 8 percent and 16 percent, respectively, seeing it as a preferred approach. See Table 2 below. Table 2: Q6 - In your opinion, which strategy contributed the most to reducing host community and refugee-based violence? (choose one) Beirut Beqaa Akkar Tripoli Promoting peaceful alternatives to violence. 0% 9% 13% 6% Reducing isolation of marginalized groups (e.g., poor communities, refugee communities). 25% 34% 50% 19% Supporting “moderate” voices who promote nonviolent dispute resolution. 25% 14% 13% 19% Reducing the influence of extremist messages promoting violence. 50% 9% 0% 6% Involving youth in civic activities. 0% 32% 25% 31% Not possible. 0% 2% 0% 0% 26 Not applicable. 0% 0% 0% 13% Other: (please explain) 0% 0% 0% 6% Improved Understanding of People from Different Religions, Not Countries: Awardee and beneficiary survey results found improved understanding of people from different religions (i.e., Sunni, Shia, Christian), but not so for those from different countries (i.e., Syria). For example, almost half of respondents agreed or strongly agreed with the statement: “LCRI has helped to improve beneficiaries’ understanding of people from different religions.” However, more than two-thirds of respondents (64 percent) disagreed or strongly disagreed with the notion that the program improved their understanding of people from different countries, one of the highest negative response in the survey. On this question, LCRI staff were more aligned with awardees and beneficiaries on the limited success of the program in helping to improve understanding of people from different countries. Objective 2: Counter the Influence of Violent Extremist Groups Through its analysis of KIIs and survey responses, the ET found that awardees and beneficiaries were split on the impact of the program in CVE, but also that LCRI experienced some modest successes at the community level with regard to the CVE objective and sub-objectives. Two frequently cited examples of the program’s effective CVE activities follow below: • One CVE activity, “Empowering Youth to Play a Visible Role in Middle Dreib Communities” (LCRI065), was effective in targeting very sensitive areas in Akkar, first by employing an extremely detailed and nuanced beneficiary targeting strategy. that placed high emphasis on working with embedded, influential local partners to identify those who are most vulnerable to recruitment. Specifically, the organization used informants who could informally provide information on the locations where recruitment took place (e.g., street corners, religious institutions) and the residents who frequented these places. This organization then found ways to reach out to these individuals to recruit them to participate in their activities - e.g., through family or friendship ties (more information on this approach can be found in Box 4: “Toward a More Nuanced CVE Approach”). • The organization also developed a set of public events during Ramadan that engaged and empowered the youth beneficiaries as well as their respective communities. Specifically, five beneficiaries who participated in an FGD with the ET felt that the massive engagement of various groups in those activities produced a kind of “power balance,” which made opponents think twice before committing any act of coercion, intimidation, or retaliation. In other words, despite the fact that news of LCRI065’s components – e.g., five Ramadan-related public events, a communal dinner, Eid festival – provoked some Salafist groups to make threats against the 75, participating youth (“since they consider music, gathering and festivities as ‘Haram’,” said one FGD participant), the design of and support for the activity acted as enough of a bulwark against these forces to enable these events to proceed. The potential of this activity therefore derived from several key components that could be leveraged in future OTI or USAID activities: an experienced, established partner with strong local ties, collaboration and support from local moderate actors – e.g., municipal officials, community groups, the identification of particularly vulnerable youth for the project – e.g., those who were unemployed, had not finished their basic schooling, and possessed only rudimentary literacy skills – using the awardee’s aforementioned beneficiary targeting strategy. The activity was an effective example of LCRI’s ability to genuinely engage and empower vulnerable youth to play a visible, positive role in their communities, and, in this local context, at least challenge extremist actors and messaging. The activity supported both sub-objectives 2.1 (counter extremist messaging) and 2.3 (strengthen youth empowerment and civic participation). • LCRI implemented a suite of moderate actor and space-strengthening activities in Aarsal and surrounding towns, which the ET found, through KIIs with OTI and LCRI staff, as well as its document review, helped 27 these communities break down social barriers and build bridges among vulnerable groups – e.g., youth – amidst the increased isolation, influx of refugees, and extremist activity that was prompted by the start of the 2011 Syrian conflict and exacerbated by the 2014 battle of Aarsal. These actvities included the opening of the youth center and gender-mixed gym in Aarsal by veteran partner Joussour El Nour (LCRI055), which worked with OTI in the past – e.g., establishing a soccer field (under LCSI) – and would continue working with the Office later in the life of LCRI – e.g., setting up a public library (LCRI200). This youth center and gym provided a neutral space for both men and women across religious groups to participate in sports and public awareness sessions. • Similarly, LCRI’s rehabilitation of and subsequent building of a roof to cover a soccer field in Ras Baalbeck (LCRI188, LCRI258) enabled earlier LCRI-supported groups, such as the Education Scouts of Ras Baalbeck, to continue building ties through sports and arts in a non-partisan location (sub-objective 2.2: Strengthen moderate actors and increasing moderate space). The visibility and utility of these spaces served several purposes in Aarsal and the surrounding towns, providing different segments of the community with neutral locations to interact, including women and girls for the first time at the new gym, and strengthening the view of CSOs like Jossour and the Scouts as capable, moderate actors within the community. These activities were also good examples of integrating “hard” and “soft” activities, with the youth center/gym combining facilities and equipment with workshops on civic engagement and awareness sessions for youth (and free three-month memberships to those who attended at least two of these sessions); similarly, the soccer field combined the rehabilitation and roofing with sports and music training of youth from diverse backgrounds in six towns. In terms of the ET’s survey findings, Awardee and beneficiary respondents were split on LCRI’s contribution to this objective or its sub-objectives: Greater Community Resilience to Violent Extremism: A clear majority (60 percent) believes that most communities are more resistant to violent extremism than when LCRI started in 2014. But many may not be giving credit to LCRI for this increased resistance, with 57 percent of respondents disagreeing or strongly disagreeing that LCRI helped to reduce the influence of violent groups.24 See Figure 4 below. Figure 4: Q8 – In your opinion, how would you describe these communities in regard to mobilization toward - i.e., resistance to - violent extremism? 24 The ET notes that this percentage could also be the result of beneficiaries not being aware of LCRI, as many of its activities were not branded. 28 Strategy Involving Youth in Civic Activities Credited: Awardees and beneficiaries were the most positive about the program’s strategy that involved youth in civic activities. In the survey, both groups felt the “involving youth in civic activities” strategy had contributed the most in comparison to other approaches in reducing sectarian violence (47 percent) and in CVE (31 percent), and second-most important in reducing host community and refugee-based violence (28 percent). LCRI staff agreed with awardees and beneficiaries on this strategy, with staff respondents perceiving involvement of youth in civic activities as having been the second strongest strategy for reducing host-community and refugee-based violence (28 percent) and the strongest approach for CVE (33 percent). Table 4: Q7 - In your opinion, which strategy contributed the most to countering violent extremism? (choose one) Beirut Beqaa Akkar Tripoli Promoting peaceful alternatives to violence. 75% 20% 0% 24% Reducing isolation of marginalized groups (e.g., poor communities, refugee communities). 0% 24% 50% 6% Supporting “moderate” voices who promote nonviolent dispute resolution. 0% 17% 0% 29% Reducing the influence of extremist messages promoting violence. 13% 7% 0% 0% [PERCENT AGE] (5) [PERCENT AGE] (1) [PERCENT AGE] (3) Beirut More resistant Same level of resistance Less resistant [PERCENT AGE] (25) [PERCENT AGE] (9) [PERCENT AGE] (15) Beqaa More resistant Same level of resistance Less resistant [PERCENT AGE] (8) [PERCENT AGE] (0) [PERCENT AGE] (2) Akkar More resistant Same level of resistance Less resistant [PERCENT AGE] (14) [PERCENT AGE] (1) [PERCENT AGE] (4) Tripoli More resistant Same level of resistance Less resistant 29 Involving youth in civic activities. 0% 28% 50% 41% Not possible. 13% 0% 0% 0% Not applicable. 0% 2% 0% 0% Other: (please explain) 0% 2% 0% 0% Beirut Supports Different Strategies Than Other Program Regions: Geographically, respondent views on this strategy were consistent across the four surveyed regions, with those from both areas in the North (Tripoli and Akkar) and in Beqaa seeing it as the biggest contributor within their respective groups to reducing sectarian violence (67 percent, 50 percent, and 42 percent, respectively) and CVE (41 percent, 50 percent, and 28 percent, respectively) within their respondent groups (respondents from Beirut saw the “youth” strategy and “promoting peaceful alternatives to violence” strategy as equal contributors [33 percent each]). Beirut respondents also differed from the other regions in their views on the most effective strategies for reducing host community and refugee-based violence (reducing the influence of extremist messages promoting violence [50 percent]) and CVE (promoting peaceful alternatives to violence [75 percent]). For reducing host community and refugee-based violence, those from Beqaa and Akkar believed “reducing isolation of marginalized groups (e.g., poor communities, refugee communities)” was the best strategy within their respondent groups (34 percent and 50 percent, respectively), while those from Tripoli opted for involving youth in civic activities (31 percent). 30 V. SYNTHESIS AND CONCLUSIONS QUESTION #1 - DID LCRI UTILIZE AN APPROPRIATE STRATEGY TO ACHIEVE USAID/OTI’S PROGRAM GOAL IN THE COUNTRY? The LCRI’s CVE strategy overall was limited—the approach would be better described as PVE. While OTI intended to pivot from “preventing” to “countering” violent extremism under the LCRI program, this only partly took place under LCRI. Furthermore, the strategy lacked specific guidance at the tactical level on how activities were supposed to achieve the sub-objectives, such as clearer guidance on how to identify “vulnerable youth” or “moderate actors.” The LCRI strategy was effective in identifying the geographic areas for program interventions, both for mitigating sectarian tensions/host communities-refugee issues and areas in which violent extremism posed an issue. The ET thus concurs with the LCRI strategic decision not to program in the Hezbollah-controlled South., the ET also notes that continuing limited programming in Beirut could have yielded opportunities for CVE work with specific groups – e.g., prisoners, religious leaders – by building on certain successful programs with legacy partners in sensitive areas of the city where the concerns of the program were relevant (e.g., Tariq al Jdideh). However, the ET also recognizes that limited program resources compelled LCRI to make strategic decisions to direct funds elsewhere (U.S. regulations and legal constraints can also play a role in limiting programming with these audiences). Nevertheless, Beirut programming merits consideration for future programming – e.g., by USAID/Lebanon (please see ET suggestions on such programming with these audiences in the “Recommendations and Lessons Learned” section). LCRI’s strategy does not describe the organizations most appropriate to carry out the program strategy, specifically with regard to the CVE objective. Organizations with an operational base and presence in the community are more likely to succeed in implementing the CVE work and sustain some, if not most, of the outcomes and results. The program’s focus on seeking and supporting moderate actors was aligned with best practices in the field25, but there was no consistent definition of what constitutes “moderate” in the program. The selection of youth was also aligned with best practices, as they are highly susceptible to recruitment.26 However, as noted above, the specifics of ‘vulnerability’ were not well defined in the strategy. See Recommendation 1 for some potential criteria for defining vulnerability, as well as suggestions on how to work with local partners to more specifically assess and prioritize potential beneficiaries. The ET found that the LCRI strategy provided the space for regular assessment and responsiveness to windows of opportunity. The program also reacted quickly and adapted effectively to changing dynamics and exogenous factors (e.g., conflict drivers, drivers of radicalization and recruitment, changing political situation, and other constraints on the program) in high-priority areas. 25 For example, see: International Cooperation and Development (2017). STRIVE: Lessons Learned. Strengthening Resilience to Violence and Exteremism. https://rusi.org/sites/default/files/201704_rusi_strive_lessons_learned.pdf 26 For example: Chahine, Ali et al. “Situation Analysis of the Youth in Lebanon Affected by the Syrian Crisis” United Nations Interagency. April, 2014. http://www.unfpa.org.lb/Documents/Situation-Analysis-of-the-Youth-in-Lebanon-Affecte.aspx; Mercycorps. “Youth and Consequences,” 2015. https://www.mercycorps.org/sites/default/files/MercyCorps_YouthConsequencesReport_2015.pdf; Morris, Sharon. “Teenage Wasteland.” Foreign Policy, November 4, 2014. https://foreignpolicy.com/2014/11/04/teenage-wasteland/ 31 While not related explicitly to the program goal, LCRI provided important access and information to the USAID Mission and the US Embassy in Lebanon, and supported USAID/USG awareness in Washington of conditions in sensitive parts of Lebanon that few other agencies could access. QUESTION #2 - DID LCRI EFFECTIVELY IMPLEMENT ITS STRATEGY TO ACHIEVE THE INTENDED RESULTS? The program had detailed and nuanced analysis of local drivers of conflict or community tension in all areas considered in this evaluation. This nuanced understanding of the local dynamics improved geographic targeting and enhanced the probability of activity-level successes by giving project staff more opportunity to design and implement successful activities. However, a significant percentage (31 percent) of sampled LCRI activities were found to be only indirectly linked to the program’s objectives, and more than half of these focused solely on tangible deliverables, demonstrating a significant overlap in and strong connection between activities that were both focused on tangibles and less directly linked to LCRI’s objectives. The ET’s KIIs and activity assessments indicated that this development was unintentional on the part of awardees. This suggests that program planning could have focused more intensively to align activities with LCRI’s objectives. LCRI field teams lacked staff expertise or sufficient training on CVE, resulting in limited effectiveness in translating program strategy into designed activities or clear communication of objectives to awardees. Recommendation 3 includes relevant guidance. The ET found that tangibles were more likely to improve program effectiveness when combined with “soft” activities. In contrast, stand-alone tangibles analyzed by the Team as part of its activity assessment sample – e.g., LCRI238, LCRI128, even when supporting a moderate actor, had minimal and potentially short-lived results. As for standalone soft activities, the ET’s activity assessments showed that these were generally judged to be successful, including LCRI162, LCRI126, LCRI208. Interestingly, interviews with a number of interviewed LCRI and awardee staff led to the conclusion that tangibles were not necessary as “transaction costs” to build confidence with awardee organizations in new communities, although several interviewed LCRI staff argued the opposing view.27 The ET found that consecutive layering was generally successful in locations such as Aarsal and Majdal Anjar because LCRI awardees created sequences of activities here that gradually built up communities’ perceptions of supported community groups and target audiences – e.g., local youth – as credible, moderate actors in the eyes of their communities. although program objectives were more quickly achieved when the program targeted existing community-based organizations or groups with an existing operational base, such as the Majdal Anjar Cooperative, Hadatha, SGA, and SADA. Awardees, field staff, and OTI staff reported issues in the management of the program until roughly mid-2016, including IP challenges in transitioning to the OTI operational model, a lack of engagement with some awardees, and a relatively weak monitoring regime. While these issues were seen as having negatively influenced the implementation of the program in its earlier years, the ET also notes from these same interviews that LCRI implemented a number of management reforms after roughly mid-2016 to address, generally successfully, these issues. A weak monitoring regime negatively affected oversight of and reporting on LCRI’s effectiveness in the first phase of the program.. The net effect was greater difficulty in monitoring activity implementation and in 27 Please see relevant results in “Findings” section’s “Tangibles Stronger When Combined with ‘Soft’ Activities sub-section” for additional information on this conclusion. 32 tracking progress toward LCRI objectives and sub-objectives, although it was noted that its monitoring practices improved later in the life of the program The ET offers some suggestions on strengthening data collection practices in Recommendation 7. QUESTION #3 - TO WHAT EXTENT DID LCRI CONTRIBUTE TO THE ACHIEVEMENT OF PROGRAM GOALS AND OBJECTIVES? With respect to objective 1, activities such as those implemented in Qaa and Aarsal succeeded within their communities in no small part because LCRI identified and engaged with established and willing local partners who could mitigate intra- and inter-community tensions and weather crises when attempting to build bridges among traditionally antagonistic groups. Such activities were able to help LCRI demonstrate evidence of progress on objective 1, especially sub-objectives 1 (promote peaceful alternatives to violence) and 2 (reduce marginalization and isolation of community groups). Please see ET findings for Objective 1 under Findings section. Several factors detailed under the ET’s responses under evaluation questions 1 and 2, reduced the program’s ability to effectively implement activities under objective 2. These factors included the program’s less strategic focus on countering (versus preventing) violent extremism (1.2), limited guidance for program staff on CVE activity design and awardee and beneficiary selection (2.2), and related challenges for LCRI in communicating the program’s CVE strategy to awardees (2.4). Please see ET findings for Objective 2 under Findings section. Responsiveness to changing dynamics in local communities. Because OTI project staff were constantly aware of the conditions in local areas, they were well-placed to respond in the event of a crisis. For example, after the suicide bombings in Qaa, the LCRI responded by quickly engaging the community to keep projects running and to support new initiatives that helped the community feel supported in a time of crisis. Nuanced understanding of local issues. Each activity description demonstrated a detailed and well-informed understanding of the key issues/tensions in each community. This was reflected by the fact that the ET found awardee and beneficiary respondents at most activity assessments able to describe the issues in their community in similar terms as were described by activity notes. Having this understanding about the drivers of conflict and points of contention is a pre-requisite to successful programming. LCRI’s presence in conflict-affected areas. OTI was praised by awardee organizations, particularly in Aarsal, for 'being there' when no other organization or donor agency would be willing to support programs. A corollary to this was that OTI was also the 'first one there', which gave it the chance to get more return on their programs because awardees recognized their effort and interest and gave them credit for helping turn areas around. One example was the souks in Tebbaneh, Tripoli. 33 VI. RECOMMENDATIONS AND LESSONS LEARNED 1. OTI should develop clearer standards for how the program defines and utilizes key concepts related to awardee and beneficiary targeting (i.e., “vulnerability” and “moderate actor”). These definitions need not be rigid but should give implementing staff a useful framework for identifying the most relevant awardees and beneficiaries given program objectives and specific programmatic opportunities and operational constraints. Potential criteria for defining vulnerability, as well as suggestions on how to work with local partners to more specifically assess and prioritize potential beneficiaries, are found in B o x 3 . 2. O T I s h o u l d e x p l o r e s u p p o r t i ng additional activities conducted by LCRI that show strategic promise. The fact that LCRI was able to implement at least one activity with Palestinian camps, prisons, religious leaders, and former fighters suggests a basis for more strategic follow-up work, despite what the ET acknowledges are substantial practical and legal challenges to these types of activities. Suggestions on strategic engagement and creative solutions for implementing programming in these areas include: Box 3: Defining “Vulnerability” in CVE USAID’s 2011 policy on The Development Response to Violent Extremism identifies at least five “push” factors for joining violent extremist organizations, including high levels of marginalization, poorly governed or ungoverned areas, the presence of government repression/human rights violations, corruption and elite impunity, and cultural threat perceptions. This policy also identifies at least six “pull” factors, including financial motivations, a desire to improve one’s social standing, to find a sense of belonging or adventure, personal empowerment, the prospect of achieving glory and fame, and social networks (USAID, 2011). Meanwhile, scholarship has demonstrated that high male/female ratios (Hudson and Den Boer, 2002), youth bulges (Urdal, 2006), transitions from autocracy to democracy (Abadie, 2006), and limited education/employment (World Bank, 2016) are risk factors for conflict or terrorism. There is also substantial literature on individual motivations for joining groups, such as shared values (Atran, 2010), shared ‘in-group’ grievances (Stern, 2016), or the desire to overturn the status quo (Morris, 2014). The process of defining ‘vulnerability’ for future programs should focus on using criteria such as those outlined above to identify the most pressing issues in each community. Because motivations for joining violent extremist organizations manifest differently in each community, programs should work with local partners to develop a nuanced targeting of beneficiaries based on a well￾informed sense of the community-level dynamics regarding radicalization. Diagnostic tools, e.g., checklists based on risk factors such as those above, can also be developed or adapted to help facilitate and operationalize analysis in the field to identify these factors and guide appropriate program strategies. While discretion should be given to local partners to determine how the above factors apply in their particular programmatic and operational contexts, creating overall methodological “guardrails” using these factors will help guide local partners in their identification and ongoing assessments of potential and current beneficiaries, and appropriate CVE strategies for their specific environments. 34 Table 2. Recommended Areas for Future CVE Programming in Lebanon Issue Area Creative Solutions for Future Programming LCRI Program Examples (Activity # - Title) Rationale for Programming Work in Palestinian Camps Rather than work in Palestinian camps, future activities could take place outside camps, but include strategic targeting of vulnerable communities within the camps as part of the program’s direct beneficiary base. LCRI119 - Bringing Lebanese and Refugee Youth Together Through the Arts LCRI213 - Engaging Lebanese and Refugee Youth in Positive Messaging through Arts SMEs and work on Lebanon suggests Palestinian camps may be the most fruitful place for recruitment by radical groups. Work in Prisons Rather than work in prisons, which is challenging due to onerous permission requirements, future activities could target families of prisoners. These families are promising future beneficiaries because they often harbor resentments against state and law enforcement institutions and can be an easy target for recruitment by violent groups. LCRI082 - Fostering Critical Thinking Skills Among Vulnerable Youth in Tarik Jdideh (e.g. through training) LCRI189 - Engaging Tariq Jdideh Youth in Skills Training & Neighborhood Revitalization Family members of prisoners can be vulnerable to recruitment due to resentments against the state for possible imprisonment without due process, or to unexpected income loss if the imprisoned family member is a breadwinner. Work with Religious Leaders Future programming could consider working with religious leaders with a strong local constituency to counter violent extremist recruitment taking place in local mosques. These leaders could be considered moderate actors under the program’s current strategic framework. Note: Lebanese religious institutions, such as Dar al Fatwa, are generally not influential at the community level.28 LCRI064 - Engaging Religious Leaders in the North LCRI266 - Sustainable Network for Moderate Leaders in North Lebanon Community-based religious leaders with strong links to constituencies can serve as a bulwark against radical preachers recruiting in their mosques. 28 Henley, Alexander. “Religious Authority and Sectarianism in Lebanon.” Carnegie Middle East Center. December 16, 2016. http://carnegieendowment.org/2016/12/16/religious-authority-and-sectarianism-in-lebanon-pub-66487 35 Work with Former Fighters Rather than work with former fighters from Syria, future programming could consider engaging non-violent advocates among former combatants in the Lebanese Civil War. Since Lebanon does not yet have a formal demobilization, disarmament, and reintegration (DDR) program for former combatants returning from fighting in Syria,29 programming cannot engage with former fighters from Syria. LCRI286 - Promoting constructive collaboration between rival communities Literature on peacebuilding and conflict management, as well as KIIs & FGDs with awardees & beneficiaries in Lebanon, indicate that former combatants of Syria’s conflict or Lebanon’s civil war can be good advocates against recruitment to armed groups. 3. OTI should support or encourage training for IP staff on how to socialize sensitive concepts with awardees, such as program objectives related to CVE, as well as how and when to communicate those objectives with beneficiaries. Relatedly, consider more training by CVE experts for staff and more expert involvement in: 1) designing CVE-specific activities; 2) selecting and approaching suitable partners; and 3) identifying and targeting vulnerable/at-risk populations. Finally, develop a more informed, systematic and standardized approach to working with awardee organizations on CVE. such an approach if implemented discretely could have several benefits, including greater consistency in awardee selection, improved beneficiary targeting and outreach, and a common understanding of what the activity and program are seeking to accomplish (also see suggested CVE guidance in Box 4 below). 29 One KII with a SME indicates that the Swiss and a locally-based research institute are working with Prime Minister Saad Hariri to develop a more holistic strategy for addressing this issue that may include DDR-type components. Currently, the Lebanese government prosecutes returning fighters who fought with anti-Assad forces in Syria as criminals. 36 Comments on Box 4: 4. To improve CVE results, IPs should target organizations that are established and locally connected. The absence of local partners with these characteristics raises the cost of implementing sensitive programming because training and building relationships with new partners on sensitive issues like CVE requires: a) Training them on running such a specialized program; and b) Time to build trust in order to more clearly communicate intentions. Therefore, work in areas without capable local partners should have a strong justification (e.g., high geographic priority). The ET acknowledges the importance of diversifying partners in certain areas to avoid the concentration of power or influence in the hands of just a few partners that could then be exploited for purposes that are not in alignment with program objectives, but still recommends that such diversification take place among established, locally connected partners where possible. Box 4: Toward a More Nuanced CVE Approach Careful awardee and beneficiary selection is crucial to successful CVE programming. Based on activity assessments and KIIs with two awardee organizations whose beneficiary targeting were found to be highly advanced, this inset recommends some effective strategies for awardee and beneficiary targeting: • Consider Greater Emphasis on Established, Local Organizations. Established organizations that are local to the community are more likely able to achieve programmatic success because they are: 1. Already in a potential position of influence, either through their work or their leadership; and 2. Already have basic operational capacity, which means the program can devote more attention to implementing a deliberate, local, CVE strategy straightaway. NOTE: local influence and capacity can be developed with new organizations but will cost more in terms of time and attention away from directly achieving CVE objectives. • Ensure Community Embeddedness. Organizations that are best equipped for effective beneficiary targeting are those with deep local ties. This embeddedness allows organizations to a) socialize sensitive topics with participants, b) solicit support from community influencers in the event of possible challenges or threats during activities, and c) gain the trust of participants and their families, a particularly important factor for targeting female participation in conservative communities. • Emphasize Nuanced Beneficiary Targeting. Programs should place high emphasis on working with embedded, influential local partners to identify those who are most vulnerable to recruitment. One awardee organization used informants who could informally provide information on the locations where recruitment took place (e.g., street corners, religious institutions) and the residents who frequented these places. This organization then found ways to reach out to these individuals to recruit them to participate in their activities (e.g., through family or friendship ties). • Build Diversity in Beneficiaries. Beneficiary targeting need not exclusively recruit from among highly vulnerable individuals even when working with such individuals is part of the program’s objective. Creating a diverse beneficiary pool allows for the opportunity for peer-to-peer relationships to form that can dissuade joining extremist organizations. One awardee organization pursued this strategy, recruiting resilient individuals to their program so that they could provide role models for the truly vulnerable in the program. These resilient participants also provided the organization with informal information on the progress of the vulnerable individuals through the life of the activity. 37 5. OTI and IPs should avoid the use of programs that only involve tangibles without pairing them with “soft” activities that directly relate to the program objectives. Justifications among IP field staff that tangibles were a necessary “entry cost” to start working in a community only raise the cost of the program doing business everywhere without a commensurate return, as other beneficiary communities also eventually came to expect similar tangibles (e.g., solar lights). 6. OTI should continue to emphasize rapid responsiveness of program activities in high priority areas for program objectives. Being the first to implement programs in a conflict-affected community improved LCRI’s reputation and yielded high returns on minimal program investments. 7. To help address program management issues identified under Evaluation Question #2 (see Finding 2.6) future OTI programming in Lebanon (or similar country environments) should establish a more robust oversight plan and regular schedule of visits to awardees and project sites for all field and other relevant staff. Such “ground truthing,” while not a panacea for identifying issues with staff members, could increase the chances of discovering any problems with staff or the activities under their purview earlier. Additionally, any delays in activity implementation related to staff turnover could be avoided or minimized through improved oversight of handover notes and a more structured final handover meeting to ensure smoother staff transitions. 8. OTI and IPs should design and enforce a more consistent and tangible M&E system that would provide more feedback on what worked and what did not, especially to encourage active learning in the emerging CVE field (see also Box 5). 38 Box 5: Strengthening OTI Data Collection for Impact Future programs should consider experimenting with more innovative data collection efforts that could provide staff with more objective, consistent, dynamic and verifiable indicators of program success. Interviews with IP staff found high demand for this type of research: few field staff reported using the research collected for LCRI. M&E staff reported that they only referred to the research conducted for the program at the end of an activity to contextualize its impact. Meanwhile, OTI country representatives and OTI-Washington found the research and reporting helpful, but the reporting could not be contextualized over time for proper trend analysis. As interagency stakeholders lauded the program’s access and information in sensitive parts of Lebanon as one of its chief successes, there should be great interest in continuing to improve knowledge collection and dissemination in future programs. Carefully crafted research can better utilize knowledge extant within IP staff and deploy innovative research methods that can both continue to meet the demand for information from OTI and its interagency partners, and meet the need among field staff and senior management for better indicators to track their program’s success. To improve research, future programs could consider two types of data collection improvements: 1. Strengthen current collection methods. Ensure that staff involved in the drafting of political timelines are also recording this information in tabular format, which will allow a kind of event data analysis that can help identify trends or activities of key actors over time. 2. Consider new data sources with a focus on objective, observable data. Objective data, or data that are based on observable behaviors, when combined with subjective, opinion-based data can provide much more insight about local conditions than opinion research on its own. Such data might include: regularly collected micro-economic price data in key target areas, tracking the number of who had left and/or died in Syria, or conducting sentiment analysis on transcripts of focus group discussions to get top-line results quickly. 39 ANNEX I: EVALUATION STATEMENT OF WORK BACKGROUND Political and Security Context Lebanon is facing a persistent complex crisis that undermines its fragile stability. The Syrian conflict, now in its sixth year, continues to spill across the country’s permeable borders. It has brought more than 1.2 million refugees to Lebanon, along with foreign extremist fighters. As a result, Lebanon is in a precarious situation politically, socially, and economically. The Government of Lebanon’s paralysis due to a long Presidential vacuum and subsequent political infighting has been detrimental to its ability to effectively manage the refugee crisis and recruitment and radicalization of youth. The Syrian conflict has exacerbated Lebanese sectarian tensions as groups side with different parties engaged in the conflict and refugees place stress on already limited resources and services. While tensions initially manifested between host communities and refugees, the proliferation of militant and extremist groups seeking to exploit sectarian grievances has resulted in a growing internal crisis. Sectarian grievances are producing community-level violence and polarization, particularly where militants have gained an active foothold. The instability has fueled an acute perception of existential threat to communities’ social, economic, political and security well-being, creating fertile ground for the indoctrination and recruitment of youth into extremist groups. Lebanon Community Resilience Initiative OTI has been in Lebanon since 2007. The proposed evaluation focuses on the Lebanon Community Resilience Initiative, which began in August, 2014, and will run through September 2017. The goal of USAID/OTI’s Lebanon Community Resilience Initiative (LCRI) is to strengthen the resiliency of Lebanon's most vulnerable communities in the face of two critical destabilizing factors: sectarian and host community-refugee tensions and susceptibility to violent extremism. In doing so, the program seeks to contribute to U.S. foreign policy interests that Lebanon remains a sovereign, stable, democratic, and independent nation at peace with its neighbors. The program has two primary objectives and three sub￾objectives: 3. Mitigate Rising Sectarian and Host Community-Refugee Tensions o Promote peaceful alternatives to violence o Reduce marginalization and isolation of community groups o Strengthen moderate actors 4. Counter the Influence of Violent Extremist Groups o Counter extremist messaging o Strengthen moderate actors and increase moderate space o Strengthen youth empowerment and civic participation The two primary objectives are mutually reinforcing and addressed through clusters of activities that promote each sub-objective. LCRI focuses heavily on youth aged 15-29, the age group most targeted by extremist messaging and recruitment efforts and manipulated to fight on behalf of political and sectarian groups. LCRI includes girls and women in most activities, challenging prevailing gender norms and addressing the overlooked issue of female marginalization and vulnerability to radicalization. Moderate actors are also targeted through the program, particularly in relation to youth who are disillusioned with political and sectarian manipulation and in areas infilitrated by extremist groups or facing the strain of Syrian refugees. 40 Program-level Theories of Change The goal of the Lebanon Community Resilience Initiative (LCRI) is to strengthen Lebanese host communities’ resiliency in the face of destabilizing factors, the most critical being sectarian and host community-refugee tensions and susceptibility to increasing violent extremism. Through this goal, the program seeks to directly contribute to U.S. foreign policy interests that Lebanon remains a sovereign, stable, democratic, and independent nation at peace with its neighbors. The program objectives of mitigating sectarian and host community-refugee tensions and countering the influence of violent extremist groups are mutually reinforcing and will be achieved through multiple clusters of activities, which aim to promote peaceful alternatives to violence, reduce the marginalization and isolation of community groups, strengthen youth empowerment and civic participation, increase moderate space and strengthen moderate actors, and counter extremist messaging. The program design rests on the assumptions that there will be some level of security and stability in communities in order for these activities to achieve the desired impact. Well-designed and targeted activities are expected to have a catalytic effect and contribute to increased resilience. However, the sustainability of stabilization and CVE activities will be limited as long as the Syrian conflict continues and, despite best intentions and well-designed activities, communities may not be able to withstand overwhelming force. Program Objective I: Mitigate Rising Sectarian and Host Community-Refugee Tensions • Sub-Objective: Promote peaceful alternatives to violence: The growing fragmentation and polarization of Lebanese sectarian groups and the increasing strain on host communities is creating heightened insecurity between various groups such that their reactions to each other are increasingly violent. Violence is also being used more often as a means to resolve community-based problems. Activities within this sub-objective seek to increase positive interaction between different community groups and provide non-violent methods and skills for resolving grievances. Theory of Change: If youth and communities are given the skills and opportunities to interact positively, then they will experience a mutual positive benefit from (non-violent) interaction and will be more likely to cooperate because the current security and political instability created by the Syrian crisis is triggering latent historical fears and prejudices within communities as well as instigating context-specific fears and prejudices which is increasing polarization, discrimination and negative narratives. • Sub-Objective: Reduce isolation and marginalization of community groups: Communities are becoming increasingly polarized, particularly along sectarian lines and between host communities and refugees. The association of specific sectarian groups with violent extremism has resulted in corresponding narratives, which exacerbates a sense of insecurity and perceptions of isolation and marginalization both within communities and between communities on a local and national level, leading to violence and in some cases alignment with extremist groups. In the case of Aarsal specifically, isolation is an actual physical separation from surrounding communities. Activities within this sub-objective seek to address both real and perceived isolation and marginalization of community groups, including providing alternatives to narratives which stereotype and discriminate specific groups and their communities. Theory of Change: If information discrediting stereotypes and providing positive alternative narratives is disseminated or if opportunities for dialogue and positive interaction within groups or between groups are created, then current stereotypes and perceptions dividing the Lebanese social fabric and communities will be mitigated, reducing mistrust within and between communities and geographical areas, because the 41 negative perceptions are not based on facts or traditional perceptions but on misinformation, rumor and rising mistrust. • Sub-Objective: Strengthen Moderate Actors: The dramatic influx of refugees has placed considerable strain on host communities, the majority of which were already marginalized and disadvantaged. The Lebanese Government has been ill-equipped to manage the refugee crisis and the UN agencies are severely underfunded to address the needs, creating competition over resources between refugees and Lebanese. In other communities, the increase in sectarian tensions has destabilized communities with increased tension and violent conflict. In both cases, moderate actors, including government actors, can play a critical stabilizing role in communities by proactively addressing points of tension to prevent groups and individuals from resolving issues on their own, which is often done through violence or marginalization. Theory of Change: If moderate leaders are strengthened and perceived as effective, then they will be better able to maintain community stability because communities will turn to them to resolve grievances rather than resort to violence and conflict. Program Objective II: Counter the Influence of Violent Extremist Groups • Sub-Objective: Counter extremist messaging: Well-funded and organized, extremist groups are using virtual and physical channels to recruit youth, promoting violent extremism as a way to address dissatisfaction and disillusionment. Activities within this sub-objective seek to provide alternative narratives to violent extremist rhetoric through media (traditional and social media) as well as through tangible activities. Theory of Change: If extremist propaganda is countered by exposing the exploitation of recruits or misleading narratives, then youth will be less likely to join these groups as an option for creating a positive future because youth being recruited do not believe that they have the ability to create positive change, and the type of future extremists groups can provide (including a highly controlled lifestyle and top-down driven society) would not be desirable to youth. • Sub-Objective: Increase moderate space and strengthen moderate actors: The growing presence of violent extremist groups is shrinking tolerant, open and diverse space in which people can freely associate and express themselves. In some cases, it is resulting in the imposition of restrictive measures on people’s daily lives and limiting their self-determination. Simultaneously, communities are increasingly doubtful of civil society organizations, which conduct assessments and make promises but do not always deliver, as well as political actors who seek to impose their own agenda on communities. Weak or non-existent moderate leaders (defined as nonviolent, tolerant and neutral) provide an opportunity for extremist groups to co-opt and control space. Activities within this sub-objective seek to strengthen the presence of moderate leadership and increase moderate space, particularly after extremist groups have been pushed out, to provide alternative leadership and demonstrate the benefits of moderate and open community space. Theory of Change: If moderate space is reclaimed and moderate actors are strengthened, then people will choose and protect moderate space/actors because (a) moderate space provides greater social, cultural, and economic benefits than extremist space and (b) moderate actors are more representative of community interests and aspirations than extremist groups. • Sub-Objective: Strengthen youth empowerment and civic participation: Well-funded and organized, extremist groups are actively recruiting youth, often exploiting a number of vulnerabilities and grievances, 42 including a sense of disillusionment and lack of belonging. This sub-objective seeks to counter this influence and manipulation by reducing youth vulnerability to recruitment and providing positive opportunities for youth participation in various aspects of their community, including decision-making, problem solving, positive messaging, conflict mitigation, and leadership. Theory of Change: If youth are empowered to play a positive role in their communities, then they will be less vulnerable to recruitment because they are joining these groups due to social, economic and political marginalization and a belief that they cannot create positive change. This theory of change assumes that addressing one or two drivers will be sufficient, in some cases, to prevent youth recruitment. Activities LCRI has implemented nearly 250 activities with over 100 local partners since it began in 2014. Partners are closely tied to the communities in which they operate, and include youth groups, CSOs, NGOs, cooperatives, and local government entitites. LCRI field staff work with these partners to develop a diverse set of activities— from small infrastructure to theater to capacity development—to meet sub-objectives. While each activity is unique, many follow similar approaches, especially within clusters. Geographic Areas Geographic areas of focus for the LCRI program include Tripoli, Akkar, the Beqaa and Beirut. LCRI prioritizes areas based on active extremist recruitment; increased levels of community tension directly resulting from the Syrian crisis with potential to manifest violently; and where outbreaks of violence may have significant subsequent effects and pose a viable threat to the stability of population centers and economic corridors. Specific geographic areas include: • North: Conflict in the region includes ongoing incidents of violence and clashes in and around Tripoli. A continued source of conflict is tension between Sunni and Alawite communities in the Bab el Tebbaneh/Jabal Mohsen districts of Tripoli, coupled with extremist recruitment in these areas and surrounding neighborhoods. Violent extremist groups are also active in pockets of Akkar, particularly as fighting on the border increases. Strengthening moderate actors, increasing moderate space, empowering youth, and decreasing sectarian tensions and marginalization are the highest priorities. Specific clusters in the North include: • Tripoli’s Neighborhoods: o Tebbaneh, Qoubbe, Jabal Mohsen, Mina outskirts, Qalamoun, Zahrieh and Old City o Abou Samra • Akkar: o Sahel Akkar, Wadi Khaled, Middle Dreib, and Fneideq • Beqaa: The Beqaa hosts the highest number of Syrian refugees in Lebanon, mostly living in confessionally homogenous communities. Tensions exist between host communities and refugee populations due to intense strain on limited resources, and are evidenced by clashes in Baalbeck, Aarsal and in central Beqaa. Intra-Lebanese sectarian tensions have also escalated along Sunni-Shiite fault lines as a result of the Syrian conflict, with most Sunni communities siding with the Syrian opposition while most Shiite communities support Hezbollah’s intervention alongside the Syrian regime. Christian communities feel increasingly threatened by the presence of militant groups and are self-arming for protection. The reverberations of the 2014 battle of Aarsal continue to be felt, leading to intense isolation of the Sunni-majority town and tense relationships with the neighboring Shiite town of Laboueh. There are also increasing pockets in central and west Beqaa in which violent extremist groups have grown their presence and recruitment efforts. Specific clusters in the Beqaa include: • North Beqaa: 43 o Aarsal and Laboueh o Baalbeck o Hermel, Ras Baalbeck and Qaa • Central and West Beqaa o Bar Elias, Qab Elias and Jeb Jennine o Saadnayel, Taalabaya and Jalala o Majdel Anjar • Beirut: Beirut and its suburbs host the second highest number of refugees in Lebanon. The majority of these refugees are located in areas which are traditional poverty pockets in Beirut and historically neglected. The influx of refugees is straining already limited resources and adding to the mix of sectarian tensions, creating fertile ground for extremist recruitment amongst an increasingly vulnerable youth population. LCRI conducted a 6-month pilot, now finished, in Beirut to address increased sectarian tensions and youth vulnerability to extremist recruitment and strategic host community￾refugee issues to mitigate outbreaks of violence. Specific clusters in Beirut included: o Tariq Jdideh, Mazraa and Bashoura OTI Approach to Transition and Stabilization Programming The Office of Transition Initiatives (OTI) is part of the Bureau for Democracy, Conflict and Humanitarian Assistance (DCHA) at USAID and aims to "support U.S. foreign policy objectives by helping local partners advance peace and democracy in priority countries in crisis. Seizing critical windows of opportunity, USAID/OTI works on the ground to provide fast, flexible, short-term assistance targeted at key political transition and stabilization needs.” The overall goal of any USAID/OTI program is to support a peaceful, democratic transition. Transition programs, however, cannot reasonably be expected to make an impact on all the complex dynamics present within a transition. USAID/OTI, therefore, develops a program rationale that articulates how the program’s actions, both activities and processes, are expected to achieve program objectives and, more broadly, contribute to the transition. Program objectives are often built around a theory of change that provides a logical explanation of how a chosen intervention or set of interventions will achieve a desired outcome and why. Based on this theory of change, USAID/OTI is then able to evaluate whether individual activities contribute to the program’s objectives and/or whether the program objectives contribute to the strategic goals of the transition. That said, initial program objectives and activities can, and likely will, shift throughout the life of the program based on ongoing, iterative assessments and improved understanding of the political dynamics of the transition. USAID/OTI primarily implements its programming through small, quick-impact activities that are intended to address specific opportunities or challenges based on the program’s theory of change. By using small activities, DCHA/OTI is able to take small, calculated risks on activities and then use ongoing monitoring and evaluation to assess impact and then adapt programming accordingly. USAID/OTI, therefore, is less concerned with the outputs from individual activities and more concerned with generating positive outcomes to support a transition. PURPOSE The purpose of this activity is to conduct an independent final performance evaluation of USAID/OTI’s Lebanon Community Resilience Initiative. The evaluation will assess the extent to which USAID/OTI’s program achieved its intended results and will identify where the opportunities are greatest for USG actors to continue building on OTI’s program strategy and approach. 44 The evaluation will result in a product that concisely explains program achievements, provides recommendations to key evaluation users, and identifies lessons learned. Where applicable, it will employ a policy-oriented lens, particularly on the program’s relevance to countering violent extremism (CVE) policy. Recommendations and lessons learned should focus on how USG actors in Lebanon can build on the approaches, partnerships, and geographic areas that are most effective and relevant in OTI programming; relevance to US CVE policy; and implications for USAID/OTI’s programming in other countries. The final product will be used to inform the design and implementation of future USG assistance in Lebanon and will be shared with the USAID/Lebanon mission and the U.S. Embassy in Beirut. A public version will be shared with local stakeholders and other interested groups and will be made available via the USAID Development Experience Clearinghouse (DEC). It will also be used to inform USAID/OTI programming in other contexts, and as a reference for the interagency on CVE programming in transition environments. EVALUATION QUESTIONS The report shall seek to answer the following evaluation questions: 1) Did LCRI utilize an appropriate strategy to achieve USAID/OTI’s program goal in the country? (LOE 20%) In considering this question, evaluators should examine the logic connecting the program's primary objectives, theories of change and chosen intervention(s) to achieve the desired outcome(s). Evaluators should also consider the identification of relevant geographic areas, partners, and beneficiary groups; responsiveness to windows of opportunity and the effect of exogenous factors (e.g., conflict drivers, drivers of radicalization and recruitment, changing political situation, and other constraints on the program); and USAID/OTI’s role within the larger USG assistance portfolio. 2) Did LCRI effectively implement its strategy to achieve the intended results? (LOE 40%) In considering this question, evaluators should address whether LCRI implemented pertinent activities/actions toward the achievement of identified sub-objectives. Evaluators should also consider what approaches and practices were effective in achieving successful results and which were not. Approaches and practices may include but are not limited to partner selection, stakeholder partnerships and creation of linkages, beneficiary targeting, geographic targeting, accessibility, procurement mechanisms, and use of tangibles, among others. The evaluation will not focus on activity-level effects, but rather the additive effects of activities toward the achievement of program sub-objectives. 3) To what extent did LCRI contribute to the achievement of program goals and objectives? (LOE 40%) Due to the lack of baseline data or a sound counterfactual, determining the impact of the program on its objectives will be difficult. However, given the available data and determinations in (1) and (2), the evaluators should assess whether goal, objective, and cluster-level change occurred and the extent to which USAID/OTI plausibly contributed to those changes; and, the significance of those changes to the instability OTI endeavored to address and to USG assistance priorities in the country. METHODOLOGY This evaluation will be non-experimental and largely qualitative in nature, but mixed methods may be used as appropriate. In answering the evaluation questions, the contractor shall utilize data that is disaggregated and 45 analyzed by sex, age, location, and Lebanese/refugee whenever such data is available and appropriate. Methodological specifics will be agreed upon among the evaluators, USAID/OTI-Lebanon, and OTI￾Washington; the evaluators are encouraged to suggest creative approaches that will satisfy the purpose, goals, and lines of inquiry of the evaluation. USAID/OTI’s activity database is a rich source of information on individual projects and should be utilized by the evaluation team. Possible methods for the evaluation include: • Facilitated workshop with key program staff to reflect on program implementation, challenges and successes; • Field visits to the implementation areas; • Interviews with key program stakeholders, including U.S. Embassy and USAID staff, community leaders, awardees and beneficiaries; • Focus group discussions and interviews with beneficiaries, awardees, and others; • Direct observation; • Surveys; • Documentation review, e.g., quarterly and annual reports, reports from Strategy Review Sessions (SRSs), Rolling Assessments, and Program Performance Reviews (PPRs), perception surveys and other existing data, and USAID/OTI-Lebanon’s activity database. TEAM COMPOSITION The evaluation team should be comprised per the specifications described below. The positions to be filled by Social Impact require USAID/OTI concurrence of the proposed candidates. • One senior evaluation specialist with research and/or evaluation experience in political transition programming. The specialist should have extensive experience related to and a deep understanding of the Lebanon and/or Middle East context. The senior evaluation specialist shall be designated the evaluation team leader and will be responsible for liaising with USAID/OTI, the preparation of all deliverables, and leading briefings with relevant stakeholders. • One senior policy expert with significant experience in policy and/or research focused on countering violent extremism (CVE). The policy expert should have a deep understanding of current evidence on the drivers of CVE and sources of resilience to radicalization and extremism, recognizing the localized nature of the same. The expert will work with the senior evaluation to set the evaluation methodology, prepare deliverables, and brief relevant stakeholders. • One mid-level evaluation specialist with research and/or evaluation experience in complex political and/or social change. This evaluation specialist should be a host country national and have local research and/or evaluation experience relevant to the themes or methods of the evaluation. Knowledge of USAID/OTI-type programming is preferred. The mid-level evaluation specialist will support the team leader and participate in the field review, interviews, the draft and final evaluation reports, and debriefs in the field only. • One administrative coordinator, locally-hired to provide administrative and logisitical support for meetings as well as interpretation and translation as needed. TIMELINE and LEVEL OF EFFORT (LOE) 46 The field work of the evaluation will take place over approximately three weeks. The team will travel to Lebanon on/about April 9, 2017 and depart on/about May 3, 2017. In addition, the team will have a total of two weeks of LOE in advance of departure for reading and some Washington-based interviews in March and April. This activity will also include three weeks of LOE for writing and completion of deliverables. Dates* Task LOE March 23 – April 5 Reading on Lebanon context. Key materials to be provided by OTI. Key Washington-based interviews. 10 Days April 6 – April 7 Kick-off meetings/calls with OTI team to refine evaluation questions, methodology, travel and logistics; database training. 2 Days April 9 Travel to Lebanon 1 Day April 10 - May 3 Field-based interviews and analysis, debriefs to senior management, staff and U.S. mission as appropriate. 22 days May 4 Travel from Lebanon 1 Day May 7-15 Report writing; submit first draft to OTI within 10 days of return to US 7 Days May 16-22 Receive comments from OTI within one week of delivery of first draft. Submit final report within 7 days of receipt of comments. 5 Days May 23 Travel to Lebanon 1 day May 24 Debrief with OTI, USAID/Lebanon 1 day May 25 Travel from Lebanon 1 day May 29-May 30 Debriefs with OTI, MSI and USG interagency as determined by OTI. 2 Days May 31-June 1 Presentation at public event, as determined by OTI. 2 Days 47 TOTAL: 55 days LOE * All dates listed above are illustrative and subject to change. A six-day work week is approved during the field work for this activity. TASKS and DELIVERABLES The evaluation team, under the direction of the team leader, is responsible for completing the following Tasks and submitting the following Deliverables: a) Evaluation Plan. An evaluation plan that includes: staffing determinations, timeline and methodology, including which methods or approaches would be used to address the specific constraints above in addition to others identified by the evaluation team. The plan will also include a draft schedule and potential interviewee list for field visits. Any substantial changes to the evaluation plan after approval will need to be consulted with and approved by OTI. Due within 3-4 weeks of award. b) Data Collection Tools: The assessment team will prepare and submit all the tools used for data collection to USAID/OTI for review and concurrence. c) Progress reports. Weekly in person or via email updates with the Evaluation Manager and monthly 1-page progress reports. d) Draft evaluation report. The draft evaluation report should include findings, conclusions, recommendations, and key lessons learned from each cluster. OTI will provide feedback on this report to the evaluation team. Due within two weeks of the second (final) field visit. e) Second draft report. Second draft report, incorporating (as appropriate) feedback from OTI on the first draft report. Due within two weeks of receiving OTI’s final feedback. f) Final conclusions presentation with Executive Summary with key findings (5-page max). A final PowerPoint presentation summarizing key findings, conclusions and recommendations, to be presented by the Senior Evaluation Expert to the primary users of the evaluation reports in up to four separate presentations (e.g., OTI-Lebanon, MSI, OTI-DC). To be scheduled as the report is being finalized, but within three weeks of receiving OTI’s final feedback. Up to four presentations on the final report will be given to USAID/OTI, implementing partners, and/or key stakeholders as requested by USAID/OTI. g) Complete Final Report (not to exceed 30 pages) excluding annexes but including findings, conclusions, and recommendations for OTI programming more broadly. Including, as well, any photos of activity sites submitted (electronically). Due within two weeks of final presentation. All deliverables will require USAID/OTI concurrence prior to their finalization. The final public version of this evaluation report will be posted on USAID’s Development Exchange Clearinghouse (DEC) website by Social Impact. Social Impact will be responsible for the following logistical matters with respect to the evaluation team engaged in this activity: • Social Impact is responsible for all scheduling, arranging meeting locations, and confirming times/dates/locations with the team and interviewees and awardees. USAID/OTI and implementing partner Management Services International, Inc. (MSI) will provide references and can assist with scheduling as needed. When the evaluators are in country, MSI will make available a conference room 48 in its offices for the evaluators to use in Beirut. However, meetings at partner organizations or alternative locations are preferable. • USAID/OTI will facilitate access to the U.S. Embassy for meetings with other USAID and Embassy officials and will arrange meeting space in the Embassy. • Social Impact shall arrange and purchase all international and U.S. travel. TOCOR concurrence is required for all international travel not included in the original activity budget; • Social Impact shall provide per diem (lodging and M&IE) for the evaluation team in Lebanon; • Social Impact shall fund in-country air travel and ground transportation; • Social Impact will obtain visas;30 and • Social Impact and the evaluation team will work with USAID/OTI and implementing partners to arrange interviews in Washington, DC and in Lebanon. REPORT STRUCTURE The evaluation report should adhere to USAID guidelines and be structured as follows: 1. Cover Page with photo 2. List of Acronyms 3. Table of Contents, which identifies page numbers for the major content areas of the report. 4. Executive Summary (2 to 3 pages): should be a clear and concise stand-alone document that gives readers the essential content of the evaluation report, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. Thus, the Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of objectives and intended outcomes; areas of meaningful under- or over-achievement. 5. Methodology: Describe the data collection methods used including strengths and weaknesses, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken. This section should also address constraints and limitations of the evaluation process and rigor, including what can and cannot be concluded from the evaluation. All actual or potential conflicts of interests among members of the evaluation team should be noted in this section. In accordance with ADS 203, the report should also state why a performance evaluation was conducted in lieu of an impact evaluation. 6. Findings: The evaluation team should determine how best to organize this section based on the evaluation questions. 7. Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to discuss the data and results, pull out lessons learned for USG Lebanon programming and for USAID/OTI, and generally comment on data and results. 8. Annexes: List of stakeholders interviewed with number and type of interactions; the scope of work; data collection protocols developed and used; any data sets can be provided in electronic format; other special documentation needed. POINT OF CONTACT 30 MSI is able to assist in arranging business visas for the evaluation team, if required. 49 The USAID/OTI-Lebanon Program Manager (otilebanon@usaid.gov) in Washington will serve as the point of contact for overall coordination of the Washington meetings. There will be some occasions where s/he arranges the meeting and location and others where s/he may provide contact information for the consultants to arrange their own logistics. While in Lebanon, the USAID/OTI team in Beirut will identify a point of contact for in-country logistics and scheduling. The TOCOR will remain the point of contact for all technical direction and requests requiring formal concurrence. ANNEX II: DATA COLLECTION PROTOCOLS Key Informant Interview Guide U.S. GOVERNMENT (USG) USAID, U.S. DEPT OF STATE Estimated Length of Interview = 1 hour DATE OF INTERVIEW: NAME OF EVALUATION TEAM MBR: TRANSLATOR: RESPONDENT NAME: TITLE/POSITION: ORGANIZATION: LOCATION: REGION: TELEPHONE/EMAIL: SEX: RELIGION: OVER 29 YEARS? ______ INTRO: Thank you for meeting with us as part of the final evaluation of the Lebanon Community Resilience Initiative (LCRI). As you may know, Social Impact has been contracted by LCRI to conduct this third-party evaluation prior to the end of the project. The evaluation focuses on program performance, strategic approach, and impacts that LCRI has had on social cohesion and countering violent extremism in Lebanon. Your input will support OTI’s assessment of past activities and provide Lessons Learned for any future engagement in Lebanon. The information you provide will be confidential in that anything you share with us will not be attributed to you personally or your organization. All data collected will be evaluated in aggregate. However, we’d like to list you with the other people interviewed for this evaluation; would this be acceptable for you?  [RESPONDENT BACKGROUND ON LCRI] To what extent are you involved with or familiar with the Lebanon Community Resilience Program? [NOTE: For respondents without direct involvement with LCRI, ask about their length of involvement in work related to Lebanon generally and CVE specifically. Adapt focus of following questions based on timeframe of involvement with LCRI and specific role.] 50 [RESPONDENT’S ROLE] What is (or has been) your role with LCRI or in relation to Lebanon and CVE policy or research? Relevance of Objectives to Conflict Drivers 1. To what extent does sectarian tension lead to violent conflict in the four targeted areas? How has this issue evolved from 2014 to present? How has USG policy on this issue evolved? 2. To what extent does host-refugee tension lead to violent conflict in the four targeted areas? How did the program address conflicts arising from competition for resources? How has this issue evolved from 2014 to present? How has USG policy on this issue evolved? 3. To what extent does violent extremism lead to violent conflict in the four targeted areas? How has this issue evolved from 2014 to present? How has USG policy on this issue evolved? 4. To what extent do other dynamics lead to violent conflict in the four targeted areas? What are these dynamics? Are there any blind spots in the areas that your program considers for programming, in your opinion? [NOTE: Probe to be clear whether these dynamics were addressed by LCRI.] 5. There were some major events in and around Lebanon that might have catalysed confessional violence or galvanized recruitment by violent extremist groups: • Government vacuum, • Trash protests in Beirut, • Russian intervention into the Syrian conflict, • Battles around Homs and Aleppo. Can you describe the effects of these events on the conditions above? Are there others that should be considered? How should LCRI have taken into account these or other factors? Knowledge of LCRI [NOTE: If respondent is more familiar with LCRI, then adapt the question to probe about the project’s specific sub￾objectives. They are: Sub-Objective 1: Promote Peaceful Alternatives to Violence; 2: Reduce the Marginalization and Isolation of Community Groups; 3: Strengthen Moderate Actors; 4: Increase Moderate Space and Strengthen Moderate Actors; 5: Strengthen Youth Empowerment and Civic Participation; 6: Counter Extremist Messaging) 6. How familiar are you with the LCRI program? 7. What do you understand to be the goals of LCRI? 8. In what ways have you seen the program influencing these goals? 9. To what extent was the program successful in your opinion? In what ways what it unsuccessful Why? Relevance and Effectiveness of LCRI Key Components 10. To what extent were the four geographic regions priority locations for addressing inter-communal tension and social cohesion? Violent extremism? What other areas were considered? [NOTE: Prompt for whether Shia’, Hezbollah-controlled, Palestinian, Christian, Druze areas were relevant? Are any sub-regional locations especially relevant?] 11. How would you characterize development programming in coordination with the USG inter-agency? With other multilateral donors (e.g., UN), bilateral donors (e.g., GIZ), 51 12. How would you characterize coordination on development/assistance activities with the Lebanese central government? At the municipal level? [NOTE: Be sure to note whether a non-answer is “no coordination” or “don’t know.” Describe any specific examples of coordination.] USG/LCRI Impacts [NOTE: If respondent is not sure of LCRI’s contributions, ask what OTI approaches would have been useful.] 13. How has USG policy in Lebanon approached mitigating issues related to inter-communal conflict or host-community/refugee tensions? How has USAID been involved in this effort? [NOTE: Is local capacity accrued to an individual? An identify group? An organization?] 14. How has USG contribute to the U.S. Government’s broader policy priorities for countering violent extremism in Lebanon? Were there any missed opportunities? Any priorities that proved particularly difficult to address? 15. How did LCRI contribute to the USAID’s development priorities in Lebanon? Were there any missed opportunities? Any priorities that proved particularly difficult for LCRI to address? Big Picture 16. Over the next five years, what would you recommend in terms of mitigating sectarian tension in Lebanon? host community-refugee tensions? 17. Over the next five years, what would you recommend in terms of CVE in Lebanon? 18. Is there anything else you would like to add? Closing Thank you for taking the time to meet with us. We will be preparing an evaluation report for USAID/OTI that will aggregate all our interviews. USAID/OTI anticipates making a version of this report publicly available so you will have an opportunity to see the results some time later this year. If you have any questions or would like to follow-up with the evaluation team, please feel free to contact me at ___________________. 52 Key Informant Interview Guide LCRI LEADERSHIP OTI (CR, STA, DC), MSI (DC, COP, DCOP, DIR-OPS, M&E, PD, FIELD OFFICER) Estimated Length of Interview = 1 hour DATE OF INTERVIEW: NAME OF EVALUATION TEAM MBR: TRANSLATOR: RESPONDENT NAME: TITLE/POSITION: ORGANIZATION: LOCATION: REGION: TELEPHONE/EMAIL: SEX: RELIGION: OVER 29 YEARS? _______ INTRO: Thank you for meeting with us as part of the final evaluation of the Lebanon Community Resilience Initiative (LCRI). As you may know, Social Impact has been contracted by LCRI to conduct this third-party evaluation prior to the end of the project. The evaluation focuses on program performance, strategic approach, and impacts that LCRI has had on social cohesion and countering violent extremism in Lebanon. Your input will support OTI’s assessment of past activities and provide Lessons Learned for any future engagement in Lebanon. The information you provide will be confidential in that anything you share with us will not be attributed to you personally and all data collected will be evaluated in aggregate.  [RESPONDENT BACKGROUND ON LCRI] When did you begin and end your work with LCRI (month/year, if possible)? [NOTE: For respondents without direct involvement with YSI, ask about their length of involvement in work related to Yemen. Adapt focus of following questions based on timeframe of involvement with LCRI and specific role.] 53 [RESPONDENT’S ROLE] What is (or has been) your role with LCRI? Relevance of Objectives to Conflict Drivers 1. To what extent does sectarian tension lead to violent conflict in the four targeted areas? What are the drivers of this issue, in your opinion? How has this issue evolved from 2014 to present? 2. To what extent does host-refugee tension lead to violent conflict in the four targeted areas? What are the drivers of this issue, in your opinion? How has this issue evolved from 2014 to present? 3. To what extent does violent extremism lead to violent conflict in the four targeted areas? What are the drivers of this issue, in your opinion? How has this issue evolved from 2014 to present? 4. To what extent do other dynamics lead to violent conflict in the four targeted areas? What are these dynamics? [NOTE: Probe to be clear whether these dynamics were addressed by LCRI.] 5. There were some major events in and around Lebanon that might have catalysed confessional violence or galvanized recruitment by violent extremist groups: • Government vacuum, • Trash protests in Beirut, • Russian intervention into the Syrian conflict, • Battles around Homs and Aleppo. Can you describe the effects of these events on the program? Are there others that should be considered? Relevance, Effectiveness and Impacts of Sub-Objectives to Conflict Drivers 6. What were LCRI’s approaches to promoting peaceful alternatives to violence? How effective were these approaches? What are examples of results from this approach? 7. What were LCRI’s approaches to reducing marginalization and isolation of community groups? How effective were these approaches? What are examples of results from this approach? 8. What were LCRI’s approaches to strengthening moderate actors? How effective were these approaches? What are examples of results from this approach? How did the program address the potential for undermining effectiveness of moderate actors through their affiliation with USG￾supported activities? [NOTE: How does respondent define “moderate actor”? To what extent did LCRI strengthened the identified actor(s)? Are different actors more or less relevant in different locations or under different circumstances?] 9. What were LCRI’s approaches to countering extremist messages? How effective were these approaches? What are examples of results from this approach? [NOTE: How does respondent define “extremist messages”? Who disseminates these messages? How effective are these messages in mobilizing toward violence? To whom are messages being targeted?] 10. What were LCRI’s approaches to youth empowerment and civic engagement? How effective were these approaches? What are examples of results from this approach? [NOTE: How does respondent define “youth”, “empowerment” and “civic engagement”?] 11. How would you describe the implementation of the different sub-objectives? Were the protocols for assigning projects to protocols clear? Which SOs were most and least effective? Did the LCRI team recognize this? If not, why not? If so, how did they respond? Relevance and Effectiveness of LCRI Key Components 54 12. To your knowledge, what lessons or approaches from the previous program (Lebanon Civil Society Initiative) were integrated into LCRI? How did these contribute to or undermine LCRI’s effectiveness? 13. To what extent were the four geographic regions priority locations for addressing inter-communal tension and social cohesion? Violent extremism? What other areas were considered? [NOTE: Prompt for whether Shia’, Hezbollah-controlled, Palestinian, Christian, Druze areas were relevant? Are any sub-regional locations especially relevant?] 14. To what extent did LCRI develop its geographic and objective targeting in coordination with multilateral donors (e.g., UN, World Bank), bilateral donors (e.g., GIZ)? 15. How would you characterize coordination on development/assistance activities with the Lebanese central government? At the municipal level? [NOTE: Be sure to note whether a non-answer is “no coordination” or “don’t know.” Describe any specific examples of coordination.] 16. LCRI worked with several different types of local partners (awardees). What, if any, differences in results were explained by the type of awardee? [NOTE: List of Awardee Types: Community group, Host local government, Host national government, Local community group, Local NGO, Media, Private entity, STTA.] 17. How would you describe, overall, how well LCRI activities reached OTI’s intended beneficiaries? What factors supported effective targeting? What factors blocked inclusion of intended beneficiaries? How did LCRI address challenges to reaching the right participants? 18. Are there examples where deteriorating security conditions (e.g., high-risk operating environment, intensification of violence, constraints on movement, threats) imposed on local partners or their activities resulted in their adapting to the new environment? If so, what strategies were effective or ineffective? What 19. In what ways awardees start to brand their activities as supported by USAID? How did branding impact effectiveness of the activities? Effectiveness or status of the awardee? LCRI Impacts 20. How did LCRI contribute to strengthening local capacity to address inter-communal conflict or countering violent extremism during the project? Beyond the scope of the project? Provide examples of successes and failures. [NOTE: Is local capacity accrued to an individual? An identify group? An organization?] 21. How did LCRI contribute to changes in attitudes or behaviours towards inter-communal conflict? Refugees? Violent extremist actors or messaging? Provide examples of successes and failures. 22. How did LCRI contribute to the U.S. Government’s policy priorities for countering violent extremism? Were there any missed opportunities? Any priorities that proved difficult for LCRI to address? 23. How did LCRI contribute to the USAID’s development priorities in Lebanon? Were there any missed opportunities? Any priorities that proved difficult for LCRI to address? Big Picture 24. In terms of inter-communal violence between host and refugee communities and between confessional groups in the four targeted areas (Beirut, Tripoli, North Lebanon, Beqaa), would you say that the situation has improved, gotten worse, stayed the same? What best explains these changes or lack of change? 55 25. In terms of countering violent extremism in the four targeted areas (Beirut, Tripoli, North Lebanon, Beqaa), would you say that the situation has improved, gotten worse, stayed the same? What best explains these changes or lack of change? 26. What would you recommend in terms of mitigating sectarian tension in Lebanon over the next five years? 27. What would you recommend in terms of mitigating host community-refugee tension in Lebanon over the next five years? 28. What would you recommend in terms of countering violent extremism in Lebanon over the next five years? 29. Is there anything else you would like to add about LCRI? Closing Thank you for taking the time to meet with us. We will be preparing an evaluation report for USAID/OTI that will aggregate all our interviews. USAID/OTI anticipates making a version of this report publicly available so you will have an opportunity to see the results some time later this year. If you have any questions or would like to follow-up with Informant Interview Guide AWARDEE Estimated Length of Interview = 2 hours DATE OF INTERVIEW: NAME OF EVALUATION TEAM MBR: TRANSLATOR: RESPONDENT NAME: TITLE/POSITION: ORGANIZATION: LOCATION: REGION: TELEPHONE/EMAIL: SEX: RELIGION: OVER 29 YEARS? _______ INTRO: Thank you for meeting with us as part of the final evaluation of the Lebanon Community Resilience Initiative (LCRI). As you may know, Social Impact has been contracted by LCRI to conduct this third-party evaluation prior to the end of the project. The evaluation focuses on program performance, strategic approach, and impacts that LCRI has had on social cohesion and countering violent extremism in Lebanon. Your input will support LCRI’s assessment of past activities and provide Lessons Learned for any future engagement in Lebanon. The information you provide will be confidential in that anything you share with us will not be attributed to you personally and all data collected will be aggregated in the final report.  [RESPONDENT BACKGROUND ON LCRI] When did you begin and end your work with LCRI 56 (month/year, if possible)? [NOTE: Adapt focus of following questions based on timeframe of involvement with LCRI and specific role.] [RESPONDENT’S ROLE] What is (or has been) your role with LCRI? What role, if any did you or your organization play in the prior project (Lebanon Civil Society Initiative)? Relevance of Objectives to Conflict Drivers 1. Describe community conditions in the locations you work in. 2. How would you characterize these conditions specifically with regards to sectarian tensions? host￾refugee tensions? extremist recruitment? If present, do they break out in violence? How have these issues changed since 2014? 3. Are there other dynamics that lead to violent conflict? [NOTE: Probe to be clear whether these dynamics were addressed by LCRI.] 4. There were some major events in and around Lebanon that might have catalysed confessional violence or galvanized recruitment by violent extremist groups: • Government vacuum, • Trash protests in Beirut, • Russian intervention into the Syrian conflict, • Battles around Homs and Aleppo. Can you describe the effects of these events on sectarian tensions/host-refugee tensions/extremist recruitment? Are there other events that should be considered? How should these or other factors influence implementation of your activity(ies)? Relevance, Effectiveness and Impacts of Sub-Objectives to Conflict Drivers [NOTE: Not all sub-objectives were addressed by each activity. Be sure to ask about the relevant sub￾objective (cluster) assigned to the activity. Then probe as to whether any of the other 5 sub-objectives (underlined below) were relevant. Continue to ask for details on any sub-objective that the respondents says the activity addressed.] 5. In implementation of your activity(ies), what approaches did you take to promote peaceful alternatives to violence? How effective were these approaches? What are examples of results from these approaches? 6. In implementation of your activity(ies), what approaches did you take to reduce marginalization and isolation of community groups? How effective were these approaches? What are examples of results from this approach? 7. In implementation of your activity(ies), what approaches did you take to strengthen moderate actors or increase moderate space? How effective were these approaches? What are examples of results from this approach? How did involvement of these moderate actors in your activity impact their reputation, status or effectiveness in the community? [NOTE: How does respondent define “moderate actor”? To what extent did activity strengthen identified actor(s)? Are actors more or less relevant in different locations or different circumstances?] 8. In implementation of your activity(ies), what approaches did you take to involve youth, particularly in civic activity? How effective were these approaches? What are examples of results from this approach? 57 9. In implementing your activity(ies), what approaches did you take to counter extremist messages? How effective were these approaches? What are examples of results from this approach? [NOTE: How does respondent define “extremist messages”? Who disseminates messages? How effective are these messages in mobilizing toward violence? To whom are messages being targeted?] 10. Did your activity(ies) include girls and women? If so, how were they involved? How did their involvement impact results in addressing inter-communal conflict and social cohesion? Did they involvement impact results in addressing violence in the community, particularly the influence of violent groups? 11. Describe your activity’s(ies’) intended beneficiaries. To what extent were you able to reach them? What factors made this successful/unsuccessful? How did you address challenges to reaching the ‘right’ beneficiaries? What factors blocked inclusion of intended beneficiaries? 12. For your activity(ies), what aspects of implementation best supported successful results? What aspects undermined them? [NOTE: Aspects of implementation may include participatory consultation, validation by key stakeholders, bundling of multiple activities, outreach to beneficiaries, transparency of decision-making, awardee selection, procurement process, etc.] 13. For your activity(ies), how did security conditions (e.g., high-risk operating environment, intensification of violence, constraints on movement, threats) impact implementation? How were you able or not able to adapt to these threats? 14. FOR THOSE BRANDING ACTIVITIES: How did branding your activities as being supported by USAID impact effectiveness of the activities? Effectiveness or status of your organization? Big Picture 15. How would you recommend incorporating your activity’s successes into future programs to mitigate inter-communal conflict, promote social cohesion or counter violent extremism? What would you recommend changing or doing differently? Do you expect to continue your work? Describe in detail. 16. In terms of inter-communal violence between host and refugee communities in your activity area, would you say that the situation has improved, gotten worse, stayed the same since 2014? How did the activity contribute to this? Who or what organizations were given credit for this? 17. In terms of inter-communal violence between sectarian communities in your activity area, would you say that the situation has improved, gotten worse, stayed the same since 2014? How did the activity contribute to this? Who or what organizations were given credit for this? 18. In terms of countering violent extremism in your activity area, would you say that the situation has improved, gotten worse, stayed the same since 2014? How did the activity contribute to this? Who or what organizations were given credit for this? 19. Is there anything else you would like to add about LCRI or your activity? 20. In terms of inter-communal violence between host and refugee communities and between confessional groups in the four targeted areas (Beirut, Tripoli, North Lebanon, Beqaa), would you say that the situation has improved, gotten worse, stayed the same? What best explains these changes or lack of change? 21. In terms of countering violent extremism in the four targeted areas (Beirut, Tripoli, North Lebanon, Beqaa), would you say that the situation has improved, gotten worse, stayed the same? What best explains these changes or lack of change? 58 22. What would you recommend in terms of mitigating sectarian tension in Lebanon over the next five years? 23. What would you recommend in terms of mitigating host community-refugee tension in Lebanon over the next five years? 24. What would you recommend in terms of countering violent extremism in Lebanon over the next five years? 25. Is there anything else you would like to add about LCRI? Closing Thank you for taking the time to meet with us. We will be preparing an evaluation report for LCRI that will aggregate all our interviews. LCRI anticipates making a version of this report publicly available so you will have an opportunity to see the results some time later this year. If you have any questions or would like to follow-up with the evaluation team, please feel free to contact me at ___________________. 59 Key Informant Interview Guide SUBJECT MATTER EXPERT UN, BI-LATERAL DONOR, RESEARCHER/ANALYST, NON-USG IMPLEMENTER Estimated Length of Interview = 1 hour DATE OF INTERVIEW: NAME OF EVALUATION TEAM MBR: TRANSLATOR: RESPONDENT NAME: TITLE/POSITION: ORGANIZATION: LOCATION: REGION: TELEPHONE/EMAIL: SEX: RELIGION: OVER 29 YEARS? ______ INTRO: Thank you for meeting with us as part of the final evaluation of the Lebanon Community Resilience Initiative (LCRI). As you may know, Social Impact has been contracted by LCRI to conduct this third-party evaluation prior to the end of the project. The evaluation focuses on program performance, strategic approach, and impacts that LCRI has had on social cohesion and countering violent extremism in Lebanon. Your input will support LCRI’s assessment of past activities and provide Lessons Learned for any future engagement in Lebanon. The information you provide will be confidential in that anything you share with us will not be attributed to you personally or your organization. All data collected will be evaluated in aggregate. However, we’d like to list you with the other people interviewed for this evaluation; would this be acceptable for you?  [RESPONDENT BACKGROUND ON LCRI] To what extent are you involved with or familiar with the Lebanon Community Resilience Program? [NOTE: For respondents without direct involvement with LCRI, ask about their length of involvement in work related to Lebanon generally and CVE specifically. Adapt focus of following questions based on timeframe of involvement with LCRI and specific role.] [RESPONDENT’S ROLE] What is (or has been) your role with LCRI or in relation to Lebanon and CVE policy or research? Relevance of Objectives to Conflict Drivers 1. To what extent does sectarian tension lead to violent conflict/extremist recruitment in the four targeted areas of Beirut, Tripoli, Northern Lebanon and the Beqaa? Describe what you think are the key drivers of this dynamic. Is it also relevant in other parts of Lebanon? How has this issue evolved from 2014 to present? 60 2. To what extent does host-refugee tension lead to violent conflict/extremist recruitment in the four targeted areas? How does competition for resources feed into this tension? Describe what you think are the key drivers of this dynamic. Is it also relevant in Other parts of Lebanon? How has this issue evolved from 2014 to present? 3. To what extent does violent extremism lead to violent conflict/extremist recruitment in the four targeted areas? Describe what you think are the key drivers of this dynamic. Is it also relevant in Other parts of Lebanon? How has this issue evolved from 2014 to present? 4. To what extent do other dynamics lead to violent conflict/extremist recruitment in the four targeted areas? What are these dynamics? Other parts of Lebanon? How has this issue evolved from 2014 to present? 5. There were some major events in and around Lebanon that might have catalysed confessional violence or galvanized recruitment by violent extremist groups: • Government vacuum, • Trash protests in Beirut, • Russian intervention into the Syrian conflict, • Battles around Homs and Aleppo. Can you describe the effects of these events on the conditions above? Are there others that should be considered? Relevance and Effectiveness of LCRI Key Components [NOTE: The SME may have similar responses to each of these or may have very different recommendations for conflict resolution mechanisms or reducing marginalization. If it is the same, let them say so. But interviewer should get need them to think about each individually in order to then determine if they are consistent with LCRI’s approaches or actually quite different. Overlap is not only possible but likely. But interviewer will still need to invite the respondent to think about each of these specific challenges because these relate directly to LCRI’s sub-objectives and approaches.] 6. To what extent have you or your organization interacted with LCRI in developing its geographic and objective targeting in coordination? With any other non-USG stakeholders? With government? What mechanisms are available to facilitate this kind of coordination? [NOTE: Other stakeholders might be multilateral donors (e.g., UN), bilateral donors (e.g., GIZ), and the host government. Be sure to note whether a non-answer is “no coordination” or “don’t know.” Describe any specific examples of coordination.] 7. In the priority areas, what are effective approaches to promoting peaceful alternatives to violence? How closely related is this with reducing marginalization and isolation of community groups, such as refugees? 8. In the priority areas, what are effective approaches to strengthening moderate actors? Is there a concern that certain types of support undermine the effectiveness or credibility of moderate actors through their affiliation with internationally-supported activities? Does strengthening these actors serve as the best way to counter extremist messages? Or are there other approaches that we should consider? [NOTE: How does respondent define “moderate actor”? Are different actors more or less relevant in different locations or under different circumstances? [NOTE: How does respondent define “extremist messages”? Who disseminates these messages? How effective are these messages in mobilizing toward violence? To whom are messages being targeted?] 61 9. Is a focus on youth an effective approach to mitigating intercommunal tensions (host community￾refugee and confessional)? Is it effective in countering violent extremism? If so, how? If not, why not? [NOTE: Identify any geographic variation, especially with Tripoli and Akkar.] 10. What role or potential role do girls and women actors play in efforts to address these issues? 11. To what extent are “communities increasingly doubtful of civil society organizations and political figures”? What strategies and approaches are effective in overcoming these suspicions? LCRI Impacts – Questions About Programming 12. Are you familiar with any successes or failures in terms of activities that serve to address sources of misinformation about different communities? What action could serve to marginalize these sources? 13. Are you familiar with any successes or failures in terms of activities that serve to prevent youth from joining extremist groups? If so, describe them. If not, describe how you might address the issue. 14. What programs do you see as having the most immediate impact on the issues of host￾community/refugee tension and confessional tension? What programs do you see as being most important to address these issues in the long term? 15. What programs do you see as having the most immediate impact on the issues of radicalization? What programs do you see as being most important to address these issues in the long term? Big Picture 16. Over the next five years, what would you recommend in terms of mitigating sectarian tension in Lebanon? host community-refugee tensions? 17. Over the next five years, what would you recommend in terms of CVE in Lebanon? 18. Is there anything else you would like to add? Closing Thank you for taking the time to meet with us. We will be preparing an evaluation report for LCRI that will aggregate all our interviews. LCRI anticipates making a version of this report publicly available so you will have an opportunity to see the results some time later this year. If you have any questions or would like to follow-up with the evaluation team, please feel free to contact me at ___________________. 62 Focus Group Discussion Guide BENEFICIARIES Estimated Length of FGD = 1 hour 15 minutes DATE OF FGD: NAME OF FACILITATOR: TRANSLATOR: LCRI ID NO.: LOCATION OF ACTIVITY: REGION OF ACTIVITY: OVER 29 (#): TOTAL MALE FEMALE UNDER 29 (#): TOTAL MALE FEMALE RELIGIONS (#): NATIONALITY (#): LEBANON SYRIA OTHERS SIGN-IN: Notetakers should be sure to sign-in the participants as they arrive, listing the name they would like to be called during the FGD and demographic information. First and last names are not necessary, as their names will not be used in the report. Be sure to transfer the names to seating charts to provide the facilitator so that she will be able to call on participants. Keep a second seating chart for yourself so that you know which name to reference in notetaking. Ideally, the Awardee will provide the names and demographic information for all expected attendees. TRANSPORTATION REIMBURSEMENTS: Be sure to receive guidance from the Awardee on how reimbursements should be handled. Reimbursements are not made until after the session is completed. Participants must stay until the end of the FGD to be compensated. Bring exact change for each recipient. Only those on the list of expected beneficiaries should receive a reimbursement and all recipients should sign a document verifying receipt. FACILITATOR INTRO: Hello. My name is _______. Thank you for participating in this focus group discussion to talk about your participation in the ______________________ activity [read a brief summary of the activity]. I’m here with ________ and ______ who will be taking notes. Because I will be leading this discussion in Arabic, ________ will be providing translation for the notetakers. We are an evaluation team conducting a final evaluation of the Lebanon Community Resilience Initiative (LCRI). LCRI is conducting this evaluation to better understand project results and areas needing improvement. Your input will provide Lessons Learned for any future activities in Lebanon. I will be asking you about 10 questions about this activity. It is important to give everyone an opportunity to speak about his or her experience, so I will do my best to be sure everyone has a chance to respond to each question. CONFIDENTIALITY: The information you provide will be confidential: anything you share with us will not be attributed to you personally and all data collected will be aggregated in the final report. Your names will not appear anywhere in the report. 63 Does anyone have any questions before we begin?  1. I am going to begin by asking you a few questions about the level of participation you had with this activity: a. How many of you have heard of the _______________ activity? How did you hear about it? b. How many of you participated in some way in this activity? In what ways did you participate? c. How many of you have participated in other LCRI activities? What were those other activities? 2. What motivated you to participate in this activity? 3. How would you describe the activity’s benefit to the community, in general? 4. How would you describe the activity’s benefit to you, personally? NOTE: The following are the sub-objectives, which you can use as prompts if there is no response after asking the question. But there is not need to ask about each one: a. Promoting peaceful alternatives to violence? b. Reduce marginalization and isolation of community groups? c. Strengthening moderate actors? d. Expanding moderate spaces? e. Countering extremist messages? f. Strengthening youth empowerment and civic participation? 5. You’ve described the benefits to the activity, but I want to talk about the things you think could have been improved. What do you believe the activity should have included but didn’t in order to better achieve its goals [NOTE: Facilitator should describe the goal as the sub-objective assigned to the specific activity]? 6. What were the biggest challenges that the activity faced? How did the activity organizers address these challenges? 7. Do you think the activity included the ‘right’ people in the activity? If so, who do you think those people are and why? What strategies did the organizers use to include these members of your community in this activity? NOTE: Be sure to probe about relevant identity group: a. Girls or women? b. Young or elderly people? c. People from different religions/ethnicities? d. Refugees from Syria or other countries? 8. What aspects of the activity have continued, even after the activity ended? NOTE: this applies differently to different activities, however some purchased new equipment, facilities, systems, regulations. Also probe examples of evidence of changes in attitudes, behaviors; new or improved relationships or understanding 9. If you were to participate in a similar activity in the future, what would you recommend that the organizers do differently? 10. Is there anything else you would like to add about the activity or any other topic we have discussed? Thank you for your taking the time to participate in this discussion. If you have any questions about this evaluation, please feel free to talk with my co-workers or me after this discussion. 64 Focus Group Discussion Guide AWARDEE Estimated Length of Interview = 1 hour 15 minutes DATE OF FGD: NAME OF FACILITATOR: TRANSLATOR: RESPONDENT NAMES/TITLES/POSITIONS: LOCATION OF ACTIVITY: REGION OF ACTIVITY: OVER 29 (#): TOTAL MALE FEMALE OVER 29 YEARS? _______ INTRO: Thank you for meeting with us as part of the final evaluation of the Lebanon Community Resilience Initiative (LCRI). As you may know, Social Impact has been contracted by LCRI to conduct this third-party evaluation prior to the end of the project. The evaluation focuses on program performance, strategic approach, and impacts that LCRI has had on social cohesion and countering violent extremism in Lebanon. Your input will support LCRI’s assessment of past activities. The information you provide will be confidential in that anything you share with us will not be attributed to you personally and all data collected will be aggregated in the final report.  [RESPONDENT BACKGROUND ON LCRI] When did you begin and end your work with LCRI (month/year, if possible)? [NOTE: Adapt focus of following questions based on timeframe of involvement with LCRI and specific role.] [RESPONDENT’S ROLE] What is (or has been) your role with LCRI? What role, if any did you or your organization play in the prior project (Lebanon Civil Society Initiative)? What do you know about the objectives of the LCRI? Relevance of Objectives to Conflict Drivers 1. Describe community conditions in the locations you work in. 2. How would you characterize these conditions specifically with regards to sectarian tensions? host￾refugee tensions? extremist recruitment? If present, do they break out in violence? How have these issues changed since 2014? 3. Are there other dynamics that lead to violent conflict? [NOTE: Probe to be clear whether these dynamics were addressed by LCRI.] What part of these objectives were most relevant in your area? Question about the effectiveness of the program? Its impact? 65 In your community what are the drivers? Did you program have an impact on those drivers? Relevance, Effectiveness and Impacts of Sub-Objectives to Conflict Drivers [NOTE: Not all sub-objectives were addressed by each activity. Be sure to ask about the relevant sub￾objective (cluster) assigned to the activity. Then probe as to whether any of the other 5 sub-objectives (underlined below) were relevant. Continue to ask for details on any sub-objective that the respondents says the activity addressed.] 4. In implementation of your activity(ies), what approaches did you take to promote peaceful alternatives to violence? How effective were these approaches? What are examples of results from these approaches? 5. In implementation of your activity(ies), what approaches did you take to reduce marginalization and isolation of community groups? How effective were these approaches? What are examples of results from this approach? 6. In implementation of your activity(ies), what approaches did you take to strengthen moderate actors or increase moderate space? How effective were these approaches? What are examples of results from this approach? How did involvement of these moderate actors in your activity impact their reputation, status or effectiveness in the community? [NOTE: How does respondent define “moderate actor”? To what extent did activity strengthen identified actor(s)? Are actors more or less relevant in different locations or different circumstances?] 7. Describe your activity’s(ies’) intended beneficiaries. To what extent were you able to reach them? What factors made this successful/unsuccessful? How did you address challenges to reaching the ‘right’ beneficiaries? What factors blocked inclusion of intended beneficiaries? 8. Describe how you assessed the needs in your community? How you presented those needs to MSI? How did you know about MSI’s program? [NOTE: Aspects of implementation may include participatory consultation, validation by key stakeholders, bundling of multiple activities, outreach to beneficiaries, transparency of decision-making, awardee selection, procurement process, etc.] 9. For your activity(ies), how did security conditions (e.g., high-risk operating environment, intensification of violence, constraints on movement, threats) impact implementation? How were you able or not able to adapt to these threats? 10. To whom did the community attribute the activity(ies) results? Explain. How did your role in the community change as a result of the program? Big Picture 11. How would you recommend incorporating your activity’s successes into future programs to mitigate inter-communal conflict, promote social cohesion or counter violent extremism? What would you recommend changing or doing differently? Do you expect to continue your work? Describe in detail. 12. Is there anything else you would like to add about LCRI or your activity(ies)? Closing 66 Thank you for taking the time to meet with us. We will be preparing an evaluation report for LCRI that will aggregate all our interviews. LCRI anticipates making a version of this report publicly available so you will have an opportunity to see the results some time later this year. If you have any questions or would like to follow-up with the evaluation team, please feel free to contact me at ___________________. Checklist – Activity Site Visits DATE OF SITE VISIT: ACTIVITY NO: PLACE: REGION: Description of Project: Item Status Comment Quality High Medium Low N/A Utilization Many people Most people A Few people Not in use N/A Relevant to LCRI objectives Meets multiple sub-objectives Meets at least 1 sub-objective Meets no sub-objectives N/A Capacity Over capacity Meets capacity Under capacity N/A General Accessibility Accessible to all Accessible to most Accessible to a few Not accessible 67 N/A Equity of Access Males Females Sunni Shia’ Christian Druze Host community Syrian Refugees Palestinian Low income Other: Other Observations: 68 Survey Guide BENEFICIARIES Estimated Length of Survey = 5-10 minutes, SurveyMonkey disseminated by SMS, email, Facebook ANONYMOUS BUT WITH ACTIVITY ID TAG DATE OF SURVEY: LCRI ID NO: LOCATION OF ACTIVITY: REGION OF ACTIVITY: Introduction LCRI wants your feedback on [ACTIVITY (NO ID)] so that it can improve its programs in the future. This multiple choice survey will only take 5 minutes to complete and is completely anonymous. If you finish the survey, you will receive a summary of all responses to date. THANK YOU FOR YOUR PARTICIPATION! Survey Questions 1. Which sentence best describes your involvement in the activity? a. I participated in all aspects of the activity. b. I participated in most aspects of the activities. c. I only participated in a few aspects of the activities. d. I did not participate in this activity. 2. Please indicate if you Strongly Agree, Agree, Disagree or Strongly Disagree with the following statements about the activity you participated in: a. LCRI’s activity helped to promote peaceful alternatives to violence where I live. b. LCRI’s activity helped to reduce isolation of marginalized groups in where I live. c. LCRI’s activity helped to strengthen people who promote nonviolent means for resolving disputes where I live. d. LCRI’s activity helped to reduce the influence of extremist messages that encourages violence where I live. e. LCRI’s activity helped to encourage young people to be more active in civic activities where I live. f. LCRI’s activity helped to reduce sectarian-based tension where I live. g. LCRI’s activity helped to reduce host community and refugee tensions where I live. h. LCRI’s activity helped to reduce the influence of violent groups where I live. i. Because I participated in this activity, I have a better understanding of people from different religions. j. Because I participated in this activity, I have a better understanding of people from different countries. 3. Please indicate if you Strongly Agree, Agree, Disagree, or Strongly Disagree with the following statements about your community. a. Since the end of 2014, sectarian-based tensions increased in my community. 69 b. Since the end of 2014, host and refugee community tensions increased in my community. c. Since the end of 2014, there has been more violence where I live. d. Since the end of 2014, there are fewer people who my community can turn to to resolve disputes. e. Since the end of 2014, I am concerned about the number of people joining violent groups. 4. Which of the following statements best describes your opinion about the activity? a. I would not have changed anything about the activity. b. I would suggest only minor changes to the activity. c. I would suggest major changes to the activity, but it was better than no activity at all. d. It would have been better not to have run the activity at all. 5. What motivated you to participate in the activity (check all that apply): a. There was nothing else to do. b. My friend or family member was participating. c. My friend or family member told me to participate. d. I was interested in the activity theme. e. It was something I had never tried before. f. I wanted to make a positive contribution to my community. g. I wanted to learn something new. h. I wanted to earn money. i. I wanted to meet new people. j. I wanted to learn about different types of people. k. I wanted to reduce violence where I live. l. I thought it would be fun. 6. If the activity were offered again, would you participate in it? (yes/no) 7. In your opinion, how would you describe the communities where you worked with regards to mobilizations toward violence? (choose one) a. Most communities are more resistant to violence than when LCRI started at the end of 2014. b. Most communities have about the same level of resistance to violence than when LCRI started in 2014. c. Most communities are less resistant to violence than when LCRI started at the end of 2014. 8. [Linked Q] Based on your previous response, to whom do you most attribute this change/lack of change? (e.g., individuals, organizations, communities, political parties) 9. Name one thing about the activity that you would keep if it were run again: [text, limited to 140 characters] 10. Name one thing about the activity that you would change if it were run again. 70 [text, limited to 140 characters] Demographic Questions 9. Sex: Male // Female 10. Age: [Number or Range options] 11. Residence: [Name of town] 12. Nationality: Lebanese // Syrian // Palestinian // Other (text) Closing Thank you for your taking the time to participate in this survey. If you have any questions about this evaluation, please feel free to email us at __________________________. 71 Survey - LCRI Awardees Date of Survey: ______________________________________________________________________ Name (optional): ____________________________________________________________________ Position (optional): __________________________________________________________________ Organization (optional): ______________________________________________________________ Introduction: The evaluation team would like to get your feedback / opinion on the LCRI project. This survey will only take only 10minutes to complete. Please note that the information you provide is treated as confidential and will be used only for the purpose of the analysis. THANK YOU FOR YOUR PARTICIPATION! Survey Questions 1. When did you begin working with LCRI (month/Year)? ----------------------------------- 2. Which regions are you working in (check all that apply)? a. Beirut : (specify): ---------------------------------- b. Beqaa : (specify): ---------------------------------- c. North : (specify): --------------------------------- 3. Please indicate if you Strongly Agree, Agree, Disagree or Strongly Disagree with the following statements: Strongly Agree Agree Disagree Strongly Disagree a. In this(ese) region(s), LCRI helped to promote peaceful alternatives to violence. b. In this(ese) region(s), LCRI helped to reduce isolation of marginalized groups. c. In this(ese) region(s), LCRI helped to strengthen people who promote nonviolent means for resolving disputes. d. In this(ese) region(s), LCRI helped to reduce the influence of extremist messages that encourages violence e. In this(ese) region(s), LCRI helped to empower young people to be more active in civic activities. 72 f. In this(ese) region(s), LCRI helped to reduce sectarian-based tension. g. In this(ese) region(s), LCRI helped to reduce host community and refugee tension. h. In this(ese) region(s), LCRI helped to reduce the influence of violent groups. i. In this(ese) region(s), LCRI helped to improve activity beneficiaries’ understanding of people from different religions. j. In this(ese) region(s), LCRI helped to improve activity beneficiaries’ understanding of people from different countries. 4. What do you think motivated people to participate in the LCRI activities (check one): m. They had nothing else to do. n. They wanted to make a difference in their community. o. They wanted to challenge the traditional/current authority. p. They wanted to change ways of addressing community issues. q. They wanted to challenge armed group authorities. r. Other (please explain): ---------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------- 5. In your opinion, which strategy contributed the most to reducing sectarian violence? (choose one) a. Promoting peaceful alternatives to violence. b. Reducing isolation of marginalized groups (e.g., poor communities, refugee communities). c. Supporting “moderate” voices who promote nonviolent dispute resolution. d. Reducing the influence of extremist messages promoting violence. e. Involving youth in civic activities. f. Not possible. g. Not applicable h. Other: (please explain): --------------------------------------------------------------------------------------------------- ------------------------------------------------------------- 6. In your opinion, which strategy contributed the most to reducing host community and refugee-based violence? (choose one) a. Promoting peaceful alternatives to violence. b. Reducing isolation of marginalized groups (e.g., poor communities, refugee communities). c. Supporting “moderate” voices who promote nonviolent dispute resolution. d. Reducing the influence of extremist messages promoting violence. e. Involving youth in civic activities. f. Not possible. g. Not applicable h. Other: (please explain): --------------------------------------------------------------------------------------------------- ------------------------------------------------------------- 73 7. In your opinion, which strategy contributed the most to countering violent extremism? (choose one) a. Promoting peaceful alternatives to violence. b. Reducing isolation of marginalized groups (e.g., poor communities, refugee communities). c. Supporting “moderate” voices who promote nonviolent dispute resolution. d. Reducing the influence of extremist messages promoting violence. e. Involving youth in civic activities. f. Not possible. g. Not applicable h. Other: (please explain): --------------------------------------------------------------------------------------------------- ------------------------------------------------------------- 8. In your opinion, how would you describe these communities with regards to mobilizations toward violent extremism? (choose one) d. Most communities are more resistant to violent extremism than when LCRI started at the end of 2014. e. Most communities have about the same level of resistance to violent extremism than when LCRI started in 2014. f. Most communities are less resistant to violent extremism than when LCRI started at the end of 2014. 9. In your opinion, how would you describe these communities with regards to mobilization toward tensions in the community (host-community/refugee or sectarian tensions? (choose one) a. Most communities are more resistant to inter-communal tensions than when LCRI started at the end of 2014. b. Most communities have about the same level of resistance to inter-communal tensions than when LCRI started in 2014. c. Most communities are less resistant to inter-communal tensions than when LCRI started at the end of 2014. 10. Based on your response to question #8 and #9, to whom do you most attribute this change/lack of change? (e.g., individuals, awardees, communities, political parties) 11. Name one thing about LCRI that you would keep if possible: ---------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------- 12. Name one thing about LCRI that you would change if possible. ---------------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------- Thank you for providing the time to participate in this survey. For any further clarification, please do not hesitate to communicate with us via e-mail: ___________________ 74 ANNEX III: DOCUMENTS REVIEWED Programmatic Documents Cluster Review Workshop (December 11, 2015). Cluster Review: Increase Moderate Space and Strengthen Moderate Acgors (undated). Cluster Review: Promote Peaceful Alternatives to Violence (undated). Cluster Review: Reduce the Marginalization and Isolation of Community Groups. (undated). Cluster Review: Strengthen Youth Empowerment and Civic Participation (undated). Field Research to Support CVE Activities. L7 (January 2, 2016). Focus Group Discussions (December 2014) Focus Group Discussions (December 2015). Focus Group Discussions (February 2016). Focus Group Discussions (July 2015). Focus Group Discussions (June 2015). Focus Group Discussions (May 2015). Focus Group Discussions (November 2015). Focus Group Discussions (September 2015). Improving Economic Livelihoods in the North and Beqaa (For Internal Use Only) OTI/Lebanon (undated). LCRI Annual Report Year 1 (October 31, 2015). LCRI Annual Report Year 2 (October 31, 2016). LCRI Wave I: Analytical Report. Pursue (July 2015) Mitigating Sectarian Tensions in the Beqaa (For Internal Use Only) OTI/Lebanon (undated). MSI Cluster Evaluation. Pursue (October 2016). MSI Lessons Learned Annual Report (2015). North Strategy Revision Note to file (February 2017) North Strategy Updated (2016-2017) OTI Partners Help Reduce the Marginalization of Aarsal (For Internal Use Only). Report on 2015 Aarsal Activities. OTI/Lebanon (undated). Popular Blog Challenges Negative Perceptions of Aarsal (For Internal Use Only) OTI/Lebanon (undated). PPM Introduction Program Document (August 2015) Program Document (February 2016) Program Document (January 2016) Program Document (June 2015) Program Document (June 2016) 75 Report on 2015 Ramadan Activities: OTI-Supported Ramadan Activities Bring People Together, Reaching Close to 22,500 People in the North and Beqaa (For Internal Use Only) USAID/OTI – Lebanon (undated). Report on Nascent Groups in Tripoli: Nascent Youth Groups in Old Tripoli Take on a Life of Their Own (SBU) USAID/OTI-Lebanon (January 2016). Rolling Assessment (December 2015) Rolling Assessment (March 2015) Rolling Assessment (November 2016) Special Report: OTI Partners Counter Negative Image of Tripoli via Social Media, the Arts and Culture (For Internal Use Only) USAID/OTI – Lebanon. (September 2015). Strategic Review Session (July 205) Strategic Review Session (May 2016) Target Audience analysis and Communications Recommendations for CVE. L7 (January 2, 2016). USAID/OTI Lebanon CTPF FTF Concept Paper. (November 13, 2015). USAID/OTI Lebanon, Countering Violent Extremism, Program Strategy (November 2015). Subject Matter Documents Alami, Mona. 2015. The Rise of ISIS in Lebanon. The Atlantic Council. Retrieved from http://www.atlanticcouncil.org/blogs/menasource/the-rise-of-isis-in-lebanon. Atran, Scott. (2010). Talking to the enemy: Violent extremism, sacred values, and what it means to be human. New York, NY: Penguin. Atran, Scott. “Here’s What the Social Science Says About Countering Violent Extremism.” The Huffington Post, April 25, 2015. http://www.huffingtonpost.com/scott-atran/violent- extremism-social￾science_b_7142604.html Bahout, Joseph. “The Unraveling of Lebanon’s Taif Agreement: Limits of Sect-Based Power Sharing,” Carnegie Middle East Center. May 16, 2016. http://carnegieendowment.org/2016/05/16/unraveling-of-lebanon-s￾taif-agreement-limits-of-sect-based-power-sharing-pub-63571 Berman, Eli, Joseph H. Felter, Jacob N. Shapiro, and Erin Troland. “Modest, Secure, and Informed: Successful Development in Conflict Zones.” American Economic Review 103, no. 3 (2013): 512–17. doi:10.1257/aer.103.3.512 Brisard, Jean-Charles. (2005). Zarqawi: The New Face of Al Qaeda. New York: Other Press. Cammett, Melanie. Compassionate Communalism: Welfare and Sectarianism in Lebanon. London: Cornell University Press, 2014. Carter, Lynn, and Guilain Denoeux. “Development Assistance and Counter Extremism: A Guide to Programming.” U.S. Agency for International Development, October 2009 Carpi, Estella. “Prisms of Political Violence, ‘Jihads’ and Survival in Lebanon’s Tripoli.” Civil Society Knowledge Centre. Lebanon Support. December 2015. http://cskc.daleel-madani.org//paper/prisms-political￾violence-%E2%80%98jihads%E2%80%99-and- survival-lebanon%E2%80%99s-tripoli 76 Chahine, Ali et al. “Situation Analysis of the Youth in Lebanon Affected by the Syrian Crisis” United Nations Interagency. April 2014. http://www.unfpa.org.lb/Documents/Situation-Analysis-of-the-Youth-in￾Lebanon-Affecte.aspx Corbeil, Alexander. “The Syrian Conflict and Sunni Radicalism in Lebanon/” Carnegie Sada. January 9, 2014. http://carnegieendowment.org/sada/54132 “Terorism Havens: Lebanon.” (2007). Council on Foreign Relations. Retrieved from https://www.cfr.org/backgrounder/terrorism-havens-lebanon Dawson, Laura, Charlie Edwards, and Calum Jeffray. “Learning and Adapting: The Use of Monitoring and Evaluation in Countering Violent Extremism.” Royal United Services Institute for Defence and Security Studies (RUSI), 2014. https://www.rusi.org/downloads/assets/201406_Learning_and_Adapting.pdf. “Drivers of Instability, Conflict and Radicalization: A snapshot from Akkar,” L7 CVE Lab, January 2015. http://data.unhcr.org/syrianrefugees/documents.php?page=1&view=grid&Country%5B%5D=122&Type% 5B%5D=16 Dodwell, Brian, Milton, Daniel, and Rassler, Don. (2016). The Caliphate’s Global Workforce: An Inside Look at the Islamic State’s Foreign Fighter Paper Trail. West Point, N.Y.: Combating Terrorism Center, West Point. Elbakidze, L., and Y. H. Jin. 2015. Are economic development and education improvement associated with participation in transnational terrorism? Risk Analysis, Vol.35(8): pp.1520-35. Fink, Naureen Chowdhury. Barakat, Rafia. Shetret, Liat. “The Roles of Women in Terrorism, Conflict, and Violent Extremism: Lessons for the United Nations and International Actors” Center on Global Counterterrorism Coordination Policy Brief. 2013. Fink, Naureen Chowdhury, Peter Romaniuk, and Rafia Barakat. “Evaluating CVE Programming: Practice and Progress.” Center on Global Counterterrorism Cooperation, September 2013. http://globalcenter.org/wp- content/uploads/2013/07/Fink_Romaniuk_Barakat_EVALUATING-CVE￾Fishman, Brian. (2017). The master plan: ISIS, al-Qaeda, and the Jihadi strategy for final victory. Yale University Press: New Haven CT. Gambetta, Diego, and Steffen Hertog. 2016. Engineers of jihad: The curious connection between violent extremism and education. Princeton, NJ: Princeton University Press. Harald Doornbos and Jenan Moussa, “Present at the Creation: The Meeting that Led to the Beginning of ISIS,” Foreign Policy, August 2016. Accessed at http://foreignpolicy.com/2016/08/16/present-at-the-creation/ Hassan, Hassan and Weiss, Michael. (2016). ISIS: Inside the Army of Terror. New York: Regan Arts. Hegghammer, Thomas. (2008). Why Jihad Went Global. Warwick Institute of Advanced Study. Retrieved from http://hegghammer.com/_files/Hegghammer_-_IAS_Newsletter_-_Why_jihad_went_global.pdf Henley, Alexander. “Religious Authority and Sectarianism in Lebanon.” Carnegie Middle East Center. December 16, 2016. http://carnegieendowment.org/2016/12/16/religious-authority-and-sectarianism-in-lebanon￾pub-66487 Horowitz, Richard. (2013). Hezbollah's Military and Politics: Any Difference?. World Policy Blog. Retreived from http://www.worldpolicy.org/blog/2013/09/25/hezbollahs-military-and-politics-any-difference Khalil, James, and Martine Zeuthen. “A Case Study of Counter Violent Extremism (CVE) Programming: Lessons from OTI’s Kenya Transition Initiative.” Stability: International Journal of Security & Development 3, no. 1 (September 30, 2014). doi:10.5334/sta.ee. Kilcullen, David. (2016). Blood Year. London: Hurst Publishers. 77 Lefévre, Raphaël. “A New Direction for Lebanon's Muslim Brothers.” Carnegie Middle East Center. February 11, 2016. http://carnegie-mec.org/diwan/62740 Lefévre, Raphaël. “Lebanon’s Dar al-Fatwa and the Search for Moderation.” Carnegie Middle East Center. January 5, 2015. http://carnegie-mec.org/2015/01/05/lebanon-s-dar-al-fatwa-and-search-for-moderation-pub￾57627 Lefévre, Raphaël. “Tackling Sunni Radicalization in Lebanon.” Carnegie Middle East Center. December 24, 2014. http://carnegie-mec.org/2014/12/23/tackling-sunni-radicalization-in-lebanon-pub-57592 Lefévre, Raphaël. (2014). “The Roots of Crisis in Northern Lebanon.” Carnegie Middle East Center http://carnegie-mec.org/2014/04/15/roots-of-crisis-in-northern-lebanon/h9yc Luca, Ana Maria. “Shadi al-Mawlawi: from hero to terrorist.” NOW Lebanon. February 25, 2015. https://now.mmedia.me/lb/en/reportsfeatures/564884-shadi-al-mawlawi-from-hero-to-terrorist Lyall, Jason, Graeme Blair, and Kosuke Imai. “Explaining Support for Combatants during Wartime: A Survey Experiment in Afghanistan.” American Political Science Review 107, no. 4 (November 2013). http://imai.princeton.edu/research/files/afghan.pdf. McCants, William, and Clint Watts. “U.S. Strategy for Countering Violent Extremism: An Assessment.” Foreign Policy Research Institute. Accessed August 2, 2015. http://www.fpri.org/articles/2012/12/us-strategy￾countering-violent-extremism- assessment. McPherson, M., Smith-Lovin, L. and Cook, J.M. (2001). Birds of a Feather: Homophily in Social Networks. Annual Review of Sociology, Vol.27: pp.415-44. Mercycorps. “Youth and Consequences,” 2015. https://www.mercycorps.org/sites/default/files/MercyCorps_YouthConsequencesRepor t_2015.pdf. Milligan, Maren. “Making Sense of Tripoli I: The Security Trap or Conflating Cause and Effect.” Jadaliyya. December 9, 2014. http://www.jadaliyya.com/pages/index/20179/making-sense-of-tripoli-i_the-security￾trap-or-con Milligan, Maren. “Making Sense of Tripoli II: The Institutional Catch 22.” Jadaliyya. December 11, 2014. http://www.jadaliyya.com/pages/index/20180/making-sense-of-tripoli-ii_the-institutional-catch Milligan, Maren. “Tripoli’s Troubles to Come.” Middle East Research and Information Project. August 13, 2012. http://www.merip.org/mero/mero081312 Morris, Sharon. “Teenage Wasteland.” Foreign Policy, November 4, 2014. https://foreignpolicy.com/2014/11/04/teenage-wasteland/. Neumann, Peter R. (2016). Radicalized: new jihadists and the threat to the west. London: I.B. Tauris. Rabil, Robert G. Salafism in Lebanon: From Apoliticism to Transnational Jihadism. Washington, DC: Georgetown University Press, 2014. Rosenblatt, Nathaniel. (2016). All Jihad is Local. New America International Security Program Policy Paper. Retrieved from https://www.newamerica.org/international-security/policy-papers/all-jihad-is-local/ Rothenberg, R. (2001). From whole cloth: making up the terrorist network. Connections, Vol.24(3): pp.36-42. Sageman, Marc. (2004). Understanding Terror Networks. Philadelphia: University of Pennsylvania Press. Sageman, Marc. (2008). Leaderless Jihad: Terror Networks in the Twenty-First Century. Philadelphia: University of Pennsylvania Press. 78 Salem, Paul. “Lebanese Army’s Defeat of Salafists Buys Only Short Respite.” Carnegie Middle East Center. June 27, 2013. http://carnegie-mec.org/2013/06/27/lebanese-army-s-defeat-of-salafists-buys-only-short￾respite-pub-52232 Schmitt, Caroline. (2016). “Lebanon, the next failed state?” Retrieved from http://www.dw.com/en/lebanon-the￾next-failed-state/a-19066242 Simon, Alex. “The city that didn’t know where to start: Tripoli’s Revolving Stalemate.” Synaps, January 30, 2017. http://www.synaps.network/the-city-that-didnt-know-where-to-start￾Stern, Jessica. (2016). Radicalization to Extremism and Mobilization to Violence: What Have We Learned and What Can We Do About It? Annals of the American Academy of Political and Social Science. Vol.668. Swedberg, Jeffrey, and Lainie Reisman. “Mid-Term Evaluation of Three Countering Violent Extremism Projects.” U.S. Agency for International Development, February 2013. http://pdf.usaid.gov/pdf_docs/PDACX479.pdf. Swedberg, Jeffrey, and Steven Smith. “The Development Response to Violent Extremism and Insurgency,” September 2011. http://pdf.usaid.gov/pdf_docs/PDACS400.pdf. “Terrorism continues to shake Lebanon’s security.” (2013). Al-Monitor. Retrieved from http://www.al￾monitor.com/pulse/originals/2013/12/new-terrorist-lebanon-attacks.html#ixzz4keCf6dFU Urdal, Henrik. 2006. A clash of generations? Youth bulges and political violence. International Studies Quarterly Vol.50: pp.607-29. Warrick, Joby. (2016). Black Flags: The Rise of Isis. New York: Anchor Books. “What Is the Evidence on the Impact of Employment Creation on Stability and Poverty Reduction in Fragile States. A Systematic Review.” Overseas Development Institute, May 1, 2013. http://r4d.dfid.gov.uk/Output/192937/Default.aspx. Wood, Graeme. (2016). The Way of Strangers: Encounters with the Islamic State. New York: Allen Lane. World Bank Group. (2016). Economic and Social Inclusion to Prevent Violent Extremism. World Bank Middle East and North Africa Region MENA Economic Monitor. Retrieved from http://documents.worldbank.org/curated/en/409591474983005625/pdf/108525-REVISED-PUBLIC.pdf 79 ANNEX IV: PERSONS INTERVIEWED Due to the sensitive nature of this program and evaluation, names and position titles for persons interviewed by the ET have been redacted. KIIs Organization # Male Staff Interviewed # Female Staff Interviewed Total MSI/LCRI 6 6 12 USAID/OTI 3 6 9 Miscellaneous (Subject Matter Experts) 3 1 4 Beirut Awardees 0 2 2 Beqaa Awardees 8 3 11 Northern Lebanon Awardees 6 3 9 TOTAL 26 21 47 FGDs Location # of Beneficiary FGDs # Male Beneficiary Participants # Female Beneficiary Participants Total # of Participants Beirut 1 6 7 13 Beqaa 5 29 10 39 North 6 25 15 40 Total 12 60 32 92 Location # of Awardee # of Awardee Organizations # Male Awardee # Female Awardee Total # of Participants 80 FGDs Represented Participants Participants Beirut 1 6 5 1 6 Beqaa 2 11 8 3 11 North 2 7 7 2 9 Total 5 24 20 6 26 Note: The relatively low number of awardees who participated in the focus group discussions is due to: (1) Awardees selected for an activity deep dive assessment, which includes KII, have been called upon by the ET not to attend to avoid repetitive assessment of the same evaluation questions; and (2) Akkar FGD had a confirmed attendance of 8 organizations but only one organization actually attended and participated in the discussions. 81 ANNEX V: DEEP DIVE ASSESSMENTS CVE Activity Assessment Deep Dives (Ref. to Box 1) Region Grant Number Grantee Activity Name Location Grant Size Beirut LCRI189 Rescue Me Engaging Tariq Jdideh Youth in Skills Training and Neighborhood Revitalization Tariq el Jdideh $ 94,277.08 Relevance of LCRI189 to CVE objective: Strong. Justification: Rescue Me not only works in an area of Beirut that is a risk of radical group recruitment, but the organization has a detailed beneficiary targeting strategy that uses local informants to carefully identify and support the most at-risk individuals in the area. Beqaa LCRI237 Municipality of Qaa Installing Solar-Powered Street Lights in Qaa Qaa $ 180741.18 Relevance of LCRI237 to CVE objective: Strong. Justification: The 2016 suicide bomb attacks in Qaa were the result of an infiltration of the predominantly Christian town by extremists at night. This created significant concern among residents about their safety, which was supported by the program through this project. LCRI116 Youth Initiative Association Using the Arts to Voice Youth Concerns in Majdal Anjar Majdel Anjar $ 70,283.86 Relevance of LCRI116 to CVE objective: Moderate. Justification: Majdal Anjar has been a hotbed of recruitment by violent extremist organizations since the start of the 2003 Iraq war. The Youth Initiative Association is a known community-based organization providing alternative activities for youth in the community. However, beneficiaries of the program include university students interested in volunteer work and are not considered significant risks for recruitment. North Lebanon LCRI128 Governorate of Akkar Rehabilitating the Civil Registration Office in Aabde Aabde $ 53124.42 Relevance of LCRI128 to CVE objective: Weak. Justification: While there is recruitment to extremist groups in Akkar, this one-off project installing tangibles did not support the governorate office in any meaningful way related to the program objectives. Interviews with the awardee found little to no awareness of the program’s CVE objective. 82 LCRI153 Akkarouna Establish a Public Park and Youth Center in Bire Bire $ 122616.42 Relevance of LCRI153 to CVE objective: Weak. Justification: Activity assessment of this project found little to no awareness by the awardee organization of the program objective and no awareness among beneficiaries. LCRI174 Muslim Scouts￾Abu Samra Branch Strengthening a Scouts Group in Abu Samra Abu Samra $ 109325.21 Relevance of LCRI174 to CVE objective: Moderate. Justification: Abu Samra neighborhood is a neighborhood in Tripoli where recruitment takes place; however the awardee would have benefitted from better guidance from the program on how to recruit beneficiary participants. LCRI111 One Voice Team "Peace Stairs" in Marginalized Tripoli Neighborhoods Jabal Mohsen $ 74784.69 Relevance of LCRI111 to CVE objective: Strong. Justification: The program recruited youth volunteers to renovate or beautify staircases that connect neighborhoods in Tripoli that experienced clashes from 2011-2014 (e.g., Jabal Mohsen, Bab Tebbaneh). The process involved painting over pro-extremist propaganda with peace messages. LCRI180 Jeel Al Amal (Generation Hope) Establishing a Public Park in Fnaideq Fnaideq $ 129,837.19 Relevance of LCRI180 to CVE objective: Weak. Justification: Although Fnaideq is one of the most deprived parts of Lebanon, an assessment of this project found minimal connection to the program objectives. LCRI065 Hadatha Association Empowering Youth to Play a Visible Role in Middle Dreib Communities Khirbet Daoud $ 53395.77 Relevance of LCRI237 to CVE objective: Strong. Justification: The project management for Hadatha deployed an extremely detailed and nuanced beneficiary targeting strategy for their programs, though this was developed without coordination or support from the program. LCRI126 Qoubbe Youth Academy Qoubbe Youth Engagement in Community Development and Street Arts Qoubbe $ 86164.00 83 Relevance of LCRI126 to CVE objective: Moderate. Justification: Qoubbe is an area where extremist group recruitment is prevalent, though less extensive than in other parts of Tripoli (e.g., Bab Tebbaneh, Mankoubin, Abu Samra). The Qoubbe Youth Academy attracts youth in the town interested in participating in football, arts, and other community activities, though it does not specifically seek out individuals who might be vulnerable to recruitment. LCRI052 Tebbaneh Youth Council Rehabilitating Tebbaneh's Clothing Market Jabal Mohsen $ 101077.19 Relevance of LCRI052 to CVE objective: Strong. Justification: The Tebbaneh clothing market rests on the fault lines between the Tripoli neighborhoods of Jabal Mohsen and Bab Tebbaneh that saw extensive clashes from 2011-2014. Although these clashes were based primarily on sectarian tensions, extremist group recruitment is also widespread. While the reconstruction of the clothing market did not necessarily target at-risk beneficiaries, the symbol of restoring the market brought together parts of the community that had previously been fighting. 84 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST 85 86 ANNEX VII: OTI STATEMENT OF DIFFERENCE: LEBANON COMMUNITY RESILIENCE INITIATIVE FINAL EVALUATION 1/10/2018 USAID’s Office of Transition Initiatives (OTI) welcomed the final evaluation of its Lebanon Community Resilience Initiative (LCRI) by Social Impact. OTI was eager for an independent assessment of the extent to which the program achieved its intended results, and with the program closing, wrote the SOW with the aim of identifying lessons learned and opportunities for other USG actors in Lebanon to continue building on LCRI’s strategy and approach. While the final evaluation report explored themes that fell under the three evaluation questions—on appropriateness of strategy, efficacy of implementation to achieve results, and contribution to intended impact—OTI was disappointed by a number of shortcomings that affected the credibility of the findings and thus the utility of the recommendations to internal and external parties. The two primary shortcomings were turnover on the evaluation team and a sampling strategy that was not well suited to the OTI model. Taken together, these contributed to the collection of data that was not well-suited to describing the program in its entirety, and generated concerns about the reliability of the analysis and conclusions throughout much of the report. The evaluation team designed a rigorous, thoughtful sampling strategy, but one that unfortunately did not account for either the OTI model or the program’s maturation over time – resulting in a sample that was biased toward early, pilot activities and was not able to assess cumulative impact. OTI provided feedback on the first draft of the sampling strategy in an attempt to explain and correct these biases, but felt compelled to approve the revised strategy despite its shortcomings because the team was already in the field and on a tight timeline when OTI received it. By sampling individual activities rather than looking holistically at clusters, the evaluation was unable to capture the aggregate impact of clusters of programming, a core component of the OTI model. The strategy also oversampled activities from early in the program, resulting in a sample that skewed toward early, entry-point, or pilot activities—a period and set of activities in which theories of change and assumptions were being developed, assessed, and refined, and during which systems and processes were similarly emerging, adapting, and improving. The evaluation team’s intent to adhere to methodological rigor is commendable, but it resulted in a methodology that did not fit the model and was unable to fully address the evaluation questions. In addition to a sampling strategy that was unable to capture the model and maturation of the program, the evaluation team experienced significant turnover—which resulted in a loss of information that would have contributed to a stronger strategy, more informed fieldwork, and led to a more nuanced analysis. The team leader left the project after an in-depth orientation on the program’s background, OTI goals for the evaluation, and training in the activity database, which is comprised of foundational information on the program and each activity. SI selected an excellent replacement from within the team, but due to distance and time constraints, he did not receive the same deep dive into the program and database as the original team lead, resulting in an incomplete transfer of contextual and programmatic knowledge. Another member of the team resigned at OTI’s request after a series of events in the field, which was followed by the departure of the team leader after he drafted an outline of the report. SI provided a skilled evaluator from the DC office to write the final report, but at this point not one member of the original team was involved in the report writing and the lead author had not been involved in fieldwork. This turnover resulted in loss of contextual knowledge and nuance, leaving some of the findings unanchored in the OTI model and program context, which negatively affected the credibility of the findings, and the report lacking a clear sense of who the findings were intended to inform. OTI appreciates the willingness of the evaluators to continue their efforts despite the challenges posed by the high turnover, their commitment to engaging in substantive discussions about OTI and Implementer comments on the report, and their participation in discussions about how to improve the design of final evaluations in the future. OTI also recognizes that the country context and a number of circumstances both within and beyond the program and SI’s control—such as staff turnover, a tight timeline, and high non-response rates—presented very serious challenges to the team’s ability to obtain, review, and analyze all relevant contextual and programmatic information. OTI appreciates the efforts of all parties to find solutions to these challenges and to consider feedback on the interpretation of the report’s findings. The final version of the report demonstrated significant improvement over early drafts, but nonetheless, OTI maintains that the evaluation misses the mark on the credibility of the findings and their utility to the design of future programming in the USAID-Lebanon Mission and within OTI. ANNEX VIII: MSI’S STATEMENT OF DIFFERENCE TO THE FINAL EVALUATION OF THE LEBANON COMMUNITY RESILIENCE INITIATIVE 200 12th Street South, Suite 1200 Arlington, VA 22202 USA This statement of difference is in response to the final evaluation report on the Lebanon Community Resilience Initiative (LCRI) delivered by Social Impact on August 17, 2017. USAID’s Office of Transition Initiatives (OTI) contracted Management Systems International (MSI) to implement LCRI from 2014 to 2017. LCRI worked in areas of the country most affected by the Syrian crisis to meet the following two program objectives (POs): mitigate rising sectarian and host community-refugee tensions (PO1) and counter the influence of violent extremist groups (PO2). MSI’s statement of difference illustrates how the final evaluation’s significant methodological limitations undermine the validity and relevance of certain findings, conclusions, and recommendations, resulting in a missed learning opportunity. This response presents methodological and contextual programmatic issues related to the evaluation team’s (ET’s) findings on the three evaluation questions: • Question 1: Did LCRI utilize an appropriate strategy to achieve USAID/OTI’s program goal in the country? • Question 2: Did LCRI effectively implement its strategy to achieve the intended results? • Question 3: To what extent did LCRI contribute to the achievement of program goals and objectives? METHODOLOGICAL ISSUES The evaluation methodology fails to create a robust framework through which to answer the evaluation questions due to: 1) a lack of transparency regarding how the ET analyzed the data it collected; 2) a weak sampling strategy; and 3) a poorly designed mini-survey instrument. As a result, it is not possible to make a reliable connection between the data collected and certain findings and conclusions derived from those data. A lack of transparency on the ET’s data analysis methodology weakens the report’s validity. The evaluation report is missing a data analysis section to explain the framework through which the ET coded and analyzed the different data sources, as well as how it triangulated the findings across data sources. Given the different actors involved in the key informant interviews (KIIs), focus group discussions (FGDs), and “deep dive” activity assessments, an explanation of how the ET analyzed the data and weighted the responses would have lent credibility to the findings and extended their value beyond the anecdotal claims the report presents. The sampling strategy did not provide for a representative examination of the LCRI strategy, resulting in unreliable conclusions based on incomplete data. The evaluation based its selection of awardee participants and activities on the inaccurate assumption that the activities were implemented independently; it did not account for the layering and clustering approach that was integral to LCRI’s strategy. For example, the sampling strategy for the FGDs consisted of randomly selecting individual activities from each region. Moreover, the activities for the “deep dive” assessments were purposively selected based on “location, program objectives, and largest budgets based on amount disbursed.” Lastly, for the KIIs, the ET selected awardees involved in the same activities it visited for the “deep dive” assessments. These sampling strategies assumed that the activities were implemented independently and were all equally expected to affect the strategic objectives. However, LCRI’s layering approach ensured that activities were grouped into predefined geographic and programmatic clusters within the program’s strategic framework in order to aggregate and amplify small grant outcomes. LCRI’s strategy, which followed USAID/OTI’s guidance, was not based on individual activities, but on the intentional clustering of activities. In the context of this layering approach, a stratified random sample that divided the awardees into smaller groups by cluster would have provided a more nuanced and reliable representation of LCRI’s strategy than a simple random sample by region could, particularly for the FGDs. In addition, the selection criteria for the “deep dive” activity assessments could have accounted for the clustering approach to examine a critical mass of activities more systematically. These changes to the sampling strategy would have enabled a more robust evaluation, particularly since evaluation questions 1 and 2 specifically asked about LCRI’s strategy to achieve the program goals and intended results. This was a missed opportunity for the ET to take a more nuanced look at LCRI’s strategic approach, and as a result, the evaluation reached unreliable conclusions based on incomplete data. The mini-survey used to draw conclusions for evaluation question 3 was poorly designed in ways that undermine the credibility of the associated findings and conclusions. Examples include: • The mini-survey was administered to a small sample of beneficiaries, which threatened the internal validity of the evaluation and resulted in unreliable findings. The survey responses cited in the findings under question 3 were from only 59 beneficiaries and 49 implementing partner and awardee points of contact, mostly from the same small set of activities that the ET visited during the “deep dive” assessments. Fifty-nine people represent a tiny proportion of the beneficiaries from a program that had 11,797 youths directly participating in activities. Not only was the mini￾survey sample size small relative to the number of beneficiaries the program reached, but it also was small in absolute terms, making it inappropriate to extrapolate the responses into generalizable conclusions of LCRI’s achievement of program goals and objectives. Nonetheless, the ET used the mini-survey data to answer evaluation question 3 without including a discussion of sample size limitations. Even if sample size could not be increased, a more rigorous analytical approach would have been to consistently triangulate the survey responses with the other data sources, such as the KIIs and FGDs. While the ET did triangulate the data for a couple of findings, it did not do so consistently throughout this section of the report. • Several questions in the mini-survey were poorly framed, resulting in invalid measurements and misleading findings. For example, the mini-survey instrument did not properly account for LCRI’s strategic approach and assumed that each activity was meant to address all subobjectives. It asked whether the pertaining LCRI activity helped to achieve a variety of intended results regardless of whether the specific activity was intended to address each particular subobjective. Other questions asked awardees involved in specific LCRI activities that were not intended to address all subobjectives, which subobjective contributed the most to each PO. Therefore, it is inaccurate to aggregate responses to these questions to conclude whether LCRI, in general, contributed to these changes and which strategy contributed the most. • The mini-survey was not piloted in advance, compromising the robustness of the survey results. The evaluation report does not describe the process for validating the questionnaire, particularly the development, translation, and piloting of survey questions. This weak methodological aspect of the survey design further exacerbates the small sample size and flawed question framing issues highlighted above. CONTEXTUAL PROGRAMMATIC ISSUES This section clarifies key contextual factors that are missing from the evaluation. These omissions create in an incomplete picture of LCRI and diminish the relevance of certain findings. Accounting for the program setting would have better contextualize those findings. The evaluation includes findings on the role of “tangibles” in engaging new communities, despite acknowledging weak evidence to support such findings. The ET concludes that “tangibles” — defined as community goods or small-scale infrastructure projects — were not a necessary “transaction cost” to gain entry in new communities. It reaches this conclusion even after confirming opposing views among the small number of interviewees who commented on this point (eight key informants including five implementing partner staff members and three awardees). In addition, MSI would like to present an alternative finding related to the program’s use of tangibles based on a countering violent extremism (CVE) case study,1 commissioned by LCRI and led by CVE experts Dr. Guilian Denoeux and Dr. Lynn Carter. This case study explored the insights and lessons on CVE programming gained through LCRI by drawing on a multiplicity of sources: project documentation review, analysis, and coding; research into dynamics and developments related to violent extremism (VE) in Lebanon; extensive field interviews with key stakeholders; and interviews and FGDs with direct and indirect beneficiaries. Based on this research, the study made conclusions about LCRI’s use of “tangibles” that counter those from the LCRI final evaluation; in particular, it concluded that “small￾scale tangibles were an effective tool for entry into resistant and suspicious communities. Through quick-impact, high-visibility activities, communities were able to see the value of the work being done by the grantee. Tangibles strengthened grantee standing in and relationship with the community and gave them the ability to expand their efforts and draw more community members into softer CVE programming.” The CVE case study also found that LCRI used “tangibles” effectively because they “were not an end in themselves; [tangibles] helped strengthen relationships within the community, between participants and the community, and between the grantee and the community. The process was important for building resilience and a platform from which future programming could draw in more members of the community and realize greater CVE gains.” The evaluation misclassifies some tangible activities as “one-offs,” thereby undermining the credibility of its analysis and weakening its findings on LCRI’s effective implementation of its strategy. The evaluation concludes that “stand-alone tangibles analyzed by the ET as part of its activity assessment sample – e.g., LCRI238 and LCRI128 – even when supporting a moderate actor, had minimal and potentially short-lived results” (p. 32). However, MSI would like to clarify that the two activity examples cited were associated with clusters of activities and should not be considered “stand￾alone tangibles.” 1 Innovative Approaches to CVE Programming: Insights and Lessons from the USAID Office of Transition Initiatives Lebanon Community Resilience Initiative (2014-2017). • Specifically, LCRI238 was part of the Aarsal-Labboueh cluster aimed at improving relations between two rival communities (also including LCRI054, 101, 132, 151, 173, and 179), and LCRI128 was part of the Akkar governorate cluster aimed at strengthening a key moderate actor along the Lebanese-Syrian border (also including LCRI048, 128, 233, and 268). The evaluation also highlights its analysis of LCRI’s activity database and states that “18 (34 percent) of the projects considered tangibles, were ‘one-offs’, either because a) there was no follow up activity in the location; b) this was the only grant awarded to the local partner; or c) the partner only implemented tangible activities” (p. 20). While the ET does not provide a list of which projects it considers tangibles, LCRI’s activity database shows that the two examples of “one-off tangible” activities referenced in the evaluation report — LCRI153 and LCRI180 — do not fall under the ET’s own definition of a “one-off tangible.” • Specifically, LCRI153 was implemented in the town of Bire along with two other activities (LCRI058 and 277), and Akkarouna, the local partner, also implemented another LCRI grant in the Akkar region (LCRI250). The second example, LCRI180, was implemented in the town of Fnaideq by Jeel al Amal, a partner that also implemented two other LCRI activities in Fnaideq (LCRI078 and 276). The evaluation overlooks the interdependence between LCRI’s two POs, which limits the relevance of certain findings associated with evaluation question 1. The ET reports a “mixed interpretation of sub-objectives,” where “LCRI categorized similar activities across sub- objectives and objectives” (p. 16). However, MSI would like to clarify that in the Lebanon context, some of the main drivers of VE stem from sectarian tensions and instability, which need to be mitigated (PO1) as part of the effort to counter the influence of VE groups(PO2). Research commissioned by LCRI and conducted by the firm Levant 7 (and included in the ET’s document review) provided evidence that showed how VE groups exploit sectarian tensions to radicalize and recruit youth in Lebanon. Through this research, LCRI recognized that an activity categorized as falling under one PO could in fact have an effect on the other. Thus, LCRI’s two POs were interconnected, and LCRI built this interconnectedness into the program strategy, as the program documentation explained. Both LCRI’s first annual report in September 2015 and notes from the strategic review session held in 2016 clearly documented and analyzed the interconnectedness of the two POs, and both of these documents were included in the ET’s document review. However, the ET did not acknowledge this aspect in the discussion of its findings and therefore presented an incomplete analysis of LCRI’s strategy. The evaluation does not account for how LCRI’s overall strategy utilized CVE and preventing violent extremism (PVE) approaches to pursue USAID/OTI’s program goal. The ET concludes that “the LCRI’s CVE strategy overall was limited—the approach would be better described as PVE” (p. 31). It states that LCRI’s CVE strategy “improved community resilience to the proliferation of extremist groups, rather than addressing their influence directly.” MSI would like to clarify that LCRI’s strategy aimed to counter VE in geographic target areas where known actors, dynamics, and recruitment were present by supporting the development of more moderate actors and dynamics (i.e., CVE) and at the same time worked in other communities to prevent the spread of VE (i.e., PVE). Therefore, the particular approaches taken to address extremism varied based on the context of the communities: some activities in Tripoli directly aimed to counter VE groups where recruitment and radicalization were taking place, and others in places like central Beqaa aimed to limit the influence of those groups where communities were vulnerable. The ET’s emphasis on separating PVE from CVE dismisses the complex nature of addressing VE. It is indeed important to identify and be clear on which challenges the programming activities were intended to address, since the types of interventions called for could differ accordingly. However, focusing the discussion about the relevance of LCRI’s strategy on a narrowing definition of CVE that dismisses how CVE and PVE are part of the overall strategy to address VE undercuts USAID/OTI’s program goal to “strengthen Lebanese communities to more effectively cope with destabilizing factors of increased community tensions and violent extremism.” CONCLUSION While MSI recognizes the notable limitations that the ET faced, the methodological and contextual programmatic issues presented in this statement of difference expose the lack of reliability in the evaluation findings and thus bring into question the validity and relevance of some key conclusions and recommendations. These issues limit the learning that can be derived from this evaluation report. ANNEX IX: SI RESPONSE TO THE LETTERS OF DIFFERENCE ISSUED BY USAID AND MSI REGARDING LCRI FINAL EVALUATION 6/21/18 SI acknowledges the points raised in the letters of difference issued by USAID/OTI and MSI. Regarding OTI’s concerns, SI agrees with some of the challenges noted regarding turnover of evaluation team members on this assignment, and SI appreciates OTI’s recognition of significant efforts that were taken to minimize disruption and consequences of that unanticipated turnover. However, SI regrets that other substantive issues noted in the letters were not raised earlier, when SI would have had the opportunity to respond constructively or provide additional information during the evaluation or report review period. Additionally, though we recognize that MSI was dissatisfied with our findings, which noted several challenges in the LCRI, we disagree with many of the methodological concerns raised. SI addresses the primary criticisms from the letters of difference below: Sampling Approach In its pre-fieldwork Inception Report, SI initially proposed sampling based on geographic clusters. OTI recommended that sampling be conducted at the grantee level to better capture experiences and outcomes from across more activities (especially layered activities). In response to OTI’s guidance, SI moved to a dual-staged sampling approach: 1. First, the evaluation team sampled grantees to participate in focus groups discussions that covered all the activities grantees had been responsible for implementing. Grantees were randomly selected to provide as representative a sample as possible and broad coverage of the layered activities. 2. Next, for site visits and in-depth assessments, the evaluation team purposively sampled specific activities that had been implemented by the grantees selected above. Unfortunately, very few grantees agreed to participate in evaluation fieldwork. As such, the final sample was smaller than envisioned (a limitation noted in the evaluation report). The letters of difference note concerns about sampling having only occurred at the activity level and overrepresenting older activities. This is incorrect as sampling first occurred at the grantee level, covering a broad range of layered activities. Additionally, the sample closely matched the distribution of activities across the LCRI period of performance and thus is representative of MSI’s activities. These data are presented in the table below. Sampling Comparison Chart Coverage Year of Activity Completion % of Grantees % of Activities % of Total Grant Value31 2015 2016 2017 Selected Sample 40% 49% 46% 37% 51% 11% Final Grantee Respondents 20% 23% 23% 34% 55% 9% Final Site Visits N/A N/A N/A 18% 64% 18% All LCRI Activities N/A N/A N/A 36% 54% 7% Analysis While MSI noted dissatisfaction with the level of detail regarding the evaluation’s analysis methods, it never requested further detail, which SI could have provided. As the analysis process relied on tally sheets to document significant themes and patterns in a reproduceable manner, which is commonly done for such studies, SI does not believe the analysis method is a weakness of the study. SI concurs that there were limitations to the mini-survey, as outlined in the report’s limitations section. However, as the name suggests, a mini-survey serves a different purpose than a large-scale quantitative survey. Mini-surveys are meant to supplement specific aspects of a primarily qualitative evaluation, providing additional evidence to substantiate or contrast with the qualitative findings. Thus, a statistically representative sample was never the intention. MSI also expressed concern about the survey asking all beneficiaries and awardees about each of the intended benefits of LCRI rather than focusing on the specific sub-objective linked to a particular activity. The survey asked about the effect of the program as a whole to capture data on both the effects of multiple, clustered activities as well as the potential for interrelated objectives. This particular concern is contrary to MSI’s later concern about the evaluation not having taken the clustered nature of activities and the interrelatedness of the primary objectives into account. As MSI notes, several activities may have been implemented in the same areas (thus beneficiaries and communities may have benefited from multiple activities) and a single grantee may have implemented multiple activities. Similarly, the interrelatedness of the objectives suggests that, even if a particular activity had been categorized as addressing a particular primary or sub-objective, it may have also impacted other, interrelated objectives. For these reasons, SI had asked survey respondents about all potential outcomes. 31 Percentage of total grant value Contextual Understanding MSI also expressed concern regarding a series of findings related to the role of “tangibles.” Tangibles are grants that include durable components (construction of a building, equipment, etc.). Some grants included only tangibles, while others included both tangibles as well as other soft-side components such as capacity building or training. SI appreciates MSI’s offer to share the report it procured regarding the role of tangibles in the program. Unfortunately, the evaluation team did not receive this internally￾procured data during the course of the evaluation, and as such was unable to assess its validity or incorporate any relevant information into the findings. MSI also raised concerns about SI having potentially miscategorized some activities, notably LCRI238 and LCRI128, which MSI notes as having been implemented as a part of a broader cluster. However, SI’s definition of a “one-off” activity included three potential qualifying criteria. And, while other activities were implemented in the same community as these activities, LCRI238 was implemented by the Governorate of Baalbeck-Hermel, which did not implement any other activities (criteria #2) and LCRI128 was implemented by the Governorate of Akkar, which implemented three activities, all of which were for “tangibles” (criteria #3). Thus, these activities were properly categorized in the report. MSI also raised concerns about LCRI153 and LCRI180 being cited as “one-off” tangibles. These activities were cited in two places: once in regards to a finding about activities focused on building moderate spaces or public spaces (regardless of whether they were one-offs) where beneficiaries did not seem to have a strong grasp of the intended objectives and the effect of the overall activity being limited, and later in reference to the finding that 18 activities were considered “one-off” tangibles and were found to have limited effect on program objectives. With regret, though the overall finding regarding the 18 one￾off tangible activities remains, SI acknowledges that LCRI153 and LCRI180 were mistakenly cited in the second context. LCRI238 and LCRI128 (or others of the 18 “one-offs”) should have been cited instead. Lastly, MSI raised concerns about SI’s finding that the LCRI approach would be better described as a Preventing Violent Extremism (PVE) approach rather than as a Countering Violent Extremism (CVE) approach. SI concurs with MSI’s description of how PVE and CVE can both be used to address violent extremism. However, the objectives of LCRI were CVE-focused rather than PVE-focused, based on the intention of OTI to pivot from a resiliency focus to efforts that would more directly confront the influence of violent extremist organizations. In assessing the extent to which this pivot took place under LCRI, the team concluded that the activities placed a higher emphasis on PVE than the intended CVE. Concluding Remarks While this assignment faced numerous challenges, both within and outside the evaluation team’s control, SI appreciates the willingness by OTI and MSI to actively engage in discussions and find solutions throughout the course of and after the conclusion of the evaluation. SI documented lessons learned in an After Action Review that it shared with OTI, and SI will continue to apply learning from the evaluation process when implementing ongoing and future evaluations for OTI. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523