Lebanon Civic and Civic Support 
Initiatives
Final Evaluation (Public Release)
December 2012
This publication was produced for review by the United States Agency for International 
Development. It was prepared independently by Pursue, Ltd. on behalf of The QED Group, 
LLC, under Task Order #65 of the Program Development Quickly II (PDQ-II) IQC, contract 
number DOT-I-00-08-00017-00. The author’s views expressed in this document do not
necessarily reflect the views of USAID or the United States Government. 
[This page intentionally left blank.] 
CONTENTS 
I. PURPOSE OF THE EVALUATION ........................................................................... 4
1.1 Intended Use of the Evaluation..................................................................................... 4
II. LCI & LCSI PROGRAM RATIONALE AND OVERVIEW.................................. 4
2.1 Program Goals ..................................................................................................................... 5
2.2 Activities ................................................................................................................................. 6
2.3 System Conditions ............................................................................................................. 7
2.4 Logic Model and Impact Chain...................................................................................... 8
III. EVALUATION QUESTIONS & METHODOLOGY...........................................12
3.1 Evaluation Questions ..................................................................................................... 12
3.2 Sample Selection.............................................................................................................. 13
3.3 Key Informant Interviews and Site Visits............................................................... 14
3.4 Focus Groups .................................................................................................................... 14
3.5 Workshop Exercises....................................................................................................... 15
IV. FINDINGS: RELEVANCE.......................................................................................16
V. FINDINGS: EFFECTIVENESS ...............................................................................20
VI. FINDINGS: IMPACT ................................................................................................. 27
VII. FINDINGS: LESSONS LEARNED AND BEST PRACTICES....................31
VIII. CONCLUSIONS AND RECOMMENDATIONS..............................................34
APPENDIX A: EVALUATION MATRIX.....................................................................36
APPENDIX B: BENEFICIARY ORGANIZATION SCRIPT ................................. 40
Acronyms 
ADS Advocacy Development Specialist 
CSO Civil Society Organization 
IT Information Technology 
LAF Lebanese Armed Forces 
LCI Lebanon Civic Initiative 
LCSI Lebanon Civic Support Initiative 
MSC Most Significant Change 
NGO Non-Governmental Organization 
OTI Office of Transition Initiatives 
PDS Project Development Specialist 
PM Prime Minister 
PPS Probability Proportionate to Size Sampling 
SMT Senior Management Team 
USAID United States Agency for International Development 
USG United States Government 
1 
Executive Summary 
The goal of this program evaluation was to document the relevance, effectiveness, and impact of 
the Lebanon Civic Initiative (LCI) and Lebanon Civic Support Initiative (LCSI) programs (2007-
2012). The evaluation assessed the manner in which the Office of Transition Initiatives (OTI) 
responded to US foreign policy priorities and shifting dynamics in Lebanon, met programmatic 
objectives, and achieved both intended and unintended impact. The evaluation also identified 
best practices and approaches from the LCI and LCSI programs. 
The Evaluation Team found that LCI and LCSI programs were based on a coherent and 
logically connected set of assumptions about how change might occur. The Evaluation Team 
endorsed OTI’s theory of change and agreed with OTI’s balancing of what was ideal with what 
was possible. OTI pushed boundaries, working on more political issues and in more contentious 
areas of the country than previous USAID programming. The advocacy component introduced 
with LCSI programming in 2010 was a key factor in this success. 
The delivery of OTI’s program of development assistance to civil society in Lebanon was 
appropriate and based on an accurate understanding of conflict dynamics in the country and at 
the sub-national level. The Senior Management Team (SMT) was aware of shifting political 
realities in the country and revised its strategy appropriately given these shifts and in response to 
on-the-ground learning. Above all, OTI’s focus on working with smaller, community-based 
organizations outside of Beirut was a strategic move that maximized impact and filled a gap in an 
already crowded donor environment. 
OTI enjoyed a friendly and constructive relationship with most of its project partners. Project 
partners were generally satisfied, not only with the funding they received from OTI, but also 
with the technical assistance provided. OTI capacity building – helping conduct needs 
assessments, set goals, implement activities, network, and report on activities in a timely manner 
– was critical to the growth of partner organizations. Some, though not all, of OTI’s project 
partners developed sufficient capacity over the course of their involvement with OTI to 
compete in competitive bid processes for future funding from USAID or other international 
donors. 
The program was effective and largely successful at the outcome level. Both organizational and 
individual learning took place as a direct result of OTI programming. Organizations and 
individuals reported positive growth as a result of their work with OTI. Some two hundred 
organizations and thousands of youth participated in an OTI-funded project activity at some 
point from 2008-2012. 
With regards to the attainment of advocacy skills, this was more pronounced in individuals than 
in organizations. Among youth, we found an increased understanding of the roles and 
responsibilities of municipal authorities. When asked what they took away from their 
involvement in either an OTI training workshop or in an activity implemented by an OTI 
project partner, most stressed personal growth. Some said they felt more self-confident; others 
said they had become more open and learned to respect other perspectives, even around 
2 
sensitive issues. Many said they developed new interpersonal skills, learned to be more 
responsible, and lost their fear of saying what they thought, even in front of an audience. 
Impact at higher levels was somewhat more difficult to determine and to attribute to the 
program. Project activities did help create new, open space for advocacy, but there was little 
evidence that this space significantly challenged political party dominance. Where participants 
demonstrated new advocacy skills or the values of active citizenship, this was primarily around 
highly local issues or municipal affairs. It remains to be seen if the youth who gained new skills 
or values as a result of their participation in an OTI-funded project will go on to also engage 
regional or national issues in the future. 
The depth and breadth of OTI’s programming alone was sufficient to help strengthen civil 
society in Lebanon. If considered only in terms of a dollar amount, this large injection of funds 
into civil society helped bolster this sector, which has historically championed democracy. Where 
the LCI and LCSI programs excelled, however, was in reaching out to new partners. 
Based on its review of the OTI program in Lebanon, the Evaluation Team set forth the 
following recommendations. These refer to learning from the OTI experience in Lebanon and 
also represent some best practices reflected in the OTI program which may be instructive for 
future development assistance programming in Lebanon or for the provision of development 
assistance in countries in a similar state of transition. 
1. Continue to engage with civil society in Lebanon. Actively seek out new 
partners in addition to supporting established NGOs. In the mentoring of 
youth who are not civically engaged and of less professional organizations, 
OTI excelled over the program period. Aside from funding, OTI was able to 
contribute less to larger and well-established NGOs. It is with smaller groups 
in strategic geographic locations outside of Beirut that OTI can offer the 
greatest added value. 
2. Ensure that this and similar small grant programs have sufficient staff to 
maintain a similarly high level of engagement with partners. New organizations 
require far more assistance than do established organizations. OTI had a 
relatively high staff-to-project ratio over the implementation period. Certainly, 
projects of comparable size have been implemented by USAID with fewer 
staff. However, the very close mentoring relationship between OTI staff and 
partners was specific to the Lebanon program and required a comparatively 
high ratio of staff-to-project partners. Furthermore, consistent staff 
engagement with project partners was a critical component of the LCI and 
LCSI programs. 
3. Allocate sufficient resources for research to be conducted before the start of 
similar programs and throughout implementation for monitoring and 
evaluation purposes. OTI’s success in effectively revising strategy was based in 
large part on adequate access to information. Focus groups, public opinion 
surveys, and other research are critical tools for measuring progress and also 
for superior strategy design and revision. 
3 
4. Assist beneficiary organizations in conducting systematic needs assessments 
rather than on an ad hoc basis. Ensure that sufficient financial and time 
resources are available for the proper conduct of needs assessments. This will 
help ensure that project activities respond to real community demands. 
5. In seeking to network multiple beneficiary organizations, consider also the 
risks of negative competition. Relationships, positive or negative, between 
individuals and organizations are likely to outlast any USAID program. 
Mitigate negative competition risk through transparent granting. Re-granting 
some organizations but not others can create false expectations about future 
funding. With organizations that are not re-granted, communicate clearly why 
this decision was made. If the organization failed to meet some expectation, 
this must be made clear. 
4 
I. PURPOSE OF THE EVALUATION 
The goal of this program evaluation was to document the relevance, effectiveness, and impact of 
the Lebanon Civic Initiative (LCI) and Lebanon Civic Support Initiatives (LCSI) programs 
(2007-2012). The evaluation assessed the manner in which the Office of Transition Initiatives 
(OTI) responded to US foreign policy priorities and shifting dynamics in Lebanon, met 
programmatic objectives, and achieved both intended and unintended impact. The evaluation 
also identified best practices and approaches from the LCI and LCSI programs. 
This report is structured to provide an overview of the evaluation mandate, a description of the 
LCI and LCSI programs, a description of OTI’s theory of change, a review of the evaluation 
methodology and activities, a detailed assessment for each of fourteen evaluation questions, and 
finally a summary and conclusion section with detailed recommendations for future US 
Government (USG) support of civil society in Lebanon or for USG support of civil society in 
other conflict-prone environments. 
1.1 Intended Use of the Evaluation 
This evaluation will be used in support of the USAID mission in Lebanon, in particular, to help 
better target development assistance with civil society actors in the country. Best practices and 
approaches discerned through this evaluation will be fed into OTI programming in other 
countries in similar states of transition. 
The primary intended user of this evaluation is OTI. However, we also acknowledge a more 
comprehensive enumeration of stakeholders (evaluation readers), including the program 
implementer Chemonics International, local OTI program staff, over two hundred beneficiary 
organizations, as well as the Lebanese public and the Government of Lebanon, both of which 
have benefited from project activities implemented by OTI’s non-governmental organization 
(NGO) partners. The thematic focus of this evaluation has been adjusted to provide information 
that is relevant and useful to the future decision-making of OTI and which may help identify 
best practices for other evaluation stakeholders. 
II. LCI & LCSI PROGRAM RATIONALE AND OVERVIEW 
In September 2007, OTI initiated programming in Lebanon to further cross-confessional 
interaction and reduce community level sectarian conflicts. OTI began programming with the 
broad mandate of “preserving democratic space,” with specific objectives to be determined by 
the field team at a later date. The task order also explained that, 
the initial approach is for OTI and its SWIFT Partner [Chemonics International] 
to work on initiatives to encourage cross-confessional dialogue and activities at 
the community level, with the view that dialogue can improve connectedness 
between the local and national levels, and ultimately improve governance 
prospects at the national level. 
5 
Both the LCI and LCSI programs have provided small, in-kind grants to a wide range of 
Lebanese organizations working with youth in marginalized and conflict-prone areas. Between 
September 2007 and August 2012, OTI staff awarded and administered 296 cash or in-kind 
grants to 196 organizations. The dollar amount of grants over both OTI programs ranged from 
a minimum of $4,000 to a maximum of $250,000, with an average grant of $77,000 (standard 
deviation: $55,000). 
2.1 Program Goals 
Program literature demonstrated how program goals changed over the course of the program, 
responding to on-the-ground learning and changes in social and political conditions. A challenge 
of this evaluation was assessing indicators of success against changing program goals. However, 
we do recognize a clear continuity between goals at different stages of the program and also that 
goals were modified only as a result of significant consideration and in consultation with a broad 
array of stakeholders, including with senior USAID/OTI and Chemonics International staff in 
Washington D.C. 
The original LCI program goals are recorded in Section IV of the 2007-2008 annual report. 
These guided the first several months of implementation and approximately the first twenty 
grants of the program: 
1. Strengthen civil society to be a more effective proponent of reform. 
2. Promote community consensus building through community development. 
In a May 2008 strategy review session, however, OTI significantly reformulated its program 
goals. This retargeting was based principally on the conclusion that that “cross-confessional 
dialogue was not a strategic area for OTI intervention” and that a political stalemate at the 
national level suggested that intervention would be more effective at the local level. 
New objectives regarding conflict mitigation were intended to respond to US foreign policy, and 
decision makers believed that these goals addressed sensitive political issues that other donors 
were unwilling to broach in addition to maximizing OTI’s quick impact approach to 
intervention. OTI stated that it could add value by targeting specific conflict “hot-spots” in the 
country. Concluding early on that sectarian tension writ large was too broad of a conflict catalyst 
to be addressed at the national level, OTI chose to concentrate activities in specific geographic 
locations with recurring conflict or with recent exposure to conflict. It was believed that 
intervention here would also have implications on the security and political environment at the 
national level. 
The next significant adjustment to the program goals accompanied the transition from the LCI 
to the LCSI program in January 2010. With the extension of OTI civic initiative programming 
in Lebanon, the team added a third goal to those agreed in June 2008; this was “to build the 
capacity of CSOs to advocate for local or national issues.” OTI staff promoted internally the 
need for this additional focus, citing a lack of advocacy and organizing skills as a recurring 
deficiency with potential organizational partners. 
6 
The addition of an advocacy and capacity building component to OTI’s work was the primary 
feature differentiating LCSI programming from LCI programming. An additional senior staff 
member joined OTI to direct these program activities. This advocacy programming was built 
around the theory of change, and with the assumption that, 
programming on local level issues will contribute to a broadening, deepening and 
strengthening of the abilities of activists to advance reform more successfully 
and to withstand attempts to neutralize their work. It was necessarily a long￾term undertaking that may be issue – rather than policy – oriented and in any 
case may not see concrete reform outcomes emerge over the three to four years 
of such a USAID program. Instead, smaller victories will likely result, along with 
a more pronounced understanding of how to both collaborate and advocate 
more effectively. 
In 2010, OTI also introduced four “intermediate results,” which could be used to measure 
progress against the three leading program goals: 
1. Youth successfully influence decision-makers. 
2. Independent space is created as a challenge to political party dominance. 
3. CSOs create linkages to lay the groundwork for collective action. 
4. Youth’s attitudes and behaviors toward conflict change positively. 
In the 2010 annual program report (March 2011), program goals were listed: 
1. Promote leadership, critical thinking, activism, and advocacy. 
2. Mitigate tensions in conflict-prone areas through reconciliation work, collective memory 
activities, and dialogue on key national and community-level issues. 
3. Enhance civil society organizations’ capacity to advocate for local or national issues.
The second of these three goals was dropped in 2011. 
2.2 Activities 
The primary program activity for OTI was the recruitment of partner organizations and the 
awarding of small grants. In the first years of the program OTI staff sought out potential 
partners; in the later years of the program, staff worked primarily with previously identified 
partners. At no time did OTI publish an open request for proposals. The program’s project 
development specialists (PDS) described their jobs as primarily to seek out capable groups of 
youth, to help them develop and refine their project ideas, and to assist them in the 
implementation of their projects over the grant period. The OTI staff we consulted with 
emphasized processes; individual and organizational learning was expected to take place through 
the implementation of projects. 
Grants were awarded according to several criteria, namely the alignment of the organization’s 
project goals with one or more of OTI’s program goals, the potential of the group to grow and 
develop, the potential for the organization to attract target youth, and the financial viability of 
7 
the proposed project. Some staff also said they considered geographic components, like 
catchment size or whether or not specific villages were politically “symbolic.” Organizations 
were not generally expected to have clearly refined project ideas, and OTI staff were committed 
to a period of consultation before each award to refine project goals and activities. Senior OTI 
staff described the process of “green lighting” a specific project as a consensual activity, whereby 
all members of the OTI team were given the opportunity to offer constructive ideas, object, or 
to otherwise contribute. Final decisions on funding were made by the Senior Management Team 
(SMT). 
With the incorporation of the advocacy component into the LCSI program in 2010, OTI placed 
additional emphasis on working with youth to engage local decision makers. For example, 
health or traffic-focused projects, which made up a sizable portion of LCSI grants, also included 
meetings with municipal leaders, like mayors, to obtain local buy-in for project goals but also to 
give youth the opportunity to formally engage with civic officials. 
The SMT was responsible for balancing the portfolio of program grants so that it reflected 
program goals at any given time and for ensuring synergy between grants. In the last year of the 
program, senior OTI staff placed additional emphasis on establishing linkages between project 
partners, with the conviction that this would help partners continue activities and resist 
cooptation in the years following the close of the LCSI program. 
2.3 System Conditions 
The LCI program responded to a period of severe political crisis for Lebanon following the 
assassination of former Prime Minister Rafik Hariri in February 2005, the withdrawal of Syrian 
forces from Lebanese territory later that year, and then a devastating 2006 war with Israel, in 
which Hezbollah and not the Lebanese Army was the primary combatant. 
During and after the launch of the LCI and LCSI programs, Lebanon continued to face serious 
threats to its political stability. These included long periods of stalemate with no president, no 
prime minister, or no cabinet, as well as heated parliamentary elections with a high risk of 
violence. A series of unsolved bombings from 2005-2008 targeted high profile figures affiliated 
with the US-backed March 14 Coalition; these included the assassinations of Gebran Tueni, 
Samir Kassir, Walid Eido, Pierre Gemayel, Antoine Ghanem, Francois al-Hajj, and others. A 
Hezbollah-led occupation of downtown Beirut shut down this major economic center for much 
of the period from December 2006 to May 2008. And in the summer of 2007, a protracted 
battle in the Palestinian refugee camp of Nahr al-Bared between the Lebanese Armed Forces and 
the militant Islamist group Fatah al-Islam resulted in the deaths of over four hundred. 
Throughout the period of implementation, conflict along sectarian and ethnic cleavages occurred 
sporadically, with major episodes of violence including the May 2008 Hezbollah and Amal 
clashes with mobilized Druze and Sunni forces in Beirut, the Chouf, and Halba. Violent conflict 
within and around the country’s twelve Palestinian refugee camps was also routine. The camps 
of Nahr al-Bared and Beddawi in the North, Bourj al-Barajneh near Beirut, and ‘Ayn al-Hilweh 
in southern Lebanon have been the camps with the most conflict internally, between Palestinian 
factions, and externally, with Lebanese security forces. 
8 
Though somewhat calmer domestic conditions prevailed in 2009-2011, political tensions 
persisted and provided the backdrop for the majority of the LCI and LCSI implementation 
period. In the last year and a half of the program, conflict in neighboring Syria threatened to
further destabilize an already fragile Lebanon. The country saw an influx of Syrian refugees 
beginning in March 2011, who primarily fled to the Wadi Khaled and eastern Bekaa areas of 
Lebanon. Providing aid to these refugees was regarded as a US political and humanitarian 
priority. 
Much of the political violence in Lebanon in the period 2011-2012 may be associated with the 
conflict in Syria. The history of Syrian intervention in Lebanon and lasting political and sectarian 
tensions in the country made Lebanon particularly vulnerable to a Syrian “spillover” effect. 
Clashes between Sunni and Allawi residents of the Bab al-Tabbaneh and Jabal Mohsen 
neighborhoods in Tripoli escalated throughout the year, with Sunni residents supporting the 
Syrian Opposition and Allawi residents supporting the regime of Syrian President Bashar al￾Assad. The killing of prominent Sunni cleric and advocate of the Syrian Opposition Sheikh 
Ahmad Abdel-Wahid by the LAF at a checkpoint in Tripoli in May 2012 provoked large 
protests, particularly within the Sunni community in Beirut and Tripoli. 
Also in May 2012, the Shia Meqdad clan retaliated to the kidnapping of member Hassan al￾Meqdad by the Free Syrian Army with a large scale kidnapping operation in which Syrian, 
Lebanese, and one Turkish citizen were targeted. Both pro-regime and anti-regime forces in 
Syria have allegedly been responsible for the kidnapping of Lebanese citizens, on both Lebanese 
and Syrian soil. 
In August 2012, former Lebanese minister Michel Samaha was indicted for his involvement in an 
apparent plot to bomb several targets in the Akkar region of northern Lebanon. And in October 
2012, in the first car bomb assassination since 2008, ISF intelligence officer Wissam al-Hassan 
and seven bystanders were killed in an explosion near Sassine Square in the Achrafieh 
neighborhood of Beirut. In November 2012, at least three were killed in Sunni-Shia clashes in 
the southern city of Sidon. 
OTI programs, globally, are valued for their flexibility and their ability to seize windows of 
opportunity. A focus of this evaluation was to determine to what extent OTI was able to 
respond rapidly to changing system conditions in line with the OTI intervention model. System 
conditions often determined which programmatic objectives and approaches were feasible. 
Effectively responding to system conditions demanded a nuanced understanding not only of the 
Lebanese political scene but also of Lebanon’s role in the region. 
2.4 Logic Model and Impact Chain 
Based on a review of project documentation and conversations with OTI staff regarding 
program theories of change, the Evaluation Team drafted a Logic Model (Figure 1) and an 
Impact Chain (Figure 2). These were reviewed by OTI in its approval of the Evaluation Plan 
and served throughout the evaluation as a reference and as a concise representation of program 
assumptions, activities, and theories. 
9 
We found that, though program goals varied slightly over the implementation period as a result 
of OTI responding to on-the-ground learning and changing system conditions, OTI maintained 
a consistent vision of how program activities might lead to the primary goal of preserving 
Lebanon’s democratic space and civil peace. This vision is represented in the Impact Chain. The 
analytical value of an impact chain in evaluation research is in the disaggregation of impact 
across various levels of intervention. The vast majority of OTI’s work over the program period 
was with individuals in small NGOs and community groups, yet the objective was to affect 
change at the national level through depth and breadth of programming. The Impact Chain 
represents a series of causal and testable links, stringing together OTI program activities with 
intended outcomes and impact at micro-, meso-, and macro-levels. 
The task of the Evaluation Team was to discern how far “up the Impact Chain” OTI program 
activities created change and also to discern, where possible, how much change at higher levels 
may be attributed to OTI programming and not to other exogenous factors. The attainment of 
impact at lower levels on the impact chain is regarded as a necessary condition for establishing 
impact at higher levels. 
As depicted in Figure 2, OTI envisioned a theory of change whereby the capacitation of both 
individuals and organizations at the local level would lead to more effective civil society 
engagement with state officials and political parties at the village, district, governorate, and 
ultimately national levels. First, both youth and organizations would learn new skills and 
strengthen or acquire key citizenship and democratic values. Second, the cumulative effect of 
individual and organizational change across thousands of engaged youth project participants and 
community members and hundreds of organizations would lead to a critical mass of youth and 
community organizations who could influence decision makers and who might support peaceful, 
democratic calls for reform over violent means of political redress. Third, these youth and 
organizations would be able to promote these values within their communities, leading to 
changes in individuals, organizations, and political structures beyond those directly engaged by 
OTI. Lastly, a shift in societal values would lead to changes at the national level in how politics 
are conducted, with new respect for democratic processes and democratic political institutions. 
Greater respect for democratic values among citizens, within political parties, and in all branches 
of government would lead to more effective national governance, thereby preserving Lebanon’s 
civil peace and democratic space. 
10
Figure 1: Logic Model 
System Conditions Program Components Program Activities Intended Impact
Program Eligibility
1.1 Local NGOs, local private entities, 
local associations, US organizations, 
public international organizations 
Target Population
1.2 Organizations working with youth 
in marginalized and conflict-prone 
areas
System Conditions
1.3a Significant governmental 
instability in period 2007 – 2012
1.3b Post-2005 Cedar Revolution, 
2006 Israel-Hezbollah war 
1.3c Successive bombings and 
assassinations, 2005 - 2008 
1.3d Nahr al-Bared conflict, 2007
1.3e Salient regional, sectarian, and 
ethnic social cleavages
1.3f High unemployment, particularly 
among youth 
1.3g Strong NGO role in social 
welfare provision
1.3h Influx of Syrian refugees
Stakeholders 
1.4a USAID / Department of State
1.4b Government of Lebanon
1.4c Beneficiaries, i.e. grant awardees, 
and local communities
1.4d Lebanese civil society, NGOs, 
CSOs, associations
1.4e Chemonics International
1.4f International providers of foreign 
development and conflict prevention 
assistance in Lebanon, e.g. UN, DfID
3.1a Identify organizations matching target population profile; 
initial contact with organizations
3.1b Invite proposals for funds
3.1c Receive and evaluate proposals against program criteria, 
including link between project activities and expected 
outcomes 
3.1d Accept, reject, or request amendments to bids
4.1 Civil society is a more effective 
proponent of reform (2007-2008)
4.2 Communities reach consensus on 
key social and political issues (2007-
2008)
4.3 Tension in conflict-prone areas is
mitigated (2008-2011)
4.4 Civil society organizations’ 
capacity to advocate for local or 
national issues is enhanced (2010-
2012)
4.5 Youth are engaged as activists and 
demonstrate leadership, critical 
thinking, and advocacy skills (2008-
2012)
2.1 Recruitment
3.2a Work with prospective awardees identified in 
recruitment phase to refine project concepts
3.2b Stakeholder engagement
3.2c Identify advocacy component of some projects
2.2 Consultation
3.3a Build capacity of awardees as needed
2.3 Implementation 3.3b Mentor organizations through project implementation
3.4a Monitor progress and reporting
3.4b Renew funding or close projects at end of project cycle
3.4c Seek publicity for grant awardee activities, in particular 
for media and campaign awards
2.4 Program Management
3.5a Maintain regional and thematic balance in grant 
allotment
3.5b Adjust strategy with changing environment
3.5c Promote inter-award synergies
2.5 Strategy and Design
11
Figure 2: Impact Chain 
National
Impact
Lebanon's democratic space and civil peace is preserved.
Community
Impact
OTI project activities and the youth engaged by these projects 
successfully change community attitudes and behaviors. 
1. Civil society is strengthened and becomes a more effective 
proponent of reform (2007 - 2008). 
2. Communities reach consensus on meaningful social and 
political issues (2007 - 2008). 
3. Tension in conflict prone areas is mitigated; less conflict takes 
place (2008 - 2011). 
4. Civil society has increased capacity to advocate for local and 
national issues (2008 - 2012).
Participant
Impact
The behavior and attitudes of youth engaged by OTI projects are 
changed, with youth embracing citizenship values. Youth engage in 
activism and display leadership, critical thinking, and advocacy skills 
(2008 - 2012).
Intermediate
Results
Independent 
space is 
created as a 
challenge to 
political party 
dominance.
CSOs create 
linkages to lay 
the ground￾work for 
collective 
action.
Youth 
successfully 
influence 
decision￾makers.
Youth’s 
attitudes and 
behaviors 
toward 
conflict 
change 
positively.
Effective
Implementation
Organizations use new or 
developed skills to implement 
effective programs.
Youth effectively implement 
projects with learned skills.
Organizational learning on 
engaging at-risk youth and 
advocating for change takes 
place.
Youth participate in OTI 
programs and learn civic 
activism and advocacy skills.
Inputs
OTI Program Activities 
(see Figure 1)
12
III. EVALUATION QUESTIONS & METHODOLOGY 
Based on a review of program documentation and initial consultations with OTI staff, 
the Evaluation Team identified twelve primary evaluation questions under four headings: 
Relevance, Effectiveness, Impact, and Lessons Identified and Best Practices. These 
reflected the mandate of the evaluation and were the primary questions guiding this final 
evaluation of the LCI and LCSI programs in Lebanon. 
3.1 Evaluation Questions 
Relevance: 
1. Was the program based on a coherent and logically connected set of assumptions 
about how change might occur, i.e. was the program designed in such a way that 
results might theoretically occur? 
2. Were OTI’s areas of programmatic and geographic focus appropriate given the 
political realities in Lebanon and OTI’s role within a larger USG assistance portfolio? 
3. Were OTI’s revisions to strategy appropriate given shifting political realities and 
windows of opportunity? 
Effectiveness: 
4. Did the program reach the stated objectives? Was the program implemented 
according to its design? 
5. Did individual activities logically address intended objectives? 
6. Did activities respond to beneficiary needs and priorities? 
Impact: 
7. To what extent were intermediate results met and successfully fed into program 
objectives? 
8. Did impact of OTI activities differ between geographic areas of interventions? If so, 
what were the distinguishing factors between districts or approaches taken? 
9. Were there significant unintended impacts that resulted from the OTI program 
beyond its stated goals and objectives? 
10. What impact, both intended and unintended, might be expected beyond the life of 
the OTI program? 
Lessons Learned and Best Practices: 
11. To what extent was OTI’s approach to civic advocacy instructive for civil society 
strengthening programming in Lebanon? 
12. Did OTI’s branding approach influence attitudes about the United States or US 
assistance in targeted communities or beneficiaries? 
13
A matrix of indicators for each evaluation question and data collection tools is provided 
in Appendix A. The Evaluation Team approached each focus area with somewhat 
different methodologies reflecting the nature of the evaluation questions. In the main, 
however, this evaluation relied on qualitative methodologies, including document review, 
focus groups, semi-structured key informant interviews, site visits, and workshop 
exercises with OTI staff, beneficiary organizations, and community members. 
Interview, focus group, and site visit notes, as well as workshop exercise results were 
translated into English. Most interviews and focus groups were also audio recorded. In 
transcription, organizational names were retained, but the names of individuals were 
redacted to preserve respondent anonymity. Interviews with the SMT were not 
anonymized. All data was manually coded to identify trends. The evaluation mandate 
was to assess the overall performance of the OTI program rather than to assess the 
performance of individual grants or specific partners. In our findings section, we utilized 
data selectively to illustrate more general findings regarding the relevance, effectiveness, 
and impact of the overall program. Where project partners are named, this should not be 
regarded as a comprehensive assessment of the named partner. 
3.2 Sample Selection 
Given the large size of the program, with nearly three hundred grants awarded over five 
years, the Evaluation Team wished to draw some general inferences regarding the 
effectiveness of the program over time and across thematic areas. In order to do so, we 
relied on a random sample of approximately ten percent of the program grants 
(approximately 25-30 grant awards). We selected this sample probability proportionate to 
grant size (PPS), and thus random, larger grants were somewhat more likely to be 
included in the sample. We chose this approach so that most of the grants selected 
would reflect a significant investment of OTI resources. This case selection accurately 
reflected the distribution of OTI grants over the five-year period, the geographic and 
thematic distribution of grants, as well as all OTI program goals. 
However, the Evaluation Team did not limit its analysis to just the random sample of 
cases. While randomization was required to draw generalizable inferences regarding the 
impact of the entire OTI program, we were also interested in determining what set the 
highest impact grants apart from “average” grants. Thus, we also drafted a High Impact 
Case Selection roster. Here, we included partners recommended by OTI staff as well as 
partners named in quarterly and annual reports as success stories. 
14
3.3 Key Informant Interviews and Site Visits 
The Evaluation Team began with a series of interviews with current and former OTI 
senior staff. The Evaluation Team queried six senior staff and four programmatic teams1
on their working assumptions, i.e. how they believed projects under their management 
might lead to the desired program objectives. Staff were asked to walk through their 
decision making process with the Evaluation Team: detailing how they recruited youth 
and partner organizations, how they set project objectives with youth, what criteria they 
used to evaluate the feasibility and potential impact of projects, etc. 
Interviews were also conducted with one or more staff for each randomly selected case 
and for each High Impact Case. A total of 29 key informant interviews were conducted 
with beneficiary organizations. Staff were asked to describe their project, their working 
relationship with OTI staff, their satisfaction with the OTI program, and their 
perception of US government assistance to civil society in Lebanon, among other 
subjects. 
In addition to the 29 site visits which accompanied key informant interviews with project 
partners, the Evaluation Team attended six OTI partner events, including trainings, 
conferences, and ceremonies. 
3.4 Focus Groups 
The Evaluation Team fed back in results from the 40 key informant interviews with OTI 
and beneficiary organization staff to establish a script for a series of follow-up focus 
groups with youth participants in OTI-funded projects and community members in 
prioritized geographic areas. The Evaluation Team moderated a total of fourteen focus 
groups. One focus group was conducted with project partners in Beirut, one focus group 
with partners in the North, one focus group with partners in the South, and two focus 
groups with partners in the Bekaa. Two additional beneficiary organization focus groups 
were conducted with one sample of media-themed grants and another sample of 
advocacy-themed grants. One focus group with community members was conducted in 
each of the North, Bekaa, and South. Furthermore, three focus groups were conducted 
with youth in the North, Bekaa, and the South. A separate focus group was also 
conducted with advocacy trainers who had worked closely with OTI partners. 
In regional focus groups with project partners, representatives of different organizations 
were brought together to discuss common experiences in working with OTI. We believe 
this component of the evaluation synergized well with the current OTI objective to build 
networks between different partner organizations. Satisfaction with the project activities 
was assessed, as were characteristics of the youth participants. The Evaluation Team in 
these focus groups looked, in particular, for evidence of leadership, critical thinking, and 
advocacy skills. 
 
1
 The Evaluation Team met with members of the North, South, Bekaa, Advocacy and Media teams in 
addition to all members of the SMT. 
15
In focus groups with community members, the Evaluation Team queried individuals on 
their reception of OTI projects in their neighborhood or village. The Evaluation Team 
was concerned not only with the relevance of CSO projects vis-à-vis objectives to instill 
citizenship values in youth participants but also in whether or not individual projects 
targeted clear needs in the communities in which they were implemented. Individuals 
were encouraged to discuss problems facing their community and whether or not they 
thought the OTI projects in their area addressed these problems. 
3.5 Workshop Exercises 
In addition to guided conversation through focus groups, moderators conducted a 
variety of workshop exercises with focus group participants. These included a Most 
Significant Change (MSC) exercise, whereby individuals were asked about the most 
significant change they could attribute to the project in the following domains:
1. Changes in the way our NGO operates 
2. Changes I’ve seen in myself
3. Changes in my community 
4. Changes in someone I work with 
Other workshop exercises included the drawing of Impact Grids and Organizational 
Lifelines. In both exercises, participants were asked to map out both positive and 
negative changes in their organization and to assess how these changes may or may not 
be related to their partnership with OTI. 
A final four hour self-review workshop was held with the entire OTI staff on November 
2, 2012. Staff were provided with some preliminary findings from the evaluation and 
were invited to reflect on their experience of working with OTI over several years. Four 
primary questions guided this conversation: 
1. Did we do what we said we would do? 
2. Did it make any difference? 
3. Was it the right thing to do? 
4. What did we do differently that helped us succeed? 
All staff also completed a questionnaire and responded to items related to strategy 
design, grant management, their relations with grantees, their relations with civil society 
more broadly, and their perception of OTI’s outreach to the Lebanese public. Staff were 
invited to reflect more generally on US government assistance to civil society in 
Lebanon. 
16
IV. FINDINGS: RELEVANCE 
1. Was the program based on a coherent and logically connected set of assumptions about how change 
might occur, i.e. was the program designed in such a way that results might theoretically occur? 
LCI and LCSI program goals and the program’s theory of change are summarized in 
sections 2.1-2.3 of this report and also depicted in a Logic Model (Figure 1) and Impact 
Chain (Figure 2). Though the initial mandate of “preserving democratic space” was 
vague, we found that OTI’s initial goals of (1) strengthening civil society to be a more 
effective proponent of reform and (2) promoting community consensus building 
through community development were a logical starting point for OTI in 2007. 
However, a timely strategy review based on a pilot program with twenty initial grants 
(2007-2008) produced valuable results. OTI correctly identified that progress against 
these two goals alone would be insufficient for achieving substantial and lasting impact. 
OTI’s strategy review session in May 2008 was a key event and reflected best practices in 
both development and conflict management programming. The SMT carefully 
considered their working assumptions about how change might theoretically occur. Four 
key conclusions of this exercise were: 
1. Even though confessional mistrust was at the heart of Lebanon’s 
political crisis, there was very little will to reform the system. Lebanon’s 
sectarian tensions were deeply structural and could not be meaningfully 
challenged with short-term funding “solutions.” Further, confronting 
sectarianism head-on with development programming was not consistent 
with OTI’s commitment to a “quick-impact” approach. 
2. Working at the local or community level held more promise than 
working at the national level. Previous attempts to reform the system 
from above had met with questionable success. 
3. Working to build consensus between confessional groups on apolitical 
issues alone would be insufficient, as “sects interact and agree on a daily 
basis on apolitical issues but always take refuge in confessional positions 
when the going gets tough.” OTI would have to directly engage with 
political issues. 
4. Working exclusively with established NGOs would produce minimal 
change, as “the donor environment in Lebanon was extremely crowded.” 
There was no “gap” here that OTI could address; significant 
programming in support of civil society, generally, was already 
widespread. In order to achieve impact, OTI would have to not only 
work with existing civil society partners but also seek out new partners. 
With the benefit of four years of hindsight, the Evaluation Team agreed that this was an 
accurate assessment of the situation at the time. Challenging the program’s initial 
assumptions in a timely manner – within six months of inception – enabled the SMT to 
recalibrate their intervention strategy to reflect a better understanding of the 
environment in which they worked. This included setting new goals. These goals 
17
represented a significant departure from OTI’s initial goals, but they were more 
appropriate to the context and reflected an accurate understanding of what was 
achievable. We found in the documentation of this strategy review exercise evidence of 
the SMT’s understanding of the dynamics of politics, conflict, and sectarianism in 
Lebanon. 
OTI’s small grant approach was a rational “bottom-up” approach to supporting change 
in Lebanon. Change could theoretically occur at both the individual level, through those 
youth engaged by OTI-funded projects, and at the organizational level, through project 
partners. Identifying “youth” and “youth organizations” rather than civil society more 
generally as the group of target beneficiaries also enabled OTI to narrow its focus, which 
was essential for maximizing OTI’s quick-impact approach. An analysis of OTI’s 
portfolio also reveals that OTI staff succeeded in working with new groups of youth at 
the very local level and not just with established NGOs. Seeking out new groups of 
youth, primarily through the personal contacts of local staff, theoretically enabled OTI to 
work within a “gap,” reaching a population that was, by and large, not being reached by 
established NGOs or by other international donors. 
The focus on individual youth, small and often informal organizations, and local or 
municipal politics was appropriate. However, one shortcoming of this highly localized 
approach was the decreased potential to achieve quick-impact at the national level, which 
was a key objective at the start of the program. While a review of OTI’s thinking and 
system conditions in Lebanon does support the conclusion that a bottom-up approach 
was a more reliable approach to achieving lasting change, any change at the national level 
would necessarily have to be a long-term goal. In general, the more local the 
intervention, the longer it will take for change to work its way up to the national level. If 
affecting change at the national level was a primary objective, then working primarily at 
the very local level was a risk, given that OTI programs are typically of short duration. 
However, in this case, this risk was mitigated by two important factors: (1) that funding 
for additional years of programming was in fact made available, making the OTI 
program in Lebanon one of OTI’s longest running programs and (2) that OTI engaged a 
very large number of beneficiaries in a limited amount of time, potentially quickening any 
cumulative effect. 
Both the LCI and LCSI programs were based on coherent and logically connected sets 
of assumptions, and these assumptions and OTI theories of change were clearly 
documented in the program literature provided to the Evaluation Team. 
2. Were OTI’s areas of programmatic and geographic focus appropriate given the political realities in 
Lebanon and OTI’s role within a larger USG assistance portfolio?
Both a conflict assessment and civil society assessment were conducted in 2008 and 
2009, respectively, near the beginning of the LCI and LCSI programs in Lebanon. These 
assessments, in addition to other OTI documentation, stressed the need to engage 
individuals and organizations across the country and at the local level: 
18
Donors should not focus their developmental work in specific regions to 
the exclusion of others. They should try to target the broadest possible 
array of regions and communities, so as maximize impact and cultivate an 
image of fairness among all Lebanese.2
 
Further, OTI emphasized in its design of the program that target geographies should be 
the North, South, and Bekaa regions, as these regions were where sectarian groups were 
most likely to recruit youth and also where there was a history of conflict which had not 
already been targeted with development programming. This prioritization was not meant 
to exclude Beirut and Mount Lebanon but rather done in response to the correct 
conclusion that the donor scene was crowded in the capital and that there was not a gap 
here that OTI could address. 
Table 1: Geographic Distribution of Grants and Funds 
Sector or Region In Evaluation 
Sample
All LCI and LCSI 
Grants
Total Grant 
Amount*
Percent of all 
Spending
Beirut 0 10 $743,534 2.8%
Civic Advocacy ** 43 $3,735,382 14.3%
Cross Regional 1 38 $3,134,950 12.0%
Media 4 23 $2,657,446 10.1%
North 8 75 $6,633,632 25.3%
South 6 47 $3,956,379 15.1%
Bekaa 7 61 $5,348,533 20.4%
Totals 26 297 $26,209,855 100.0%
* Subtotals based on estimated grant amount in OTI database (August 2012), not disbursed funds.
** Sampled separately. See methodology section.
The focus on regional programming outside of Beirut also acknowledged an important 
conclusion in the civil society assessment conducted by OTI: that national-level civil 
society programming would have been ineffective or faced debilitating opposition. The 
Evaluation Team agreed with this assessment: 
Standard advocacy-type activities involving civic organizations or 
coalitions seeking to engage, lobby or pressure the state on policy reform 
issues at the national level are unlikely to yield significant results as the 
system has shown a strong ability to deflect, derail or ignore such 
pressures throughout Lebanon’s modern history.3
 
 
2 Denoeux, Guilain, Lebanon Conflict Vulnerability Assessment: A Preliminary Summary of Findings, Management 
Systems International for USAID, 2008.
3
Jennings, Ray Salvatore, Guilan Denoeux, and Amalia Prado, An Examination of the Prospects for USAID 
Engagement Supporting Civil Society Development in Lebanon, USAID Office of Transition Initiatives, 2009.
19
OTI largely adhered to its geographic commitments. As documented in Table 1, 
approximately 60% of program funds were directly in support of projects in the North, 
South, and the Bekaa. Most Civic Advocacy Grants, Media, and Cross Regional Grants 
were awarded to Beirut or Mount Lebanon-based partners but with the expectation that 
communities in the North, South, and the Bekaa would also benefit from this 
programming. 
We found that the program was based on an accurate assessment of conflict “hot-spots.” 
OTI projects were implemented in areas of the country which had seen significant 
conflict prior to the inception of the OTI program and which remained contentious 
areas throughout the duration of the program. OTI succeeded in recruiting partners and 
implementing projects in areas of the country that had hitherto been largely inaccessible 
to USAID and to other Western donors. Some particular villages and neighborhoods 
stand out in this respect: Anjar, Mejdal Anjar, Bint Jbeil, Baalbeck, Wadi Khaled, Haret 
Hreik, Ouzai, Chiah, and Nabatieh, among others. 
3. Were OTI’s revisions to strategy appropriate given shifting political realities and windows of 
opportunity? 
As discussed above, OTI demonstrated facility in responding to on-the-ground learning 
and updated both implementation approaches and program goals accordingly. In 
addition to the recalibration of program goals in 2008 following OTI’s initial ground￾truthing exercise, the most significant revision to OTI strategy came with the 2010 
transition from the LCI program to the LCSI program. This included the addition of an 
advocacy component to the OTI program. 
In terms of program design, the Evaluation Team found that this additional component 
was merited and improved the overall strength of the program. Equipping project 
partners with additional advocacy skills would enable them to extend their reach both 
geographically and up the impact chain. The addition of the advocacy component helped 
mitigate the risk that OTI would effect change at only the very local level. We discuss 
the efficacy and impact of OTI’s advocacy grants in the sections which follow. 
The Evaluation Team questions the appropriateness of one strategy revision. In 2011, 
OTI dropped the goal to “mitigate tensions in conflict-prone areas.” The SMT explained 
that the goal had been identified as a lesser priority for several reasons. The removal of 
this goal primarily reflected the natural evolution of the OTI program in 2011 and 2012, 
where advocacy programming was considered the most efficient use of resources. Staff 
also agreed that conflict reduction was more of an indirect goal, that they envisioned this 
being accomplished as a result of progress towards the other two goals. The SMT also 
mentioned differing perceptions between OTI and its partners of the need to mitigate 
conflict. Whereas the SMT favored programming with a conflict mitigation objective, the 
local community and partner NGOs were not disposed to work on these issues as 
proposed by OTI. OTI awarded a total of 34 grants in which conflict mitigation was 
identified as a relevant country objective; however, only one grant active in 2012 was 
funded under this objective. Removing this objective from the list of program goals in 
20
2012 accurately reflected the composition of OTI’s current portfolio. Nevertheless, 
Lebanon witnessed a sharp increase of conflict in the latter half of 2012 relative to the 
period 2010-2011. It is the opinion of the Evaluation Team that the potential to reduce 
tensions in a conflict-prone area remains a compelling criterion on which to grant future 
awards. This is not a goal that should be addressed only indirectly through advocacy 
programming. 
V. FINDINGS: EFFECTIVENESS 
4. Did the program reach the stated objectives? Was the program implemented according to its design? 
A key structural point in this evaluation is the differentiation between effective 
implementation and program impact. Effective implementation refers to the 
implementation of all program activities as planned and as detailed in the Impact Chain 
(Figure 2), thereby fulfilling the following four objectives: 
1. Organizational learning on engaging at-risk youth and advocating for 
change takes place. 
2. Organizations use new or developed skills to implement effective programs. 
3. Youth participate in OTI projects, and youth learn civic activism and 
advocacy skills. 
4. Youth effectively implement projects with learned skills. 
Both organizations (NGOs and CBOs) and individuals (participants) were intended 
beneficiaries of the OTI program. The first two objectives concern organizational 
learning, and the second two concern individual learning. In this section, we evaluate 
whether or not these objectives were met. Our findings on organizational learning 
derive primarily from key-informant interviews with project staff, and our findings on 
individual learning derive primarily from focus groups with youth who participated in the 
OTI program or in activities sponsored by OTI project partners. In the following section 
(Findings: Impact), we then evaluate whether or not achieving these objectives also led to 
the fulfillment of the program’s intermediate results, higher-level outcome and impact 
measures, and plausible change at the national level. 
What set this OTI program apart from other development activities with civil society in 
Lebanon was the team’s commitment to working with new partners, informal 
associations of youth, and groups with very local concerns. In contrast to more 
conventional small-grants programs, which tend to award grants to established and more 
professional NGOs which are accustomed to bidding for awards, OTI staff sought out 
partners that would not generally have had the capacity to win an award in a competitive 
application process. OTI staff took on an intensive coaching role with project partners, 
working closely with them to identify needs in their community and to plan and 
implement activities. This entailed far more work than soliciting proposals, evaluating the 
merit of these proposals, awarding grants, and monitoring progress – the typical work 
21
process in a small grants program. One project manager said that she would recommend 
OTI as a donor to other organizations like hers, 
because they are not like the typical donor. They sit with you and work 
with you, and they are flexible. It is not only the output that matters to 
them. They are not complicated like the EU. 
The most significant change the Evaluation Team observed was in the domain of 
organizational learning. When asked to depict their organizational growth or decline 
since establishment through an organizational lifeline, all CSOs strikingly drew an 
upward trend from the beginning of their relationship with OTI. In many cases, this was 
also the point of creation of the group or organization. Partners spoke about OTI grants 
as milestones in the life of their group or organization that had much to do with the 
organizational learning and growth experienced through working on an OTI grant. The 
words of one organization are particularly reflective: 
Work with OTI helped us to develop an identity. If they had not helped 
us, we would have worked for much longer before we achieved this level 
of popularity and credibility in the area. 
Those we interviewed highlighted what they learned about conducting needs 
assessments, booking venues, finding office space, keeping financial records, purchasing 
equipment and supplies, organizing events, and preparing progress reports for a major 
donor like USAID. In the words of one project leader: 
Everything related to paperwork; this is the first things I can tell you that I 
learned. 
Organizational staff also clearly recognized that these were the skills their organization 
needed in order to obtain grants from other donors in the future. A project partner in 
the North commented: 
[OTI] opened us the space to submit other projects. The grant is over 
now, but we managed to get funding to continue our work, we are 
working with the US Embassy. 
Another partner recounted: 
When we got the grant, [the organization] was like a child. Then we 
developed a strategic plan and a business plan for five years. Also, when 
we started, we were a number of enthusiastic people. Now, we have 
about 70 committed people working with us on various projects and we 
expanded our network of donors, we now receive funding from Italy, 
Germany, Norway, Denmark and the UN. 
On interacting with OTI staff, partners voiced mixed opinions. The strength and quality 
of the working relationship between project staff and the PDS at OTI varied significantly 
across projects. However, this did not appear to be a function of staff competence. The 
same staff were regarded warmly by some partners and received critically by others. 
Rather, this appears to have been a function of individual relationships. No specific staff 
22
or regional teams were consistently criticized or celebrated more than another. One 
organizational partner commented: 
Sometimes, we have to justify too much the activities that we are doing. 
Also, if staff is changing, it can be confusing, because you have to explain 
the same thing to the new person. 
On average, organizations said that the OTI staff with whom they worked were 
professional and available when needed, bringing strong regional knowledge and helpful 
ideas. Several organizations expressed an initial concern that OTI would impose an 
American agenda, dictating what types of activities were or were not acceptable. 
However, most applauded OTI for helping their organization develop their own ideas 
and for not imposing specific types of activities. Several organizations also commented 
on the professional growth of the OTI staff with whom they worked; as the program 
progressed, the OTI staff came to better understand the structure, needs, and priorities 
of their organizations, thus making it easier to work with OTI. 
Complaints with OTI staff generally centered on procedural affairs, for example, having 
to obtain multiple estimates from different vendors for the in-kind purchase of goods. 
One representative from an organization in the South complained: 
I don’t see any impact from OTI on our administration. Simply, we worked 
more, because of the detailed OTI requirements. 
Compared to a conventional small-grants program, OTI staff spent a great deal of time 
with project partners. This was a comparative advantage of the program that allowed it 
to successfully capacitate small, nascent groups in previously unreached parts of the 
country in order to enable them to effectively undertake project activities. This 
mentoring approach necessitated a larger proportion of staff time spent in field 
compared to other grant-making programs to ensure the impact of grants. Still, some 
grantees said they would have liked even more face time with OTI staff and, in 
particular, members of the SMT, which reflects the benefit derived from OTI staff 
support and the importance placed on it by partner organizations. In the words of one 
organization: 
Several times, we had important activities related to the project, and no 
one from OTI showed up to attend that event. We would have 
welcomed more OTI presence, to feel more encouraged. 
A key factor in the program’s ability to leverage this advantage and function effectively 
and efficiently was that ex-pat OTI and Chemonics staff were co-located with the local 
team. In multiple interviews, both staff and beneficiaries cited this arrangement as a 
major benefit to the program, greatly facilitating day-to-day management and decision￾making. It further allowed for international and local staff to develop a deeper shared 
understanding of local realities and progress, thus enabling a rapid response from the 
senior management to changing circumstances on the ground. 
Frequent field visits by ex-pat SMT members to partner grantees went a long way 
building trust between grantees and the program management; this relationship 
23
development would have not been as possible had international staff been confined in 
movement or in alternative arrangements: 
They [OTI staff] are always participating in discussing ideas and agreeing 
to the program […] continuous support from them has helped the 
organization […] their involvement was very good… and it was very 
important that in these circumstances [the OTI Country Director] 
participated in our events. 
Regarding skills building, the attainment of advocacy skills was more pronounced in 
individuals than in organizations. Organizational learning was primarily the domain of 
project management; however, youth participants demonstrated an increased knowledge 
of and commitment to advocacy. One of the most effective advocacy activities, 
implemented by multiple organizations (most notably in the South), entailed setting an 
appointment with the mayor or with other municipal authorities to discuss issues in the 
community. For example, one group of youth made an appointment with their mayor to 
discuss traffic safety in their community and to agree on a strategy toward improvement. 
Reportedly, some youth were shocked to find that they could get an appointment with 
the mayor, despite not having a personal connection (wasta) to him. Simple activities such 
as this increased the confidence of youth in municipal authorities and demonstrated how 
civic activism could take place outside of party politics. Youth focus groups consistently 
cited meeting with the mayor as a point of pride, transforming their perception that it 
was impossible for youth to meet with local authorities and empowering them to do so. 
Among youth, we found an increased understanding of the roles and responsibilities of 
municipal authorities. Youth discussed how mayors should be accountable to their city 
or village and not just to the political party that backed them in elections. However, it 
was less clear that this newfound active citizenship reached beyond the municipality. 
Youth were (perhaps rightly) skeptical when, for example, it came to setting a meeting 
with their member of parliament (MP). 
When asked what they took away from their involvement in either an OTI training 
workshop or in an activity implemented by an OTI project partner, most stressed 
personal growth. Some said they felt more self-confident; others said they had become 
more open and learned to respect other perspectives, even around sensitive issues. Many 
said they developed new interpersonal skills, learned to be more responsible, and lost 
their fear of saying what they thought, even in front of an audience. An anecdote from 
one young man who participated in a theatre workshop is illustrative: 
I developed the capacity to freely express my own opinion in front of other 
people. My personality became stronger, and I became someone who could 
stand up in front of the community and say what I want. This change in 
myself came through theatre, where I was able to stand up and say what I 
wanted to say in front of an audience. 
Another youth participant shared: 
24
I don’t use my watch anymore when I am working. We worked hard and 
gave a lot of time for the project, the project became part of my social life. 
The main change was that I really began to live the project. 
Aside from a commitment to inclusivity, most OTI programming did not have an 
explicit focus on gender issues. However, especially in the North, the Evaluation Team 
heard youth, both male and female, tell compelling stories about how much it meant to 
them to be able to work in a safe space with youth of the opposite gender – an 
opportunity that was not elsewhere available to them. At least one friendship fostered by 
an OTI partner organization led to an engagement, and one of the advocacy trainers 
mentioned a marriage between the children of two formerly-feuding families – the youth 
had met in an activity organized by an OTI partner. Youth affiliated with one NGO in 
the North told a story about how they were plagued by rumors that males and females 
were interacting inappropriately; they confronted this challenge creatively, erecting glass 
walls in their office space to ease parents’ worries. In another instance, one young 
woman’s father disapproved of her involvement in an organization that also had male 
members. She invited him to attend the next meeting, and he was reportedly so won over 
that he became an active supporter of the organization and also began to volunteer his 
own time. 
5. Did individual activities logically address intended objectives? 
As the focus of this evaluation was on the entirety of the OTI program in Lebanon, the 
Evaluation Team did not conduct a detailed evaluation of activities implemented by OTI 
project partners. However, we did ask a representative sample of organizations about 
what goals they set with OTI staff, what activities they implemented, and how successful 
they felt these activities were. The activities implemented by project partners were 
extremely diverse. Among those we interviewed, some examples of activities included 
recycling campaigns, street theatre, workshops and training sessions, mural painting and 
photography, opening a public park, and setting up shelters at bus stops. Most project 
activities were not explicitly political, though advocacy component projects, especially 
those in Beirut, tended to include more political activities than those projects 
implemented by small CSOs in the North, South, and Bekaa. 
One OTI staff member commented, “The program objectives are so wide that, whatever 
we do, we will hit one of the goals.” And indeed, the correlation between project partner 
objectives and OTI’s larger programmatic goals, e.g., mitigating conflict, was not always 
obvious. This is perhaps a valid criticism, but we recognize that OTI’s theory of change, 
as discussed in the previous section, did not rely heavily on the nature of individual 
project activities but rather on the assumption that collective action around virtually any 
objective, not matter how apolitical, could create open space and instill values of active 
citizenship. 
Not all organizational representatives we met with were able to clearly articulate what 
specific activities they implemented looked like and provided vague descriptions of 
“youth empowerment activities” or “citizenship activities.” 
25
Unsurprisingly, organizations were much more willing to discuss their success with the 
Evaluation Team than their failures. In general, however, most partners we met with 
appeared to have successfully implemented activities they agreed with OTI. Where 
partners expressed disappointment in an activity, this generally had to do with minor 
failures, such as lower than expected attendance at a workshop or training session. Some 
partners also identified family pressure (the Akkar) and political pressure (the Bekaa) as 
having posed an obstacle to successful completion of one or more activities. But where 
these were identified as obstacles, project staff also demonstrated a commitment to 
overcoming these through trust building with the community, either by seeking positive 
media coverage or stressing the transparency of their organization to the community. 
In our interviews with OTI’s regional teams and PDSs, some staff noted that projects 
sometimes changed dramatically between the design phase and implementation phase. 
Larger, more established NGOs demonstrated greater fidelity to program design than 
did new or informal CBOs. The risk of non-fidelity with these smaller organizations, 
however, was mitigated by constant contact with OTI staff. Though these partners 
complained of too much paperwork (e.g. attendance lists and meeting minutes), it is 
likely that these project management checks on performance significantly improved 
fidelity to program design. 
The Advocacy Team at OTI was notably less concerned with project fidelity than were 
OTI’s regional teams. This team stressed process over performance. One member 
explained that the team had three primary objectives with each grant. Only one goal was 
achieving objectives of the campaign; equally or perhaps more important were the goals 
of “getting the advocacy skills” and “the grantees themselves working on other 
campaigns, spreading advocacy.” We felt this accurately reflected OTI’s theory of 
change. In fact, the Advocacy Team was the team of local staff that appeared to best 
understand their role in the “bigger picture” of OTI’s program. Team members were 
able to articulate how their work might not only increase capacity at the local level – at 
the level of the individual or organization – but also lead to changes in the community or 
society at large. Constant reference to OTI’s Advocacy Index appears to have helped 
with this. Indicators on this index like Coalition Building, Engagement with Decision 
Makers, and Outreach may have made higher level outcomes more proximal to the 
Advocacy Team, whereas OTI regional teams were primarily committed to also 
important, but lower level, outcomes related to capacity building, organizational learning, 
and project fidelity. 
6. Did activities respond to beneficiary needs and priorities? 
In focus groups, the Evaluation Team asked both youth and adult community members 
about what sorts of problems they saw in their community. Both youth and community 
members identified major social and economic problems: unemployment, political 
violence, school dropout, and fear and uncertainty about the future. The primary word 
that adults used to describe youth, across the North, South, and the Bekaa was “lost.” 
Community members were not generally aware of apolitical youth groups in their area 
26
and tended to associate both youth groups and NGOs with political parties. This was 
especially true in the North. 
There was no direct overlap between individual activities implemented by OTI project 
partners and needs identified by the community. This was largely because community 
members were focused on systemic problems, like unemployment, whereas OTI partners 
tended to tackle more modest local problems, like littering. However, community 
members did respond positively to the idea of apolitical youth organizations and 
activities with the argument that anything was better than youth spending time idle on the 
streets. 
Taken alone, individual project activities did not clearly address community needs, but 
taken as part of the overall OTI program, the goals of creating independent space for 
youth and the promotion of non-violent civic activism were commensurate with 
community grievances. In other words, while activities like a recycling campaign or 
street theatre did not directly appeal to the community, OTI’s larger programmatic goals 
did. 
As for addressing the specific needs of beneficiary organizations, which largely had to do 
with increasing capacity, it was the process of working with OTI that met this goal, not 
the specific activity implemented by the organization. 
The quality of the needs assessment conducted in partnership with OTI varied 
significantly case-by-case. All beneficiary organizations had access to needs assessment 
trainings or to OTI staff with needs assessment expertise, but not all organizations took 
equal advantage of these resources. Advocacy grant partners conducted the most 
thorough needs assessments. Some organizations had a clearer understanding of their 
own needs and needs in their community than did others. Some project partners spoke 
dismissively of a needs assessment as just one more piece of paperwork they had to 
complete for OTI; these were, perhaps unsurprisingly, also the organizations that would 
have benefited most from a more thorough needs assessment. However, OTI staff also 
mentioned that conducting thorough needs assessments requires a certain amount of 
resources – both in terms of time and money – and that particularly for partners who 
received small and short-term grants, these resources were not available. Thus, despite 
the provision of needs assessment trainings, the needs assessment might not always have 
been conducted effectively. Most project staff we met with spoke positively of their 
interaction with OTI in discussing needs and setting goals. However, in many cases, this 
appears to have been a very informal process. A more formal needs assessment would 
have benefited a significant number of project partners. The Evaluation Team 
recommends that this be standardized in the future. 
27
VI. FINDINGS: IMPACT 
In Figure 3, we reproduce the Impact Chain in Section 2 but employ a traffic light 
scheme to demonstrate the extent to which we believe impact was obtained and the 
extent to which change may be attributable to the OTI program. The color green 
indicates that the Evaluation Team was able to observe significant impact and to 
attribute this to the OTI program; yellow indicates that some impact was observed but 
that not all objectives were met or significant limitations apply; and orange indicates that 
the attribution of any change to the OTI program is tenuous or unclear. 
Implementation was effective at both the organization level and at the individual level. 
Given the depth and breadth of the program, coupled with this effective 
implementation, we can conclude that many organizations and many more individuals 
benefited from the OTI program. Both organizational and individual learning took place, 
and both individual and organizational capacity to plan and implement social 
development and advocacy projects increased as a direct result of the program. 
7. To what extent were intermediate results met and successfully fed into program objectives? 
The intermediate results introduced by OTI in 2010 represent a direct linkage between 
project and program activities and higher level impact measures. The Evaluation Team 
found that all intermediate results were met but that some notable qualifications apply. 
Without a doubt, significant independent space was created; many youth and 
organizations discovered valuable new means of active citizenship and political 
participation which were relatively free from many of the constraints of Lebanon’s 
political party system. However, we found little evidence that this new independent space 
meaningfully challenged political party dominance, even at the local level.4
 While in OTI 
programming, many youth may have found an alternative to participation in party 
politics, it is not clear that participation was so high that it encroached on political party 
recruitment, nor was it clear that beneficiary organizations were so influential that they 
posed a challenge to political party dominance. As some representatives from one 
partner organization noted, 
We do talk about politics in the NGO, because parties are part of society, 
and even if there are differing opinions, we managed to spend time 
together, which is a positive thing. Also, there are parties that have 
similar social goals as ours. We do not take the approach that being active 
in the NGO is an alternative to being active in a party. Youth can do 
both things. Parties also do social work, not only political work. We do 
not tell the parties: “your work is wrong, and we are right.”
 
4
It is worth adding the caveat here that political parties in Lebanon spend considerable funds on 
mobilization and recruitment across the country and remain the civil society organizations with the most 
reach locally. 
28
Figure 3: Impact Chain 
National
Impact
Lebanon's democratic space and civil peace is preserved.
Community
Impact
OTI project activities and the youth engaged by these projects 
successfully change community attitudes and behaviors. 
Civil society is strengthened and becomes a more effective 
proponent of reform (2007 - 2008). 
Communities reach consensus on meaningful social and political 
issues (2007 - 2008). 
Tension in conflict-prone areas is mitigated; less conflict takes 
place (2008 - 2011). 
Civil society has increased capacity to advocate for local and 
national issues (2008 - 2012).
Participant
Impact
The behavior and attitudes of youth engaged by OTI projects are 
changed, with youth embracing citizenship values. Youth engage in 
activism and display leadership, critical thinking, and advocacy skills 
(2008 - 2012).
Intermediate
Results
Independent 
space is 
created as a 
challenge to 
political party 
dominance.
CSOs create 
linkages to lay 
the ground￾work for 
collective 
action.
Youth 
successfully 
influence 
decision￾makers.
Youth’s 
attitudes and 
behaviors 
toward 
conflict 
change 
positively.
Effective
Implementation
Organizations use new or 
developed skills to implement 
effective programs.
Youth effectively implement 
projects with learned skills.
Organizational learning on 
engaging at-risk youth and 
advocating for change takes 
place.
Youth participate in OTI 
programs and learn civic 
activism and advocacy skills.
Inputs
OTI Program Activities 
29
CSOs established new linkages with other organizations and with other donors as a result 
of their involvement with OTI. Almost all organizations we spoke with reported 
increasing the size of their network over the course of their involvement with OTI. 
Those who were awarded an advocacy grant reported the greatest increase in their 
network size. Some organizations partnered with others to host joint events, and 
recipients of advocacy grants oftentimes were involved in joint actions. 
Youth were able to successfully influence decision makers, but this was exclusively at the 
local level. Mayors and local officials demonstrated a willingness to work with youth to 
implement specific activities. But, we did not find widespread evidence of youth 
influencing decision makers at the regional or national level. 
Youth attitudes and behaviors toward conflict did change positively. The most significant 
changes reported by youth participants were personal changes. Individuals spoke of 
feeling less anger, of stronger relationships with friends and family, and also of forming 
new friendships with members of other sectarian and ethnic groups. Youth 
demonstrated the ability to think critically about social issues and to consider alternative 
perspectives. We would expect this change in attitudes and behaviors to also influence 
youths’ position on political violence. However, in focus groups with the Evaluation 
Team, youth did not discuss sectarian or political violence and were instead more eager 
to discuss personal relationships. 
8. Did the impact of OTI activities differ between geographic areas of interventions? If so, what were the 
distinguishing factors between districts or approaches taken? 
OTI was confronted with unique conditions and challenges in different geographic 
regions. For example, political parties posed more of a challenge in the South, while 
suspicious or conservative families posed more of a challenge in the North. But in the 
attainment of impact, the Evaluation Team observed no noticeable differences that could 
be attributed to different approaches in different regions. OTI’s program in the Bekaa 
relied somewhat more heavily on the personal connections of local staff, but the 
personal connections of local staff were also a key to success in the North and the South. 
Advocacy grantees and NGOs in Beirut appeared to have been somewhat more efficient 
in their implementation of project activities, but we do not attribute this exclusively to a 
different OTI approach. These organizations also tended to be bigger, more established, 
and many also received funding from other donors. Their increased efficiency may also 
be explained by more experience and greater access to resources. 
 
9. Were there significant unintended impacts that resulted from the OTI program beyond its stated goals 
and objectives? 
Some OTI staff suggested that the OTI program in Lebanon, which was the largest 
foreign program in support of civil society in the country, may have upset some balance 
in the NGO scene. They suggested that OTI may have “spoiled” some partners by 
working with them so closely to plan projects and prepare project documentation, giving 
30
them an inaccurate perception of how donors work. When presented with the following 
statement, most OTI staff agreed that 
Even though our in-kind system and partner-based, rolling application 
approach may have unintentionally created “negative competition” and a 
“sense of lethargy” amongst civil society organizations, our close 
mentoring and support met the needs of civil society actors in Lebanon 
who would not have had the chance to succeed otherwise. 
OTI also paid above the standard NGO wage, perhaps setting false expectations for 
future remuneration. Staff pointed out that some NGOs they have worked with for as 
long as five years “still don’t get their paperwork right” and may be unprepared for a 
competitive bid process. 
In future work entailing small grants to a large number of partners, OTI might make 
specific reference to the threat of negative competition and outline strategies for 
mitigating this risk where it is perceived. 
10. What impact, both intended and unintended, might be expected beyond the life of the OTI program? 
Later LCSI grants had explicit capacity building goals, and there was some interest in 
how organizations not only met immediate project goals but also in how beneficiary 
organizations were able to leverage their learning with OTI to implement new projects 
and raise funds without assistance from OTI staff. Some staff also voiced concerns 
regarding the future success of OTI beneficiary organizations without continued support 
from OTI after 2013. There was some concern that, without continued access to OTI 
mentorship and financing, organizations might fail to meet project goals, disband, or in 
some instances, be co-opted by political actors who do not share OTI’s vision of civic 
activism. 
OTI programming is conventionally very short-term and focused on delivering quick 
impact projects in countries experiencing political crises or emerging from conflict. OTI 
programming prepares the way for future, longer-term developmental assistance from 
USAID or other donors. Thus, sustainability is not generally a high priority objective for 
OTI programs, which instead seek immediate reduction in conflict and increases in 
accountability and transparency. The LCI and LCSI programs in Lebanon, however, 
differed somewhat from OTI programs in other countries. The combined LCI and LCSI 
program in Lebanon was active for five years (more than twice the OTI average) and in 
later phases incorporated explicit capacity building objectives. Logically, the program 
envisioned medium-term and long-term change which would result from the increased 
capacity of youth and youth organizations to advocate for meaningful social and political 
change, in both local and national arenas. Typically, programs of this scope, with such 
investments of time and money and with medium-term and long-term goals, would also 
invest in sustainability. The award of “legacy grants” to select beneficiary organizations 
31
will ensure funding for some of the most promising OTI projects even after the close of 
the current program. 
VII. FINDINGS: LESSONS LEARNED AND BEST PRACTICES 
11. To what extent was OTI’s approach to civic advocacy instructive for civil society strengthening 
programming in Lebanon? 
The depth and breadth of OTI’s programming alone was sufficient to help strengthen 
civil society in Lebanon. If considered only in terms of a dollar amount, this large 
injection of funds into the civil society helped bolster what has historically been a 
strongly pro-democratic sector. Where the LCI and LCSI programs excelled, however, 
was in reaching out to new partners. CSOs performing advocacy work were heavily 
concentrated in the capital, as one might expect, given that Beirut is by far the most 
populous city in the country and also the seat of government. In contrast, CSOs in more 
rural areas or in smaller cities or villages have tended to cater to basic service provision 
or to socio-economic development demands: health services, clean water, vocational 
training, etc. OTI added significant value to civil society in Lebanon by extending 
advocacy work to the periphery – to areas of the country where no such work was being 
done. 
The message that advocacy work in cities and villages far from Beirut could increase 
confidence in local government, help citizens make appropriate demands of their 
government, and motivate young people was instructive. OTI’s focus on sub-national 
dynamics in civil society – the recognition that needs and demands vary significantly by 
region – was likewise instructive. Both Lebanese NGOs and international donors should 
heed this example. To limit development assistance in rural areas to basic service 
provision is to miss out on significant opportunities for meaningful advocacy work. 
Political parties in the country tend to use social service provision, through politicized 
NGOs or other associations, for two purposes: to buy votes in election years and to 
attract participants in other forms of political contestation, for example, protests, 
demonstrations, strikes, or occasionally, violent action.5
 Lebanese tend to regard NGO 
work as highly politicized, with NGOs functioning as vehicles for a political party (e.g., 
Future, Hezbollah) or a country with outside interests (e.g., the US, Iran) to distribute 
awards to supporters. Advocacy work with new or established service providers in areas 
outside of Beirut has real potential to transform citizen-party interactions by 
undermining this perception. In contrast to the politics of distributing private goods (e.g., 
handouts to supporters, either as cash or social services), the politics of advocacy are 
about both government and political party accountability, as well as about democratic 
means of securing public goods (e.g., clean streets, safe traffic). A preference for public-
 
5
Cammet, Melani and Sukriti Issar, “Bricks and Mortar Clientelism: Sectarianism and the Logics of 
Welfare Allocation in Lebanon,” World Politics 62(3), 2010.
32
good social welfare over private-good social welfare in a political system is strongly 
associated with democratic processes.6
 
12. Did OTI’s branding approach influence attitudes about the United States or US assistance in 
targeted communities or beneficiaries? 
OTI targeted primarily small, local CSOsbut also worked with a number of larger, well￾established NGOs. Among the latter, the Evaluation Team saw little evidence suggesting 
that the program produced any change in attitudes regarding US assistance. In this 
arena, USAID is already a well-known donor and is regarded as just one among many 
international donors (albeit the largest of them). The influence on these NGOs was 
minimal, but these larger NGOs tended to already have favorable perceptions of US 
assistance. 
As for emerging NGOs, the flexibility in the question of branding was important for the 
development of the partner’s credibility in the community. Activities were not branded 
where the partner’s image in the community might have been threatened. Thus, a 
community’s perception of US assistance can be influenced positively if linked to a 
NGO that is endorsed by the community. 
Where the Evaluation Team saw the greatest shift in attitudes about the US or US 
assistance was with smaller project partners. From these organizations, we heard a 
common story. The summary from one partner in the South is illustrative: 
I don’t have any concerns [about US government funding]. I find it 
normal to have USAID funding. In the end, I try to reach the objectives 
of my NGO… Most of the people know who is funding us, but we do 
not make it very visible… Every donor has an agenda. But I know that I 
will not follow any donor’s agenda, because I follow the agenda of the 
NGO.
There was some perception that US funding might be stigmatizing for some NGOs, and 
this was truer in the Bekaa than in other regions, but most organizations had no grave 
concerns about their work being associated with US funding. Only one of the 
approximately thirty organizations we met with stated that they carefully kept their US 
funding a secret. About half said they advertised the source of their funding by hanging a 
USAID sign in their offices or putting OTI branding on publications. 
Those organizations that did discuss concerns about advertising their relationship with 
USAID to the community, on average, said that this was more of a concern in the 
beginning of their partnership with OTI. The collaborative work process, with weekly 
instead of quarterly contact with OTI staff, did a great deal to build trust between OTI 
and partner organizations. Organizations did not feel pressured to implement certain 
activities or to subscribe to certain political positions because of their relationship with 
 
6
Bueno de Mesquita, Bruce, Alastair Smith, Randolph M. Siverson, and James D. Morrow, The Logic of 
Political Survival (Boston, MA: The MIT Press, 2004).
33
OTI. Organizations felt like OTI staff worked with them to help them realize their 
organization’s goals. Many representatives stated that they “knew” the US had an agenda 
in Lebanon but were staunch in their defense of project activities, stating that the ideas 
were their own. 
The OTI team developed a guideline document for use by staff to determine when to 
brand. Data on how branding decisions were reached for each grant was not shared with 
the Evaluation Team. However, given the large size of the program and the length of 
implementation, knowledge of OTI’s support of civil society in the country was 
widespread and generally positive. Press coverage over the period of implementation 
linking project activities to USAID funding was extensive and generally presented as a 
neutral statement of fact – not an allegation. 
34
VIII. CONCLUSIONS AND RECOMMENDATIONS 
The LCI and LCSI programs were based on a coherent and logically connected set of 
assumptions about how change might occur. The Evaluation Team endorsed OTI’s 
theory of change and agreed with OTI’s balancing of what was ideal with what was 
possible. OTI pushed boundaries, working on more political issues and in more 
contentious areas of the country than previous USAID programming. The advocacy 
component introduced with LCSI programming in 2010 was a key factor in this success. 
The delivery of OTI’s program of development assistance to civil society in Lebanon was 
appropriate and based on an accurate understanding of conflict dynamics in the country 
and at the sub-national level. The SMT was aware of shifting political realities in the 
country and revised its strategy appropriately given these shifts and in response to on￾the-ground learning. Above all, OTI’s focus on working with smaller, community-based 
organizations outside of Beirut was a strategic move that maximized impact and filled a 
gap in an already crowded donor environment. 
OTI enjoyed a friendly and constructive relationship with most of its project partners. 
Project partners were generally satisfied, not only with the funding they received from 
OTI, but also with the technical assistance provided. OTI capacity building – helping 
conduct needs assessments, set goals, implement activities, network, and report on 
activities in a timely manner – was critical to the growth of partner organizations. Some, 
though not all, of OTI’s project partners developed sufficient capacity over the course of 
their involvement with OTI to compete in competitive bid processes for future funding 
from USAID or other international donors. 
The program was effective and largely successful at the outcome level. Both 
organizational and individual learning took place as a direct result of OTI programming. 
Organizations and individuals reported positive growth as a result of their work with 
OTI. Some two hundred organizations and thousands of youth participated in an OTI￾funded project activity as some point from 2008 – 2012. 
Impact at higher levels was somewhat more difficult to determine and to attribute to the 
program. Project activities did help create new, open space relatively free from political 
party influence, but there was little evidence that this space challenged political party 
dominance. Where participants demonstrated new advocacy skills or the values of active 
citizenship, this was primarily around highly local issues or municipal affairs. It remains 
to be seen if the youth who gained new skills or values as a result of their participation in 
an OTI-funded project will go on to also engage regional or national issues in the future. 
Based on its review of the OTI program in Lebanon, the Evaluation Team set forth the 
following recommendations. These refer to learning from the OTI experience in 
Lebanon and also represent some best practices reflected in the OTI program which may 
be instructive for future development assistance programming in Lebanon or for the 
provision of development assistance in countries in a similar state of transition. 
35
1. Continue to engage with civil society in Lebanon. Actively seek out 
new partners in addition to supporting established NGOs. It is in the 
mentoring of youth who are not civically engaged and of less 
professional organizations that OTI excelled over the program period. 
Aside from funding, OTI was able to contribute less to larger and well￾established NGOs. It is with smaller groups in strategic geographic 
locations outside of Beirut that OTI can offer the greatest added value. 
2. Ensure that this and similar small grant programs have sufficient staff 
to maintain a similarly high level of engagement with partners. New 
organizations require far more assistance than do established 
organizations. OTI had a relatively high staff-to-project ratio over the 
implementation period. Certainly, projects of comparable size have 
been implemented by USAID with fewer staff. However, the very 
close mentoring relationship between OTI staff and partners was 
specific to the Lebanon program and required a comparatively high 
ratio of staff to project partners. Furthermore, consistent staff 
engagement with project partners was a critical component of the LCI 
and LCSI programs. 
3. Allocate sufficient resources for research to be conducted before the 
start of similar programs and throughout implementation for 
monitoring and evaluation purposes. OTI’s success in effectively 
revising strategy was based in large part on adequate access to 
information. Focus groups, public opinion surveys, and other research 
are critical tools for measuring progress and also for superior strategy 
design and revision. 
4. Assist beneficiary organizations in conducting systematic needs 
assessments rather than on an ad-hoc basis. Ensure that sufficient 
financial and time resources are available for the proper conduct of 
needs assessments. This will help ensure that project activities respond 
to real community demands. 
5. In seeking to network multiple beneficiary organizations, consider also 
the risks of negative competition. Relationships, positive or negative, 
between individuals and organizations are likely to outlast any USAID 
program. Mitigate negative competition risk through transparent 
granting. Re-granting some organizations but not others can create 
false expectations about future funding. With organizations that are 
not re-granted, communicate clearly why this decision was made. If the 
organization failed to meet some expectation, this must be made clear. 
36
APPENDIX A: EVALUATION MATRIX 
Evaluation Question Indicators Data Source
Relevance
R1. Was the program based on a coherent and logically 
connected set of assumptions about how change might 
occur, i.e. was the program designed in such a way that 
results might theoretically occur?
The program was based on an accurate analysis of conflict 
in the country and understanding of civil society.
Each link in the logic model and impact chain is 
theoretically realizable.
Project document review
Conflict and civil society 
academic literature review
Expert review
R2. Were OTI’s areas of programmatic and geographic focus 
appropriate given the political realities in Lebanon and OTI’s 
role within a larger USG assistance portfolio?
The program was based on an accurate assessment of 
conflict “hot-spots.”
The majority of OTI resources were utilized on projects in 
the North, South, and Bekaa rather than in Beirut.
There is evidence that national-level civil society 
programming would have been ineffective or faced 
debilitating opposition.
Sectarian groups exert influence over or recruit individuals 
from the locations where OTI implemented project 
activities.
Project document review
Conflict and civil society 
academic literature
Interviews with USAID –
Washington D.C.
Expert review
R3. Were OTI’s revisions to strategy appropriate given shifting 
political realities and windows of opportunity?
Revisions of programmatic focus (goals) were based on 
changing social and political conditions and reflected real 
on-the-ground learning.
OTI set new goals as windows opened and scaled back 
other goals when windows closed.
OTI had an accurate understanding of how system 
conditions were changing during implementation.
Project document review
Conflict and civil society 
academic literature
Expert review
37
Effectiveness
E1. Did the program reach the stated objectives? Was the 
program implemented according to its design?
Organizational learning took place
Organizations implemented projects as agreed with OTI 
staff (i.e., planned activities actually occurred).
Individual learning took place
Youth participated in projects.
Beneficiary organization 
interviews
Participant interviews
Focus groups
Consultations with OTI staff
E2. Did individual activities logically address intended 
objectives?
Each project activity was associated with a specific program 
goal, and project activities could theoretically have 
contributed to meeting the identified program goal.
Project goals were clearly aligned with program goals.
Database review (random 
case selection)
Consultations with OTI staff
Beneficiary organization 
interviews (random case 
selection)
E3. Did activities respond to beneficiary needs and 
priorities?
A needs assessment took place with each civil society 
partner.
Beneficiary organizations felt that OTI understood their 
needs and priorities.
Communities thought that OTI projects responded to their 
needs and priorities.
Beneficiary organization 
interviews
Participant interviews
Focus groups
Consultations with OTI staff
Survey results
38
Impact
I1. To what extent were intermediate results met and 
successfully fed into program objectives?
OTI staff demonstrate knowledge of intermediate results.
Youth successfully influence decision-makers.
Independent space is created as a challenge to political 
party dominance.
CSOs create linkages to lay the groundwork for collective 
action.
Youth’s attitudes and behaviors toward change positively.
Beneficiary organization 
interviews
Participant interviews
Focus groups
Consultations with OTI staff
Change stories (MSC)
I2. Did impact of OTI activities differ between geographic 
areas of interventions? If so, what were the 
distinguishing factors between districts or approaches 
taken?
The magnitude of change stories (MSC) stories varies by 
region (e.g. North, South, Bekaa) or catchment size (e.g. 
village/city population).
Differences can be explained by OTI action rather than 
unique geographic features.
Beneficiary organization 
interviews
Focus groups
Change stories (MSC)
I3. Were there significant unintended impacts that resulted 
from the OTI program beyond its stated goals and 
objectives?
Individuals or organizations attribute change to OTI 
intervention which was either not part of the OTI theory of 
change or incongruent with the broader goal of preserving 
democratic space and civil peace.
Beneficiary organization 
interviews
Participant interviews
Focus groups
Change stories (MSC)
I4. What impact, both intended and unintended, might be 
expected beyond the life of the OTI program?
Partners state that they believe they can/cannot continue 
activities beyond their relationship with OTI.
Past partners (organizations with closed grants) 
demonstrate lasting ability to advocate, provide youth with 
alternatives to conflict.
Beneficiary organization 
interviews
Participant interviews
Focus groups
Consultations with OTI staff
39
Lessons Identified and Best Practices
L1. To what extent was OTI’s approach to civic advocacy 
instructive for civil society strengthening programming 
in Lebanon?
Organizations awarded grants through the OTI program 
implemented additional projects with non-OTI civil society 
partners.
Civil society organizations which were not-funded by OTI 
imitated or adopted activities implemented by OTI 
partners.
Beneficiary organization 
interviews
Consultations with OTI staff
L2. Did OTI’s branding approach influence attitudes about 
the United States or US assistance in targeted 
communities or beneficiaries?
Beneficiaries (partners, participants and communities) are 
aware that project activities are funded by USG
Change stories credit USG assistance
Change stories (MSC)
Beneficiary organization 
interviews
Participant interviews
Focus groups
40
APPENDIX B: BENEFICIARY ORGANIZATION SCRIPT 
1. Based on our conversations with OTI, we understand that your organization in 
the year _______ was awarded a grant entitled __________________________. 
Your organization may have been awarded more than one grant. Could you tell 
us a little more about the broad parameters of your collaboration with OTI? 
a. What is your role in the organization? How long have you been with the 
organization? 
b. How did your organization first come into contact with OTI? 
c. Why do you think that OTI chose to work with your organization? Do 
you think you organization’s objectives match OTI’s objectives?
d. How did you identify needs in your community? Was this done with a 
particular assessment methodology? What was the conversation like with 
OTI staff? 
e. Do you remember the exact project goals you agreed with OTI? 
f. What sort of activities have you implemented with the funds or material 
provided by OTI? 
2. We are interested in learning more about the kinds of people your organization 
works with. Many of OTI’s partners work mostly with young people. Can you 
tell us a little about the people involved in your organization? 
a. What kind of people are working on this project? Teenagers, young 
adults, adults, the elderly? Were they from the area? Of average means? 
Well educated, still in school? 
b. Who does what? Can you describe the division of labor? 
c. What kind of people does your organization reach out to? That is, who 
would you describe as your beneficiaries or clients? Or, who do you try 
to attract for special events? Community members, students, municipal 
leaders, the poor or disadvantaged, drug users? 
d. Would you describe your organization as local (village or neighborhood), 
regional (e.g., North, South, Bekaa), national, or international? 
3. Can you tell me more about some of the activities you planned with OTI’s 
support? 
a. What activities do you think were the most successful? 
b. Which were you and your team most proud of? Which did you wish had 
gone better? 
c. With any sort of activities, some things go well, and other things don’t 
quite go as we planned. What do you think you and your organization 
have learned from your success and failures? 
41
d. To what extent do you think this partnership with OTI has helped your 
organization meet its goals? 
4. We would like to know a little bit more of what you thought of the OTI staff you 
worked with. What you tell me will not be directly associated with your name or 
the name of your organization, nor will what you tell me be considered in future 
applications for funding by your organization. 
a. Overall, would you say your interactions with OTI staff were positive, 
negative, or a combination of both? Why? 
b. Were OTI staff capable and professional in your opinion? 
c. Do you feel like the OTI staff you met with really understood your 
organization and what it was trying to accomplish? If not, what do you 
think they failed to recognize? 
d. How did you set project goals with OTI? Do you feel like your 
organization and the OTI staff you worked with shared similar values and 
were interested in achieving similar goals? 
e. Were OTI staff minimally involved, somewhat involved, or very involved 
in your project? Do you wish they had been more/less involved? 
f. Would you recommended OTI as a partner/funder to other 
organizations like your own in Lebanon? 
5. Have any of your activities been covered by the press. In a local newsletter or 
newspaper? On TV? 
a. If yes, can you give us a copy of this? 
b. If yes, did anyone at OTI help you in your publicity efforts? Can you tell 
us a little more about the nature of this assistance? 
c. Did you and OTI discuss any media plan related to your project? How 
helpful was this conversation? 
6. Most OTI grants to organizations like yours are only for a few months, maybe a 
year. Many OTI organizational partners also have other sorts of funding, for 
example, from other donors or contributions from the community. 
a. How critical was OTI funding in your larger financial portfolio? Did 
OTI provide you with funds when you needed them most? 
b. Did you have to significantly alter the way you work after you received 
the grant? After the end of the grant? (If grant ongoing, ask about 
expectations.) 
7. Different people in Lebanon may have different opinions about how appropriate 
US government funding is for Lebanese civil society organizations like yours. 
a. Was the source of funding ever a concern for you? If so, what were your 
major concerns? How were these concerns addressed? 
42
b. Did you advertise your organization’s relationship with USAID, for 
example, on posters or in newsletters? 
c. Do you regard USAID as a impartial, or do you worry that USAID has 
an unhelpful political agenda? 
8. We are engaged in a program evaluation of OTI work in Lebanon. This means that 
we are trying to determine if OTI’s partnerships with organizations like your own 
succeeded in making any change. Do you have any material, printed or 
otherwise, on your organization that you think we might find useful?