DRG LEARNING, EVALUATION AND RESEARCH ACTIVITY: TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT DECEMBER 2018 This publication was produced for review by the United States Agency for International Development. It was prepared by NORC at the University of Chicago. Author views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT FINAL REPORT December 2018 Prepared under Contract No.: GS-10F-0033M / AID-OAA-M-13-00013 Submitted to: USAID Submitted by: Sahar Tabaja (Evaluation Specialist) Nehad Fattah (Evaluation Specialist) Audra Grant (Team Leader) Contractor: NORC at the University of Chicago Attention: Renee Hendley Bethesda, MD 20814 Tel: 301-634-9489: Email: Hendley-Renee@norc.org DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | iii TABLE OF CONTENTS EXECUTIVE SUMMARY.................................................................................................................................1 EVALUATION PURPOSE AND EVALUATION QUESTIONS ........................................................1 CORE EVALUATION QUESTIONS .......................................................................................................2 KEY FINDINGS AND CONCLUSIONS ................................................................................................3 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS ................................................9 1.1 EVALUATION PURPOSE..............................................................................................................9 1.2 EVALUATION QUESTIONS........................................................................................................9 2.0 PROJECT BACKGROUND ........................................................................................................... 11 2.1 POLITICAL CONTEXT.............................................................................................................. 11 2.2 PROGRAM CONTEXT, DESIGN AND MANAGEMENT................................................. 12 3.0 EVALUATION METHODS AND LIMITATIONS ..................................................................... 14 3.1 STUDY DESIGN ........................................................................................................................... 15 3.2 TARGET POPULATIONS .......................................................................................................... 17 3.4 LIMITATIONS ............................................................................................................................... 18 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ............................................... 20 4.1 EVALUATION QUESTION 1.................................................................................................... 20 4.2 EVALUATION QUESTION 2.................................................................................................... 27 4.3 EVALUATION QUESTION 3.................................................................................................... 36 4.4 EVALUATION QUESTION 4.................................................................................................... 42 4.5 ADDITIONAL EVALUATION QUESTIONS ........................................................................ 56 5.0 SUGGESTIONS FOR FUTURE USAID PROGRAMMING ..................................................... 62 Annex 1: Data Collection Instruments........................................................................................................0 Annex 2: Scope of Work............................................................................................................................. 66 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | iv ACRONYMS AA Autonomous Administration in Northern Syria AOR Agreement Officer Representative CEC Civic Education Center CEPPS Consortium for Elections and Political Processes Strengthening CSO Civil Society Organization DQA Data Quality Assurance DRG-LER Democracy, Human Rights, and Governance Learning, Evaluation, and Research Activity FGD Focus Group Discussion GOS Government of Syria HNC High Negotiations Committee HTS Hayat Tahrir Al-Sham IFES International Foundation for Electoral Systems IRI International Republican Institute IS Islamic State ISIS Islamic State in Iraq and Al-Sham KII Key Informant Interview LACU Local Administrations Council Unit LC Local Council M&E Monitoring and Evaluation NGO Non-Governmental Organization NDI National Democratic Institute NORC National Opinion Research Center PRIDE Promoting Inclusive Democratic Engagement PMP Performance Management Plan PYD Kurdish Democratic Union Party SIG Syrian Interim Government SNC Syrian Negotiations Committee SOC Syrian Opposition Council CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | v SOW Statement of Work START Syria Transition Assistance Response Team USAID United States Agency for International Development USG United States Government CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 1 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS As part of the DRG Learning, Evaluation, and Research (DRG-LER) Activity, the United States Agency for International Development (USAID)-Middle East Bureau (MEB) requested that NORC design and budget a performance evaluation of the USAID Syria Transition Assistance Response Team (START) Promoting Inclusive Democratic Engagement (PRIDE II) Project. The project began in 2015 and is implemented by the Consortium for Elections and Political Processes Strengthening (CEPPS), comprised of the National Democratic Institute, (NDI), the International Republican Institute (IRI) and International Foundation for Electoral Systems (IFES). This performance evaluation assesses to what extent the PRIDE II program interventions have changed representation and participation among vulnerable segments of the Syrian population – women, youth, and minorities – and have enhanced the capacity of government to facilitate inclusive democratic processes and institutions in Syria. With Syrian institutions collapsed as the civil war continues, elected officials and citizens lack the mechanisms to engage in democratic governance and understand their respective roles and responsibilities in the governance processes. Officials may not fully grasp the importance of inclusive government, particularly towards traditionally marginalized groups such as women, youth, and minorities. Such deficiencies have led to a widespread lack of accountability, responsiveness, and inclusivity on local, sub-national and national governance initiatives to identify and address community priorities. Similarly, the Syrian people’s experience with political and civic participation remains nascent, and emerging citizen groups are in need of continued technical assistance. Syrians lack faith in the efficacy of civic participation efforts and are unfamiliar with the processes of collaborating to identify and forge solutions to common concerns—both bedrock principles for successful local-level governance in the near and longer terms. Consequently, Syrian citizens must build a knowledge base and skillset of democratic practices that enable them to deliberate, negotiate, make informed decisions, and participate in democratic processes such as community organizing and advocacy at the local level. The PRIDE II project seeks to increase the capacity of government and political leaders, civil society actors, women’s groups, and citizens at the grassroots level to support the eventual establishment of inclusive and effective democratic processes and institutions in Syria. The current performance evaluation is intended to help USAID START and the USAID-MEB better understand the effects of key types of PRIDE II interventions. The findings will also be used by USAID START and the USAID-MEB to develop future activities that contribute to peace-building and democratization. The primary audience for this evaluation is the USAID￾MEB. Other central consumers include START, the Middle East Regional Platform (MERP), and CEPPS partners, NDI, IRI and IFES. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 2 CORE EVALUATION QUESTIONS The evaluation focuses on PRIDE II project activities implemented between July 2015 and June 2017, emphasizing the period during which on-the-ground implementation activities occurred. The evaluation questions address the performance of the PRIDE II interventions and are based on the main program objectives and intermediate results outlined above. The core questions, therefore, are summarized as follows:1 What outcomes (anticipated and unanticipated) do staff at target CSOs perceive as resulting from NDI, IRI and IFES-supported activities? Have levels of civic engagement increased, particularly among under-represented groups as a result of enhanced understanding of processes related to the political transition?  Are there improvements in awareness of democratic principles for those exposed to IFES civic and voter messaging?  Has IFES technical assistance led to more CSO engagement in consensus building?  How many and what kinds of citizen-led advocacy efforts emerged among groups targeted by NDI’s citizen participation activities?  To what extent have these advocacy efforts been successful in achieving their intended results?  What factors were associated with the emergence of citizen-led advocacy efforts among groups targeted by NDI’s activities? Do national and sub-national representatives better understand their democratic roles and responsibilities and their role in peacebuilding?  To what extent has the IRI-supported program contributed to accomplishing goals of enhancing skills and understanding of democratic roles and responsibilities among program participants?  How has the IRI-supported program participant knowledge of consensus building strategies and communications skills improved?  To what extent has the Musharaka Forum been effective in the context of shifting conflict dynamics?  To what extent does the present Musharaka model continue to be viable in the current context? Are any elements of the model promising for continued 1 In order to address the core evaluation questions and capture the dynamics of the PRIDE II project, the team was asked to include questions that measured more specific aspects of each implementer program. Those questions are elaborated on in the subsequent section. 2 1 3 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 3 implementation? Have any elements of the model proven to be ineffective in the current context? What activities are programs are working well and why? Which are not working well and why? What examples can be identified as successful stories or failures? A summary of the key evaluation findings is below, based on qualitative and quantitative data collection from key informant interviews (KII), focus groups, and a rapid assessment survey. KEY FINDINGS AND CONCLUSIONS Fieldwork findings suggest that both within PRIDE II beneficiary households, communities, and local-level – as well to a small degree, national-level institutions – women beneficiaries in Syria and Turkey have the potential to become an important locus of activism, information and learning, diplomacy, and change. Results reveal that beneficiaries have greater confidence in their ability to speak publicly, understand democratic processes, engage in advocacy, and lead in planning and management with local councils. Women, furthermore, increased understanding among their men counterparts. Women beneficiaries’ role in the family and in public life, likewise, expanded, in further reflection of women’s vital potential role.  NDI. NDI women beneficiaries, for example, indicate changing perceptions of their traditional role in society and a greater desire to engage in political processes. As a result of NDI’s interventions, women beneficiaries believe they are more effective in other aspects of their lives. In addition, the NDI program facilitated mobilization and the initiation of awareness sessions on women’s role in political and decision-making processes among other issues. Women, furthermore, have increased understanding of political processes and related concepts among their male counterparts.  IRI. Women assert they are better able to exercise advocate on behalf of their communities, and influence local decision makers following their involvement in IRI activities. The women also are, now, more comfortable voicing their opinions in public settings. IRI women beneficiaries’ advocacy efforts led to the establishment of women’s sections in over 20 local councils. While attitudes among local council members may still reflect traditional gendered views, the offices could become mechanisms for the institutionalization of Syrian women’s local interests and future engagement. At the national level, IRI women beneficiaries’ engagement through the IRI women’s network informed recommendations on the role of women in the future transition of Syria that reached the office of Staffan de Mistura, the Syrian Opposition Council (SOC), and the High Negotiations Committee / Syrian Negotiations Committee (HNC / SNC).  IFES. The evaluation similarly finds promise among women beneficiaries of the IFES Musharaka Forum. Women and girls are just as likely as men to believe that they can be effective leaders in Syria in the future, and also be agents of positive change. According to interviews with forum staff, young women are especially interested in participating in the forum. Women linger on the margins of political participation, 4 1 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 4 both in Syria and Turkey. The forum successfully reaches these young women in Turkey, who, along with their male counterparts, developed a greater sense of purpose and expressed a strong desire to contribute to events in Syria in the future. Thus, the Syria PRIDE II project is a useful point of departure for facilitating a larger role for women in public life. Women PRIDE II beneficiaries show increased knowledge, mobilization capacity, and participation in politics, and, importantly, illustrate Syrian women’s capacity to emerge as key decision makers and influential democratic actors in Syria as well as Turkey. At the individual-level, increased knowledge transfer on gender, democracy, and the Syrian political transition is widespread across the entire PRIDE II project, as evidenced in KIIs, focus groups and the rapid assessment survey. In an outcome that performed especially well, beneficiaries across all three programs, on the main, indicated greater knowledge and understanding of key democratic concepts and processes, as well as a capacity to share that knowledge with others in their households and community. The project appears to produce more sustainable outcomes on these results, as beneficiaries report a more nuanced understanding of concepts and processes and confidence in actually applying them in their communities. Concepts such as “human rights”, “citizenship”, “elections”, “constitution”, and “women’s rights”, among others, resonated strongly with beneficiaries. The evaluation also finds that local council engagement successfully translated into increased local council responsiveness to citizen-led demands. Though local councils often lacked the financial ability to support projects, local councils supported grassroots efforts to mobilize funds for and to implement projects that met citizens’ demands.  NDI. Local councils and decision makers under the NDI program became more responsive to citizens, as they attended civic education sessions with beneficiaries, and similar to citizen beneficiaries, adopted terms and concepts in their other meetings and engagements with the community. The NDI program produced over 40 citizen-led advocacy campaigns that culminated in improvements in local infrastructure and services following beneficiaries’ direct engagement with local decision makers. Councils did not always have the resources even if they had the willingness to assist citizens. Instead, responsive local councils also supported grassroots efforts to mobilize funds for and implement projects that met citizens’ demands. Relatedly, NDI beneficiaries feel more educated on the roles and responsibilities of local council members and key decision makers in their locales since their involvement with the NDI program. Accordingly, they feel more secure and willing to approach decision makers due to their improved understanding of political processes and terminology. Local council members and other local decision makers found they were more receptive to meeting with concerned citizens, and displayed enhanced willingness to grant permission for initiatives and assist group advocacy efforts. In the rapid survey of 21 local decision-makers engaged by NDI, 78 percent said they had been asked to provide funding, 30 percent were asked to build support in the community, and 26 percent were 2 3 4 5 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 5 asked to participate. All said they had agreed to the request.2 An overwhelming 96 percent stated that their levels of engagement with citizens had improved and increased in frequency after their experience with citizens’ advocacy. Increased engagement was attributed to enhanced comprehension of the role of citizens, as well as their relationship to local representatives.  IRI. While the opening of women’s offices in local councils was an important achievement and a positive step forward for the IRI program’s women beneficiaries, both beneficiaries and project staff were skeptical of deepened attitudinal change amongst the male-dominated local councils, however. Several interviewees noted that the women’s offices delegated gender stereotyped functions such as women and child programs, a gesture that some felt was an effort to placate women rather than institutionalize women’s interests in a formal way.  IFES. Though Musharaka Forum engagement with local decision makers was more limited, Musharaka Forum participants met with Syrian local council members in Turkey, and planned for future meetings and engagements with council members. Subnational and national-level actor change is also evident.  NDI. The NDI program led to local-level decision maker responsiveness to citizen￾led demands. At the national-level, engagement was limited, as the program’s focus is local-level engagement.  IRI. IRI did very well in connecting women in country with national- and international-level representatives. Indeed, it provides perhaps the only instance of a concrete linkage created between citizens inside Syria and their representatives in Geneva that evaluators observed as a result of PRIDE II activities. Through IRI, women’s groups galvanized grassroots consultations across governorates with Syrian women and reflect their aspirations and demands for potential transition to the Geneva negotiations through a paper that was submitted to de Mistura’s office.  IFES. Most prominent for IFES, is the change in the knowledge, attitudes and behavior of the national opposition actors that can be attributed to the electoral technical assistance provided by IFES to SOC leadership, the former HNC members, and Syrian Interim Government (SIG) staff. Formal opposition group beneficiaries spoke highly of the quality of the training and technical assistance provided, and credited the program with several initiatives undertaken by the SIG, SOC, and now, SNC, to advance democratic governance in opposition-held Syria. The Syrian conflict undoubtedly impacted PRIDE II activities across NDI, IRI, and IFES implementer programs, however. While results suggest that each of the partners 2 This was not a random sample of decision makers as access to all decision makers in the areas of operation was not possible or feasible. The evaluation team asked to be provided with a list of decision makers who were approached by citizens as part of advocacy activities. Given the project was likely to keep in contact with those decision makers who had supported advocacy efforts, we expect bias in this group of decision makers. 6 7 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 6 adapted to the conflict through creative use of remote technologies for monitoring, trainings, meetings, and information-sharing, some activity implementation remains complicated by conflict dynamics inside and outside of Syria. Specifically, IRI and IFES’s work with opposition groups is challenged by fragmentation within the national formal Syrian opposition, and members that are currently in exile. As a more immediate consequence of the ongoing violence, partners are difficult to access for programs, and in addition, their standing inside of Syria may have eroded among the broader population. In addition, the link between citizens and national-level actors and structures remains weak in the deteriorated Syrian environment. The evaluation finds that the operational environment also affected the rhythm of some PRIDE II activities in other ways, however. IFES partners in Washington, DC and in Turkey, for example, pointed to difficulties with processing financial transactions, namely payments to partners, due to restrictions on transferring funds to Turkey and Syria. Partners also mentioned the lengthy USAID vetting process for candidate partner organizations – a process that is acknowledged as unrelated to the conflict. These issues slowed down or delayed project activities and have thus, affected project work plans. On occasion, information requests from USAID are also challenging, as the originating information, in some cases, has to come from Syria. Depending on the security situation and on-the-ground program demands, quick partner response may be difficult, and pull managers from other critical activity tasks. Several program weaknesses revealed themselves during the evaluation, however.  NDI. Despite inroads of engagement with communities with local- and national-level decision makers, effecting substantive attitudinal change amongst these actors appears more challenging. While local decision-makers were willing to meet with NDI groups to discuss their demands and take action on service-based issues, they were less open to exploring or negotiating changes of a more political nature. Thus citizens’ ability to advocate for change around political issues such as representation, inclusion, participation, or transparency remained limited. Citizen-led service-based advocacy was relatively more successful, by contrast. Even IRI women beneficiaries acknowledge their successful advocacy with local leaders (most of whom were men), they nevertheless expressed concern that leaders ultimately only offered token gestures to women.  IRI. The significant gains made at a grassroots level towards enhanced knowledge and awareness of democratic principles, and citizen advocacy, did not palatably translate into concrete linkages with national level decision makers. The various opposition bodies – SOC, SIG and SNC – remain distant from Syrians in Turkey, and even further still from citizens inside Syria. Overall, less than five national-level decision-makers participated in program activities or connected through them to beneficiary citizens in any way. With the exception of the work of the IRI-supported network activity among women, the evaluation found no evidence of any feedback mechanism created between citizens and their views, positions, and activities on the ground, and national opposition leaders negotiating on citizens’ behalf. 8 9 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 7 Although, IRI partners expanded formal opposition learning around their democratic roles and responsibilities, the capacity of key actors – the SOC – is limited, as is their impact on a national-scale. Funding to the SOC is also minimal, and individuals differ in their ideas on how to accomplish goals. These weaknesses, however, are largely due to the political context and the Syrian opposition, which is internally fragmented and in exile. Increases in women’s engagement in the SOC and SIG was not very robust. Though this goal face political impediments unrelated to women’s engagement per se, female SOC staff reported a lack of change in SOC and SIG leadership attitudes towards the importance of women’s participation. The majority-male SOC political committee is supportive of the issue, but in practice, takes very little action internally, SOC staff stated. Indeed, the number of women on the political committee of the SOC declined during the life of the PRIDE II project. Despite the immense challenges of working in Syria and of the conflict, the PRIDE II project accomplished a great deal in term of key outcomes, amid threats to beneficiaries’ safety and a myriad of political factors that limit the political and social space for beneficiaries in both Syria and Turkey. Beneficiaries even perceive the conflict as a source of opportunity. For instance, while beneficiaries believe that more could certainly be achieved without the conflict, they believe the conflict provides an unprecedented opportunity to be involved in democratization activities that would otherwise be impossible under the Assad regime. PRIDE II project implementation across NDI, IRI, and IFES activities is facilitated by a strong sense of trust in program personnel, sub-grantee staff, and consultants. Many beneficiaries describe the ongoing concern of being associated with opposition groups or destabilizing forces, thus for some, joining PRIDE II activities comes with risk. Across the board, trust established by partners encouraged participation by adopting strategies that bolstered beneficiaries’ sense of security. According to the rapid assessment survey, nearly all (98%) PRIDE II beneficiary respondents said trust in activity staff influenced their decision to participate in their respective program activity, and similarly large portion (94%) say that trust will likewise influence their participation in future PRIDE II activities. Despite the complexity of the project environment and breadth of the PRIDE II project, coordination between NDI, IRI, and IFES appears rather limited. Though partner activities are complementary and operate in similar geographic areas in Syria and Turkey, there appears little regular coordination related to when activities are conducted, or on the possibility of capitalizing on existing intervention synergies. Communication itself tends to be minimal, and when it does occur according to one CEPPS officer, it is usually at the initiative of office managers and personnel, rather than regular meetings or discussions. Similarly, findings suggest that coordination and communication between PRIDE II and USAID could also be improved in some areas. This would keep USAID and PRIDE II partners informed of implementers’ respective activities, developments and program 10 11 12 13 14 15 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 8 challenges. Communication frequency is also critical, given the fluidity of the operational environment, and would facilitate more information-sharing, joint problem-solving, learning, and ultimately smooth program execution. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 9 1.0 EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE This performance evaluation assesses to what extent the PRIDE II program interventions have changed representation and participation among vulnerable segments of the Syrian population – women, youth, and minorities – and have enhanced the capacity of local government to facilitate inclusive democratic processes and institutions in Syria. The evaluation will help START and the USAID-MEB better understand the effects of key types of PRIDE II interventions. The findings will also be used by USAID / START and the USAID-MEB to develop future activities that contribute to peace-building and democratization. The primary audience for this evaluation is the USAID-MEB. Other central consumers include the USAID, START, and CEPPS partners, NDI, IRI and IFES. 1.2 EVALUATION QUESTIONS The evaluation focuses on PRIDE II activities implemented between July 2015 and June 2017, emphasizing the period during which on-the-ground implementation activities occurred. The core evaluation questions address the performance of the PRIDE II interventions, and are based on the main program objectives and intermediate results outlined above. The evaluation emphasized questions 1 through 4, but included other the queries, in order to fully understand individual program dynamics and outcomes that were related to the core questions. All of the questions, therefore, are summarized as follows: What outcomes (anticipated and unanticipated) do staff at target CSOs perceive as resulting from NDI, IRI and IFES-supported activities? Have levels of civic engagement increased, particularly among under-represented groups as a result of enhanced understanding of processes related to the political transition?  Are there improvements in awareness of democratic principles for those exposed to IFES civic and voter messaging?  Has IFES technical assistance led to more CSO engagement in consensus building?  How many and what kinds of citizen-led advocacy efforts emerged among groups targeted by NDI’s activities?  To what extent have these advocacy efforts been successful in achieving their intended results?  What factors were associated with the emergence of citizen-led advocacy efforts among groups targeted by NDI activities? Do national and sub-national representatives better understand their democratic roles and responsibilities and their role in peacebuilding?  To what extent has the IRI-supported program contributed to accomplishing goals of enhancing skills and understanding of democratic roles and responsibilities among program participants? 1 2 3 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 10  How has the IRI-supported program participant knowledge of consensus building strategies and communications skills improved?  To what extent has the Musharaka Forum been effective in the context of shifting conflict dynamics?  To what extent does the present Musharaka model continue to be viable in the current context? Are any elements of the model promising for continued implementation? Have any elements of the model proven to be ineffective in the current context? What activities are programs are working well and why? Which are not working well and why? What examples can be identified as successful stories or failures? To what extent have results been achieved in the area of increasing leadership capacity among women, and minorities, if relevant?  Have there been improvements in women’s ability to exercise leadership, engage in dialogue and transition processes and consensus building (IFES/IRI)? Is there evidence of peaceful agreement on democratic reform being reached via consensus building?  Have mechanisms to establish networks and consensus-building forums been effective? Are Syrian stakeholders better informed of democratic governance and women’s rights?  How effective are IRI- and IFES-supported mechanisms and materials in creating networks that inform key stakeholders about democratic governance and women’s rights?  How useful are IFES civic education materials? How well has the Syria PRIDE program CEPPS partners (IRI/NDI/IFES) achieved goals of improving dialogue and participation among stakeholders in the areas of planning, assistance and policy? To what extent have approaches to increase engagement of women in political processes and negotiations succeeded across the three activities?  Has women’s leadership increased at the national level?  Were approaches to increase women's engagement implemented as originally designed?  Which challenges, if any, affected the implementation of these approaches?  Considering in particular how these approaches were implemented at local, national, and international levels, were any particularly successful in encouraging women's engagement in political processes? How has the conflict, overall, affected accomplishment of results and outcomes? What outcomes and results have not been achieved? How has the conflict affected the ability to anticipate outcomes? Have CEPPS partners been responsive to beneficiaries’ needs in the changing conflict￾affected environment? 11 10 9 8 7 6 5 4 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 11 What activity and program models and approaches might be duplicated in other conflict￾affected settings? 2.0 PROJECT BACKGROUND 2.1 POLITICAL CONTEXT Amid widespread political intransigence, the Syrian civil war continues to have a dire impact on an already inflamed humanitarian and security crisis. The Government of Syria (GoS) regained Aleppo from the opposition in late-2016, and GoS use of chemical weapons on civilians in opposition areas continues despite international condemnation. Peace talks held in Geneva have produced little progress. Russia, Iran, Turkey, and parties involved in the conflict, convened in January 2017 in Kazakhstan in a series of meetings to discuss ways to minimize the violence, with very limited success.3 Progress toward a resolution to the conflict is further hindered by the weakening of moderate armed groups at the hands of the Islamic State of Iraq and al-Sham (ISIS) and its affiliates and Hayat Tahrir al-Sham – including the group formerly known as Jabhat al-Nusra –as political negotiations show little sign of gaining traction. Syria’s governing structures at the local, sub-national, and national levels alike struggle to fill the vacuum left by the Assad regime. Elected officials and citizens lack the capacity to engage in democratic governance and understand their respective roles and responsibilities in the governance processes. Officials may not fully grasp the importance of inclusive government in the difficult context of conflict resolution, particularly towards traditionally marginalized groups such as women, youth and minorities. The impact of such deficiencies has led to a widespread lack of accountability, responsiveness, and inclusivity on local, sub-national and national governance initiatives to identify and address community priorities. Similarly, the Syrian people’s experience with political and civic participation remains nascent, and emerging citizen groups are in need of continued technical assistance. Syrians lack faith in the efficacy of civic participation efforts and are unfamiliar with the processes of collaborating to identify and forge solutions to common concerns—both bedrock principles for successful local-level governance in the near and longer terms. Consequently, Syrian citizens must build a fundamental knowledge base and skillset of democratic practices that enable them to deliberate, negotiate, make informed decisions, and participate in democratic processes such as community organizing and advocacy at the local-level. Under the PRIDE II award, CEPPS partners seek to increase the capacity of government and political leaders, civil society actors, women’s groups, and citizens at the grassroots level to support the eventual establishment of inclusive and effective democratic processes and institutions in Syria. 3 Human Rights Watch, World Report 2018: Syria, at https://www.hrw.org/world-report/2018/country￾chapters/syria 12 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 12 2.2 PROGRAM CONTEXT, DESIGN AND MANAGEMENT The PRIDE II partner theory of change hypothesizes that if Syrian democratic actors – especially women, youth, minorities, civil society actors and government officials – better understand their roles and responsibilities and have the capacity to constructively engage with each other and understand the incentives to create and work within an inclusive environment, then local, subnational and national structures will more effectively represent and respond to the priorities of their constituents, and citizens will be able to play more meaningful roles in governance processes. The program’s theory of change assumes relevant stakeholders will be willing and able to attend CEPPS partners’ workshops, and the security situation will allow CEPPS partners to conduct in￾country programming through their Syrian partners. Thus, the objectives of CEPPS PRIDE II objectives are described below:  CEPPS Objective 1: Increase political and civic participation and representation of women youth, and minorities, to foster public and stakeholder confidence in peaceful and representative transitional political processes, and bolster opposition credibility.  IR 1.1 (IRI/IFES): Leadership and representation of women, youth and minority groups, on national and sub-national governing bodies and in policy discussions is increased.  IR 1.2 (IRI): Peaceful agreement on democratic reform is promoted through consensus building.  CEPPS Objective 2: Increase knowledge and understanding of democratic processes amongst the Syrian population including consensus building, coalition formation, citizen and stakeholder engagement, and elections, which will enhance an inclusive Syrian-led transition.  IR 2.1 (NDI/IFES): Public understanding of processes related to Syria’s political transition produce higher levels of civic engagement and participation, particularly among marginalized and underrepresented groups  IR 2.2 (IRI/IFES): Local councils, the Syrian Opposition Coalition (SOC), the Syrian Interim Government (SIG) CSOs, Syrian citizens, and international stakeholders are better informed about democratic governance and women’s rights  IR 2.3 (IRI/IFES): Local, sub-national, national representatives and their constituents better understand their democratic roles and responsibilities, and their role in peacebuilding.  IR 2.4 (IRI/NDI/IFES): National and sub-national representatives and their constituents openly and regularly engage each other on matters of planning, assistance and policy. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 13 The three CEPPS PRIDE II partners implement activities in select regions in Northern Syria, Rif Damascus, and in Turkey (see Table 1). Critically, the perilous security conditions in Syria necessitate the implementation of project activities in neighboring Turkey. In Syria, PRIDE II program activities are implemented in Aleppo, Idlib, Latakkia, Hasakah, RIF Damascus, and Raqqa Governorates. In Turkey, program sites include Istanbul, and Gaziantep. For a period of time, the ability of beneficiaries to travel from Syria to Turkey to participate in activities as needed enabled PRIDE II project activity implementation. Changing regional and domestic conditions, however, created significant challenges for PRIDE II programming. Turkey closed its borders in 2015 to all Syrians, except injured asylum seekers. Both U.S.-backed rebel efforts to thwart Islamic State fighters, and Kurdish rebel groups maintaining a foothold in Syria led to even tighter border restrictions. The border clampdown has severely restricted Syrians trying to flee the civil war – and has prohibited travel of many PRIDE beneficiaries from Syria to Turkey for program activities. As a result, CEPPS PRIDE II partners redesigned project activities to maintain access to beneficiaries, relying on remote technologies to conduct trainings, meetings, and disseminate project information. Table 1: CEPPS PRIDE II Project Activity Activity Name USAID Office Implementer Life of Activity Active Geographic Regions PRIDE II START NDI June 2015- June 2018 Aleppo, Idlib, Latakkia, Hasakah, RIF Damascus, Raqqa Governorates PRIDE II START IFES June 2015- June 2018 Istanbul and Gaziantep, Turkey and Deir Ezzour PRIDE II START IRI June 2015- June 2018 Istanbul and Gaziantep, and Northwest Syria and Rif Damascus The MEB houses PRIDE II while both the MEB and START manage the project. The CEPPS￾USAID Syria PRIDE II project is designed around a number of key intervention activities (see Table 2). Table 2: PRIDE II Project Interventions Implementer Country Intervention NDI Syria ▪ Civic education sessions ▪ Local participatory needs assessments ▪ Advocacy and mobilization campaign planning and implementation ▪ Public forums and meetings with decision-makers IRI Syria ▪ Women’s political empowerment training ▪ Local advocacy campaigns, especially around women’s political participation CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 14 Implementer Country Intervention Turkey ▪ Women’s leadership: training on project and proposal writing, communication, public speaking for potential women leaders, SIG and SOC women staff IFES Syria ▪ Training on basic and vocational skills ▪ Training on electoral systems and electoral observation ▪ CEC trainings on communications and advocacy and workshops on gender, local governance and challenges, advocacy, and CSO coordination and management Turkey ▪ Dialogue forums for youth, adolescents, women and broader Syrian community members in Gaziantep ▪ Technical support to SOC, SIG, HNC / SNC for negotiations on electoral systems and frameworks, constitution development 3.0 EVALUATION METHODS AND LIMITATIONS To gather data required for this evaluation, NORC’s evaluation team used several techniques which entailed a combination of mutually reinforcing qualitative and quantitative methodologies that reflect the program design, research questions addressed, and indicators. The team integrated the results of each of the methods to capture the diversity of opinions and perceptions of beneficiaries and stakeholders regarding the PRIDE II project and progress accomplished during the evaluation period. The qualitative analysis, which includes in-depth KIIs and focus group discussions (FGDs), gathers in-depth responses on individual-level experiences, perceptions, opinions, feelings, and knowledge. Qualitative analysis provides the local context and information on why outcomes occur. The evaluation qualitative approach also entails PRIDE program document review. In turn, the qualitative findings informed the survey instrument. The quantitative survey, by contrast, includes a rapid assessment survey of available CEPPS PRIDE beneficiary respondents. It attempts to capture an indication of the magnitude of opinions among beneficiaries. Qualitative outcomes supported by quantitative methods contributes to mutual validation of the data and a more complete picture of the investigated domain. The approach to selecting the appropriate methodology is based on the USAID Evaluation Policy as well as NORC’s experience conducting evaluations in the field. The NORC evaluations team conducted its work in a participatory manner, which involved engaging USAID, CEPPS implementing partners, program beneficiaries, and other stakeholders. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 15 3.1 STUDY DESIGN The current performance evaluation draws on fieldwork conducted in Washington, DC, Turkey and Syria, between January 30 and February 10, 2018, and between March 26 and April 9, 2018. Qualitative fieldwork included in-depth KIIs with PRIDE II beneficiaries, core CEPPS implementers, Syria PRIDE II CSO partner organization staff, and cognizant USAID personnel. In total the evaluation team reached, 136 respondents through qualitative data collection (see Tables 3 and 4, below). The sampling for the qualitative in-depth KII draws purposive samples and uses a variety of strategies to select respondents. First, NORC team received beneficiary lists for all CEPPS programs.4 Dependent on the sensitivities involved, and the size of the lists, NORC was able to draw random samples from lists that were sufficiently large.5 There was not sufficient data to stratify, though the team endeavored to select beneficiaries from different age groups and locales in Syria, and ensure that the samples comprised at least 50 percent women and men. For other beneficiaries, the NORC team was able to interview the entire sample provided, thus there was no random selection.6 As well, the NORC evaluation team conducted eight focus groups among men and women PRIDE II beneficiaries. The focus groups, likewise, were based on purposive samples drawn from beneficiary lists provided by CEPPS. Each group comprised five to nine respondents and was moderated by an Arabic speaker with expertise in conducting focus groups. The group discussions were roughly two hours in duration, while individual interviews lasted approximately one hour. NORC led the development of the beneficiary and partner questionnaire in collaboration with USAID and the Syria PRIDE implementers. The questionnaires were designed for each individual PRIDE activity (See Annex 1).  NDI  IRI Women’s Program  The Musharaka Forum  Civic Education Center 4 NDI provided a list of 128 beneficiaries for the KIIs and survey and IFES provided 124. IRI provided a list of 20 beneficiary women who were available and willing to speak with the team for qualitative fieldwork and an additional 13 for the quantitative survey. 5 These include the NDI Program and the IFES Musharaka Forum. 6 Specifically, the team was able to reach all of the PRIDE II NDI coordinators, facilitators and IRI women beneficiaries targeted for interview. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 16  Formal Opposition Group members Instruments were translated into Syrian Arabic when necessary and implemented in the appropriate language. NORC’s team designed the KII and focus group protocols to ensure all instruments addressed relevant, common themes. The mixed method evaluation design facilitates NORC’s ability to capture different dimensions of similar phenomenon, leading to richer, triangulated findings. In analyzing the qualitative data, the team observed detailed notes written during interviews and focus groups and identified key themes and patterns in responses. When possible, the team drew general trends in the findings across the two methods.7 The team debriefed after each interview / discussion. The evaluation team participated in a one-day training, January 31st, in Gaziantep, Turkey prior to fieldwork. Fieldwork among PRIDE II stakeholders in Turkey was conducted face-to-face from Gaziantep and Istanbul, Turkey. The interviewers communicated with beneficiaries in Syria through a variety of secure remote technologies familiar to respondents, such as WebEx, Skype, Google Hangouts and WhatsApp. RAPID ASSESSMENT SURVEY In order to ensure the evaluation adequately captured views of a large scope of attitudes regarding PRIDE II programming including from within Syria, the data collection included a rapid assessment quantitative survey conducted among 139 Syrian beneficiaries residing in Syria, as well as in Turkey.8 The rapid assessment survey is based on purposive samples, and as with the KII and FGDs, the NORC selected respondents from beneficiary lists provided by CEPPS implementers. Although rapid assessment surveys do not have the analytic rigor of large random probability surveys, they nevertheless provide “quick,” useful preliminary information that can inform programming and future research, rather than support the development of large databases.9 The NORC team administered the rapid survey in Syrian Arabic from Gaziantep, Turkey using NORC’s secure Liberty online platform. Evaluation team members contacted beneficiaries via WhatsApp. WhatsApp communications are encrypted, and for many beneficiaries, it was a comfortable mode of communication. In addition, the connections were reliable and therefore enabled more effective implementation. Still, the risk of interrupted connectivity, and the additional risk of shelling, required a short, closed-ended survey, approximately 15 questions in 7 Most interviews and focus groups were recorded when possible in order to have access to detailed information. The team gained consent from beneficiaries. 8The NORC team attempted to contact 152 PRIDE beneficiaries, and completed 140 surveys, however, following data cleaning, there were 139 valid surveys. 9 In a comparison of a national health survey with a rapid assessment survey including the same questions, McIntyre found that results between the two were remarkably similar, though cautioned that a short-coming of rapid surveys is selection bias. This, she concludes, makes all the more important, the triangulation of quantitative findings with qualitative results, which is used in the current evaluation. See Katherine McIntyre, “Rapid Assessment and Sample Surveys: Trade Offs in Precision and Cost” in Health Policy Plan December 14 (4): 363-73. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 17 length, lasting seven to 10 minutes. The qualitative research findings informed the survey questionnaire. NORC conducted a remote training for the rapid assessment survey on March 23, 2018. 3.2 TARGET POPULATIONS As the Syria PRIDE II beneficiary population is very broad, with respondents residing in Turkey and inside Syria, the NORC team sought to ensure a diversity of viewpoints. Therefore, the evaluation included target populations representing the main programs of the three core PRIDE II implementers: IFES, IRI, and NDI. A summary of beneficiaries is outlined below (see Table 3). Table 3: PRIDE II Evaluation Target Population Beneficiary Program Participants Methodology Mode IFES Musharaka Forum FGDs (2); KIIs, Survey Face-to-Face: GZT Civic Education Center KIIs Only Remote: Syria Training Groups FGD (1); KIIs, Survey Remote: Syria IRI IRI Women’s Program FGDs (3); KIIs, Survey Remote: Syria Formal National Syrian Opposition Group Members KIIs Face-to-Face and Remote: Turkey NDI NDI Program FGDs (2); KIIs, Survey Remote: Syria Key Local Decision Makers KIIs, Survey10 Remote: Syria Table 4, shows the overall number of those interviewed for the qualitative data collection exercise. The total beneficiary sample (136 interviewees) of the qualitative data collection is roughly divided between women (48%) and men (51%), and includes 49 in-depth KIIs and 46 beneficiary focus group participants. 10 The survey among key decision makers included closed-ended and open-ended questions, where subjects were asked to participate in a brief KII after the survey for elaboration on responses. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 18 Table 4: Qualitative Data Collection: PRIDE II Total Evaluation Participant Population Total Beneficiaries11 Total CSO Partner Staff Total CEPPS Implementing Staff Total Cognizant Personnel IFES 15 (7 women; 8 men) 10 1 N/A IRI 28 (23 women; 5 men) 4 6 N/A NDI 52 (16 women; 36 men) 10 4 N/A USAID N/A N/A N/A 6 TOTAL 95 24 11 6 The rapid survey sample consists of beneficiary participants from each of the three key PRIDE II programs: NDI;12 the Musharaka Forum (IFES, Gaziantep, Turkey); and the IRI women’s program (IRI, Syria).13 Due to the security sensitivities, the sampling strategy is both purposive and random.14 Roughly half of the rapid survey sample is almost evenly divided between men (51% or 71) and women (47% or 66).15 The rapid assessment survey reached 139 respondents. 3.4 LIMITATIONS The volatile – and sometimes dangerous – security and political environment presented several challenges for the evaluation. They are listed below:  Social desirability or halo bias: There is a tendency among respondents to under￾report socially undesirable answers and alter their responses to approximate what they perceive as the social norm (halo bias). The extent to which respondents shared their true opinions may have varied for some questions that call upon the respondents to assess the attitudes and perceptions of their peers or people on whom they depend upon for the provision of services or support. To mitigate this limitation, the NORC evaluation team provided appropriate confidentiality and anonymity assurances to all KIIs and FGD respondents and to all survey respondents 11 Fieldwork included KIIs among eight IRI-supported opposition group members; three of whom were also engaged by IFES. 12 The team also conducted the survey among 22 of 27 local decision makers affiliated with the NDI program. 13 The IRI survey is comprised of all women. 14 The Musharaka Forum participants were selected randomly. For the survey among NDI beneficiaries, those who were not selected for individual interviews were interviewed for the survey. IRI’s sample is based on purposive sampling. 15 Ethnic and religious minorities were under-represented in the survey as well as the KIIs. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 19 through an informed consent process. Interview protocols were also adjusted to include probative questions that served as a check on responses.  Political instability in Syria. One significant constraint with regards to data collection was the political and security context in Syria. As Syria is in the midst of a civil war, data collection was at risk of being interrupted due to shelling. In some instances, the team had to reschedule or cancel interviews or focus groups, particularly with IRI and NDI beneficiaries within Syria, due to air strikes. In other cases, respondents were not available due to their need to evacuate areas.16 Respondents also were wary of the interviewers, caused by concern that anti￾opposition forces were either posing as interviewers or conducting surveillance of conversations. There was also the possibility of “revictimizing” respondents, as they shared their experiences during interviews. The issue was especially relevant for IRI PRIDE II women beneficiaries, who may have been sexually assaulted following the conflict.17 Also, a number of beneficiaries recently lost family members to the conflict. To mitigate these issues, the team worked with CEPPS partners to socialize respondents about the evaluation, and also relied on partners to recommend individuals who would be willing to participate in the evaluation process. In addition, NORC ensured that the evaluation team was comprised mostly of women, and the interviewers were from Syria and the Levantine region. This strategy made the evaluation team more accessible and relatable to respondents, engendering more trust and greater comfort levels. Finally, beneficiaries were reminded that they could stop the interview at any time, reconvene later, or not participate at all if they were uncomfortable answering questions. Interviewers debriefed with respondents afterward, if necessary.  Underrepresentation of views inside of Syria. On the whole, the unstable nature of the security situation inside of Syria complicated acquisition of beneficiary views inside the country. Specifically, in late-2017, the Autonomous Administration in Northern Syria (AA) also announced that data collection activities in some areas of the country were prohibited, namely Hasakah. The directive had a direct impact on IFES data collection in Qamishli City, according to IFES program staff. The NORC team, as a result, could not access those IFES beneficiaries and some key program personnel located in Syria.18 For other reasons related to security conditions, access 16 For example, just before survey implementation in late-March, 13 NDI beneficiaries from the besieged Eastern Ghouta of rural Damascus were evacuated, and were, therefore, unable to participate in the rapid assessment survey. Additionally, AA data collection prohibitions in IFES areas prevented the team from interviewing beneficiaries inside of Syria. Other key informants also were not in the country. 17 Women were not assaulted as a result of their participating in PRIDE II programming, however. 18 Most IFES CEC program staff and beneficiaries in Syria were unavailable for interview, as a result of the unexpected AA regulation on data collection activities in select areas. The team, therefore, focused on Musharaka Forum activities. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 20 to other PRIDE II beneficiaries was limited at various times during the evaluation due to shelling or evacuation.  Generalizability of results. Since some respondents were underrepresented, findings may not be generalizable across the entire beneficiary population. The qualitative results and rapid assessment survey are not based entirely on random probability samples, and the survey sample is also small. The team endeavored to create as diverse a sample as possible by conducting a large number of KIIs, focus groups, as well as the survey. The team also sought to triangulate qualitative and quantitative results in order to confirm findings and identify consistency across results. None of these challenges, however, prevented the evaluation team from gathering the relevant, evidenced-based information and data needed to produce valid findings and conclusions for this performance evaluation. 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS The following section discusses the evaluation findings based on qualitative and quantitative data collection. The evaluation questions serve as a frame for presentation. Each of the four main research questions are addressed and then a final section for discussing findings related to other proposed research questions concludes the section. For each research question, we first outline the overall findings noted across the entire program, and then we discuss the more specific program findings according to each activity. Finally, we draw several conclusions and suggest recommendations for the overall program. 4.1 EVALUATION QUESTION 1 What outcomes (anticipated and unanticipated) are perceived to be the result of NDI, IRI and IFES-supported activities? FINDINGS According to results across all three PRIDE II programs, advocacy capacity improved, producing enhancements in infrastructure and enhanced women’s leadership capacity. Consistent with anticipated PRIDE II program goals, the evaluation finds that PRIDE II beneficiary participants across NDI, IRI, and IFES programs demonstrate increased capacity for advocacy and engagement, most notably at the local-level. PRIDE II staff report an increase in successful beneficiary advocacy among all beneficiaries for service-based initiatives, attributed to skills acquired in PRIDE II programing. The large proportion of service-based advocacy efforts reflects, partly, citizen demands for projects, such as improved infrastructure in villages, as well as the reality of an increasingly restrictive and unstable political environment that precludes broader, more political forms of advocacy. Advocacy also enhanced leadership capacity and CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 21 inclusion for women beneficiaries, both at the local- and the national-level. PRIDE II staff also note greater inclusion of women and of other marginalized groups as result of the program. PRIDE II beneficiaries are eager to disseminate acquired knowledge and information. This improved level of capacity and confidence may be, in part, related to beneficiary knowledge and willingness – even eagerness – to disseminate the knowledge and information gained since their exposure to the PRIDE II interventions. Following participation in Syria PRIDE II programming, beneficiaries report an increased desire to discuss a variety of sensitive topics related to democracy and governance in semi-public settings and among other members of the community. Beneficiaries, similarly, are more open than in the past to discussing political topics with family members within the household. Women, in particular, noted an improved ability and level of comfort with engaging the men both within their families and communities. Retention and utilization of democratic concepts and practices. Beneficiaries, according to in-depth KIIs with program staff and program participants, accept, comprehend, and have retained knowledge of key democratic concepts and practices learned since their involvement in Syria PRIDE II. Moreover, they exhibited a greater ability and willingness to use democratic concepts and terms learned in training sessions and consultations in non-project government engagements, especially at the local level. Unanticipated outcomes: internalization of democratic ideas. There are some unanticipated outcomes identified by PRIDE II program staff and respondents, specifically, beneficiaries’ ability to meaningfully internalize the democratic concepts and practices learned. Both respondents and program staff across each program, explained that, heretofore, many ideas were completely unknown to PRIDE II beneficiaries. Since their involvement in the program, however, beneficiaries show an unexpected level of enthusiasm and understanding of political and civic concepts and issues. Evaluation survey findings substantiate this observation. At least 90 percent of NDI, IRI, and IFES respondents say the PRIDE II project enhanced their understanding of democratic words, concepts, and civic practices, as well as their familiarity with democratic roles and responsibilities. NDI Democratic concepts have taken hold. In specific analysis of how outcomes manifest across each PRIDE II program, NDI beneficiary participants in Syria say they can now more confidently discuss democratic concepts, such as “political participation”, “representation”, “constitution”, “citizenship”, “separation of powers”, and “human rights” in public settings without fear. There is a strong demand for civic education sessions, despite precarious security conditions in Syria that render discussion of such topics very sensitive. “The respondents had never heard of these words before. But after the trainings, not only to they understand the meaning of the concepts, they use them with no fear at all. They have remembered them and understand how it is relevant for them.” (Staff Member, NDI Program KII, GZT) “Before the sessions, my knowledge about democracy, civic education, women rights, election, and citizenship was limited but now at least I have more awareness and CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 22 understand about these concepts.” (Male Youth, Beneficiary, KII, NDI Program Syria) Citizen-led campaigns have produced improvements in local infrastructure and service delivery in Syria. There have been 41 NDI citizen-led advocacy campaigns that have, likewise, produced tangible improvements in local infrastructure and services in program sites, as a result of beneficiaries’ direct engagement with key local decision-makers. The category of “decision makers” often includes members of local councils, but beneficiaries also engage in advocacy with other influential decision making actors, such as notable families, private business owners, and civil society organizations. The targeted focal point of advocacy engagement is determined by the nature of the project. Beneficiaries’ capacity for advocacy also improved as a result of the advocacy campaign activities. Rapid assessment survey results substantiate these observations. Among NDI’s beneficiaries, 85 percent believe they were part of successful campaigns that addressed citizens’ needs. An even larger majority (93%) believe that their efforts helped increase decision-maker willingness to meet with citizens to discuss issues of concern and consider policy solutions. Unanticipated outcomes. NDI program staff said they were most surprised at the number of eager participants willing to engage the program in the difficult environment inside Syria. Personnel cited the willingness of beneficiaries to attend civic engagement sessions, and engage various topics, including those around democracy and governance, which are sensitive. This enthusiasm extends to participation among women program participants also as facilitators and coordinators. A number of NDI sessions are held in locales with a large militant Islamist presence, such as Atareb and Idlib. Despite heightened risk of scrutiny and possible confrontation, a large number of women attended NDI sessions in these areas. Moreover, in the men-only sessions, participants discussed violence against women, an issue that is normally the concern of women in Syria. The large number of participants may be owed to the program’s creative and flexible strategy for accessing potential beneficiaries, which will be discussed in subsequent sections. Sustainability. Evaluation findings indicate that beneficiary acquisition of knowledge and democratic practices, and the willingness to discuss issues are perhaps key sustainable outcomes. The program contributes to the development of linchpin actors in Syria who, as potential influencers, could emerge as critical sources of information and leadership. Linchpin actors are individuals that can play the role of influential connectors in networks, bringing people together or aiding in the proliferation and dissemination of information. IRI Increased women’s engagement with local actors. Improvement in PRIDE II beneficiary advocacy capacity is also notable among the IRI-supported women beneficiaries. For example, women’s engagement with local councils led to the establishment of women’s sections in 22 local council offices, moving councils a step closer –albeit a small one – towards institutionalization of women’s interests. In one case, women’s advocacy efforts resulted in a local council office opening a women’s restroom. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 23 Perhaps a by-product of increased women’s engagement, is a greater sense of efficacy in the public domain among IRI women beneficiaries. The survey among a small number of IRI women program participants indicates that participants now express more confidence in their ability to advocate on behalf of their community (100%) and believe their efforts have been successful in influencing local decision makers in Syria (100%). Against the backdrop of their positive assessments of their leadership and advocacy capacity, somewhat fewer (62% vs.31%) say they are comfortable publicly voicing their opinions.19 The evaluation team finds that IRI program engagements also reached women that exist on the social periphery of communities in Syria. In one example, women that faced social ostracization due to their status as unmarried women with children were invited by IRI women beneficiaries to bring their children to kindergarten. Respondents also mentioned the launch of 10 initiatives in their respective communities designed to enhance women’s role in decision-making. Women have been able to engage at the national-level. Equally significant, participants’ grassroots consultations and engagement through IRI’s work with women informed recommendations on the role of women in the future transition of Syria. The recommendations reached the office of Staffan de Mistura, the Syrian Opposition Council (SOC), and the High Negotiations Committee / Syrian Negotiations Committee (HNC / SNC).20 Unanticipated outcomes. Two women supported by IRI programming were elected to local councils. Despite the difficult conditions in Syria and persistence of patriarchy, women accomplished inroads into Syrian politics, which includes progress towards establishing linkages with national-level actors. Although subsequent evaluation questions discuss this in greater detail, examples of IRI-supported activities include, an IRI women’s delegation, which met with the HNC, and efforts to convene a national conference to facilitate election of women onto the SOC. Other national linkage activities connected SOC women employees and women activists and potential leaders in Turkey with the aim of fostering women’s leadership and broaden networks. Sustainability. The IRI program intervention within communities and its facilitation of deeper connections among women are among the activities’ more sustainable outcomes. The potential of these women to move into new roles as a result of enhanced capacity and changed perceptions of gender is noteworthy, as indicated by their enhanced confidence in engaging in advocacy and expressing their opinions, and their belief that their views have influence on others. IFES 19 As mentioned, some of the participants are former political prisoners and / or their husbands were imprisoned. The evaluation team was told that there may still be apprehension of speaking in public for fear of retribution. 20 Following the collapse of the HNC, the organization was renamed Syrian Negotiations Committee (SNC). In usage, the evaluation refers to HNC or HNC / SNC, as HNC was the organization the PRIDE program engaged. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 24 The Musharaka Forum saw a rise in levels of social and personal and political efficacy among participants, especially youth beneficiaries. According to both focus groups and KII results, involvement in the forum exposed members to groups and perspectives different from their own (e.g. Kurds). Youth beneficiaries also assert that Forum participation provided a greater sense of purpose. “In the past, I felt like nothing, I was nobody. Now I feel like the future”. (Youth Beneficiary, Male, Musharaka Forum, FGD, GZT) Asked whether the skills and information they learned can be applied to a future transition scenario in Syria, Musharaka focus group respondents expressed confidence in their ability to be involved in Syrian political processes, and looked forward to doing so. Musharaka participants believe they are now informed about the political processes related to the transition, through the forums, and have the skills to engage in transition processes as democratic citizens. Survey findings underscore focus group results (see Table 5). Table 5: Musharaka Forum Beneficiary Views of Program – Overall Results (Turkey) As I read each statement, please tell me the extent to which you agree or disagree: Musharaka Forum / Baytna Syria programming has informed me about the transition process. 92% (55) Musharaka Forum / Baytna Syria activities has made me more familiar with democratic roles and responsibilities. 95% (57) I believe I can have a say in whatever might happen in the transition of Syria. 85% (51) Musharaka Forum / Baytna Syria activities have enhanced my understanding of democratic words, practices and concepts. 93% (56) The Musharaka Forum / Baytna have helped develop and enhance my advocacy skills. 83% (50) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to engage in advocacy related to Syria or to Syrians in Turkey in a meaningful way. 65% (39) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to spread knowledge and information about Syria or about Syrians in Turkey in a meaningful way. 90% (54) I have the capacity and skills to demonstrate leadership. 93% (56) N=60 Observing views by gender, women are just as likely as men to believe they can affect change and demonstrate leadership as a result of their participation in the Musharaka Forum (see Table 6). This sentiment is also fairly consistent across age groups, though findings suggest that participants 36 years and older, are less likely to think they can engage in a way that is impactful in a future Syria (see Table 7), as do somewhat smaller majorities of both women and men – assessments likely shaped by conditions in Syria. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 25 Table 6: Musharaka Forum Beneficiary Views of Program by Gender (Turkey) As I read each statement, please tell me the extent to which you agree or disagree: Men Women Musharaka Forum / Baytna Syria programming has informed me about the transition process. 87% (20) 95% (35) Musharaka Forum / Baytna Syria activities has made me more familiar with democratic roles and responsibilities. 96% (22) 95% (35) I believe I can have a say in whatever might happen in the transition of Syria. 78% (18) 89% (33) Musharaka Forum / Baytna Syria activities have enhanced my understanding of democratic words, practices and concepts. 91% (21) 95% (35) The Musharaka Forum / Baytna have helped develop and enhance my advocacy skills. 83% (19) 84% (31) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to engage in advocacy related to Syria or to Syrians in Turkey in a meaningful way. 61% (14) 68% (25) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to spread knowledge and information about Syria or about Syrians in Turkey in a meaningful way. 78% (18) 97% (36) I have the capacity and skills to demonstrate leadership. 96% (22) 92% (34) N=23 N=37 Table 7: Musharaka Forum Beneficiary Views of Program by Age (Turkey) As I read each statement, please tell me the extent to which you agree or disagree: 15-18 19-25 26-35 36-44 45+ Musharaka Forum / Baytna Syria programming has informed me about the transition process. 67% (2) 100% (14) 100% (16) 85% (17%) 83% (5) Musharaka Forum / Baytna Syria activities has made me more familiar with democratic roles and responsibilities. 100% (3) 100% (14) 100% (16) 90% (18) 83% (5) I believe I can have a say in whatever might happen in the transition of Syria. 67% (2) 86% (12) 94% (15) 80% (16) 83% (5) Musharaka Forum / Baytna Syria activities have enhanced my understanding of democratic words, practices and concepts. 100% (3) 86% (12) 94% (15) 100% (20) 83% (5) The Musharaka Forum / Baytna have helped develop and enhance my advocacy skills. 100% (3) 86% (12) 94% (15) 70% (14) 100% (6) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to engage in advocacy related to Syria or to Syrians in Turkey in a meaningful way. 67% (2) 79% (11) 75% (12) 55% (11) 50% (3) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 26 As I read each statement, please tell me the extent to which you agree or disagree: 15-18 19-25 26-35 36-44 45+ I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to spread knowledge and information about Syria or about Syrians in Turkey in a meaningful way. 100% (3) 79% (11) 100% (16) 90% (18) 83% (5) I have the capacity and skills to demonstrate leadership. 100% (3) 100% (14) 88% (14) 100% (20) 67% (4) N=3 N=14 N=16 N=20 N=6 More women, more organization partners. Among women, the Musharaka Forum participation is typically high and personnel say that even more women have become interested in participating in the forum. Furthermore, the number of Musharaka partners also increased dramatically from three to 25, and equally important, new organizations have also grown out of the forum. Opposition training and elections monitoring reports. In IFES programming with the SOC, SIG and HNC / SNC, staff worked with members of the national formal Syrian opposition on the Geneva negotiations, and contributed to an augmented understanding of their democratic responsibilities going into a political transition. This entailed provision of technical support and guidance remotely and in-person on elections and constitution issues. IFES, as well, engaged with Syrian CSOs on elections training, observation, and voter education, and from these activities, CSOs generated two local election monitoring reports, the first of such reports produced in Syria, according to IFES program personnel.21 Unanticipated outcomes. IFES personnel identified several unanticipated consequences. First, one IFES staff member in Gaziantep mentioned the program’s influence on forum participants’ keen desire to influence events inside of Syria should conditions permit, despite being removed from the resident population. Secondly, personnel described how the forum is generating interest through word-of-mouth among other Syrians in Turkey, a mode of communication that has been especially visible among youth. Although the forums seek to increase interest inside of Turkey as a program objective, the breadth of interest among youth generated from word-of￾mouth was mentioned as noteworthy by program staff. Finally, the program has been able to reach young women, who are usually on the periphery of politics. Youth of both genders have developed a greater sense of purpose and express a strong desire to contribute to a better Syria. Sustainability. The IFES Musharaka Forum promotes diverse networks that can endure among exile or non-exile communities. A network of organizations has evolved from the forum and has begun their own initiatives, such as the Women’s Choir, Daw, and Zenobia. 21 See, NDI, Saqarib Local Council Elections Monitoring, published July 2017 and the LACU Elections Monitoring Report, “Local Council Elections, Saqarib City: Idlib Governorate,” published 2017. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 27 CONCLUSIONS The evaluation team finds that advocacy, knowledge dissemination, and retention of democratic concepts and practices at the individual-level, are key achievements accomplished during the evaluation period across all three programs implemented by NDI, IRI and IFES. The programs build social cohesion and a common understanding of the meaning of democracy and of expectations around roles and responsibilities in a democratic system. PRIDE II approaches, further, provide individual beneficiaries with access to a broader network of people. These anticipated results are consistent with PRIDE II program goals and objectives. On the whole, PRIDE II beneficiaries demonstrate enhanced levels of personal and political efficacy and confidence with applying the skills and knowledge they have acquired. Amidst a very fluid political environment and adverse security conditions, beneficiaries, in general consensus, believe that they will be able to use the information, capacities, and knowledge to influence future Syria, regardless of the outcome scenario. Unanticipated results include the extent to which democratic concepts and knowledge have been internalized, the programs’ ability to conduct activities at all in Syria, enthusiasm among women participants and their related participation in programming. For IFES programming, the interest among youth in the Musharaka Forum programming generated through word-of-mouth speaks to the program’s credibility and perceived benefits of participation. RECOMMENDATIONS To enable future program implementation, the PRIDE II program should continue to rely on implementation strategies that mitigate obstacles caused by the conflict, such as slowly establishing trust through credible on-the-ground personnel, using word-of￾mouth to inform others of various PRIDE programs, creatively using remote technologies to reach difficult-to-access beneficiaries, and adopting a flexible approach towards addressing sensitive political topics, and selecting the locale of sessions. Similarly, Musharaka Forum activities, which operate from Turkey, appear to promote a sense of solidarity and efficacy among beneficiaries. Future activities should build on this outcome by addressing how respondents might concretely influence events in a future Syria and under varying scenarios. Moreover, the forum should expand its scope of work in the future to include interventions inside of Syria. 4.2 EVALUATION QUESTION 2 Have levels of civic engagement increased, particularly among under-represented groups, as a result of enhanced understanding of processes related to the political transition? 1 2 3 1 2 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 28 FINDINGS Increased civic engagement and enhanced understanding of the political process is evident and widespread also among under-represented groups in Syria. With the conflagration in Syria ongoing, negotiations are stalled and the prospects for a concrete political transition are dim. Still, there is progress along PRIDE II program goals. As mentioned in the previous section, PRIDE II beneficiaries are more familiar with a wide array of democratic concepts and practices, and beneficiaries developed skill sets for transferring democratic knowledge to their peers and community members for continuing civic education, needs assessments and advocacy as a result of PRIDE II interventions. However, beneficiaries are now more aware of the importance and utility of broader political processes that can be relevant for a future political transition – progress noted particularly among underrepresented groups, namely women and youth. Beneficiary interest in civic participation and desire for continued civic engagement is more widespread. Importantly, women beneficiaries report now having a better understanding of democratic processes, their roles in such processes, and in related decision-making. Women and youth participants also acknowledge the importance of sharing the knowledge and skills acquired with other community members. Indeed, there is even a sense of personal responsibility to actively engage and cultivate relationships with minorities (Kurds and IDPs) and youth “for a better Syria”. In parallel, women and youth beneficiaries demonstrate a greater sense of empowerment and confidence in expressing their right to speak freely and engage with subnational decision makers (local council members). Critically, the evaluation team notes that for some PRIDE II activities, men also now see the importance of women’s participation in civic engagement, especially their role in election processes. NDI The NDI Program contributes to heightened women’s and youth awareness. Common among all the respondents of the NDI Program is the introduction to democratic processes that were previously unknown. Some participants explained they had some previous knowledge of democratic institutions, the role of citizens vis-a-vis representatives, and how governments function, but believe they were misinformed, until their exposure to NDI sessions, which clarified and / or familiarized beneficiaries with democratic political processes and related concepts. “I now know every institution has a role – one person does not represent an institution.” (Youth KII, Female, Syria) “I learned that the president can change and I understand the concept of ‘free and fair elections’.” (Youth KII, Male, Syria) “As a group, we learned to fight for our rights and conduct peaceful demonstration”. (Youth KII, Male, Syria) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 29 “I never knew what democracy meant, nor have ever seen it applied in reality. I used to think that only people in power understand democracy. After the sessions, I learned that I, myself, can use democracy in my house, with my family. I can make decisions; I have that power within me.” (Youth KII, Female, Syria) “I will have a voice. If the society changes for the better, I will at least have a voice and an opinion. I will be able to prove myself. I will try to contribute as much as I can in the transition period to make it better”. (Youth KII, Female, Syria) Additional seed funding could augment beneficiary gains in knowledge and information, by expanding to audiences in more isolated areas, in the view of program coordinators.22 One coordinator said, “The program was a success, but if there was more seed funding awarded, then more Syrians would be reached especially in rural areas where the concept of civic engagement and democratic process is less prevalent and their involvement with decision￾makers is minimal.” Another coordinator felt the NDI program well addressed basic awareness regarding civic engagement and increased beneficiaries’ knowledge about democratic processes. Yet, beneficiaries were ready to embrace advanced advocacy programing, including the delivery of more citizen-led campaigns. NDI programming facilitates enhanced understanding around the functions of local councils and importance of participation in political processes. NDI beneficiaries believe they are now better educated on the roles and responsibilities of local council members and key decision makers in their areas since their involvement with the NDI program. Accordingly, beneficiaries express a greater willingness to approach decision makers, due to their improved understanding of political processes and related terminology. Improved advocacy skills, used to engage local councils and decision makers, bolstered beneficiary confidence. Respondents point to participation in some successful campaigns for community services and rehabilitation projects, such as clearing rubble, removing landmines, and securing water tanks. These positive results, many assert, underscore the value of systematically identifying community interests and engaging decision makers as part of the advocacy process. As some participants put it: “We had an advocacy campaign about solid waste removal and requested the local council to do something about this issue, and they responded within their capacity. We felt heard.” (Youth Male, Beneficiary, KII, NDI Program , Syria) “I now know how to reach decision makers and thanks to the sessions, I know more about governance and transparency. In the transition, if there is a new constitution developed, women will know and will be able to participate [in order] to develop the constitution. All people can 22 However, the NDI Seed Award funding program is outside of the evaluation timeframe, as it occurred post￾June 2017. It is mentioned here, as it is linked by a program staff member to the effort’s overall ability to reach wider audiences. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 30 participate. If elections happen we will be able to vote.” (Female Beneficiary, KII, NDI Program, Syria) Table 8, for example, describes the value of women’s involvement in local elections. Table 8: The Outcome of Women’s Engagement: Women’s Participation Local Council Elections. A woman from Aleppo described her experience learning about elections during her involvement with the NDI Program. Prior to her exposure to elections sessions, she said, “Certain families ruled the local council, but thanks to the sessions, I learned I can choose a person who represents me and the community”. In the town where she resides, eight women nominated themselves to run in local council elections, which she attributes to NDI programing. She explained that the idea of women participating in elections is not acceptable in Syria, and the action of these women has become a model for others. This story was shared by others to the evaluation team, and for all, it is considered a positive example of women empowerment, voice, and what can result from women’s participation in political processes. During these elections, one women became president of a Women Affairs Office, while two other women were elected to administrative positions. Women gain confidence amid changing role perceptions. As NDI women beneficiaries developed a greater sense of empowerment in parallel with an acquired willingness to make decisions regarding day-to-day issues, confidence levels increased. Some women beneficiaries indicated changing perceptions of their traditional role in society and a desire to break old normative barriers. As the conflict continues, women said they struggle to find their place in society. Though many expressed a desire to engage in political processes, barriers to women’s participation persist. Indeed, combatting lack of confidence was a common theme in in-depth beneficiary KIIs. However, as a result of NDI’s work, women beneficiaries said they now feel more effective in other aspects of their lives. “I used to hesitate and think, as a woman, I shouldn’t participate in civic led activities, but now it is my right to participate.” (Female Youth, KII, NDI Program, Syria) “Thanks to the session, women have begun to mobilize and initiate awareness sessions that include women’s rights and duties in the political process, women’s role in elections, and early marriage and women roles in decision-making process.” (Beneficiary, KII, NDI Program, Syria) NDI activities also contribute to increased youth engagement. Another achievement includes youth engagement in citizen-led activities. Similar to their women beneficiary counterparts, youth believe they now have voice, and can be vital agents of change by being involved in political processes. Importantly, political awareness was viewed by youth as an important element for ultimately “getting things done.” CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 31 “Before, I didn’t bother or get involved in anything, but now, thanks to the sessions, I have more information and I get involved in small and big situations and issues related to our community.” (Male Youth, Beneficiary, KII, NDI Program , Syria) “Having a space to comfortably give my opinions and to discuss problems and solution is in a safe environment is freeing from me.” (Female Youth, Beneficiary, KII, NDI Program, Syria) IRI IRI women beneficiaries are eager to raise awareness around women civic engagement… Results from focus group discussions with participants in the IRI women’s program show women joined in their aspiration for a new and democratic Syria. In the view of respondents, there has never been a meaningful role for women in Syrian politics, due to their marginalization in public life. Women assert that the IRI’s program provides a platform for women to be more involved in public life and contributes to their ability to work towards change in Syria. Further, the IRI program offers a mechanism for substantively involving women in political processes, through the program’s various political awareness raising sessions, communications and leadership trainings, session on how to lead civic campaigns, and local council and stakeholder engagements. The respondent comments below, likewise, illustrate the IRI program’s influence. “I have a lot of interest and follow up consistently on the political conditions in Syria. I follow the news, in order to keep up with the political climate and learn political concepts and terms. However, it was not always that way. It was difficult for me because of community traditions. I was not allowed to visit local councils or meet with officials in the community. That all changed when I enrolled in the IRI program, through the trainings and skills received. I saw real impact in how I communicate with confidence my civic duties, and I was heard. [At a local council meeting] the outcome of the meeting was primary approval in accepting female members in the next local council elections.” (Beneficiary, IRI, FGD) “Personally speaking, my awareness about election mechanisms in local councils and [of] CSOs was limited, but now I understand how they work and I have better communication skills and my personality has become stronger. I can now speak and demand my rights.” (Beneficiary, IRI, FGD) “Through the visits to local councils and CSOs I have enhanced my communication skills. My work feels legitimate. (Beneficiary, IRI, FGD) …and women seek to be involved in Syria’s future local elections. Awareness of political processes, particularly those related to elections, created enthusiasm for participation in future political contests in Syria. In the words of one IRI program focus group respondent from Kafr Nabl: “Our work in this program has come at the time when conditions in Kafr Nabl and its surroundings will change soon. We will have a democratically elected local council for the first time when elections occur next. Previously, there used to be favoritism by clans or certain CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 32 families in all areas, but there will be a new local council that will be democratically elected, and the work of the program has enabled the women to secure two seats in the new local council.” (Beneficiary, IRI, FGD) IRI women’s group beneficiaries developed advocacy campaigns concerning women’s participation in elections. Such activities, evaluation findings suggest, seem helpful for encouraging discussions around political issues, and thus allow for more engagement in the political process. According to interviews and focus groups, women’s civic engagement and its role in any upcoming transition in Syria emerged as personal priorities for the women as a result of their enrollment in the IRI programing. Other IRI-led campaigns include awareness and support for women prisoners that relocated to Idlib following their release from incarceration, and as well as initiatives to assist women to find work and encourage young women to purse higher education. Overall, qualitative and quantitative findings suggest IRI women beneficiaries gained experiences that strengthened their leadership skills and capacity to implement projects that promote women’s engagement in politics – and that also promote engagement with other women. According to KIIs and focus groups, key beneficiary capacities gained, include communications skills, which contribute to building confidence and empower women beneficiaries to express themselves. Women also cited learning how to mobilize groups to engage in civic duties and how to work together to achieve a collective goal, such as participating in local elections. Relatedly, findings from the rapid survey suggest that over 90 percent of beneficiaries are now able to leverage their new-found knowledge to achieve mid-range goals, such as winning a position on a local council, and are able to look forward to longer term goals of ultimately winning a position on a democratic governing body in a new Syria. IFES The Musharaka Forum fosters communal tolerance. The Musharaka Forum creates a secure environment for positive engagement and dialogue among Syrians of different backgrounds. The diversity of the forums – which comprises Arabs, Kurds, men and women of all ages, and participants from different socio-economic backgrounds – fosters tolerance among members. The forum exposed participants to ideas and experiences different from their own through their participation in the forums. As a Musharaka Forum staff member in Gaziantep explained, the program’s intent is to facilitate between different opinions and create consensus around issues that are of interest to all. Even if complete consensus is elusive, participants learn that coexistence is possible amid disagreement. In the words of one respondent, “I sat with people on the same table although each shared different opinions than myself, and it was okay.” (Beneficiary, Musharaka Forum, KII, GZT) Locating common ground in experiences seems to not only encourage tolerance, but also empathy for others affected by the conflict. By way of example, one forum addressed alienation of mothers of ISIS youth supporters. Through discussion, participants agreed that ultimately, everyone shared the same national identity regardless of the choices of children in the family. This is the basis for solidarity: CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 33 “There was an agreement within the group that the mothers were Syrians, and we as a group should be close to each other, despite our differences. The children who joined ISIS made a mistake, and it is not the fault of the mothers.” (Beneficiary, Musharaka Forum, KII, GZT) Table 9, below, further describes the experience of Musharaka Forum women. Table 9: The Women’s Forum: Acceptance through Dialogue and Consensus The Women’s Forum considered engaging mothers from Raqqa whose children were former fighters with ISIS. Many forum members found it difficult to accept the women, due to the role they may have played in the Syria conflict. As discussions began, the women who had children involved with ISIS explained they were completely unaware of what their children were doing, and they believed they were also victims of the civil war. The women talked about the tragic loss of the children and the pain of having to flee their homeland. Through dialogue on this very sensitive issue, other forum members saw the women were like everyone else in group, and are struggling to make sense of the tragedy and build a new life, either in Gaziantep or back in Syria. The program also attempts to engage marginalized women and youth. “We have a few Kurdish participants and a few females join the sessions. It is more difficult to have young women join the Youth Forum, but we try. Young women are faced with more obstacles because of traditional norms including limited contact with non-relative males. Also, some of the sessions occur in the evening and most of the young females are not allowed to go out at night.” (Staff Member, Musharaka Forum, KII, GZT) To reach women, including those that are not as well integrated into the Syrian community, Musharaka coordinators visit women in their homes. As a first step, coordinators introduce men in the household to the objectives and activities of the Musharaka Forum. This is as much an opportunity to socialize others to the program as it is to build trust. Once coordinators have buy-in from men, they ask them to encourage or allow women household members to attend the sessions. To encourage youth participation, coordinators visit places of work and universities. Musharaka Forum participants feel more influential and able to discuss sensitive topics in public settings. Addressing topics ranging from citizenship, health care, government transfer, civil society, and secularism, to women’s role in the political process and violence against women, forum discussions challenge participants to confront sensitive issues that are usually considered taboo. The forums also cultivated a safe space for dialogue on these matters, and beneficiaries are united in their assessment that not only do they feel more confident discussing these topics, they are convinced they can be sources of change in a new Syria. For instance: “Transitional justice, democracy, and being an active member in society are just a few topics discussed in the forums. These issues are a prime focus, because the belief is if we are not developing our citizenship rights, then we will not be able to return to Syria.” (Staff Member, Musharaka Forum, KII, GZT) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 34 “If there is a political transition, I will participate to help build a better Syria. People will hear my opinion.” (Beneficiary, Musharaka Forum, FGD, GZT) “I can stand up in front of a forum of 70 people and discuss the atrocity that occurred when I was in prison. I feel it is important to tell my story.” (Beneficiary, Musharaka, FGD, Forum, GZT) Not all forum topics address political processes, however. The program considers issues related to special needs, health care, and similar concerns. Though less political, this strategy contributes to IFES’ ability to reach wider audiences and build trust slowly among apprehensive community members. As a result, the evaluation finds increased participation of women forum members. Forum expansion is also enabled by involving women speakers. A staff member explained, men began to attend, as women’s participation increased. Men believed it was their duty to support burgeoning women’s interest in politics, especially elections. In general, the IFES-supported Musharaka Forum is an important outlet for Syria’s marginalized community residing in Gaziantep. Participants have a platform to discuss and exercise their civic duties with the aim of eventually contributing to positive events in Syria. However, respondents reported challenges living in Turkey, due of their fragile resident status in the country. It seems that issues related to the daily living in Gaziantep may occasionally pose a distraction from the Musharaka Forum’s agenda, though not in a manner that impedes accomplishment of program objectives. For example, parents experience the stress of their children being segregated in schools, separated from the rest of the population. Lack of work is another looming concern for Syrians living in Gaziantep. Interviews revealed these hardships led to discussions unrelated to the forum’s civic engagement agenda. Yet, the forum allowed participants to discuss these urgent issues, demonstrating flexible responses to conditions on￾the-ground. CONCLUSIONS Consistent with PRIDE II program goals and objectives, PRIDE II interventions contributed to increased political awareness and civic engagement among underrepresented Syrian population segments, primarily women and youth. Though some achievements are small-scale, it appears activities are slowly chipping away at traditional values and roles of women and youth beneficiaries. Women recognize the added value they bring to political processes, and learn how to empower themselves through education, awareness, confidence-building and communication. Beneficiaries demonstrate a greater sense of efficacy around influencing leaders and community members. Participants are now more comfortable engaging decision-makers that previously seemed unapproachable as a result of PRIDE II programing. Musharaka Forum coordinators, however, highlight the challenge of reaching the poor, who are much harder to access and engage. This population segment lacks means of transportation, and therefore, cannot attend forum sessions or events. Musharaka 1 2 3 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 35 Forum coordinators consistently attempt to engage marginalized groups and resolve issues that limit some participants from joining the forum discussions – such as providing transportation to forum discussions whenever possible. The Musharaka Forum officers report significant progress in reaching these populations, but further engagement and support is needed moving forward. For the NDI program, demand for more sessions is high. Participants interviewed requested more sessions and dialog and stressed the importance of continued programming. According to fieldwork results, participants desire more sessions regarding roles and functions of the government that goes beyond the basic structures discussed in the current NDI program. Beneficiaries express an eagerness to understand government processes, interest in sessions focused on youth and their role in the transition process, and want sessions examining and analyzing, in-depth, the Syrian constitution and how to apply the constitution in the current Syrian context. RECOMMENDATIONS Across, each NDI, IRI, and IFES, it is worthwhile engaging other minority groups, such as Kurdish, IDP, and Christian communities.23 Some PRIDE personnel acknowledged, however, that reaching these groups is politically and physically difficult, but the interest in including them in PRIDE II programming is widespread. Continue, therefore, to provide minority-targeted programs that encourage engagement, and that facilitate more meaningful roles for minorities in political processes to the extent possible. Also, awareness sessions and campaigns targeting women and youth should increase. Participants in the three PRIDE II programs requested more sessions and trainings for longer durations of time. Many beneficiaries aged 18-45 living in Gaziantep, Latakia, Aleppo, Idlib, Hasakah, and Homs felt they now have the basic knowledge and skills related to democratic processes and terminology, and are ready for more advanced work in this area. Participants suggested sessions with experts in the fields of democracy and political processes and engaging with international organizations that can share their expertise in working with conflict-affected areas. Also, connecting the participants with other women leaders either in Syria or internationally. In this vein, the program might consider highlighting the success of women leaders living in conflict areas. For Musharaka programs, specifically, build activities that create a bridge between Syrians living outside and inside the country. This connection will help Syrians living inside and outside work together to build a better Syria, and contribute to a sense of ownership of the processes of rebuilding their country. The evaluation team notes a sense of disconnect expressed by some Syrian beneficiaries in Gaziantep. Offering programming for Syrians living inside and outside Syria, can bridge this gap, as Syrians in Gaziantep are keen to connect with Syrians still living inside, interviews and focus groups showed. Many still have family in Syria, and felt that those inside would greatly benefit from the Musharaka Forum activities. Musharaka respondents also stressed the 23 These groups were not part of the data collection. 4 1 2 3 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 36 importance of reaching other Syrians living in and around Gaziantep areas, but security conditions, political conditions in Turkey, and border constraints present formidable challenges. Noted, this would represent an addition to the forum scope of work for expanded, future programming, but could help advance steps towards blunting the communal disintegration that has affected so many. For IRI, provide more Seed Award funding and training to identify and implement citizen-led activities and campaigns that target the involvement of local decision makers. Decision maker buy-in is vital, as is their engagement in campaigns. Providing programs where women and men can work together to expand women’s involvement in politics could help women move into more leadership roles and address gender stereotypes that are shaped by the persistence of patriarchal norms and institutions in Syria. 4.3 EVALUATION QUESTION 3 Do national and sub-national representatives better understand their democratic roles and responsibilities and their role in peacebuilding? FINDINGS Across all programs, Syria PRIDE II activities inside of Syria had a positive effect on sub￾national representatives’ understanding of their democratic responsibilities. NDI and IRI activities increased local council members’ engagement with citizens. The program achieved such engagement, through recruiting representatives and other decision makers to civic education sessions alongside other citizens (NDI), and by approaching and involving decision￾makers in advocacy campaigns or locally-led initiatives (NDI and IRI ). In some instances, project staff facilitated the set-up of meetings and exchanges with citizens around specific issues of concern to beneficiaries. Such exchanges and engagement opportunities encouraged the exchange of knowledge and gave currency to democratic concepts and terms. Local decision makers interviewed reported an increase in their understanding of the significance of greater community outreach and engagement. Despite the Syrian revolution, few had experienced democratic governance, and project staff observed, that former authoritarian patterns of governance were the only norms many decision makers knew. Thus, communities found themselves returning to old, authoritarian practices. The program offered an opportunity to witness and experiment with different mechanisms for local decision making that elevated citizen engagement. “There was no engagement before – people did not go to the local council to talk or express, but now they come to us and talk, talk openly on Facebook and WhatsApp, and not through secret numbers, but not so effectively yet or comprehensively because of the security situation.” (NDI Program, Local Decision Maker, Syria, KII / Rapid Survey) Across many locales, efforts to engage local councils successfully translated into increased responsiveness to citizen-led demands. However, local councils are frequently not the only or most able decision maker in many communities; the conflict has produced a highly fragmented governance landscape, while the realities of aid mean that many CSOs are more resource rich 4 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 37 than most local councils. Therefore, council responsiveness was not always measured by direct financial support to a PRIDE II initiative as in many instances, councils did not have the resources even if they had the willingness. Thus, responsive local councils also supported grassroots efforts to mobilize funds for and implement projects that met citizens’ demands. The overall positive attitudinal changes amongst sub-national representatives did not immediately translate into improved perceptions of and attitudes towards women’s participation. Local councils succumbed to demands from IRI program advocacy campaigns calling for greater participation of women by opening women’s offices. While this is an important achievement and a positive step forward, beneficiaries and project staff expressed skepticism of fundamental attitudinal change among local councils, which are dominated by men. Several interviewees considered it a placating move, with the women’s offices delegated gender stereotyped functions such as women and child programs. “There is still some ridicule or belittling of women in the way local councils are dealing with us.” (IRI Women’s Group Activity leader, KII, GZT) Overall, PRIDE II national-level engagement was more limited, by comparison. Syrian citizens in Turkey engaged with SOC, SIG, and HNC through meetings and exchanges facilitated by the IFES Musharaka Forum and IRI women’s group activity. The Musharaka Forum held sessions that brought together the HNC Liaison Committee and Musharaka Forum representatives in coordination with national-level representatives, in order to increase citizen engagement and connect those in Istanbul and Gaziantep. While these exchanges were informative and helpful for beneficiaries involved, they remained ad hoc, and did not appear to result in more sustained interaction between them and national representatives. Still, SOC and SIG leadership interviewed described enhanced understanding of the implications of citizen engagement for legitimacy as a result of PRIDE II engagement. They report a more nuanced appreciation and understanding of the importance of citizen engagement for legitimacy. Prior to PRIDE II involvement, SOC and SIG respondents explained, they previously believed leadership simply made decisions, and people accepted those decisions. Now, they understand that even if a person is elected he / she has to return to voters and gain acceptance of decisions in order to be legitimate. While the HNC Liaison Committee and forum representatives traveled to Istanbul to engage with the HNC and Etilaf, and to attend trainings, the ability of these mechanisms to produce organizational-level changes was limited. Program activities were limited to affecting individual leaders’ knowledge and attitudes, outcomes largely due to conflict influences on political dynamics in Syria. NDI NDI programming directly influenced local decision-maker attitudes and efforts towards citizen engagement. As a direct result of NDI’s civic education and advocacy efforts, sub￾national governance actors – understood to include local councils, CSOs, local notables/heads of families, prominent businessmen or wealthy individuals, or political leaders – responsiveness to citizens improved. Local council members attended civic education sessions with CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 38 beneficiaries; similar to citizens, some have adopted terms and concepts in their other meetings and engagements with the community. “About a year and a half ago I received a report about governance and such and I had no clue what those meant. I called [the facilitator] and asked him to come over for a cup of tea and we spoke, and then we held several sessions on this.” (NDI Program, Local Council Member, KII / Rapid Survey, Syria) “Of course, had [engagement with citizens] not affected me, I wouldn’t have continued, especially if we hadn’t seen results. Interacting with people made us understand how better to understand people and communicate with them and how to work [as a local council].” (NDI Program, Local Council Member, KII / Rapid Survey, Syria) Beyond newfound receptivity to meeting with concerned citizens, sub-national actors granted permission for initiatives and assisted group advocacy efforts, whether by garnering support amongst other governance actors, providing resources directly, or by participating in the activities themselves. In the rapid survey of 21 local decision-makers engaged by NDI, 78 percent said they had been asked to provide funding, 30 percent were asked to build support in the community, and 26 percent were asked to participate. All said they had agreed to the request.24 An overwhelming 96 percent stated that their levels of engagement with citizens had improved and increased in frequency after their experience with citizens’ advocacy. The local decision-makers attributed the increased engagement to enhanced comprehension of the role of citizens, as well as their relationship to local representatives. In response to a question of whether the interaction with this group of advocating citizens affected understanding of the role of citizens in local governance, one local council member said: “Completely, they mobilized action in the village through the volunteer group and now everyone turns to them for opinion and help whenever there’s any activity in town. Involve them to discuss service provision issues with the new local council post.” (NDI Program Local Council Member, KII / Rapid Survey, Syria) And asked if the interaction with the group of citizens affected thoughts on citizen-local council / decision maker relationship, another local decision maker replied: “Completely, we have changed our mind about how decision are made. We used to just take a decision, now after sessions, I realized it’s important to talk to people, hear their opinions, communicate why certain decisions have to be made or not.” (NDI Program Local Council Member, KII / Rapid Survey, Syria) IRI 24 This was not a random sample of decision makers as access to all decision makers in the areas of operation was not possible or feasible. The evaluation team asked to be provided with a list of decision makers who were approached by citizens as part of advocacy activities. Given the project was likely to keep in contact with those decision makers who had supported advocacy efforts, we expect bias in this group of decision makers. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 39 As mentioned above, national-level and sub-national actors responded to IRI citizen-led demands to include women in decision-making, even if they were not yet entirely convinced of the importance or necessity of women’s participation. “We have seen a lot of change – greater awareness of the importance of participation, 60 percent improvement in [women’s] knowledge, but in practice, local actors have prevented us from fully utilizing this knowledge. Now they [the women] cannot show much leadership inside, but there’s a strong base [of support].” (IRI Women’s Activity Leader, KII, Syria) “In the beginning, men objected to female participation and engagement in such topics, but through discussions and women’s persistence to take their rights, this has become normal for men. They don’t negotiate over this anymore.” (IRI Women’s Activity Participant, FGD, Syria) Rapid survey results among IRI-supported women program participants speak to this sentiment (see Table 10). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 40 Table 10: IRI Women Participant Views: Local Engagement (Syria) I would like to learn about your engagement with local decision-makers, and would like to read you some statements. As I read the statement, please tell me the extent to which you agree or disagree. I am now more capable of: Strongly Agree / Agree Strongly Disagree / Disagree Engaging or interacting with local council members. 100% (13) 0 Engaging or interacting with heads of local CSOs. 100% (13) 0 Engaging or interacting with local political actors. 61% (8) 15% (2) Engaging or interacting with local business people. 15% (2) 54% (7) Engaging or interacting with local families / elders / local notables. 28% (4) 31% (4) N=13 The single, most concrete and clear result of improved national representatives’ responsiveness to citizen engagement efforts was the recommendations paper produced by the IRI program with women. Described earlier, IRI women participants, through the network, conducted wide-ranging community consultations inside Syria on women’s expectations and forms of participation demanded for the transition process. IFES On the current evaluation question, most prominent for IFES, was the change in the knowledge, attitudes and behavior of the national opposition actors that can be attributed to the electoral technical assistance provided by IFES to SOC leadership, the, then, HNC members, and SIG staff. Formal opposition group beneficiaries spoke highly of the quality of the training and technical assistance provided, and credited the program with several initiatives undertaken by the SIG, SOC, and now, SNC, to advance democratic governance in opposition-held Syria. Workshops with the SOC on constitutional development, electoral systems, their legal frameworks, and their impact on representativeness of democratic systems, and the various considerations and reforms required to ensure free, fair and transparent elections significantly impacted participating staff and members of the political committee. The SOC established two specialized committees on elections and the constitution. These committees prepared position papers on these issues for negotiations with the regime, in support of the SNC. The committees referred back to IFES for further technical support to finalize these papers, which made their way to Geneva. Similar workshops with the SIG, and before, with the Local Administrations Council Unit (LACU) at the SOC, led to concerted effort to create frameworks for provincial and local council elections. IFES technical assistance led to the issuance of electoral guidelines and a CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 41 provincial level system which was implemented across all opposition-held governorates. Trained staff members went on to instruct elections administrators and staff on the agreed system for them to conduct and oversee provincial level elections. They also attempted to establish a system for open and public local council elections: the initiative was successfully piloted in Saraqeb, a former SIG staff beneficiary asserted. However, the consequent bombing of the Saraqeb town market, close to where elections had been held, dissuaded the SIG from attempting any further open elections. “Constitution, national participation, national electoral legislation, after that good governance, local administrations, institution building… We learned all this from IFES Ninety-nine per cent of us did not know these issues. What they gave us, importantly, are the international standards.” (Former SIG Staff Member and Beneficiary, KII, GZT) Though the SOC and SIG initiatives undertaken as a result of the IFES-supported technical assistance, themselves, had varying levels of success and impact, they are a strong indication of changes in knowledge and understanding of their democratic roles and responsibilities. As mentioned, the Musharaka Forum allowed national actors engagement with citizens in outreach through its activities. SOC members attended forum sessions and met with participants. Forum members aggregated interests and concerns and presented them to the SIG, and while the SIG has yet to respond, it has declared its intention to do so, and encouraged further articulation of demands. The SIG has also reached out to the Forum to request facilitation of access to more citizens. “Although we started to know about these issues, we were still separated from the events. Nevertheless, after meeting those politicians, they are telling us about what is happening as we are people who are interested in politics. It is very valuable opportunity that we are meeting the decision makers.” (Musharaka Forum Beneficiary, FGD, GZT) CONCLUSIONS Amongst the most visible progress at the sub-national level are cognitive shifts in sub￾national leaders’ understanding of their democratic roles and responsibilities. Efforts to engage with local councils culminated in enhanced council responsiveness to citizen-led demands. Results are more muted, by contrast, on substantive change in attitudes towards women’s role in the political space. Indeed, leaders mentioned they are supportive of women’s interests, but the extent to which this will translate into deep and broad institutionalization of women’s interests is uncertain. The persistence of patriarchal norms is pervasive and gendered perceptions of roles widespread. At the national level, some leaders’ understanding of democratic systems of governance grew and with it, their comprehension of what needs to be negotiated, how citizens inside Syria should participate, and their role in transition. This was evident in the steps taken by SOC and SIG to better prepare themselves for negotiations on the elections and constitution files. However, their stated affirmations of the significance of engaging Syrian citizens in the transition-related discussions did not translate into visible action. 1 2 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 42 4.4 EVALUATION QUESTION 4 What activities are programs are working well and why? Which are not working well and why? What examples can be identified as successful stories or failures? WHAT IS WORKING WELL The evaluation team identifies several aspects of the PRIDE II project that performed especially well, based on findings from KIIs, focus groups and the rapid assessment survey. Syria PRIDE II access and recruitment of beneficiaries in Syria and Turkey is a strength program-wide. Qualitative and quantitative findings indicate NDI, IRI and IFES program activities succeeded in reaching targeted Syrian citizens, including women, youth, adolescents, activists, and marginalized community members. NDI, IRI, and IFES implementing partners working in Syria varied and adapted their beneficiary recruitment and stakeholder engagement methods, especially inside Syria, where the poor security situation constrained activities. In Turkey as in Syria, implementing partners leveraged trust networks through local staff and partners to reach beneficiaries; 98 percent of surveyed beneficiaries and all those interviewed said trust in project staff directly informed their decisions to participate in their respective program activity. A similarly large portion (94%) said that trust will likewise influence their participation in future PRIDE activities. All PRIDE II programs effectively engaged beneficiaries. Whether in civic education sessions, political empowerment trainings, advocacy campaigns, or forum dialogue sessions, once recruited, Syrian citizens engaged closely and chose to continue participating in activities. Furthermore, most participants interviewed said they talked about concepts or topics they learned through PRIDE activities with their friends and family. Repeat participants also tended to bring others with them through word-of-mouth; multiple beneficiaries interviewed said they had been introduced to the program through a friend. Across the NDI, IRI , and Musharaka Forum programs, interviewed beneficiaries attributed their continued participation to the value and usefulness of knowledge and skills gained through activities. This was equally the case for beneficiaries in Syria and in Turkey. Level of participation in additional activities is also significant. Regardless of the success of advocacy campaigns in reaching their intended results, partners achieved aims of mobilizing people to advocate for community needs – and engaged decision makers in the process. The majority of beneficiaries who attended civic education sessions (NDI), political empowerment (IRI), or dialogue sessions (IFES) also went on to participate in collective action. In the cases of NDI and the IRI programs, beneficiaries involved a wider segment of their community in advocacy campaign activities – individuals who had not participated in previous program activities. PRIDE II beneficiaries believe see dividends from their collective action in the rapid assessment survey (see Table 11). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 43 Table 11: Level of Engagement in Advocacy Activities NDI Beneficiary Views of Program (Syria) I am engaged in activities, such as advocacy efforts with the local councils. 77% (51) Musharaka Forum Beneficiary Views of Program (Turkey) I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to engage in advocacy related to Syria or to Syrians in Turkey in a meaningful way. 65% (39) IRI Women Participant Views (Syria We / our group was able to conduct advocacy activities in our community. 100% (13) The recruitment and engagement of women beneficiaries worked particularly well, as evidenced by the high level of women’s participation. As discussed above under Evaluation Question 2, women PRIDE II beneficiaries reported increases in knowledge of democratic concepts, leadership and communication skills, and a greater sense of empowerment. While NDI and Musharaka Forum activities were not exclusively directed at women, women still comprised half or more of all participants. Various strategies contributed to this result, such as: recruitment and capacitation of both men and women field-level staff and facilitators; and tailoring the implementation of certain activities – like civic educations sessions or trainings – to circumvent local security or customary restrictions on women’s mobility, thus facilitating women’s ability to attend.25 Locally, the PRIDE II project succeeded in engaging decision-makers in advocacy and awareness raising activities. Despite the poor security conditions and difficult political environment, NDI and IRI programs, especially, managed to create opportunities for their participants to interact with and engage local decision makers in their activities. To operate in Hayat Tahrir al-Sham (HTS)-controlled areas, where public discussion of politics is not possible, project staff and participants approached only certain trusted decision-makers bilaterally, and through these decision-makers, negotiated access to the local council. In other areas, council members, themselves, attended civic education sessions or hosted them, while in other areas, such sessions were held at the local council office. At the very least, local council members agreed to meet with citizens as a result of project activities in most towns. “The main challenge in all the areas in which we are working is that no one [local authority] likes the concept of democracy.” (Staff Member, NDI Program, KII, GZT) Finally, training and technical assistance to SOC and SIG leadership and staff on electoral issues was especially well-received and had a visible effect. SOC leadership and staff, as well as former SIG staff involved in IFES-provided technical assistance, spoke to the influence of IFES technical assistance on their understanding of the issues themselves as well issues’ significance. Furthermore, the assistance informed efforts by the SIG to organize open 25 This included holding sessions in private homes or women-only trainings in locales considered safe and acceptable. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 44 local council elections and establish a common framework for provincial-level council elections. The SOC and the HNC / SNC established teams to prepare for negotiations on the electoral and constitutional files. Survey results suggest that most PRIDE II beneficiaries are satisfied with the performance of PRIDE II activities, with most saying that various activities work fairly or very well. The exception is the IRI program results, which suggest women beneficiaries perceive that more work needs to be done to improve women’s engagement with national-level decision-makers (see Tables 12-14). Table 12: What is Working Well / What is Working Poorly: Beneficiary Views: NDI (Syria) Works Well / Very Well Works poorly / Very Poorly Does Not Apply to Me Political empowerment trainings 81% (54) 10% (7) 6% (4) Advocacy campaign planning 81% (54) 9% (8) 6% (4) Implementation of advocacy campaigns 75% (50) 10% (7) 10% (7) Reaching men in your community inside or outside of Syria (for example, Turkey) 87% (58) 3% (2) 7% (5) Reaching more women in your community inside or outside of Syria (for example, Turkey) 85% (57) 10% (7) 1% (1) Reaching youth in your community inside or outside of Syria (for example, Turkey) 89% (59) 6% (4) 3% (2) Reaching more Kurds in your community 49% (33) 31% (21) 15% (10) Reaching more internally displaced persons inside or outside of Syria (for example, Turkey) 79% (53) 13% (9) 4% (3) Increasing opportunities for women to express themselves in communities inside or outside of Syria (for example, Turkey) 90% (60) 6% (4) 1% (1) Influencing local decision-makers from Syria to the extent desired 74% (49) 21% (14) 3% (2) Changing local decision-makers’ attitudes towards women’s participation in public and political life 79% (53) 15% (10) 3% (2) Encouraging local decision-makers response to local demands 85% (57) 9% (6) 3% (2) N=66 Table 13: What is Working Well / What is Working Poorly: Beneficiary Views: IRI (Syria) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 45 Works Well / Very Well Works poorly / Very Poorly Does Not Apply to Me Political empowerment / awareness trainings 100% (13) 0% 0% Advocacy campaign planning 100% (13) 0% 0% Implementation of advocacy campaigns 100% (13) 0% 0% Reaching men in your community inside or outside of Syria (for example, Turkey) 76% (10) 7% (1) 15% (2) Reaching more women in your community inside or outside of Syria (for example, Turkey) 100% (13) 0% 0% Reaching youth in your community inside or outside of Syria (for example, Turkey) 83% (11) 15% (2) 0% Reaching more Kurds in your community 83% (11) 7% (1) 7% (1) Reaching more internally displaced persons inside or outside of Syria (for example, Turkey) 100% (13) 0% 0% Encouraging local decision makers responses to demands 77% (10) 0% 7% (1) Changing local decision-makers’ attitudes towards women’s participation in public and political life 77% (10) 7% (1) 7% (1) Influencing national Syria decision-makers 30% (4) 53% (7) 0% N=13 Table 14: What is Working Well / What is Working Poorly: Beneficiary Views: IFES (Turkey) Works Well / Very Well Works poorly / Very Poorly Does Not Apply to Me Political empowerment /awareness trainings 95% (57) 2% (1) 3% (2) Reaching men in your community inside or outside of Syria (for example, Turkey) 84% (51) 10% (6) 5% (3) Reaching more women in your community inside or outside of Syria (for example, Turkey) 76% (46) 17% (11) 5% (3) Reaching youth in your community inside or outside of Syria (for example, Turkey) 81% (49) 12% (8) 5% (3) Reaching more Kurds in your community 63% (38) 23% (14) 7% (4) Reaching more internally displaced persons inside or outside of Syria (for example, Turkey) 81% (49) 11% (7) 7% (4) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 46 Works Well / Very Well Works poorly / Very Poorly Does Not Apply to Me Increasing opportunities for women to express themselves in communities inside or outside of Syria (for example, Turkey) 71% (43) 21% (13) 2% (1) Influencing local decision-makers from Syria to the extent desired 71% (43) 21% (13) 3% (2) Influencing national Syria decision-makers 75% (45) 18% (11) 3% (2) N=60 Observation of individual Syria PRIDE II activities, sheds more light on strengths unique to each program. NDI NDI’s civic education sessions were widely regarded by project staff and beneficiaries alike as the most successful activities of the project. As stated in earlier sections of this report, quantitative data, KIIs and FGDs with staff, community-based facilitators, and beneficiaries indicated that these sessions have high levels of impact on participating individuals. Sessions most lauded were those on democratic governance, citizenship, rule of law, separation of powers, transitional justice, and women’s rights. Beneficiaries spoke of acquiring knowledge of concepts they had never understood, considered or even heard of before. They also felt they had gained fuller awareness of the significance of citizen engagement on democratic principles for transition in Syria. Repeatedly, beneficiaries and facilitators were keen to express the power of their newly acquired knowledge on their mindset, linking it to how their vision for a future Syria had changed and their expectations of and voice in the transition process. The sessions not only affected citizen’s knowledge and attitudes, but also influenced their behavior. In sessions, facilitators observed a difference in how repeat participants spoke about issues and debated with others: more openly, constructively and with less hostility towards opinions unlike their own26. One facilitator told the story of a lawyer and a local woman who decided to run for local council elections as a result of their attending sessions and feeling empowered and encouraged. They were both elected to their respective councils. “I have noticed that people who attended my sessions – from men and women – could participate in those public meetings in an academic manner of discussion. They could discuss the ideas in an academic manner as they now have more knowledge of topics; such as elections… So, now, there is a positive change as the local council members and the citizens meeting and talk. Through our public sessions with the people, we discuss and target any failure in any service.” (NDI Program Beneficiary, FGD, Syria) 26 KIIs with coordinators and FGDs with all NDI facilitators. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 47 In their own communities, the sessions mobilized citizens into taking action around common needs and issues, both through NDI-organized advocacy campaigns and independent local initiatives. Long-accustomed to a centralized system of governance, heavily controlled from the top with few avenues for individual or collective initiative, facilitators said, beneficiaries now feel and express greater awareness of their own agency and possess higher levels of self-efficacy. “We know so much more about so many topics – citizenship, transitional justice, women’s rights, there are a lot of rights I did not know about such as the right to express my opinions… As a result of my attending sessions, I became the coordinator for a local initiative to help women run for [LC] elections… I also worked on a campaign on early marriage, we covered half of Atareb, and did sessions in schools, clinics and meetings with parents.” (NDI Program, female beneficiary, Syria, KII) NDI civic education sessions did well in achieving high levels of participation of women. NDI personnel reported that between 40 percent and 60 percent of participants in these sessions were women. NDI’s coordinators and facilitators held both women-only as well as mixed gender groups, a strategy that ensured access to women in areas and at times when their mobility was restricted. As a result of the program, women participants “established a women’s office in the local council in Azaz” (NDI Program, male coordinator, Syria, KII) and a woman facilitator from the program in Jarablus was elected to the LC. “Women are currently manlier than the men in their participation in sessions!” (NDI Program, male coordinator, Syria, KII) Border closures in February 2015 hindered the NDI program effort’s ability to connect directly with their field team, including holding in-person meetings and trainings in southern Turkey. In response, the project developed an online training and learning platform. The platform allows NDI staff in southern Turkey to conduct remote trainings with facilitators and coordinators inside Syria, share resources and material, as well as monitor and follow-up on activities and reporting submitted from the field. Facilitators inside Syria hired after border closures credited the system for their connectedness with the rest of the team, many of whom they have never met in person. NDI also succeeded in linking citizens with governance actors and decision-makers in their communities. Decision makers attended sessions were they met and discussed topics of interest to the community; or they were approached by participants – aided by facilitators – as part of an advocacy campaign. The project was able to organize some advocacy campaigns in spite of the precarious political and security situations by selecting relatively apolitical, basic services issues around which to mobilize citizens. Furthermore, by prioritizing process over results, the project achieved the outcome of enhancing engagement between citizens and decision-makers. “When facilitators or coordinators recognize that there is an issue participants want to talk about to their decision-makers, they try to be that liaison and go to the local council.” (Staff, KII, NDI Program) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 48 IRI IRI’s support of women was highly effective and fruitful. Working through a hub-and-spoke model proved widely successful in reaching and engaging a wider range of women inside Syria: IRI worked with a core team of more experienced participants from 25 IRI women’s groups. These participants received political empowerment training in southern Turkey, and subsequently went on to mobilize and train larger groups of women in their respective villages. The fact that the IRI women’s groups were already well established – both as functioning civically active groups as well as trusted individuals in their communities – was crucial for the success of this strategy. IRI group members’ decision to target all women in their communities, regardless of level of education or activism, ensured strong community-wide engagement. Once back in their home towns, the primary IRI beneficiaries successfully recruited between 30 to 40 local women to attend political empowerment trainings that they delivered based on their own training in Turkey. As these were women only workshops, IRI held the trainings as semi-private events, away from pressures of hostile local governance actors. This was particularly important in HTS areas. Following their trainings, IRI mobilized their local participants to advocate and campaign for greater women’s participation in local councils, across 22 towns in Idlib, Aleppo, Qamishli, and Rif Damascus. In total, the IRI project funded 10 initiatives, which lasted an average of five months. The advocacy campaigns resulted in 22 local councils establishing women’s offices, and in two women getting elected to the local council itself. Women demonstrate increased ownership, efficacy, and commitment to being involved in public. Across the board, IRI participants expressed high impact of PRIDE II activities on their sense of empowerment and self-efficacy. This was the result of both knowledge gained in the training, as well as the experience of the advocacy campaigns and initiatives implemented. “Some women had family or social problems -- their husbands never let them go outdoors. But when they joined us, they became brave, they were encouraged and started to significantly follow politics and demand their rights. I am one of those women.” (IRI Participant, Syria, FGD) “We knew about the treatises around women’s rights, but we never thought to look at them. But now we know 1325 and CEDAW,27 and know what rights to defend.” (IRI Participant, Syria, FGD) “I can discuss and debate any decision with any person.” (IRI Participant, Syria, FGD) “As [IRI] women, we benefited, but the female beneficiaries [that we worked with], they changed in their view of women’s potential capacity to play a political role.” (IRI Participant, Syria, FGD) 27 Convention for the Elimination of All Forms of Discrimination against Women. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 49 The project also did very well in connecting women in country with national- and international￾level representatives. Indeed, it provides perhaps the only instance of a concrete linkage created between citizens inside Syria and their representatives in Geneva that evaluators observed as a result of PRIDE II activities. Through the IRI women’s program, the IRI women’s group galvanized grassroots consultations across governorates with Syrian women and reflect their views, hopes, and demands for potential transition to the Geneva negotiations through a paper that was submitted to de Mistura’s office. “We didn’t even know that there were women among the HNC. Then we met the women’s council of de Mistura. Even if it’s a secondary role, it’s important [that they are there].” (IRI , Syria, FGD) “I don’t know if they [international figures] will be responsive or not…the discussions were intense, but we spoke…all thanks to the civil society, which is giving us this opportunity to talk bravely and openly. We rely on it.” (IRI, Syria, FGD) IFES The Musharaka Forum succeeded in creating a safe space and a community for Syrians in Gaziantep. For many beneficiaries who have been long disconnected from their home towns, the forum’s activities and sessions rejuvenated commitment to civic engagement inside Syria. This was especially the case for youth and adolescents, most of whom had left Syria at a young age. The extent of beneficiaries’ engagement was reflected in their continued participation in forum dialogue sessions, as well as in community activities the forum undertook. “Political knowledge is critical to Syria in the future. Syria needs the coming generation to have [political] awareness. We, as the future generation, know about our responsibilities, which will help us to build Syria.” (Musharaka forum member, Gaziantep, FGD) Members not only actively participated themselves, but many brought friends to the forum, allowing the program to recruit more beneficiaries through word of mouth. One FGD participant said he joined after attending sessions for adolescents with his teenage daughter, for whom he was looking for a safe civic activity. He has since become an active member of the Dialogue Forum. The Women’s Forum is also especially active and members lauded the forum’s outreach to more marginalized Syrian refugee communities in Gaziantep. “The sessions they were carrying out in slum areas drew my attention. It was a great idea from the forum to reach out the people because we know it is not easy for some women to get access to forum sessions and meetings….It is not necessary to talk about politics always, we spoke with women about their problems, rights, and demands, and it was a very great idea from the forum to reach out to those women.” (Musharaka forum member, Gaziantep, FGD) This sense of community cohesion not only benefited Syrians in Gaziantep, but also translated to collective action into Syria. The forum provided Syrians in southern Turkey, not normally involved in politics and at risk of feeling marginalized by the political process, an avenue to remain engaged in their country’s political life. For example, a number of forum members from Deir Ezzor became involved in discussions and efforts to establish a local council for Deir Ezzor CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 50 remotely, from southern Turkey, in preparation for ISIS’s withdrawal from their governorate. Forum staff did not facilitate these efforts, but in the FGD one member credited their involvement in these efforts to increased impetus gained from the forum’s sessions. Another noteworthy outcome for IFES programming is the change in the knowledge, attitudes and behavior of the national opposition actors that can be attributed to the electoral technical assistance provided by IFES to SOC leadership, members of the former HNC, and as well, the SIG staff. The quality of the training and technical assistance provided is valued by opposition beneficiaries, and the program is credited with facilitating several initiatives undertaken by the SIG, SOC, and now, SNC, to move democratic governance forward in opposition-held Syria. As mentioned, workshops with the SIG, and the LACU at the SOC, led to the establishment of frameworks for provincial and local council elections. IFES assistance led to the issuance of electoral guidelines and a provincial level systems for all opposition-held governorates. Trained staff members shared their knowledge by teaching elections administrators and staff on the system enabling the conduct and supervision of provincial elections. WHAT IS NOT WORKING WELL Yet, the Syria PRIDE II activity, has faced challenges affecting changes in other result areas. PRIDE II is generally less successful in effecting substantive attitudinal change amongst local- and national-level decision-makers. While local decision-makers willingly met with citizen groups to discuss their demands and take action on service-based issues, they resisted exploring or negotiating changes of a more political nature. Citizens’ ability to advocate for change around political issues such as representation, inclusion, participation, or transparency remained extremely limited. Citizen-led service-based advocacy was relatively more successful. Affecting change among national-level decision-makers is also minimal, a finding supported by the survey, which finds that a smaller majority of beneficiaries (56% vs. 27%) believe that PRIDE activities have done well at efforts to influence national-level decision-makers. Likewise, the program provided only a few opportunities for citizens to connect with national-level decision makers. The significant gains made at a grassroots level towards enhanced knowledge and awareness of democratic principles, and citizen advocacy, have not palatably translated into concrete linkages with national level decision makers. The various opposition bodies – SOC, SIG and SNC – remain distant from Syrians in Turkey, and even further still from citizens inside Syria. Overall, less than five national-level decision-makers participated in program activities or connected through them to beneficiary citizens in any way. With the exception of the work of the IRI-supported women’s network, the evaluation found no evidence of any feedback mechanism created between citizens and their views, positions, and activities on the ground, and national opposition leaders negotiating on citizens’ behalf. IFES-supported engagements between Musharaka Forum representatives, and national-level decision makers in Istanbul, facilitated feedback mechanisms between citizens and the national￾level, however, the breadth of engagement is limited, due to the conflict. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 51 Increases in women’s engagement in the SOC and SIG was not very robust. While there were political hindrances to this goal unrelated to women’s engagement as such, interviews with women SOC staff reported a lack of change in SOC and SIG leadership attitudes towards the importance of women’s participation. Understanding that greater women’s participation is a donor demand, the SOC political committee – dominated by men –appears to support the issue, but in practice, takes very little action internally, SOC staff stated. Indeed, the number of women on the political committee of the SOC declined during the life of the PRIDE II project. NDI For the NDI activity, translation of gains in knowledge and attitudinal change at the individual-level into concrete change at the communal- or socio-political level is not yet evident. Citizens remain reluctant and unable to advocate for change around the more political topics discussed in the civic education sessions; questions of fair representation, open and fair elections, rule of law, transparent and democratic local governance remain too sensitive and unattainable across most areas of north and north-eastern Syria. Since even discussion and advocacy on these issues presented difficulties for staff and beneficiaries alike, there is at this point little evidence of sustainable mechanisms towards greater representation and participation at the local level. While it is acknowledged that this is not an immediate program objective, to the extent that attitudes and successful advocacy are related to establishing local institutions that reflect more liberal ideals, it is worth noting that work that expands awareness among local actors would be valuable. “We don’t discuss political issues but we are preparing a political safe space for future building of local institutions.” (Female coordinator, KII, NDI Program, Syria) IRI Though the IRI activity facilitated linkages between women and national-level actors, in a similar vein to other PRIDE II activities, the project did less well with providing sustained and continuous connections between national-level decision makers (SOC, SIG, and negotiating committees) and sub-national level decision makers and citizens inside Syria. This is partly due to weaknesses among opposition leaders, and the Syrian political context. As discussed earlier, the conflict has precluded progress on engaging formal Syrian opposition partners, like the SOC. Of all Syria’s opposition groups, IRI managers explain, the SOC is still considered the most important opposition voice, and thus remains among IRI’s key opposition partners. Although, IRI partners expanded opposition group learning around their democratic roles and responsibilities, the SOC’s capacity is limited, as is their impact on a national-scale. Funding to the SOC is also minimal, and individuals differ in their ideas on how to accomplish goals.28 28 An IRI team member in Washington, DC explained, that the SOC needs to execute programs that deliver services, in order to influence the SIG. Thus it is important that the SOC implement projects and is awaiting funding from the international community. The SIG identified project areas in fall 2017. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 52 “The SOC suffers when they can’t deliver on outcomes. They need to produce proposals that are credible and doable.” (IRI Staff Member, Washington, DC, KII). IRI, however, continued its capacity building work with the SOC and Syrian Interim Government (SIG) by encouraging collaboration among the program management teams and working groups of the SIG and SOC, which included women in decision-making processes. IFES Disconnect between partners and implementers? Qualitative interviews suggested a disconnect between the IFES’s project team’s understanding of the type of activities implemented through the CEC, and the local implementing partner (Shar for Development) understanding of how their activities contribute to the goals and objectives of PRIDE II. This disconnect in information relayed in KIIs suggests insufficient coordination over the design of Shar’s project, and its fit within the larger PRIDE program. An issue seems to be the question of how best to adapt program activities in response to emerging challenges of working within AA areas. While IFES sees the process of engaging AA staff in workshops around basic vocational skills as a stepping stone to opening channels of dialogue between citizens and the local authorities, Shar for Development prioritizes the beneficiaries’ need for this type of skills and training, and considers its primary aim to be the fulfillment of local (developmental) priorities. Thus when asked about the main goals and outcomes of the project, Shar staff spoke about the number of beneficiaries hired at international organizations as a result of their training. The evaluation team observes that Shar conducted a number of trainings, including: sessions for youth on improving resumes and obtaining employment; proposal design; media material; and effective communication. Shar also completed 16 dialogue sessions among youth and local activists on issues relevant to people in the area. Activities also included forums on advocacy, consumer safety, and advocacy for the environment on burning of fossil fuels. Prior to municipal elections, Shar, in addition, conducted workshops on electoral monitoring and electoral systems, and subsequently monitored the elections. According to Shar, the AA tended to be excluded from donor-funded trainings and Shar targeted the AA for capacity building activities. Shar explained that 40 percent of beneficiaries who attended workshops were from the AA. It was unclear, however, whether trainings for AA staff had been carried out separately from other citizens, or trained together.29 USAID Perspectives Regarding the PRIDE II program as a whole, the evaluation team also noticed several coordination and communications issues. 29 More follow-up research is needed on the CEC, however, in order to fully understand program dynamics on￾the-ground. The evaluation team conducted follow-up interviews with IFES staff, but did not have an opportunity to speak to CEC beneficiaries and other personnel in Syria, as conditions prevented access to more potential interviewees. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 53 Coordination among CEPPS partners is generally weak and key synergies between respective activities are missed. The evaluation team notes that for the duration of the PRIDE II program, CEPPS partners jointly engaged a Gender Advisor to provide assistance in mainstreaming gender across all PRIDE II project activities and to provide overall analysis and comprehensive recommendations regarding the partners' collective efforts to promote women's civic engagement and political participation. The Gender Advisor also facilitated regular CEPPS partner dialogue and information-sharing regarding each partner's efforts to integrate gender in all program activities, thereby improving coordination and communication within the consortium. However, a number of other opportunities for collaboration between activities that could enhance program effect and efficiency seem to be unexploited. For example, partners could link IRI and NDI beneficiaries in Syria. NDI beneficiaries and IRI-supported women beneficiaries do not really engage, even though they overlap on issues, advocacy, and in areas of operation. Also, the IFES Musharaka Women’s Forum and IRI-supported engagement with the SOC to increase women’s participation are complementary activities that could offer opportunity for CEPPS partner collaboration. The Musharaka Forum has established positive relationships with individuals on the SOC and SIG. These could perhaps be leveraged to foster national-level engagement with Syrian beneficiaries working with IRI and NDI. General perceptions on program coordination between PRIDE II’s CEPPS partners and USAID differ. From the view of CEPPS personnel, coordination between CEPPS partners and USAID is essentially good. CEPPS’ engagement with the USAID Agreement Officer Representative (AOR) was substantive and consistent throughout the program. From the USAID perspective, however, coordination still needs improvement in some areas. Roughly a year ago, CEPPS partners used to rely primarily on quarterly reporting to share information, consistent with the requirements of the cooperative agreement. However, there was critical information and events of which USAID was unaware. USAID, as a result, has requested more frequent engagement between CEPPS partners headquartered in Washington, DC and USAID￾Washington, which includes twice monthly updates and a monthly meeting. In the opinion of senior USAID personnel, the improvement is significant. “We have to be willing to go to the partners to get information. …The meetings were instituted to enable the AOR to defend and promote the CEPPS program. Over last seven months alone, we’ve noticed a big difference.” (USAID Senior Official, KII, Washington, DC). CEPPS quarterly reports, which were also very lengthy and difficult to read, are now more streamlined at the direction of USAID-Washington. Coordination and communication between core PRIDE II implementers that is more routine is also welcome. According to one PRIDE II program manager, little communication and coordination occurs by matter of routine. What communication does happen is the result of one’s own initiative. The comments of one CEPPS staff member underscored the challenges. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 54 “We’re just not cognizant of what other partners are doing. …and it’s a challenge. There are opportunities for coordination. I'm able to know what’s going on informally through social relationships, but not officially through [implementers]. There are more ways we can leverage opportunities together. Local-level staff may happen to coordinate, but it’s not articulated as it should be with greater coordination at the implementer level. If there could be more regular reporting between the partners that would help. In one instance, we did not know of [Program X’s] activity in Syria, until we read the quarterly report, which was after their activity there was finished. [Staff] are encouraged to reach out to citizen groups for information, but generally, we don’t find out about CEPPS program [activities]" (CEPPS Program Staff, KII). From the USAID perspective, there are other aspects of Syria PRIDE II program management that could facilitate effective program coordination and thus, implementation. Reduce complexity of PRIDE II reporting. Several USAID personnel pointed to the complexity of PRIDE II program information and reporting. The program, some say, is difficult to understand – which may be partly attributed to lack of communication between CEPPS PRIDE II Syria partners. The multiple interventions and how they relate to each other can be unclear. Though the competitive relationship between CEPPS partners may drive this dynamic, the vague information make it difficult to fully understand the program. Reconciling interagency and development perspectives. At a higher operational and management level, an additional challenge remains reconciling the differences between the development perspective and that of the U.S. interagency in program implementation. While the development perspective emphasizes aspects such as the civic education component of democracy and governance work, the interagency lens stresses supporting local council members or concerns related to public diplomacy, a disconnect that may stem from lack of familiarity with PRIDE II within the interagency, according to one senior USAID officer. The PRIDE II project, moreover, is a cooperative agreement. The nature of the arrangement means that partners have more leeway in shaping the direction of their programs, than they would under a formal contract with USAID. Yet, one USAID senior officer stressed that USAID’s need to provide guidance is an imperative, particularly in an environment as precarious as Syria. There needs to be more of a balance between CEPPS partners’ ability to maneuver as they implement projects, and USAID’s need to ensure the project is aligned with U.S. policy. CONCLUSIONS PRIDE II activities aimed at increasing individual citizens’ knowledge and awareness of democratic governance practices were highly successful for NDI, IRI and IFES, alike. There was clearly strong demand from beneficiaries for these activities, and once they attended one event, they were highly likely to continue attending, recruit others, as well as participate in other activities organized by the projects. For Syrian citizens in both Turkey and opposition-held areas of north and east Syria, newly acquired knowledge on basic, yet, key issues of democratic governance, citizenship, and women's rights is crucial for their ability to engage in their communities, in the transition process and to having a voice. Over the longer term, it cultivates democratic actors. 1 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 55 Advocacy activities undertaken at the sub-national level fulfilled their objectives of enhancing engagement and interaction between citizens and local decision makers. Beneficiaries and decision makers, each, felt that such activities were of value in improving relations, expanding inclusion in decision making, and introducing more democratic practices. Although these attitudinal changes were limited to participating PRIDE II decision makers only, there is the potential for similar impact with expansion of the activity to other local decision makers, contributing to the possibility of broader institutional development and change. While PRIDE II implemented activities at the sub-national and national levels, there were fewer connections actually made across these levels. This was true within projects and across the program. In addition to bolstering negotiation efforts, IFES and IRI support to the SOC, SIG and HNC / SNC emphasized the importance of connecting with citizens inside Syria for the legitimacy and health of the transition process. Yet, linkages created through the program between these bodies and the beneficiaries participating in PRIDE II activities inside Syria were few. Coordination and communication between core PRIDE II implementers was not regular or routine, and when it occurred, it tended to be ad hoc and as a result or personal initiative. Better coordination could pave the way for more interaction between programs. The Musharaka Forum is a good platform through which to connect citizens inside Syria with those in Turkey who seek ways to support local efforts. Likewise, the national-level connections established by IRI and IFES could be the channel through which to connect local citizens from both programs with national-level representatives. RECOMMENDATIONS Design more deliberately, activities across the three Syria PRIDE II program components to build on synergies and leverage complementary features of program interventions. Incorporate more clearly into the project theory of change, how activities undertaken at the local level will actually connect to efforts to support national-level structures and vice versa. Look to move from activities aimed at influencing individual-level change, to socio￾political change at the level of governance structures and formal or informal governance mechanisms that establish democratic practices as common practice. Build on the program successes with increasing citizen engagement with decision￾makers – locally and nationally – by replicating these experiences across different communities. Examples are civic education sessions, advocacy undertaken by NDI participants and IRI women beneficiaries, and connecting IFES work with the SOC and the HNC on the Liaison Committee with groups of citizens undertaking advocacy on relevant topics. 2 3 4 5 6 7 8 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 56 Utilize, and where possible, expand available virtual platforms to increase inter￾community engagement across the different parts of Syria that the project is working in, and with the Syrian community in southern Turkey, to enhance collective narrative of support for democratic change. Continue to routinize coordination and communication between USAID and PRIDE II. These should be more regular communication, and reporting should be clearer and easier to consume for relevant audiences. Better communication might also be established between CEPPS and USAID field based personnel. Discussions around improvement might entail protocols for requesting information from implementers. Coordination between CEPPS partners also could improve, so partners are routinely aware of activities, trainings and developments in Turkey and Syria. In one instance, both the IRI women and NDI beneficiaries were working in Atareb carrying out advocacy campaigns, and such efforts might have benefitted from coordination. IFES’ work with the HNC Liaison Committee is also an effort that IRI women and NDI beneficiaries could gain from, as these activities link citizens in Syria and Turkey with an opportunity to have voice at the negotiation table. One solution might be use of a contract vehicle as opposed to a cooperative agreement, which provide USAID with greater ability to enforce different partnership arrangements between implementers on the ground. Partners should also work towards synthesizing reporting on program activities in a manner that is clear and easily accessible to readers. 4.5 ADDITIONAL EVALUATION QUESTIONS Here, the evaluation team addresses additional questions that were not among the core queries requested, but are among the larger set of evaluation questions proposed. The additional questions speak to the core questions, and inform Syria PRIDE II program dynamics. How has the conflict overall affected accomplishment of results and outcomes? What outcomes and results have not been achieved? How has the conflict affected the ability to anticipate outcomes? The operational environment of the PRIDE II program is characterized by political instability and extremely volatile security conditions. Local conditions are shaped by whatever constellation of political and security actors are in control at a given point in time. CEPPS implementers and USAID personnel describe a very fluid context that varies dramatically across program locales. Thus, the challenges to consistent and effective implementation of program activities in a manner that matches stated program goals are significant. The conflict-affected program strategies for most PRIDE programming, but the project accomplished important outcomes under tremendous difficulty. According to interviews with CEPPS personnel, programming must consider threats to beneficiaries’ safety emanating from a myriad of actors, including militant Islamists and political forces that limit the political and social space for beneficiaries in both Syria and Turkey. The evaluation finds that while conditions caused by the conflict impacted implementation strategies and the nature of interventions, the PRIDE II project nevertheless managed to accomplish key outcomes, though 9 10 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 57 not without complications that have some bearing on results. Interviews with CEPPS personnel identify several impacts on programming and activities. NDI The presence of a myriad of actors, such as the Islamist Al-Qaeda-based Hayat Tahrir al-Sham (HTS), the Kurdish Democratic Union Party (PYD), and Turkish forces impacted activities at NDI program sites in Syria, including Aleppo, Idlib, Rif Damascus, Hasakah, Raqqa, and Jarablus. The program environment is rife with shelling and threats to the safety of beneficiaries and program staff. NDI personnel explain that, in addition to the problem of physical safety, these actors are generally hostile towards democracy. As a result, beneficiaries feared gathering for activity sessions and discussing civic education topics. “Besides border closings, we have all the political challenges. In all of these areas, no one likes the concept of ‘democracy’. …We can’t do activities when there is shelling. People were afraid to gather and talk about democracy.” (Staff Member, NDI Program, KII) To overcome these obstacles to program implementation, program personnel on the ground either rescheduled civic education sessions, or moved them to private venues, such as homes. Personnel also reduced meetings so they were smaller in size to minimize scrutiny. Flexible responses to the conflict creates an environment of trust. Program facilitators in Syria also adjusted to beneficiaries’ concerns by slowly establishing relationships of trust, and allowing NDI program participants to discuss any topic of their choosing in sessions, which may, or may not, include democratic concepts. Over time, respondents became more secure, and, in turn, more comfortable broaching civic topics with others in their communities. This has the combined effect of engendering trust, but more pragmatically, avoids, if necessary, alarming political actors. Specifically, while it may not be possible to discuss “elections” in some areas, there may be an opportunity to discuss “women’s rights” and “rule of law.” The dissemination of program information from trusted sources through word-of-mouth additionally generated interest and, therefore, expansion of program participation. The unpredictable nature of the conflict also affected the ability to anticipate program outcomes. In interviews, NDI staff believe that it is easy to alter session topics and assess issues beneficiaries may want to learn about. However, in the absence of concrete knowledge of how the political transition in Syria will evolve, it is difficult to know definitively how the rapidly changing conflict will ultimately impact outcomes. One NDI staff member considered how political shifts could influence the disposition of community members and possibly their willingness to be involved in democracy and governance projects: “If the government comes [in Rif Damascus] and says, ‘We’re here!’ will people be loyal to the government?” (Staff Member, NDI Program). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 58 Gains from advocacy would be more robust if the conflict were not so severe. If it were not for the conflict, the activity’s advocacy campaigns would likely be more effective. NDI program staff believe that participants would have access to more funds from local sources to support their advocacy projects. Given the collapsed Syrian economy, however, local actors have minimal resources to contribute to beneficiary-led activities. Meetings moved to private settings. Due to security reasons, NDI staff changed the “public” nature of meetings, moving interactions between decision makers and citizens to private settings, such as homes, in order to avoid scrutiny and to keep beneficiaries safe from harm. IRI Through Syria PRIDE II, IRI conducts the IRI women’s program among Syrian women and activities among members of the formal Syrian opposition. Like NDI, IRI’s programming has been influenced by Syria’s conflict dynamics, but the challenges are particularly exacerbated due to the Turkey-Syria border closing. The ever-changing dynamics of the conflict required a shift in activity location. While IRI’s women’s group activities are implemented now from within Syria as well as inside Turkey, IRI’s work with the formal opposition currently takes place primarily from Turkey (Gaziantep and Istanbul), though partners reside in exile elsewhere, in Europe and Tunisia. Amid the conflict, transfer of leadership skills to IRI women beneficiaries was successful. Despite the restrictions on the IRI women’s program activities imposed by the conflict, evaluation results find that the IRI program transferred critical leadership skills to beneficiaries that can be applied to various political processes and activities. Although the locus of program outcomes appears to be most visible at the local-level in Syria, the conflict did not entirely stymie women beneficiaries’ ability to connect to the national-level political dynamics, in results consistent with Syria PRIDE II goals. Women’s consultations and engagements with the national-level de Mistura’s office, the HNC, SOC and key interactions with the SIG, are aforementioned examples. IRI women beneficiaries are confident in their newly acquired leadership competencies, despite the conflict. Indeed, the small survey among a select number of IRI women program participants suggests improved confidence in engaging in collective action and community advocacy since involvement in PRIDE II (see Table 15). In a finding that suggests that some Syrian women may still be apprehensive in the current environment, a somewhat smaller number (62%) say that they now feel more comfortable voicing their opinions in public.30 This, however, had little impact on the prevailing sentiment among IRI survey IRI women respondents that they are able to conduct advocacy for their community and are successful in influencing local decision-makers. 30 The very small of women surveyed precludes generalizing to the entire beneficiary population. However, the smaller percentage on this measure is likely due to the nature of the IRI sample itself. Some of the participants are former political prisoners in Syria, and / or their husbands were political prisoners. There may be some concern regarding public speaking for fear of retribution. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 59 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 60 Table 15: IRI Women’s Program Views (Syria) As a result of my / our participation in the IRI Women’s program : Strongly Agree / Agree Disagree / Strongly Disagree I feel more confident leading others in collective action. 100% (13) 0 I have a better understanding of democratic words, concepts and practices. 100% (13) 0 I feel more capable to advocate on behalf of my community. 100% (13) 0 I feel more confident voicing my opinions publicly. 62% (8) 31% (4) We / our group was able to conduct advocacy activities in our community. 100% (13) 0 We / our group’s advocacy efforts have succeeded in influencing local decision-makers. 100% (13) 0 N=13 The conflict has complicated IRI work with formal Syrian opposition groups. IRI also works with formal opposition groups, the SOC, the HNC, and the SIG from Gaziantep and Istanbul. IRI staff explained that the conflict forced Syrian opposition partners aground and they sought exile in Turkey, Europe, Saudi Arabia, Tunisia and elsewhere. The challenge for the IRI capacity building endeavor lies therefore with working with leaders whose credibility is eroded among the general Syrian population since their departure from Syria. To adjust to the effects of partner distance, IRI PRIDE II personnel explained they assist Syrians with the creation of networks around key issues related to political processes in Syria. Limited funding, however, prevents use of interventions that might be broader in scope and longer term. The IRI program must also contend with circumstances more specific to the formal opposition movement itself. Namely, infighting among opposition group members complicates consensus￾building and cooperation. Furthermore, traditional attitudes towards women are pervasive among opposition group members and remain an obstacle for the inclusion of women in formal opposition organization activities. Longer term strategies are needed in dealing with exile communities. Regarding the issue of how to leverage exile communities, IRI managers assert the importance of working more creatively with people outside of Syria and within the exile community. Women can play a vital role in this kind of activity. On the other hand, it is useful that donors have longer term strategies that consider the political, economic and social complexities that affect outcomes in ways that are difficult for implementers to control. IFES CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 61 IFES implements the CEC from Syria, while the Musharaka Forum is executed from Gaziantep, Turkey. IFES faces the same security constraints as IRI, and as a consequence, operates forum activities from Turkey. While the civil war presented challenges for directly affecting change among beneficiaries inside Syria – as travel to and from the country is prohibited – the Musharaka Forum produced, as previously mentioned, notable individual-level outcomes in increasing efficacy and confidence among Gaziantep forum participants, particularly among women and youth, a critical demographic in Syria and the Middle East region as a whole. The conflict also influences IFES opposition activities... Most opposition partners are in exile and, the frequently changing cast of personalities within the formal opposition’s leadership affected IFES decisions regarding who is most relevant to engage. As a consequence, IFES must constantly rebuild relationships with new individuals and engage on electoral issues in negotiations that are already slow or stalled. As Russians are more involved in the Syrian conflict, opposition priorities are changed. At the time of the evaluation fieldwork, IFES program staff say that the new set of Syrian negotiators seems to have stabilized somewhat, but the situation is still unpredictable. According to program staff, IFES continues to engage with opposition groups and other actors in an attempt to craft new relationships and forms of engagement. Ultimately, the conflict prevented the establishment of systems and regulations under IFES’ work with the formal opposition on local elections. IFES phased out these activities, in response to the declining credibility and collapse of the SIG, and non-existent engagement between the opposition and regime. IFES administrators acknowledged that the ability to anticipate outcomes in the current environment has been elusive. “There have been so many shifts that we are forced into a reactive pattern. We have no idea whether the SOC will be in existence or what will be on the [negotiating] table. We have been in this pattern for the past year-and-a-half.” (Staff Member, CEC, KII) …and Musharaka Forum leadership opportunities. As another by-product of opposition partner disorganization, IFES was challenged with creating leadership opportunities through the Musharaka Forum, because the opposition is reluctant to engage in Gaziantep. Still, IFES cultivated youth leadership among young forum participants through its Building Leadership, program, in addition to supporting forum-member engagement opportunities through the HNC Liaison Committee. In order to adapt to this constraints around Syria’s weak opposition actors, IFES is currently considering sending a representative to Geneva, conducting a capstone project, or having panel discussions on political transitions. Although opposition partners retained core democratic concepts under PRIDE II, it is difficult to translate gains into a coherent platform due to in-fighting among divergent personalities and organizational malaise. Washington, DC-based IFES staff explain it is also difficult getting individual beneficiary partners themselves to consider next steps amid such political uncertainty. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 62 IFES has shifted to working with opposition individuals. To mitigate the effects of in-fighting, IFES has shifted to working with individuals rather than groups. However, this also requires building new relationships. For all PRIDE II beneficiaries, the war has inadvertently produced some opening for liberalization. As disruptive and destabilizing as the civil war has been for the Syria, the conflict also appears to have produced opportunity, in the view of beneficiaries. Specifically, 91 percent agree if the revolution never happened, they would not even be participating in activities like those of Syria PRIDE II, with the same percentage noting they would not be involved in activities that aim to improve democratic practices in Syria (91%). There is also the sense among beneficiaries that even more could be accomplished under PRIDE II, but not for the conflict and its complexities. Between 80 and 90 percent believe that, if the security situation was improved in Syria, even more progress could be achieved around involvement of women, community mobilization and other activities (see Table 16). Table 16: Impact of the Syria Security Situation on Program Implementation (Syria) If the security situation in Syria had not been so bad: More women could have been involved in activities 93% (129) I / we would have been better able to mobilize our community 91% (126) I / we would have been able to implement more advocacy activities 85% (118) I / we would have been able to address more topics around democracy and governance 87% (120) Decision-makers would be more responsive 83% (115) N=139 5.0 SUGGESTIONS FOR FUTURE USAID PROGRAMMING PRIDE II project interventions have contributed to increased political awareness and civic engagement among beneficiaries, including underrepresented Syrian population segments, women and youth. Though some achievements are small-scale, activities appear to be slowly chipping away at traditional values and roles of women and youth beneficiaries. Given the widespread interest and retention of democratic concepts and practices among beneficiaries, future PRIDE II activities should continue interventions that encourage awareness of democratic ideas. These seem to be among the more sustainable program outcomes, as respondents consistently indicate interest and willingness to disseminate knowledge around democratic ideas and use skills to shape political processes. Women beneficiaries, for their part, now recognize the added value they bring to political processes, and are learning how to empower themselves through education, awareness, CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 63 confidence-building and communication. Under PRIDE II, women have demonstrated increased leadership capacity and a desire to be involved in public life in and outside of Syria. Both within households, communities, local-level – and to smaller degree, national-level institutions, and women display the potential to become important sources of activism, information and learning, diplomacy, and change. Given the size of the female demographic, and the program progress made thus far, PRIDE II is a useful point of departure for expanded and deepened programming. Future programing should consider continuation of advocacy and leadership trainings, and link women PRIDE II beneficiaries inside and outside of Syria – perhaps across CEPPS programs – in order to share information, experiences, and establish relationships between groups in and outside of the country. This may additionally contribute over the long-term to greater community cohesion, which would benefit a future state- and nation-building process for Syria. PRIDE beneficiaries are now more comfortable engaging decision-makers that previously seemed unapproachable as a result of PRIDE II programing. However, local decision makers have been slow to adopt more open attitudes towards women. The program might work more closely with local decision makers in trainings on gender and the role of women in political processes. Participants in the three PRIDE II programs requested more sessions and trainings for longer durations of time. More trainings on effective women’s engagement with local decision makers, expanding and addressing more advanced topics on the constitution and political transition, were mentioned by beneficiaries. Equipped with basic knowledge and skills related to democratic processes and terminology, beneficiaries say they are ready for more and advanced work in this area. Sessions with experts in the field democracy and political processes and engaging with international organizations that can share their expertise in working with conflict￾affected areas are suggested ways forward. Also, connecting the participants with other women leaders either in Syria or internationally might be fruitful. In this vein, the program might consider highlighting the success of women leaders living in conflict areas. The Musharaka Forum appears to promote a sense of solidarity and efficacy among Syrian beneficiaries there. PRIDE activities should build on this positive effect by addressing how respondents might concretely influence events in a future Syria and under varying scenarios. As above, this might be also accomplished by trainings and workshops offered by experts, including academics as well as practitioners. When working to strengthen national level opposition bodies, ensure activities are directed at institutional advancement and not just at individual-level change in knowledge, attitude or skills such as those related to advocacy. This would be advanced by establishing and sustaining clear mechanisms for continuous interaction between national-level representatives and Syrian citizens abroad and in opposition-held areas in Syria. Once such mechanisms become routine and expected, continued activities with citizens will ensure pressure on national-level bodies to maintain these mechanisms regardless of changes in individuals in the SOC or the SIG. While these measures of support for the representative opposition bodies may be insufficient to establish them as strong actors in the transition process, they remain necessary nonetheless. If fact, the PRIDE project could do more to cross-fertilize project activities of implementer interventions. Several interventions occur in the same geographic areas, and activities are CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 64 concerned with similar issues. PRIDE partners could look to existing linkages and technologies to efficiently and effectively close the distance between beneficiaries in Turkey and Syria and within Syria. In recommendations related to PRIDE program management, continue regular, institutionalized coordination between USAID and CEPPS partners. This would keep USAID and PRIDE II partners informed of implementers’ respective activities, developments, and program challenges. Communication frequency is also critical, given the fluidity of the operational environment, and would facilitate more information-sharing, joint problem-solving, learning, and ultimately smooth program execution. As issues on the ground emerge in Syria, Turkey, and within the interagency agency, information enables effective responsiveness among USAID actors. Reporting requirements can be written into the award to routinize reporting. Coordination between PRIDE II partners needs improvement. Regular communication and coordination would promote information-sharing and perhaps opportunities to leverage synergies among interventions. Partners should also work towards synthesizing reporting on program activities in a manner that is clear and easily accessible to readers. Concerning information requests, USAID might consider a way to coordinate agency requests for information from implementers. The pace of project implementation and the operational environment do not always lend themselves to rapid response. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 0 ANNEX 1: DATA COLLECTION INSTRUMENTS Key Informant Interview Protocols USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Interview Guide: USAID Syria PRIDE Performance Evaluation NDI Partner Activity Hello, my name is __________________. I am working with the National Opinion Research Center, at the University of Chicago, which is a research institution based in the United States. We are conducting a study of the USAID Syria PRIDE program, which is carried out by the National Democratic Institute (NDI), the International Republican Institute (IRI), and the International Foundation for Electoral Systems (IFES). You, of course, have been involved in programming involving one of these organizations. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, since the information you provide will be used to improve the current and future programs like this. We will share your opinions only with NDI, IRI, IFES and USAID which also supports the program. However, your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please describe your role on the program. 2 Who are your main beneficiaries? • Probe: How are beneficiaries selected for the activity? 3 What would you say are the most significant challenges for your activity? 4 What challenges have been specifically associated with the conflict? • Probe: To what extent did the women participants feel safe to enough to engage in political deliberations, and consent to collecting information? • Probe: Was the political situation enabling to allow partners to safely and effectively conduct in-country democracy and governance programs? • Probe: Did the program facilitators freely conduct needs assessments in different provinces? Did the citizens feel comfortable enough to respond to questions about their needs? 5 What have been the most significant achievements and results thus far? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 1 • Probe: How effective is program implementation of the NDI program thus far? Please explain. Did the program expand to other areas in Syria? • Probe: To what extent did program activities help increase decision-makers willingness to meet with citizens to discuss issues of concern and to consider policy solutions? • Any others? Please provide examples? • Probe: What kinds of citizen-led advocacy efforts emerged among groups involved with NDI activities? • Probe: To what extent have these advocacy efforts been successful in achieving their intended results? • Probe: What enabling factors are associated with the emergence of citizen-led advocacy efforts among groups targeted by NDI activities? 6 Describe the experience of NDI and facilitators in implementing the Seed Funding projects. • Probe: What if any, was the impact of the Seed Funding project campaigns on the citizens in the select Syrian locations? • Probe: Have NDI groups been able to successfully develop campaigns addressing citizen’s needs through the Seed Funding program? 7 Over all, what is working especially well with your activity? Why? • Probe: In your view, what are the most important enabling factors that contribute to what is going well? • Probe: Are there any local actor engagements/relationships that are helpful? • Probe: Are there any relationships with partner organizations that have been beneficial? • Probe: Have there been any unanticipated outcomes as a result of the activity? 8 What is not working well? Why? • Probe: Are there any demographic groups that are underrepresented in the program activity? • Probe: Are there organization or beneficiary capacity weaknesses that affect results achievement? • Probe: Have there been any local actors that have obstructed activities? In what way? • Probe: How have you tried to address / correct what is not working well? 9 Imagine the program and your activities without the conflict. What kinds of gains would be achievable without the conflict? 10 As a partner organization, what are some capacity gaps that affect implementation? 11 And among the beneficiaries you engage, what are some of the capacity gaps that still need to be addressed – if there are any? • Probe: How might supporting organizations like USAID / NDI / IRI / IFES assist you in addressing those gaps? 12 In what ways have you had to shift strategies in response to the social, economic or political environment? 13 Describe to me ways in which supporting organizations have helped your organization be more adaptive. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 2 • Probe: And how well have beneficiaries adapted to changes in the environment? Can you provide examples? 14 How have you seen beneficiaries apply the skills they have acquired? • Probe: What skills or practices are most likely to endure when the program is over? For example, do you expect beneficiaries to be able to apply skills, engage, advocate, or disseminate knowledge in Syria or in Turkey in a way that is relevant? Anything else? • How might NDI beneficiaries apply their knowledge regarding basic democratic roles, practices, and concepts? • Probe: And as a partner organization, what capacities, skills, and practices are most likely to endure once the program is ended? 15 What are success stories? • Probe: Any failures? If so, how do you address those? • Probe: What factors have contributed to the failure or success of interventions? Thank you for taking the time to speak with me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 3 USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Interview Guide: USAID Syria PRIDE Performance Evaluation PRIDE NDI: Beneficiaries Hello, my name is __________________. I am working with the National Opinion Research Center (if asked where, at the University of Chicago). We are conducting a study of NDI/IRI/Musharaka Forum programs. You, of course, have been involved in this program. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, since the information you provide will be used to improve the current and future programs like this. Your responses are confidential and will be shared only with the project teams. Your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please tell me what activity you have been involved in / how you have been involved in [FACILITATOR] • Probe: Describe to me what you have been doing? • Probe: How did you learn about the program? • Probe: Why did you become involved? 2 What aspects of the program have you found most beneficial for you? [INTERVIEWER, ASK PROBES QUESTIONS ACCORDING TO THE ACTIVITY / PROGRAM AS APPROPRIATE] • Probe: To what extent are you more familiar with democratic practices, roles and responsibilities? Tell me more. • Probe: To what extent are you more informed about the transition and believe you have a voice? Can you give me details? • Probe: In what kinds of advocacy activities have you been involved? In ways have you developed enhanced advocacy skills? Please give me an example. • Probe: How has your capacity to demonstrate leadership changed? In what way? • Probe: Are there any aspects of the program that have been most beneficial to your community in Syria? Can you give me an example? 3 Thinking about the skills you now have since participating in [NAME OF ACTIVITY / PROGRAM] activities, how have you actually used those skills? • Probe: Please provide examples. • Probe: And how often would you say you have used the skills, all the time, sometimes, rarely? Can you give me an example? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 4 4 What are some of the challenges or obstacles you have faced participating in [NAME OF ACTIVITY / PROGRAM]? • Probe: Do you feel included in decision-making and key processes? Why or why not? • Probe: Are there any social gender barriers? Please explain. • Probe: Are there any barriers to youth? Please explain. • Probe: How have you had to adjust your participation and activities to the challenges or obstacles you face? 5 What is the biggest lesson you have learned that you did not know previously? • Probe: Thinking about your own skills and capacities, which need improvement at this point in time? • Probe: Is there anything you would like to learn more about? • Probe: Are there any new skills that you like to improve or have? • Probe: How can the program help you do that? 6 Over all, what do you like or value most about the program / trainings? 7 What do you like to least about the program / trainings? 8 What are some of the factors that contributed to your continued participation in the activity? • Probe: In deciding to become involved in the activity and remaining involved, how important was trust for you? • Probe: What factors made you trust the organization / people involved with the organization? 9 How do you think you will be able to apply the skills and concepts you gained through the civic education sessions and the campaigns in the future? What future scenario will allow you to use these skills? 10 In what ways, if any, could the program / activity improve? 11 Is there anything else you would like to add?] Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 5 USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Interview Guide: USAID Syria PRIDE Performance Evaluation IRI Partner Activity Hello, my name is __________________. I am working with the National Opinion Research Center, at the University of Chicago, which is a research institution based in the United States. We are conducting a study of the USAID Syria PRIDE program, which is carried out by the National Democratic Institute (NDI), the International Republican Institute (IRI), and the International Foundation for Electoral Systems (IFES). You, of course, have been involved in programming involving one of these organizations. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, because the information you provide will be used to improve the current and future programs like this. We will share your opinions only with NDI, IRI, IFES and USAID which also supports the program. However, your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please describe your role on the program. 2 Who are your main beneficiaries? • Probe: How are beneficiaries selected for the activity? 3 What challenges related to the conflict affected program activity implementation? 4 Aside from the conflict, what would you say are other significant challenges for your activity? • Probe: For example, are there any social, economic or political barriers that prevent women from developing the skills to become effective leaders? • Probe: Are there any other disincentives or challenges? Please explain. 5 What have been the most significant achievements and results thus far? • Probe: To what extent have interventions achieved goals of enhancing understanding of democratic roles and responsibilities among program participants? • Probe: Has knowledge of consensus building strategy improved? • Probe: Have civic activists been strengthened to address citizen’s need through program activity? Can you provide some examples? • Probe: To what extent did the self-efficacy training for women produce leaders who can effect policy discussion and change? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 6 • Probe: How effective was the pre / post-questionnaires in gauging information from participants who took part in self-efficacy supported trainings and programs? What lesson can be learned from this experience? • Probe: The activity developed a number projects to address the needs of the community. Can you list the name of the projects developed? • Probe: To what extent are you able to assess the women’s increase knowledge and skills in political leadership due to the program activity? Can you provide illustrative examples of improvement? • Probe: Has the Governance Exchange program been successful in connecting program participants directly to decision makers on both the local and national level so women receive the support necessary to succeed? • Probe: How engaged were the decision makers? What did their involvement consist of? • Probe: And how has the IRI Women’s Program enhanced women’s ability to address community needs? Can you offer examples? • Probe: Did the program teach women specific skills to ascend to leadership positions? 6 Over all, what is working especially well with your activity? Why? • Probe: In your view, what are the most important enabling factors that contribute to what is going well? • Probe: Are there any local actor engagements/relationships that are helpful? • Probe: Are there any relationships with partner organizations that have been beneficial? • Probe: Have there been any unanticipated outcomes as a result of the activity? 7 What is not working well? Why? • Probe: Are there any demographic groups that are underrepresented in the program activity? • Probe: Are there organization or beneficiary capacity weaknesses that affect results achievement? • Probe: Have there been any local actors that have obstructed activities? In what way? • Probe: How have you tried to address / correct what is not working well? 8 Imagine the program and your activities without the conflict. What kinds of gains would be achievable without the conflict? 9 As a partner organization, what are some capacity gaps that affect implementation? 10 And among the beneficiaries you engage, what are some of the capacity gaps that still need to be addressed – if there are any? • Probe: How might supporting organizations like USAID / NDI / IRI / IFES assist you in addressing those gaps? 11 In what ways have you had to shift strategies in response to the social, economic or political environment? 12 Please describe to me ways in which supporting organizations have helped your organization be more adaptive. • Probe: And how well have beneficiaries adapted to changes in the environment? Can you provide examples? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 7 13 How have you seen program beneficiaries apply the skills they have acquired? • Probe: What skills or practices are most likely to endure when the program is over? For example, do you expect beneficiaries to be able to apply skills, engage, advocate, or disseminate knowledge in Syria or in Turkey in a way that is relevant? Anything else? • Probe: How effective and relevant are activities and programs with achieving sustainability of the national network of Syrian women? • Probe: As a partner organization, what capacities, skills, and practices are most likely to endure once the program is ended? 14 What are success stories? • Probe: Any failures? If so, how do you address those? • Probe: What factors have contributed to the failure and success of interventions? 15 Is there anything else you would like to add? Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 8 USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Interview Guide: USAID Syria PRIDE Performance Evaluation IFES Musharaka Forum Partner Activity Hello, my name is __________________. I am working with the National Opinion Research Center, at the University of Chicago, which is a research institution based in the United States. We are conducting a study of the USAID Syria PRIDE program, which is carried out by the National Democratic Institute (NDI), the International Republican Institute (IRI), and the International Foundation for Electoral Systems (IFES). You, of course, have been involved in programming involving one of these organizations. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, because the information you provide will be used to improve the current and future programs like this. We will share your opinions only with NDI, IRI, IFES and USAID which also supports the program. However, your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please describe your role on the program. 2 Who are your main beneficiaries? • Probe: How are beneficiaries selected for the activity? 3 What would you say are the most significant challenges for your activity? 4 What challenges have been associated with the conflict? 5 What have been the most significant achievements and results thus far? • Probe: To what extent has the Musharaka Forum’s awareness and engagement platform been a useful tool to inform and educate the public on civic and political rights and responsibilities? • Probe: What impact if any has supported workshops, seminars and conferences had on women and youth involvement regarding local governance issues and community dialogue? • Probe: Are there opportunities for individuals living inside Syria to take part in Musharaka activities? If yes, list the opportunities. 6 To what extent are Musharaka forum goals being met over all? • Probe: To what extent do Musharaka forum participants have voice in transition? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 9 • Probe: To what extent do female Musharaka forum become more engaged in transitional processes? Do participants have a voice in transition? • Probe: To what extent are youth adopting skills in leadership? • Probe: Is there greater awareness of women’s rights? • Probe: Have there been any challenges to meeting goals? 7 What impact, if any, have the consensus-building forums implemented by CEPPS / IFES helped address key governance issues? 8 Over all, what is working especially well with your activity? Why? • Probe: In your view, what are the most important enabling factors that contribute to what is going well? • Probe: Are there any local actor engagements/relationships that are helpful? • Probe: Are there any relationships with partner organizations that have been beneficial? • Probe: Has trust been established? • Probe: Have there been any unanticipated outcomes as a result of the activity? 9 What is not working well? Why? • Probe: Are there any demographic groups that are underrepresented in the program? • Probe: Are there organization or beneficiary capacity weaknesses that affect results achievement? • Probe: Have there been any local actors that have obstructed activities? In what way? • Probe: How have you tried to address / correct what is not working well? 10 Imagine the program and your activities without the conflict. What kinds of gains would be achievable without the conflict? 11 As a partner organization, what are some capacity gaps that affect implementation? 12 And among the beneficiaries you engage, what are some of the capacity gaps that still need to be addressed – if there are any? • Probe: How might supporting organizations like USAID / NDI / IRI / IFES assist you in addressing those gaps? 13 In what ways have you had to shift strategies in response to the social, economic or political environment? 14 Describe to me ways in which supporting organizations have helped your organization be more adaptive. • Probe: And how well have beneficiaries adapted to changes in the environment? Can you provide examples? 15 How have you seen beneficiaries apply the skills they have acquired? • Probe: What skills or practices are most likely to endure when the program is over? For example, do you expect beneficiaries to be able to apply skills, engage, advocate, or disseminate knowledge in Syria or in Turkey in a way that is relevant? Anything else? • Probe: As a partner organization, what capacities, skills, and practices are most likely to endure once the program is ended? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 10 16 What are success stories? • Probe: Any failures? If so, how do you address those? • Probe: What factors have contributed to the failure or success of interventions? 17 Is there anything else you would like to add? Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 11 USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Interview Guide: USAID Syria PRIDE Performance Evaluation IFES Civic Education Partner Activity Hello, my name is __________________. I am working with the National Opinion Research Center, at the University of Chicago, which is a research institution based in the United States. We are conducting a study of the USAID Syria PRIDE program, which is carried out by the National Democratic Institute (NDI), the International Republican Institute (IRI), and the International Foundation for Electoral Systems (IFES). You, of course, have been involved in programming involving one of these organizations. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, because the information you provide will be used to improve the current and future programs like this. We will share your opinions only with NDI, IRI, IFES and USAID which also supports the program. However, your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please describe your role on the program. 2 Who are your main beneficiaries? • Probe: How are beneficiaries selected for the activity? 3 What would you say are the most significant challenges for your activity? 4 What challenges have been associated with the conflict? 5 What have been the most significant achievements and results thus far for the Civic Education Center? • Probe: To what extent were women effectively engaged in Civic Education Center (CEC) events? • Probe: Did the women demonstrate an increase in knowledge of consensus-building strategies due to the CEC events and activities? 6 To what extent have CEC activity goals being met over all? • Probe: Are there any unanticipated outcomes or results? 7 Over all, what is working especially well with the CEC activity? Why? • Probe: In your view, what are the most important enabling factors that contribute to what is going well? • Probe: Are there any local actor engagements/relationships that are helpful? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 12 • Probe: Are there any relationships with partner organizations that have been beneficial? 8 What is not working well? Why? • Probe: Are there any demographic groups that are underrepresented in the program? • Probe: Are there organization or beneficiary capacity weaknesses that affect results achievement? • Probe: Have there been any local actors that have obstructed activities? In what way? • Probe: How have you tried to address / correct what is not working well? 9 Imagine the program and your activities without the conflict. What kinds of gains would be achievable without the conflict? 10 As a partner organization, what are some capacity gaps that affect implementation? 11 And among the beneficiaries you engage, what are some of the capacity gaps that still need to be addressed – if there are any? • Probe: How might supporting organizations like USAID / NDI / IRI / IFES assist you in addressing those gaps? 12 In what ways have you had to shift strategies in response to the social, economic or political environment? 13 Describe to me ways in which supporting organizations have helped your organization be more adaptive. • Probe: And how well have beneficiaries adapted to changes in the environment? Can you provide examples? 14 How have you seen beneficiaries apply the skills they have acquired? • Probe: What skills or practices are most likely to endure when the program is over? For example, do you expect beneficiaries to be able to apply skills, engage, advocate, or disseminate knowledge in Syria or in Turkey in a way that is relevant? Anything else? • Probe: As a partner organization, what capacities, skills, and practices are most likely to endure once the program is ended? 15 What are success stories? • Probe: Any failures? If so, how do you address those? • Probe: What factors have contributed to the failure or success of interventions? 16 Is there anything you would like to add? Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 13 USAID-CEPPS Syria PRIDE II Performance Evaluation (January-April 2018) Formal Opposition Groups Hello, Sir / Madame, my name is __________________. I am working with the National Opinion Research Center, at the University of Chicago, which is a research institution based in the United States. We are conducting a study of the USAID Syria PRIDE program, which is carried out by the National Democratic Institute (NDI), the International Republican Institute (IRI), and the International Foundation for Electoral Systems (IFES). You, of course, have been engaged with one of these organizations. We would like to talk to you today about your experiences with the program. This should only take about one hour. Your feedback is very important, because the information you provide will be used to improve the current and future programs like this. We will share your opinions only with NDI, IRI, IFES and USAID which also supports the program. However, your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 In what ways has partner expertise been useful to your needs? • Probe: How and in what way? 2 How has your knowledge and capacities improved since your group’s engagement with the program? • Probe: Can you please provide specific examples of what you have accomplished as a result of participating in programs? 3 To what extent have you seen improved knowledge of democratic governance and women’s rights? 4 Are there any groups that you think are underrepresented in key opposition groups? • Probe: Which groups in particular? Are women, youth and ethnic minorities both adequately represented and involved? • Probe: What social, economic, or political barriers to including women and other groups? • Probe: Have you taken any extra steps to include other groups? What do those steps are efforts look like? 5 What obstacles, if any, may prevent your group from achieving its goals? • Probe: Capacity / skills / knowledge • Probe: Internal conflicts • Probe: The political environment • Probe: What else? • Probe: Not being able to operate in Syria. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 14 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 15 Annex 1: Data Collection Instruments Focus Group Discussion Protocols FOCUS GROUP DISCUSSION GUIDE – USAID-CEPPS Syria PRIDE II (January-April, 2018) Focus Group Discussion Guide: USAID Syria PRIDE Performance Evaluation PRIDE Musharaka Forum Beneficiaries Hello, my name is __________________. I am working with the National Opinion Research Center (if asked where, at the University of Chicago). We are conducting a study of NDI/IRI/Musharaka Forum. You, of course, have been involved in this program. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, since the information you provide will be used to improve the current and future programs like this. Your responses are confidential and will be shared only with the project teams. Your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please tell me what activity you have been involved in / how you have been involved in [NAME OF PROGRAM/ACTIVITY]? • Probe: Describe to me what you have been doing? • Probe: How did you learn about the program? • Probe: Why did you become involved? 2 What aspects of the program have you found most beneficial for you? [INTERVIEWER, ASK PROBES QUESTIONS ACCORDING TO THE ACTIVITY / PROGRAM AS APPROPRIATE] • Probe: To what extent are you more familiar with democratic practices, roles and responsibilities? Tell me more. • Probe: To what extent are you more informed about the transition and believe you have a voice? Can you give me details? • Probe: In what ways have you developed enhanced advocacy skills? Please give me an example. • Probe: Are any of you engaged in project development? If so, tell me about some of the projects you have been involved in designed. What did you learn and how what was the impact of the project? Did the project meet community needs? How? • Probe: How has your ability to demonstrate leadership changed? In what way? [MODERATOR, MAKE SURE YOU ASK FEMALES] CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 16 • Probe: Are there any aspects of the program have been most beneficial to your community in Syria and /or in Turkey? • Probe: [FOR WOMEN] Have you become part of a network of Syrian women? To what extent has this influenced your leadership and advocacy skills and belief that you can influence the transition? • Probe: What impact if any has supported workshops, seminars and conferences had on women and youth involvement regarding local governance issues and community dialogue? 3 Thinking about the skills you now have since participating in the Musharaka Forum, activities, how have you actually used those skills? • Probe: Please provide examples. • Probe: How often do you use the skills, all the time, sometimes, or rarely? Can you give me an example? 4 What are some of the challenges or obstacles you have faced participating in the Musharaka Forum? • Probe: Do you feel included in decision-making and key processes? Why or why not? • Probe: Are there any social gender barriers? Please explain. • Probe: Are there any barriers to youth? Please explain. • Probe: Do you feel that they can influence change in Syria? 5 Do you think you will use your skills, concepts and practices you have learned in a future Syria? 6 What is the biggest lesson you have learned that you did not know previously? • Probe: Thinking about your own skills and capacities, which need improvement at this point in time? • Probe: Is there anything you would like to learn more about? • Probe: Are there any new skills that you like to improve or have? • Probe: How can the program help you do that? 7 Over all, what do you like or value most about the program / trainings? 8 What do you like to least about the program / trainings? 9 What are some of the factors that contributed to your continued participation in the activity? • Probe: In deciding to become involved in the activity and remaining involved, how important was trust for you? • Probe: What factors made you trust the organization / people involved with the organization? 10 In what ways, if any, could the program / activity improve? 11 Is there anything you would like to share? Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 17 FOCUS GROUP DISCUSSION GUIDE – USAID-CEPPS Syria PRIDE II (January-April, 2018) Focus Group Discussion Guide: USAID Syria PRIDE Performance Evaluation PRIDE Beneficiaries – General Hello, my name is __________________. I am working with the National Opinion Research Center (if asked where, at the University of Chicago). We are conducting a study of NDI/IRI/Musharaka Forum programs. You, of course, have been involved in this program. We would like to talk to you today about your experiences with the program. This should only take about one hour. If we should need to contact you again for follow up interviews or research, may we contact you? Your feedback is very important, since the information you provide will be used to improve the current and future programs like this. Your responses are confidential and will be shared only with the project teams. Your name and identity will always be kept confidential and will not be shared with anyone. We are recording and taking notes of this discussion so that we can accurately record and summarize what you are saying. This is for purposes of analysis only. There are no “right” or “wrong” answers. Your participation in this interview is completely voluntary, and you can choose to stop participating at any time and leave without penalty. May I begin? 1 Please tell me what activity you have been involved in / how you have been involved in [NAME OF PROGRAM/ACTIVITY]? • Probe: Describe to me what you have been doing? • Probe: How did you learn about the program? • Probe: Why did you become involved? 2 What aspects of the program have you found most beneficial for you? [INTERVIEWER, ASK PROBES QUESTIONS ACCORDING TO THE ACTIVITY / PROGRAM AS APPROPRIATE] • Probe: To what extent are you more familiar with democratic practices, roles and responsibilities? Tell me more. • Probe: To what extent are you more informed about the transition and believe you have a voice? Can you give me details? • Probe: In what ways have you developed enhanced advocacy skills? Please give me an example. • Probe: If you are engaged in project development, tell me about some of the projects you have been involved in designed. What did you learn and how what was the impact of the project? Did the project meet community needs? How? • Probe: How has your capacity to demonstrate leadership changed? In what way? • Probe: Are there any aspects of the program have been most beneficial to your community (in Syria and /or in Turkey)? • Probe: [FOR WOMEN] Have you become part of a network of Syrian women? To what extent has this influenced your leadership and advocacy skills and belief that you can influence the transition? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 18 3 Thinking about the skills you now have since participating in [NAME OF ACTIVITY / PROGRAM] activities, how have you actually used those skills? • Probe: Please provide examples. • Probe: And how often would you say you have used the skills, all the time, sometimes, rarely? Can you give me an example? 4 What are some of the challenges or obstacles you have faced participating in [NAME OF ACTIVITY / PROGRAM]? • Probe: Do you feel included in decision-making and key processes? Why or why not? • Probe: Are there any social gender barriers? Please explain. • Probe: Are there any barriers to youth? Please explain. 5 What is the biggest lesson you have learned that you did not know previously? • Probe: Thinking about your own skills and capacities, which need improvement at this point in time? • Probe: Is there anything you would like to learn more about? • Probe: Are there any new skills that you like to improve or have? • Probe: How can the program help you do that? 6 Over all, what do you like or value most about the program / trainings? 7 What do you like to least about the program / trainings? 8 What are some of the factors that contributed to your continued participation in the activity? • Probe: In deciding to become involved in the activity and remaining involved, how important was trust for you? • Probe: What factors made you trust the organization / people involved with the organization? 9 In what ways, if any, could the program / activity improve? Thank you for taking the time to talk to me today. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 19 Annex 1: Data Collection Instruments Beneficiary Survey Hello, my name is [INTERVIEWER’S NAME]. I am a consultant working with NORC at the University of Chicago (NORC). NORC is a research institution based in the Chicago and conducts various kinds of studies around the world. We are conducting a study of some of activities you have been involved in lately. We would like to talk to you today about your experiences with the program. This is a short survey and should not take more than 30 minutes. Your feedback is very important, since the information you provide will be used to improve the current and future programs like this. We will treat all information you provide as confidential. I have signed a pledge to keep all information I collect from you secure. Only the evaluation team will have access to the study data. The information you provide will be used in summary form only with responses we receive from others answering the survey; we will never use your name when we report results of the survey. Your participation in this survey is voluntary – it is your choice to take this survey or not. If you feel uneasy with any of the questions, you may skip those questions without answering them. You may also stop the interview at any time. If you decide not to take part or to stop the interview, you will not lose any services that you are otherwise receiving. If you have any questions about this survey, you may email the [POSITION], [NAME], at [EMAIL OR PHONE CONTACT]. May I begin? 1 Yes 2 No 97 RESPONDENT NOT AT HOME OR RESPONDENT NOT ANSWERING PHONE CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 20 INTRODUCTION AND FILTERING Thank you. I would like to start by collecting some very basic information about yourself. Remember, the information you provide will be used in summary form only with responses we receive from others answering the survey and we will never use your name when we report results of the survey. [Age] What is your age? 1 ________________________________ 98 DON’T KNOW [DO NOT READ TO RESPONDENT] 99 REFUSED [DO NOT READ TO RESPONDENT] [Gender] What is your gender? 1 Male 2 Female 99 REFUSED [DO NOT READ TO RESPONDENT] [Resident] What is your city or town of residence? 1 ________________________________ 98 DON’T KNOW [DO NOT READ TO RESPONDENT] 99 REFUSED [DO NOT READ TO RESPONDENT] [Q1] First, I would like to confirm the program you are involved with. Please confirm by just answering yes or no. I understand you are with: 1 NDI Coordinator Name [GO TO THE NDI MODULE] 2 Musharaka Forum or Baytna Syria [GO TO THE IFES MODULE] 3 The IRI Women’s Program [GO TO THE IRI MODULE] CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 21 NDI MODULE [IF ANSWERED “1” TO Q1, ASK QUESTIONS Q2-Q12] [Q2] I would like to start by reading you a number of statements about your experiences with the project you are involved in with your coordinator. As I read each statement, please tell me the extent to which you agree or disagree: CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 22 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree 98 Do not know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] a. NDI activities have made me more familiar with democratic roles and responsibilities. b. NDI activities have helped increase decision￾maker’s willingness to meet with citizens to discuss issues of concern and to consider policy solutions. c. The activities have enhanced my understanding of democratic words, practices and concepts. d. I was part of a NDI group who successfully developed campaigns to address citizen’s needs. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 23 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree 98 Do not know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] a. NDI activities have made me more familiar with democratic roles and responsibilities. b. NDI activities have helped increase decision￾maker’s willingness to meet with citizens to discuss issues of concern and to consider policy solutions. c. The activities have enhanced my understanding of democratic words, practices and concepts. e. I have the capacity and skills to demonstrate leadership. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 24 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree 98 Do not know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] a. NDI activities have made me more familiar with democratic roles and responsibilities. b. NDI activities have helped increase decision￾maker’s willingness to meet with citizens to discuss issues of concern and to consider policy solutions. c. The activities have enhanced my understanding of democratic words, practices and concepts. f. I am engaged in activities, such as advocacy efforts, with the local councils. And now some questions about a slightly different topic. [Q3] Thinking about a future transition in Syria, how likely is it that you will use the skills and knowledge you have acquired since your participation in the program in a future transition scenario in Syria? READ OUT RESPONSE OPTIONS CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 25 1 Highly unlikely 2 Fairly unlikely 3 Fairly likely 4 Highly likely 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q4] Considering your involvement in the current program: 4 Very Important 3 Fairly Important 2 Fairly Unimporta nt 1 Very Unimporta nt 97 Does not apply to me [DO NOT READ TO RESPOND ENT] 98 Don’t know [DO NOT READ TO RESPOND ENT] 99 Refused [DO NO READ T RESPON ENT] How important is trust in the activity staff you are working with for influencing your decision to participate in the program / activity? How important is trust in the activity staff you are working with for influencing your decision to continue to be involved in the program / activity? [Q5] How much confidence do you now have in your ability to promote change in your community since participating in NDI program activities? 1 No confidence at all CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 26 2 Only a little confidence 3 A fair amount of confidence 4 A great deal of confidence 5 No opinion [DO NOT READ TO RESPONDENT] 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q6] How much influence do you think you can have on shaping future events in Syria? 1 No influence at all 2 Only a little influence 3 A fair amount of influence 4 A great deal of influence 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q7] I will now read to you some statements regarding the impact of the conflict on the activity you are involved in. As I read each statement, please tell me the extent to which you agree or disagree. 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the security situation had not been so bad, more women could have been more involved. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 27 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] b. If the security situation had not been so bad, I/we would have been able to better mobilize our community . c. If the security situation had not been so bad, I/we would have been able to implement more advocacy activities. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 28 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] d. If the security situation had not been so bad, I/we would have been able to address more political topics around democracy and governance in trainings and activities. e. If the security situation had not been so bad, decision￾makers would be more responsive [Q8] I will now read you some statements regarding your participation in and the level of activity of the organization you are involved with. As I read each statement, please tell me the extent to which you agree or disagree: CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 29 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the revolution never happened, I/we would not be participatin g in this activity. b. If the revolution never happened, I/we would have undertaken the same level of activity as now. c. If the revolution never happened, I/we would have been able to do more activities that aim to improve democratic practices in Syria. [Q9] We would now like to ask you how well activities are working overall. As I name the activity, please tell me the extent to which you think the program is working well or working poorly. If you are not involved in the activity I name, please say “does not apply to me.” CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 30 4 Works Very Well 3 Works Well 2 Works Poorly 1 Works Very Poorly 97 Does not apply to me [DO NOT READ TO RESPON DENT] 98 Do not know [DO NOT READ TO RESPON DENT] 99 Refused [DO NOT READ TO RESPON DENT] a. Political empower ment / awareness trainings b. Advocacy campaign planning. c. Implemen tation of advocacy campaigns and activities. d. Reaching men in your communit y inside or outside of Syria (for example, Turkey). e. Reaching more women in your communit y inside or outside of Syria (for example, Turkey). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 31 4 Works Very Well 3 Works Well 2 Works Poorly 1 Works Very Poorly 97 Does not apply to me [DO NOT READ TO RESPON DENT] 98 Do not know [DO NOT READ TO RESPON DENT] 99 Refused [DO NOT READ TO RESPON DENT] f. Reaching youth in your communit y inside or outside of Syria (for example, Turkey). g. Reaching others in your communit y like the Kurds or internally displaced inside or outside of Syria (for example, Turkey) CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 32 4 Works Very Well 3 Works Well 2 Works Poorly 1 Works Very Poorly 97 Does not apply to me [DO NOT READ TO RESPON DENT] 98 Do not know [DO NOT READ TO RESPON DENT] 99 Refused [DO NOT READ TO RESPON DENT] h. Increasing opportuni ties for women to express themselve s in communit ies inside or outside of Syria (for example, Turkey). i. Influencing local decision￾makers from Syria to the extent desired. j. Changing local decision￾makers’ attitudes towards women’s participati on in public and political life. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 33 4 Works Very Well 3 Works Well 2 Works Poorly 1 Works Very Poorly 97 Does not apply to me [DO NOT READ TO RESPON DENT] 98 Do not know [DO NOT READ TO RESPON DENT] 99 Refused [DO NOT READ TO RESPON DENT] k. Encouragi ng local decision￾makers response to local demands. l. Influencing national Syria decision￾makers. Thank you. I just have one last set of questions, we are almost done: [Q10] Were you involved in any other programs that were not sponsored by NDI? 1. Yes 2. No GO TO “CONCLUSION” 98. Don’t know [DO NOT READ TO RESPONDENT] 99. Refused [DO NOT READ TO RESPONDENT] [Q11] Can you tell me the name of the organization(s) that offered the other program(s)? 1. _______________________________ [OPEN-ENDED RESPONSE] 2. _______________________________ [OPEN-ENDED RESPONSE] 3. _______________________________ [OPEN-ENDED RESPONSE] 98. Don’t know [DO NOT READ TO RESPONDENT] 99. Refused [DO NOT READ TO RESPONDENT] [Q12] Which of the statements best describes your viewpoint regarding other programs in which you have been involved: 1 The other activities complement the program I am involved in. 2 They are basically the same 3 They do not complement each other at all 98 Don’t know [DO NOT READ TO RESPONDENT] CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 34 99 Refused [DO NOT READ TO RESPONDENT] Conclusion: Thank you, very much for taking the time to talk to me today. Your views are very important and helpful. IFES MODULE [IF ANSWERED “2” TO Q1, ASK Q13-Q25] [Q13] I would like to start by reading you a number of statements about your experiences with the project. As I read each statement, please tell me the extent to which you agree or disagree. 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. Musharaka Forum / Baytna Syria programming has informed me about the transition process. b. Musharaka Forum / Baytna Syria activities has made me more familiar with democratic roles and responsibilitie s. c. I believe I can have a say in whatever might happen in the transition of Syria. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 35 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] d. Musharaka Forum / Baytna Syria activities have enhanced my understanding of democratic words, practices and concepts. e. The Musharaka Forum / Baytna have helped develop and enhance my advocacy skills. f. I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to engage in advocacy related to Syria or to Syrians in Turkey in a meaningful way. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 36 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] g. I have applied the skills and practices gained from the Musharaka Forum program / Baytna Syria to spread knowledge and information about Syria or about Syrians in Turkey in a meaningful way. h. I have the capacity and skills to demonstrate leadership. i. I am engaged in program activities, such as advocacy with local￾level decision￾makers from Syria or local councils from Syria. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 37 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] j. I am engaged in activities that have involved national-level decision￾makers, such as the SIG and HNC. [Q14] How have Musharaka Forum events (for example, workshops, seminars and conferences) affected the involvement of women and youth in promoting community dialogue? Would you say that involvement among women and youth has: 5 Increased a lot 4 Increased a little 3 Stayed the same 2 Decreased a little, or 1 Decreased a lot? 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q15] How much impact have Musharaka Forum events had on involvement of women and youth in local governance issues in Syria? Would you say that involvement among women and youth has: 5 Increased a lot 4 Increased a little 3 Stayed the same 2 Decreased a little, 1 Decreased a lot? 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] And now some questions about a slightly different topic. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 38 [Q16] Thinking about a future transition in Syria, how likely is it that you will use the skills and knowledge you have acquired since your participation in the program in a future transition scenario in Syria? READ OUT RESPONSE OPTIONS 1 Highly unlikely 2 Fairly unlikely 3 Fairly likely 4 Highly likely 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q17] Considering your involvement in the current program: 4 Very Important 3 Fairly Important 2 Fairly Unimport ant 1 Very Unimport ant 97 Does not apply to me [DO NOT READ TO RESPON DENT] 98 Don’t know [DO NOT READ TO RESPON DENT] 99 Refused [DO NOT READ TO RESPON DENT] a. How important is trust in the activity staff you are working with for influencing your decision to participate in the program / activity? b. How important is trust in the activity staff you are working with for influencing your decision to continue to be involved in the program / activity? CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 39 [Q18] How much confidence do you have in your ability to promote change in your community since participating in the Musharaka Forum? 1 No confidence at all 2 Only a little confidence 3 A fair amount of confidence 4 A great deal of confidence 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q19] How much influence do you think you can have on shaping future events in Syria? 1 No influence at all 2 Only a little influence 3 A fair amount of influence 4 A great deal of influence 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q20] I will now read to you some statements regarding the impact of the conflict on the activity you are involved in. As I read each statement, please tell me the extent to which you agree or disagree. 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] If the security situation had not been so bad, more women could have been more involved. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 40 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the security situation had not been so bad, I/we would have been able to better mobilize our community . b. If the security situation had not been so bad, I/we would have been able to implement more advocacy activities. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 41 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] c. If the security situation had not been so bad, I/we would have been able to address more political topics around democracy and governance in trainings and activities. d. If the security situation had not been so bad, decision￾makers would be more responsive [Q21] I will now read you some statements regarding your participation in and the level of activity of the organization you are involved with. As I read each statement, please tell me the extent to which you agree or disagree: CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 42 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the revolution never happened, we would not be participatin g in this activity. b. If the revolution never happened, we would have undertaken the same level of activity as now. c. If the revolution never happened, we would have been able to do more activities that improve democratic practices in Syria. [Q22] We would now like to ask you how well activities are working over all. As I name the activity, please tell me the extent to which you think the program is working well or working poorly. If you are not involved in the activity I name, please say “does not apply to me.” CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 43 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] a. Political empowerme nt trainings b. Advocacy campaign planning. c. Implementati on of advocacy campaigns and activities. d. Reaching men in your community inside or outside of Syria (for example, Turkey). e. Reaching more women in your community inside or outside of Syria (for example, Turkey). f. Reaching youth in your community inside or outside of Syria (for example, Turkey). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 44 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] g. Reaching others in your community like the Kurds or internally displaced inside or outside of Syria (for example, Turkey) h. Increasing opportunities for women to express themselves in communities inside or outside of Syria (for example, Turkey). i. Influencing local decision￾makers from Syria to the extent desired. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 45 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] j. Changing local decision￾makers’ attitudes towards women’s participation in public and political life. k. Encouraging local decision￾makers response to local demands. l. Influencing national Syria decision￾makers. I just have one last set of questions, we are almost done: [Q23] Were you involved in any other programs that were not sponsored by IFES? 1 Yes 2 No [SKIP TO “CONCLUSION”] 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q24] Can you tell me the name of the organization(s) that offered the other program(s)? 1 _______________________________ [OPEN-ENDED RESPONSE] 2 _______________________________ [OPEN-ENDED RESPONSE] 3 _______________________________ [OPEN-ENDED RESPONSE] 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 46 [Q25] Which of the statements best describes your viewpoint regarding other programs in which you have been involved: 1 The other activities complement the program I am involved in. 2 They are basically the same 3 They do not complement each other at all 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] Conclusion: Thank you, very much for taking the time to talk to me today. Your views are very important and helpful. IRI MODULE [IF ANSWERED “3” TO Q1, ASK Q26-Q38] [Q26] I am going to read you some statements about your experiences with the IRI Women’s Group. As I read each statement, please tell me the extent to which you agree or disagree: 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. As a result of my participation in the IRI Women’s Group, I am more familiar with democratic roles and responsibilitie s. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 47 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] b. As a result of my participation in theIRI Women’s Group, I feel more confident leading others in collective action. c. As a result of my participation in theIRI Women’s Group, I have a better understanding of democratic words, concepts and practices. d. As a result of my participation in IRI’s program / , I feel more capable to advocate on behalf of my community. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 48 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] e. As a result of my participation in the IRI Women’s Group, I feel more confident voicing my opinions publicly. f. As a result of my participation in the IRI Women’s Group, I feel more informed about the transition process. g. As a result of my participation in theIRI Women’s Group, I believe I can have a voice in Syria’s future transition process. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 49 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] h. My belonging to a network of women across Syria gives me greater voice in the transition process. i. We / our group was able to conduct advocacy activities in our community. j. We / our group’s advocacy efforts have succeeded in influencing local decision￾makers. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 50 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] k. I believe I will be able to continue using the advocacy skills I gained in any transition scenario in Syria. Thank you. Now, let’s move to a different set of questions. [Q27] I would like to learn about your engagement with local decision-makers, and would like to read you some statements. As I read the statement, please tell me the extent to which you agree or disagree. 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree 98 Do not know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] a. I am now more capable of engaging or interacting with local council members. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 51 4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree 98 Do not know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] b. I am now more capable of engaging or interacting with local families / elders / local notables. c. I am now more capable of engaging or interacting with heads of local CSOs. d. I am now more capable of engaging or interacting with local political actors. e. I am now more capable of engaging or interacting with local business people. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 52 Now, some questions about other trainings you may have had. [Q28] Please tell me the extent to which you agree or disagree with each statement. 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. As a result of the political empowermen t trainings, my knowledge of consensus building strategies has improved. b. As a result of the political empowermen t trainings, I am better able to build consensus in my community. c. As a result of the political empowermen t trainings, my communicatio n skills have improved. And now some questions about a slightly different topic. [Q29] Thinking about a future transition in Syria, how likely is it that you will use the skills and knowledge you have acquired since your participation in the program in a future transition scenario in Syria? READ OUT RESPONSE OPTIONS 1 Highly unlikely 2 Fairly unlikely 3 Fairly likely 4 Highly likely CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 53 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q30] Considering your involvement in the current program: 4 Very Importan t 3 Fairly Importan t 2 Fairly Unimpor tant 1 Very Unimpor tant 97 Does not apply to me 98 Don’t know [DO NOT READ TO RESPON DENT] a. How important is trust in the activity staff you are working with for influencing your decision to participate in the program / activity? b. How important is trust in the activity staff you are working with for influencing your decision to continue to be involved in the program / activity? [Q31] How much confidence do you have in your ability to promote change in your community, since participating in the IRI Women’s Group ? 1 No confidence at all 2 Only a little confidence 3 A fair amount of confidence 4 A great deal of confidence 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q32] How much influence do you think you can have on shaping future events in Syria? 1 No influence at all CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 54 2 Only a little influence 3 A fair amount of influence 4 A great deal of influence 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q33] I will now read to you some statements regarding the impact of the conflict on the activity you are involved in. As I read each statement, please tell me the extent to which you agree or disagree. 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the security situation had not been so bad, more women could have been more involved. b. If the security situation had not been so bad, I/we would have been able to better mobilize our community . CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 55 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] c. If the security situation had not been so bad, I/we would have been able to implement more advocacy activities. d. If the security situation had not been so bad, I/we would have been able to address more political topics around democracy and governance in trainings and activities. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 56 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] e. If the security situation had not been so bad, decision￾makers would be more responsive [Q34] I will now read you some statements regarding your participation in and the level of activity of the organization you are involved with. As I read each statement, please tell me the extent to which you agree or disagree: 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] a. If the revolution never happened, I/we would not be participatin g in this activity. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 57 4 Strongl y Agree 3 Agre e 2 Disagre e 1 Strongly Disagre e 98 Do not know [DO NOT READ TO RESPONDENT ] 99 Refused [DO NOT READ TO RESPONDENT ] b. If the revolution never happened, I/we would have undertaken the same level of activity as now. c. If the revolution never happened, I/we would have been able to do more activities that improve democratic practices in Syria. [Q35] We would now like to ask you how well activities are working over all. As I name the activity, please tell me the extent to which you think the program is working well or working poorly. If you are not involved in the activity I name, please say “does not apply to me.” 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] a. Political empowerme nt trainings. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 58 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] b. Advocacy campaign planning. c. Implementati on of advocacy campaigns and activities. d. Reaching men in your community inside or outside of Syria (for example, Turkey). e. Reaching more women in your community inside or outside of Syria (for example, Turkey). f. Reaching youth in your community inside or outside of Syria (for example, Turkey). CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 59 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] g. Reaching others in your community like the Kurds or internally displaced inside or outside of Syria (for example, Turkey) h. Increasing opportunities for women to express themselves in communities inside or outside of Syria (for example, Turkey). i. Influencing local decision￾makers from Syria to the extent desired. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 60 4 Work s Very Well 3 Work s Well 2 Work s Poorl y 1 Work s Very Poorl y 97 Doe s not appl y to me 98 Do not know [DO NOT READ TO RESPONDEN T] 99 Refused [DO NOT READ TO RESPONDEN T] j. Changing local decision￾makers’ attitudes towards women’s participation in public and political life. k. Encouraging local decision￾makers response to local demands. l. Influencing national Syria decision￾makers. Thank you. I just have one last set of questions. We are almost done: [Q36] Were you involved in any other programs that were not sponsored by IRI? 1 Yes 2 No SKIP TO “CONCLUSION” 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] [Q37] Can you tell me the name of the organization(s) that offered the other program(s)? 1 _______________________________ [OPEN-ENDED RESPONSE] 2 _______________________________ [OPEN-ENDED RESPONSE] 3 _______________________________ [OPEN-ENDED RESPONSE] 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 61 [Q38] Which of the statements best describes your viewpoint regarding other programs in which you have been involved: 1 The other activities complement the program I am involved in. 2 They are basically the same 3 They do not complement each other at all 98 Don’t know [DO NOT READ TO RESPONDENT] 99 Refused [DO NOT READ TO RESPONDENT] Conclusion: Thank you, very much for taking the time to talk to me today. Your views are very important and helpful. ENUMERATOR OBSERVATIONS [Q39] Was the survey interrupted by an individual in a manner that influenced responses? 1 Yes 2 No [Q40] Was the survey interrupted by shelling? 1 Yes 2 No [Q41] Was the respondent display feelings of discomfort or distress during the survey? 1 Yes 2 No ENUMERATOR COMMENTS: PLEASE LEAVE ADDITIONAL COMMENTS AND REMARKS IN BOX BELOW CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 62 Annex 1: Data Collection Instruments Local Decision-Maker Survey USAID CEPPS SYRIA PRIDE II PERFORMANCE EVALUATION (Decision-makers Survey) April 2018 Hello, my name is __________________. I am working with the National Opinion Research Center (at the University of Chicago). We are conducting a study of some of activities that have been conducted in your area lately. We are speaking to you because, as a decision-maker in your area, you have witnessed or been involved in these activities. We would like to talk to you today about your experiences and opinions around these activities. This is a short survey and should not take more than 30 minutes. Your feedback is very important, since the information you provide will be used to improve the support such activities receive. Your responses are confidential and will be shared only with the project teams. Your name and identity will not be recorded, so your feedback will be anonymous. Your participation in this survey is completely voluntary, and you can choose to stop participating at any time without consequences to you or the groups or citizens undertaking these activities. May I begin? Q1: What kind of activity were you involved in or approached about by local citizens? a. Awareness raising campaign b. Service provision project (fixing roads, water installation, sewage system, etc.?) c. Social activity d. Other; please specify:________________________________________ Q2: What were you approached for? What did citizens ask of you? a. Provide funding or resources for the activity b. Grant them permission to carry out the activity c. Build support in the community for the activity d. Participate in the activity e. Other; please specify:________________________________________ Q3: Did you agree to their request? Yes No Q4: In your opinion, was the activity carried out successfully? 1 2 3 4 Activity did not take place Activity failed (did not achieve its goal) Activity achieved some of its goals Activity successfully achieved all its goals CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 63 Q5: Did you think the activity/campaign was relevant to the community’s needs or helpful in addressing them? 1 2 3 4 Not relevant or helpful at all Activity achieved some of its goals Extremely relevant and helpful Q6: Did the interaction with this group of citizens affect your understanding of the role of citizens in local? How? Q7: Did the interaction with this group of citizens affect your thoughts on citizen-local council/decision maker relationship? How? We are almost done. Thank you for your patience. Q8: After this experience, did your opinion on the importance of engaging citizens change? How? Q9: After this experience, did your engagement with citizens and citizen groups on various issues: 1 2 3 4 Change negatively Decrease in frequency and quality Stay more or less the same Improve (increase and change positively) Q10: Did you feel that your understanding of democratic words, practices and concepts changed as a result of this activity? Did it change favorably, remain the same, or change unfavorably? Q11: Did you feel that your community’s understanding of democratic words, practices and concepts improved as a result of this activity? Q12: Thinking about a future transition in Syria, how likely is it that the skills and knowledge these citizens acquired will be used in a future transition scenario in Syria? Is it highly likely, fairly likely, fairly unlikely, or highly unlikely? Q13: And how much influence do you think you can have on shaping future events in Syria – do you think you can have a lot of influence, some, or no influence at all? Thank you for your patience. We are in the final section. Q14: I would now like to ask you some questions regarding the impact of the conflict on the activity were approached about. As I read each statement, please tell me the extent to which you agree or disagree. 1 Strongly Agree 2 Agree 3 Disagree 4 Strongly Disagree 97 Do not know 98 Refused f. If the security situation had not been so bad, we would have been able to better serve CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 64 1 Strongly Agree 2 Agree 3 Disagree 4 Strongly Disagree 97 Do not know 98 Refused and respond to our community g. If the security situation had not been so bad, we would have been able to implement to engage in greater community outreach. h. If the security situation had not been so bad, we would have been able to discuss more political topics around democracy and governance in our communities. i. If the security situation had not been so bad, we would have the ability to be more responsive to citizen advocacy efforts. j. If the security situation had not been so bad, we would have been able to engage more women Q15: On somewhat related topic: 1 Strongly Agree 2 Agree 3 Disagree 4 Strongly Disagree 97 Do not know 98 Refused d. If the revolution never happened, we would not be supporting or participating in such activity. e. We would have supported the same type of activities as now. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 65 1 Strongly Agree 2 Agree 3 Disagree 4 Strongly Disagree 97 Do not know 98 Refused f. We would have been able to support more activities that improve democratic practices in Syria. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 66 ANNEX 2: SCOPE OF WORK Annex 2: Scope of Work USAID DRG-LER STATEMENT OF WORK Tasking Request N071: Syria PRIDE Performance Evaluation Date of Request: August, 2017, period of performance not to exceed six months. Type of Task: Performance Evaluation Services SOW: Introduction and Purpose: The Middle East Bureau seeks an impact evaluation to be conducted on the Promoting Inclusive Democratic Engagement (PRIDE II) program in order to inform future programming. We seek evaluators to use a mix of quantitative and qualitative data collection and analysis methods to generate answers to the evaluation questions listed below. Summary information on the program is below. Activity Background: The CEPPS partners’ theory of change for this program surmises that if Syrian democratic actors – especially women, youth, minorities, civil society actors and government officials – better understand their roles and responsibilities and have the capacity to constructively engage with each other and understand the incentives to create and work within an inclusive environment, then local, subnational and national structures will more effectively represent and respond to the priorities of their constituents, and citizens will be able to play more meaningful roles in governance processes. This theory of change assumes that relevant stakeholders will be willing and able to attend CEPPS partners’ workshops, and that the security situation will allow CEPPS partners to conduct in-country programming through their Syrian partners. Amid widespread political intransigence, the Syrian civil war continues to have a dire impact on an already enflamed humanitarian and security crisis. Progress remains hindered by the weakening of moderate armed groups at the hands of Islamic State of Iraq and al-Sham (ISIS) and its affiliates and Hayat Tahrir al-Sham (also known as Jabhat al-Nusra) as political negotiations show few signs of gaining traction. Syria’s governing structures at the local, sub￾national, and national levels alike have struggled to fill the vacuum left by the Assad regime. Elected officials and citizens lack the capacity to engage in democratic governance and understand their respective roles and responsibilities in the governance processes. Officials may not fully grasp the importance of inclusive government in the difficult context of conflict resolution, particularly towards traditionally marginalized groups such as women, youth and minorities. The impact of such deficiencies has led to a widespread lack of accountability, CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 67 responsiveness, and inclusivity on local, sub-national and national governance initiatives to identify and address community priorities. Similarly, the Syrian people’s experience with political and civic participation remains nascent, and emerging citizen groups are in need of continued technical assistance. Syrians lack faith in the efficacy of civic participation efforts and are unfamiliar with the processes of collaborating to identify and forge solutions to common concerns—both bedrock principles for successful local-level governance in the near and longer terms. Consequently, Syrian citizens must also build a fundamental knowledge base and skill-set of democratic practices that enable them to deliberate, negotiate, make informed decisions, and participate in democratic processes such as community organizing and advocacy at the local-level. Under this award, CEPPS partners have sought to increase the capacity of government and political leaders, civil society actors, women’s groups, and citizens at the grassroots level to support the eventual establishment of inclusive and effective democratic processes and institutions in Syria. CEPPS OBJECTIVE 1: Increase political and civic participation and representation of women youth, and minorities, to foster public and stakeholder confidence in peaceful and representative transitional political processes, and bolster opposition credibility.  IR 1.1 (IRI/IFES): Leadership and representation of women, youth and minority groups, on national and sub-national governing bodies and in policy discussions is increased.  IR 1.2 (IRI): Peaceful agreement on democratic reform is promoted through consensus building. CEPPS OBJECTIVE 2: Increase knowledge and understanding of democratic processes amongst the Syrian population including consensus building, coalition formation, citizen and stakeholder engagement, and elections, which will enhance an inclusive Syrian-led transition.  IR 2.1 (NDI/IFES): Public understanding of processes related to Syria’s political transition produce higher levels of civic engagement and participation, particularly among marginalized and underrepresented groups  IR 2.2 (IRI/IFES): Local councils, the Syrian Opposition Coalition (SOC), the Syrian Interim Government (SIG) CSOs, Syrian citizens, and international stakeholders are better informed about democratic governance and women’s rights  IR 2. (IRI/IFES): Local, sub-national, national representatives and their constituents better understand their democratic roles and responsibilities, and their role in peacebuilding.  IR 2.4 (IRI/NDI/IFES): National and sub-national representatives and their constituents openly and regularly engage each other on matters of planning, assistance and policy. CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 68 Activity Name USAID Office Implementer Life of Activity Active Geographic Regions PRIDE II START NDI June 2015- June 2018 Aleppo, Idlib, Latakkia, Hasakah, RIF Damascus, Raqqa Governorates PRIDE II START IFES June 2015- June 2018 Istanbul and Gaziantep, Turkey, Hasakah Governorate PRIDE II START IRI June 2015- June 2018 Turkey-wide and Syria-wide Evaluation Questions: What outcomes (anticipated and unanticipated) do staff at target CSOs perceive as resulting from NDI, IRI and IFES-supported activities? How many and what kinds of citizen-led advocacy efforts emerged among groups targeted by NDI’s activities? a. To what extent have these advocacy efforts been successful in achieving their intended results? b. What factors were associated with the emergence of citizen-led advocacy efforts among groups targeted by NDI activities? To what extent has the Civic Education Center model used in Qamishli and Hasakah been effective, in the context of shifting conflict dynamics? a. Which results, if any, have the Civic Education Centers continued to achieve through the changing conflict situation? b. Which results have been most affected by the conflict? What effect has the conflict had on the cost of implementing the Civic Education Centers? c. Considering the evidence for 2a and 2b, to what extent does the present CEC model continue to be viable in the current context? Are any elements of the model promising for continued implementation? Have any elements of the model proven to be ineffective in the current context? To what extent have approaches to increase engagement of women in political processes and negotiation succeeded across the three activities? a. Were approaches to increase women's engagement implemented as originally designed? b. Which challenges, if any, affected the implementation of these approaches? 4 3 2 1 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 69 c. Considering in particular how these approaches were implemented at local, national, and international levels, were any particularly successful in encouraging women's engagement in political processes? Dates of performance and timeline: Per item 1 under deliverables, USAID/ME expects the submitted proposal to include a proposed time line with the understanding that this evaluation must be fully completed within six months (December 2017). Deliverables: Evaluation Work plan: Within 2 weeks of the award of the contract, a draft work plan for the evaluation shall be completed by the lead evaluator and presented to the Middle East Bureau Syria Desk as well as the PRIDE II Agreement Officer’s Representative/Contracting Officer’s Representative (AOR/COR). The work plan will include: (1) the anticipated schedule and logistical arrangements; and (2) a list of the members of the evaluation team, delineated by roles and responsibilities. This time period should encompass preliminary document review. Evaluation Design: Within 2 weeks of approval of the work plan, the evaluation team must submit to the Agreement Officer’s Representative/Contracting Officer’s Representative (AOR/COR) an evaluation design (which will become an annex to the Evaluation report). The evaluation design will include: (1) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan, including references to specific questions or sections of draft tools; (2) draft questionnaires and other data collection instruments; (3) the list of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include calculations and a justification of sample size, plans as to how the sampling frame will be developed, and the sampling methodology); (4) known limitations to the evaluation design; (5) a dissemination plan; and (6) a conflict of interest mitigation plan if applicable. USAID offices and relevant stakeholders are asked to take up to 1 week to review and consolidate comments through the AOR/COR. Once the evaluation team receives the consolidated comments on the initial evaluation design and work plan, they are expected to return with a revised evaluation design and work plan within 1 week. In-briefing: Within 2 days of arrival in Istanbul, Turkey and/or Gaziantep, Turkey, the evaluation team will have an in-briefing with USAID for introductions and to discuss the team’s understanding of the assignment, initial assumptions, evaluation questions, methodology, and work plan, and/or to adjust the Statement of Work (SOW), if necessary. Weekly Briefings: During data collection, the evaluation team is expected to hold a weekly briefing, by phone, with USAID on the status of the evaluation, including potential challenges and emerging opportunities. The team will also provide the evaluation COR/manager with periodic briefings and feedback on the team’s findings, as agreed upon during the in-briefing. 1 2 3 4 5 CONTRACT No. GS-10F-0033M / TASK ORDER No. AID-OAA-M-13-00013 TASKING N075 PERFORMANCE EVALUATION OF THE USAID-CEPPS SYRIA PRIDE II PROJECT | 70 Presentation of preliminary results: The evaluation team is expected to hold a presentation in person (or by virtual conferencing software if necessary) to discuss the summary of the preliminary findings and recommendations to USAID and implementing partners, in advance of report drafting. This presentation will be scheduled as agreed upon during the in-briefing. Feedback and additional contextual information received during the presentation should be taken into account in the drafting of the evaluation report; this may include requests for additional analysis. Draft Evaluation Report: The report will address each of the questions identified in the SOW and any other issues the team considers to have a bearing on the objectives of the evaluation. The submission date for the draft evaluation report will be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, USAID will have five business days in which to review and comment on the initial draft, after which point the AOR/COR will submit the consolidated comments to the evaluation team. The evaluation team will then be asked to submit a revised final draft report five business days hence, and again USAID will review and send comments on this final draft report within five business days of its submission. Final Evaluation Report: The evaluation team will be asked to take no more than five business days to respond/incorporate the final comments from USAID. The evaluation team leader will then submit the final report to the AOR/COR. All project data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the intervention or evaluation. Budget: $300,000 of FY 2018 ESF. Submission Instructions: You will have seven (7) business days to submit a brief proposal and budget. Page Limit: 5 pages Budget: Excel 8 7 6