Al Sharaj School Rehabilitation Project YEMEN COMMUNITIES STRONGER TOGETHER (YCST) FINAL EVALUATION REPORT Final Report OCTOBER 14, 2022 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International (MSI), A Tetra Tech Company for the MENA MELS Activity. (THIS PAGE INTENTIONALLY LEFT BLANK) YEMEN COMMUNITIES STRONGER TOGETHER (YCST) FINAL EVALUATION REPORT Final Report Contracted under GS00F280GA, Task Order 7200AA19M00017 Middle East and North Africa Monitoring, Evaluation, and Learning Services Activity DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i | YCST FINAL EVALUATION REPORT USAID.GOV CONTENTS ACRONYMS...........................................................................................................III EXECUTIVE SUMMARY ........................................................................................1 INTRODUCTION..................................................................................................11 EVALUATION PURPOSE ....................................................................................................................................11 EVALUATION QUESTIONS...............................................................................................................................11 EVALUATION METHODS AND LIMITATIONS.............................................11 DATA COLLECTION METHODS....................................................................................................................12 DATA COLLECTION...........................................................................................................................................13 DATA ANALYSIS...................................................................................................................................................14 LIMITATIONS .........................................................................................................................................................14 YCST COMMUNITY CONTEXT........................................................................15 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS.........................17 EVALUATION QUESTION ONE: YCST’S CONTRIBUTIONS TO INTENDED OUTCOMES ......17 OBJECTIVE 1: ENGAGING COMMUNITIES ........................................................................................18 OBJECTIVE 2: BUILDING CAPACITY....................................................................................................22 OBJECTIVE 3: PROMOTING PEACE......................................................................................................22 OBJECTIVE 4: ENHANCING OUTCOMES ..........................................................................................25 EVALUATION QUESTION TWO: YCST’S DEVELOPMENT APPROACH..........................................27 OBJECTIVE 1: ENGAGING COMMUNITIES ........................................................................................27 OBJECTIVE 2: BUILDING CAPACITY....................................................................................................36 OBJECTIVE 3: PROMOTING PEACE......................................................................................................37 OBJECTIVE 4: ENHANCING OUTCOMES ..........................................................................................49 CROSS-CUTTING FINDINGS—SERVICE DELIVERY PROJECTS’ USE, ACCESS, AND INCLUSION CONSIDERATIONS AND ACHIEVEMENTS .......................................................................51 SERVICE DELIVERY PROJECTS—USE AND ACCESS .......................................................................51 YCST APPROACH AND OUTCOMES—INCLUSION CONSIDERATIONS AND ACHIEVEMENTS...........................................................................................................................................53 EVALUATION QUESTION THREE: EFFECTS OF CHALLENGES OR CONSTRAINTS FACED BY YCST..........................................................................................................................................................................58 COVID-19.......................................................................................................................................................58 CHANGES TO THE PROJECT SCOPE ..................................................................................................58 SECURITY CHALLENGES ..........................................................................................................................59 FINANCIAL CHALLENGES .......................................................................................................................59 STAFFING AND CAPACITY CHALLENGES........................................................................................59 CHALLENGES RELATED TO OBTAINING PERMISSIONS FROM AUTHORITIES ..................59 GENDER AND INCLUSION CHALLENGES........................................................................................60 INTERNAL CHALLENGES AND CONSTRAINTS..............................................................................60 EVALUATION QUESTION FOUR: LESSONS LEARNED FOR FUTURE INTERVENTIONS ..........61 YCST BEST PRACTICES TO BE CONSIDERED IN FUTURE PROGRAMMING........................61 LESSONS LEARNED ....................................................................................................................................62 CONCLUSIONS ....................................................................................................................................................66 RECOMMENDATIONS........................................................................................................................................66 USAID.GOV YCST FINAL EVALUATION REPORT | ii ANNEX 1: EVALUATION DESIGN ...................................................................69 ANNEX 2: LIST OF SECONDARY DATA SOURCES REVIEWED...............80 ANNEX 3: YCST COMMUNITIES SAMPLED ..................................................83 ANNEX 4: COMMUNITY SURVEY FINDINGS................................................85 ANNEX 5: CEC MEMBERS’ FINDINGS...........................................................103 ANNEX 6: EVALUATION INSTRUMENTS....................................................107 ANNEX 7: YEMEN POLLING CENTER COMMUNITY-LEVEL METHODOLOGY................................................................................................150 ANNEX 8: YPC COMMUNITY SURVEY CROSS TABS ...............................161 iii | YCST FINAL EVALUATION REPORT USAID.GOV ACRONYMS ACC Authority Community Committee CEC Community Enhancement Committee F2F Face-to-Face FMF Field Medical Foundation GC Global Communities HQ Headquarters IP Implementing Partner KII Key Informant Interview M&E Monitoring and Evaluation MEAL Monitoring, Evaluation, Accountability, and Learning MEL Monitoring, Evaluation, and Learning MELS Monitoring, Evaluation, and Learning Services Activity MENA Middle East and North Africa MoPIC Ministry of Planning and International Cooperation MSI Management Systems International NGO Nongovernmental Organization PACE Participatory Action for Community Enhancement PfG Partners for Good TOC Theory of Change TPM Third Party Monitoring USAID United States Agency for International Development YCST Yemen Communities Stronger Together YPC Yemen Polling Center USAID.GOV YCST FINAL EVALUATION REPORT | iv EVALUATION TEAM MEMBERS Ali Hayat Team Leader Claire Nowlin Evaluation Specialist Sandra Shaban Evaluation Specialist Sadeq Al-Nabhani Community Development/Resilience & Conflict Subject Matter Expert Ahlam Baqtian Local Evaluation Specialist Yemen Polling Center 1 | YCST FINAL EVALUATION REPORT USAID.GOV EXECUTIVE SUMMARY The United States Agency for International Development’s (USAID) Yemen office requested the services of the Middle East and North Africa (MENA) monitoring, evaluation, and services (MELS) contracting mechanism held by Management Systems International (MSI) to conduct a final evaluation for the Yemen Communities Stronger Together (YCST) program. The YCST project is a four-year (including a no-cost extension), USAID-funded, $22 million activity implemented by Global Communities (GC) together with Yemeni partner Field Medical Foundation (FMF) and specialized international partners Build Up and Partners for Good (PfG). The award is scheduled to end in September 2022. Since its inception in 2018, YCST has focused on supporting local-level initiatives that mitigate conflict, strengthen social cohesion, promote the peaceful resolution of differences, and, where feasible, capitalize on stabilization opportunities. It targets communities in the four governorates of Abyan, Aden, Lahj, and Taiz in southern Yemen. This evaluation answered the following four questions: 1. To what extent, and how, has the YCST award contributed to its intended outcomes, especially community and inter-community conflict mitigation and social cohesion, in participatory action for community enhancement (PACE) and stabilization intervention areas? 2. How, if at all, did YCST’s development approach contribute to, or detract from, the award’s effectiveness and potential for sustainable results? 3. What operational challenges or constraints did YCST face, and how, if at all, did these challenges affect outcomes and prospects for sustaining results? 4. As a result of this evaluation, what “lessons learned” or successful models should USAID consider for follow-on community-level interventions? EVALUATION DATA COLLECTION The team used both qualitative and quantitative methods as well as observations across a geographically dispersed mix of communities that included four stabilization and six PACE communities, including four Phase 1 and two Phase 2 communities. The evaluation team interviewed YCST staff, community members, formal and informal leaders, Community Enhancement Committee (CEC) members, and local leaders. The team conducted 510 interviews and 16 semi-structured direct observations at the facilities supported by the YCST project. The team faced challenges accessing two of the originally sampled 10 communities, one in Lahj and another in Abyan governorate due to security concerns. The site in Lahj was substituted with a similar site. The team was able to conduct interviews in the community in Abyan though no women participated in interviews. The sampling approach, while not representative in nature, ensured that the interviewers conduct interviews across the community (30 per community) and with male and female respondents. The final achieved sample also indicated that the sampling approach was able to capture key demographic groups. USAID.GOV YCST FINAL EVALUATION REPORT | 2 FINDINGS EVALUATION QUESTION ONE: YCST’S CONTRIBUTIONS TO INTENDED OUTCOMES Key findings are summarized by the objectives and outcomes below. OBJECTIVE 1 Outcome—Increase in social cohesion: According to key stakeholders, YCST activities improved relationships between community members and contributed to an increase in community members’ level of trust in local authorities (unlike under objective 3, an increased level of trust in local authorities was an unintended outcome of the PACE approach). Most community members, however, perceived no change in the level of trust between community members. Furthermore, close to a third of community members were dissatisfied with their local leaders and trusted them less than they did a year ago. Outcome—Mitigation of conflict resulting from service delivery activities: Most community members reported that access to basic services supported by YCST had either remained the same (39 percent) or improved (36 percent) over the last year. A plurality of community members were satisfied with the quality of YCST services, and close to a third were dissatisfied. Community members shared a mix of responses when asked if the overall peace and security situation and conflict mitigation mechanisms have improved. According to key stakeholders, by addressing critical service delivery gaps, YCST activities contributed to mitigating conflicts, though some key stakeholders also indicated that YCST activities had contributed to or created conflicts. Outcome—Enhanced capacity of the community to resolve/mitigate conflicts: Key stakeholders discussed how the mechanisms (e.g., CECs) set up by YCST enhanced the capacity of the community to resolve/mitigate conflicts. When it came to their ability to come together and resolve conflicts, however, the CEC members’ opinions were split. Most community members reported that their community is not able to address the key problems they face. OBJECTIVE 2 Outcome—Improved organizational capacity for conflict mitigation and resolution: Grantees appreciated the skills they had gained while working with YCST. Engagement in YCST allowed some grantees to work in more remote areas and on culturally sensitive issues. OBJECTIVE 3 Outcome—Enhanced capacity of local leaders to be responsive to community needs: Key stakeholders viewed YCST as offering an effective new conflict mitigation mechanism and improving the relationship between community and local authorities. A plurality of community members, however, were dissatisfied with the performance of their local leaders. Outcome—Mitigation of conflict resulting from service delivery activities: Key stakeholders mentioned that YCST activities had contributed to mitigating conflict. Access to YCST-supported basic services had also improved for a considerable number of community members. For a plurality of community members, however, the overall peace and security situation in their community had not changed. A quarter of the community members reported the presence of conflict impacting them, and a plurality of community members mentioned that local authorities, local community, and organizations had not played any role in addressing community conflicts. Most community members’ ability to 3 | YCST FINAL EVALUATION REPORT USAID.GOV participate in the political process had not changed. A considerable number of community members did not see an improvement in the conflict resolution mechanisms in their community. Most community members were of the view that their community is not able to address the problems they face. OBJECTIVE 4: ENHANCING OUTCOMES Outcome—Activity adaptations based on monitoring and evaluation (M&E) and contextual data: Key adaptations undertaken by YCST included procedural changes in how grantees were selected and engaged. The monitoring, evaluation, accountability, and learning (MEAL) team provided community feedback to other technical teams on how to improve the community engagement process including by implementing soft activities, which kept communities engaged with the YCST while they waited for the service delivery project. YCST continued to refine and adapt its own MEAL systems and products to address feedback from project stakeholders as well as the project’s knowledge gaps. YCST also improved its M&E systems considerably and transitioned from a paper-based system to electronic platforms. The project made a concerted effort to include women in the CECs (30 percent target of membership) and in all stages of the project (community engagement processes). This was more successful in some communities than others. CONCLUSIONS • Overall, stakeholders including local leaders were more aware than community members of YCST activities and held a more positive appraisal of YCST activities contributing to their intended outcomes, including an increase in community cohesion, mitigation of conflict resulting from service delivery projects, and enhanced capacity of community members to resolve/mitigate conflict. In contrast, a very small number of community members were aware of YCST activities beyond the service delivery projects. As per community members, the level of trust among the community members had not significantly changed. A considerable number of community members did not see an improvement in the conflict resolution mechanisms or the community’s ability to address its own problems. • Most community members did not view the YCST activities from the lens of conflict mitigation, community cohesion, and as enhancing the collective competence of the community to solve its own problems. Instead, they viewed these activities as improving access to services in their area. YCST’s community selection criteria also included selecting communities without conflicts. In such a context, while the service delivery activities were resolving community level stressors by supporting provision of essential services, it is not entirely likely that absent the YCST intervention, these service delivery needs would have led to significant violent conflicts within the community. This may also account for why only a small percentage of community members viewed the YCST intervention as a conflict mitigation activity. • For a three-year activity implemented during the peak of the COVID-19 pandemic and in a very challenging context, the YCST team accomplished significant results. YCST’s accomplishments included implementing 31 service delivery projects across 22 communities and four governorates. There was general acknowledgement by community leaders and members that the service delivery activities were relevant, enhanced access to essential services, and were of good quality. Grantees also appreciated the skills and experience they had gained while working with YCST. USAID.GOV YCST FINAL EVALUATION REPORT | 4 EVALUATION QUESTION TWO: YCST’S DEVELOPMENT APPROACH Findings related to how the activities were implemented and how they contributed to or detracted from the award’s effectiveness and the likelihood of sustainability of activities and outcomes are summarized by objective below. OBJECTIVE 1: ENGAGING COMMUNITIES According to most key stakeholders and community members who were aware of YCST, the YCST activity did not undermine or negatively affect the local authority structure in the community. Key stakeholders shared a mix of views on whether the YCST intervention contributed to creating issues or tensions between beneficiary and nonbeneficiary communities. CEC trainings were perceived as useful, though potential improvements were also suggested. Most subcontractors and staff at facilities supported by YCST were mostly satisfied with the YCST planning and implementation approach. The PACE approach was described as requiring a significant amount of time and level of effort. There was variation in the relationship between CEC members and local authorities. Perspectives of subcontractors on the role of CECs and local authorities also varied. Most local leaders were aware of PACE activities. Close to a quarter of the community members were aware of YCST activities in their community, and most of them had not participated in any of the activities. The small percentage of community members who were aware of YCST activities shared a positive appraisal of these activities and their effectiveness. The one exception to this was the dissatisfaction with CEC’s performance related to representing youth and women. Most CEC members and YCST staff were optimistic about the prospects of the CECs continuing to operate beyond the project. Primary and secondary data, however, highlighted challenges to potential sustainability of PACE activities and outcomes including CECs. OBJECTIVE 2: BUILDING CAPACITY Grantees found the application process to be clear and transparent. There was no clear capacity-building plan for grantees, as most grantees did not receive any trainings. The second batch of grantees were not effectively engaged in the design of activities. There was little or no linkage between the grantee activities (Objective 2) and PACE (Objective 1) and Stabilization (Objective 3) activities. Grantees expect to continue to apply what they have learned through their engagement in the YCST project. A considerable number of grantees, however, lack the resources to continue to apply what they have learned. OBJECTIVE 3: PROMOTING PEACE Most stakeholders and beneficiaries who were aware of YCST mentioned that the YCST activities had not contributed to any conflicts in the community. Most Authority Community Committee (ACC) members found the YCST activities to be relevant to the community needs. ACC members had different levels of engagement within ACCs, with some of them having a more engaged and involved role than others. Most YCST-supported facility staff were satisfied with the YCST implementation approach and rehabilitation work. Despite their overall satisfaction, facility staff also highlighted several issues and challenges. While most subcontractors were satisfied with the ACCs, one highlighted ACC interference and delays. Only 17 percent of the community members were aware of YCST activities in their community, 5 | YCST FINAL EVALUATION REPORT USAID.GOV and most of them had not participated in any of the activities. While a small percentage of community members were aware of YCST activities, those who were aware shared a very positive appraisal of these activities and their effectiveness. The one exception to this was the dissatisfaction with ACC’s performance related to representing youth and women. ACC members and YCST staff held a mix of views when it came to the likelihood of ACC’s sustaining on their own. Capacity-building and financial resources were identified as key needs by ACC members. Most facility staff indicated that they lack the resources to maintain the facilities. Field observations also highlighted instances of the supported facilities not being properly maintained. OBJECTIVE 4: ENHANCING OUTCOMES While YCST staff discussed how the PACE team adapted its approach to Yemen, the extent to which the approach was adapted to the Yemen context was less clear. The project did not make any significant adaptations in its approach to working in phase II communities including to improving the community selection process. CROSS-CUTTING FINDINGS—SERVICE DELIVERY PROJECTS’ USE, ACCESS, AND INCLUSION CONSIDERATIONS AND ACHIEVEMENTS While most community members (59 percent) were aware of one or more YCST-supported facilities, many of them (41 percent) were not aware of any of the YCST-supported facilities. Access to YCST￾supported services varied within and across communities, and satisfaction with the quality of services provided by YCST also varied across communities. Community members who were aware of YCST activities and generally held a very positive view of these activities reported dissatisfaction with CECs and ACCs performance related to representing women and youth. Additionally, compared to females and younger (under 30 years old) community members, males and older (over 30 years old) community members were more aware of YCST activities and had a greater rate of participation in and satisfaction with these activities. However, compared to the urban communities, community members in rural communities were more aware and had used or benefitted from these facilities. CONCLUSIONS • While community engagement was a key focus of the YCST development approach, YCST’s activity implementation approach did not result in widespread community awareness and engagement. Most of the community members were not aware of YCST activities in their community, and most of them had not participated in any of the activities. In contrast, most of the stakeholders engaged by YCST were satisfied with the activity implementation approach. • A sizeable number of community members (41 percent) were not aware of any of the YCST￾supported facilities. Access to YCST-supported services varied within and across communities, and satisfaction with the quality of services provided by YCST also varied across communities. • While there were some exceptions, for the most part, YCST applied a conflict sensitive approach to activity implementation that ensured that they were not undermining or negatively impacting the local authority structures and not contributing to conflict or tension between beneficiaries and nonbeneficiaries. USAID.GOV YCST FINAL EVALUATION REPORT | 6 • While the YCST activity made a very conscious effort to make its activities inclusive, female and younger community members were less aware, less engaged in, and less satisfied with the YCST activities. However, compared to the urban communities, community members in rural communities were more aware and had used or benefitted from these facilities. • YCST activities lacked a clear sustainability plan, which has impacted the likelihood of sustaining the activities (e.g., CECs, ACCs, maintenance of facilities supported by YCST) and overall project outcomes. Even among those who reported that YCST-supported CECs and ACCs will continue to meet or the grantees will continue to apply what they have learned, a significant number suggested that they will only be able to do so if they have additional resources including financial support. Visits to YCST built and rehabilitated facilities also indicated that the community and YCST mechanisms are not actively engaged in the upkeep and maintenance of these facilities. EVALUATION QUESTION THREE: EFFECTS OF CHALLENGES OR CONSTRAINTS FACED BY YCST The COVID-19 pandemic and its impact on YCST’s implementation was by far the most frequently referenced challenge negatively impacting the project. Respondents at all levels of engagement on the project consistently referenced security as a key challenge to implementing the YCST program. Both GC and USAID staff highlighted how the changes to the project scope (including budget cuts, addition of the stabilization component) limited some of YCST’s gains and undercut prospects for sustainability. Design limitations around the budgets allocated to service delivery projects within communities were also a commonly referenced challenge by project staff and stakeholders at the community levels as well as by the USAID and GC headquarters (HQ) team. For many PACE and stabilization communities, the projects proposed by CECs and ACCs to address their most prioritized need required significant financial resources that were not available to the project. The process of obtaining permissions from national, governorate, and local authorities to implement the YCST project in Yemen impeded the team’s progress considerably and was cited as a challenge by the project’s senior leadership and USAID staff. While the YCST project was very intentional in ensuring its programming was inclusive of gender and youth, several respondents emphasized the challenges they faced encouraging women’s participation in project activities. Stakeholders also highlighted challenges internal to the YCST implementation approach, including challenges related to communication, coordination, and delays. Delays caused by administrative and procurement processes and the long community entry processes were cited as a challenge by several respondents. Respondents cited the unusually long community entry process and subsequently long approval and procurement processes for the service delivery projects that caused community members to become impatient. Staffing issues related to frequent changes in YCST’s leadership and local staff recruitment and turnover in general were also widely acknowledged as a constraint to the project’s effectiveness. CONCLUSIONS • When assessing the overall performance of the YCST activity, it is very important to consider the significant operational challenges and external factors that the project encountered. The COVID-19 pandemic, the worsening security context in Yemen, and the challenges the project team faced in obtaining permissions to enter communities all seriously impacted the project’s ability to implement 7 | YCST FINAL EVALUATION REPORT USAID.GOV its activities as anticipated and were largely beyond the team’s control. While the project maintained a flexible approach and adapted its activities, these and challenges related to the change in program scope likely impacted the activity’s potential performance. EVALUATION QUESTION FOUR: LESSONS LEARNED FOR FUTURE INTERVENTIONS YCST’s participatory approach to prioritizing community needs was widely appreciated. Development programming was viewed as a welcome change by several respondents. The focus on project learning built into the project’s design was valuable. Service delivery projects were viewed as relevant to YCST community needs. The YCST project time frame, however, was too ambitious for the project scope. The focus on service delivery detracted from conflict mitigation and social cohesion outcomes. Local input was not adequately integrated into the activity design and the operating context in Yemen was not fully factored in the design. There were also differences in expectations within USAID regarding the nature of this project and what it was set out to achieve. CONCLUSIONS • Three-year activity duration was too short a time frame to apply the PACE model across several communities and governorates. COVID-19 complicated this further, as did the changes in activity scope including the inclusion of the stabilization component and changes in the activity budget. This also impacted the community selection process, ability to learn and adapt, and the prospects of sustainability of activities, mechanisms, and outcomes supported by YCST. • Dynamics within and guidance from USAID impacted the scope of the project including the addition of a stabilization component after the award, and significant budget cuts/revisions. There were also differences of opinion within USAID related to whether YCST was set up to help community members and leaders mitigate more local conflicts or whether it was set up to prepare them for the larger peace process, as stipulated in the activity Theory of Change (TOC). RECOMMENDATIONS Key recommendations and considerations for USAID and USAID Implementing Partners (IPs) are summarized below. INTEGRATING LOCAL PERSPECTIVES. • It is important that USAID engage local perspectives and expertise in activity design and in adapting activity designs to suit the local context. This requires actively engaging Foreign Service Nationals (FSNs) and other local stakeholders and allowing sufficient time to incorporate their inputs prior to finalizing the activity design. DEVELOPING A SHARED UNDERSTANDING OF ACTIVITY OBJECTIVES AND OUTCOMES. • It is important to ensure that USAID staff develop a shared understanding of the activity objectives. Interviews with USAID staff indicated considerable difference in understanding of the overall purpose of the activity, including the types of conflict it was designed to address. A shared understanding of objectives will facilitate more effective tracking of activity progress and results, and provision of USAID.GOV YCST FINAL EVALUATION REPORT | 8 consistent feedback to the IP. This will also facilitate more effective activity handover within USAID during staff transitions. ALIGNING ACTIVITY DURATION AND SCOPE. • It is important to allow more time (e.g., 5-years vs. 3-years) for implementation of activities that operate across multiple communities and involve a multi-stage implementation model within each community. Where it is only feasible to implement activities over a shorter duration, it is important to limit the activity to fewer communities to ensure effective implementation. Implementation can also be simplified by having activities focus on fewer objectives, for example solely focusing on PACE, or a combination of PACE and capacity building. It is also important to note that since the PACE model needs to be applied within each community, there are only limited economies of scale when implementing it across multiple communities in a short period. ACTIVITY DESIGN AND IMPLEMENTATION MODEL FLEXIBILITY. • At the activity design stage, it is important to consider what adapting the activity may look like and to assess whether the activity design and/or the implementation model allows for adaptation. Effective implementation of the PACE model for example requires several steps and time. Adapting the PACE model by simultaneously conducting some activities can potentially undermine the model, for example, selecting the priorities prior to community engagement and discussions. Understanding the importance of having flexibility within activity design and implementation models will ensure realistic expectations around learning and adapting. PERIODICALLY REVIEWING THE ACTIVITY THEORY OF CHANGE (TOC). • It is important that USAID and the IP periodically review the activity TOC and its assumptions to ensure that it aligns with the activity objectives and outcomes. For example, while the YCST TOC references the larger peace process, most of the community engagement work primarily focused on local level stressors and service delivery needs. Regularly examining the TOC against the project progress will help ensure that necessary adjustments to the activity implementation approach and/or TOC can be made in a timely manner. UNDERSTANDING THE NATURE OF THE COMMUNITY ENGAGEMENT PROCESS AND ITS IMPLICATIONS. • It is also important for USAID and IPs to consider the particulars of a community engagement model to determine and track activity outcomes. Community engagement can take several forms and it is important to consider how different community members engage with the activity. YCST’s two￾pronged community engagement approach—selecting CEC members to represent different segments of the community while capturing the perspectives of the broader community through surveys—may account for the different levels of awareness, participation and satisfaction between stakeholders and community members. Participation in a survey to prioritize community service delivery needs may be viewed as a light touch intervention by the community members. Additionally, given the scale of service delivery needs in Yemen a community survey may highlight several needs out of which the activity is only able to address one. While community members are likely to appreciate any service delivery support, they may not connect this support to the community engagement process or to contributing to conflict mitigation. Activity budget limitations can also limit the type of interventions that can be supported. This can also lead to some community members thinking that the needs they 9 | YCST FINAL EVALUATION REPORT USAID.GOV had identified were not addressed, and as a result to them questioning the overall effectiveness of the community engagement process. NEED FOR A BROADER AND DEEPER COMMUNITY ENGAGEMENT APPROACH TO ENHANCE INCLUSION. • Although the YCST team made a conscious attempt to engage women and youth in the CECs and ACCs, this did not result in wider awareness, engagement and participation of women and youth. Increasing the engagement of women and youth will likely require a community engagement model that goes beyond service delivery, priority surveys, and representation in the CECs and ACCs. It is also useful to consider approaches designed specifically to engage women and youth, for example by creating women or youth only committees or structures. CONNECTING THE COMMUNITY SELECTION PROCESS TO ACTIVITY OBJECTIVES. • It is important that the community selection process is connected to the activity objectives. Activities focused on conflict mitigation should be implemented in communities that are experiencing or impacted by conflicts. Additionally, if part of the aim (as specified in the TOC) of YCST was to support the larger peace process, the YCST team might have considered selecting communities that are more directly impacted by the larger conflict in Yemen. COMMUNITY SELECTION PROCESS AND THREATS POSED BY EXTREMIST GROUPS. • The presence and influence of extremist groups in two out of the ten communities selected by the evaluation team points to the risks and challenges of operating in the Yemeni context. USAID and IPs need to consider these types of risks as part of the community selection process. For example, interventions designed to improve services in areas controlled by extremist groups, may inadvertently enhance the influence of local leaders associated with extremist groups or result in undermining the outcomes and gains achieved by the activities. BUILDING SYNERGIES ACROSS OBJECTIVES AND ACTIVITIES. • It is important for USAID and the IPs to regularly assess how different activity components/objectives are linked to and reinforce each other. YCST was not able to take advantage of opportunities to link the capacity building activities to PACE and stabilization activities, such as connecting the work of grantees with the work being undertaken by community mobilizers and/or the ACCs and CECs. GRANTS COMPONENT - CAPACITY BUILDING, ENGAGEMENT AND NETWORKING. • For future activities that include a grants component, it is important to take a formal approach and develop a plan to increase networking, learning, and knowledge sharing opportunities among grantees. This includes taking a formal approach to capacity building of the grantees, including how their work can support and reinforce the work being undertaken by other activities; and creating opportunities for local grantees to contribute to the design and implementation of new activities and adapting on￾going activities. IDENTIFYING AND ADDRESSING THREATS TO SUSTAINABILITY OF ACTIVITIES AND OUTCOMES USAID.GOV YCST FINAL EVALUATION REPORT | 10 • It is important for USAID and Implementing Partners to assess the likely sustainability of mechanisms, activities, and outcomes from an early stage, including identifying threats or challenges to sustainability such as lack of resources and the potential for and willingness of community members and stakeholders to come together to sustain activity outcomes. 11 | YCST FINAL EVALUATION REPORT USAID.GOV INTRODUCTION USAID/Yemen contracted MSI to conduct the final evaluation of the YCST activity. The YCST award is a four-year (including a no-cost extension), USAID-funded, $22 million activity implemented by GC together with Yemeni partner FMF and specialized international partners Build Up and PfG. The award is scheduled to end in September 2022. Since its inception in 2018, YCST has focused on supporting local￾level initiatives that mitigate conflict, strengthen social cohesion, promote the peaceful resolution of differences, and, where feasible, capitalize on stabilization opportunities. It targets communities in the four governorates of Abyan, Aden, Lahj, and Taiz in southern Yemen. EVALUATION PURPOSE This evaluation examined the effectiveness of the YCST award in contributing to its intended outcomes around mitigating conflict and increasing social cohesion in targeted communities, as well as any positive and negative unintended outcomes. The evaluation explored the effectiveness of YCST’s four development approaches, the quality of implementation, and how operational challenges influenced the activity’s achievement of intended outcomes. The evaluation generated recommendations to guide USAID decision-making around the design of a new governance, peace, and stability activity focused on community resilience. EVALUATION QUESTIONS This evaluation answered the following four questions specified in the evaluation design (see Annex 1). 1. To what extent, and how, has the YCST award contributed to its intended outcomes, especially community and inter-community conflict mitigation and social cohesion, in PACE and stabilization intervention areas? 2. How, if at all, did YCST’s development approach contribute to, or detract from, the award’s effectiveness and potential for sustainable results? 3. What operational challenges or constraints did YCST face, and how, if at all, did these challenges affect outcomes and prospects for sustaining results? 4. As a result of this evaluation, what “lessons learned” or successful models should USAID consider for follow-on community-level interventions? EVALUATION METHODS AND LIMITATIONS This evaluation team collected and analyzed data from a wide range of primary and secondary sources. This section presents a summary of the methods the evaluation team used, the respondent sampling approach, and the data collection process. The section then provides an overview of the evaluation team’s approach to data analysis as well as key limitations of the evaluation. The evaluation team tried to conduct the same number of interviews within each community, irrespective of the population of the community. The sampling approach, while not representative in nature, ensured that the interviewers conducted interviews across the community and with male and female respondents. USAID.GOV YCST FINAL EVALUATION REPORT | 12 DATA COLLECTION METHODS The team used both qualitative and quantitative methods as well as observations, purposively selecting respondents who were engaged in different objectives and phases of the project’s implementation and represented a mix of YCST communities. In addition to the primary data, the evaluation team reviewed secondary data including YCST progress reports, MEAL team products and trackers, and third-party monitoring reports. The list of secondary data reviewed by the evaluation team is summarized in Annex 2. The evaluation team used a purposive sampling method to select and interview project stakeholders who had specific knowledge or perspectives around YCST’s activities. Key informants included USAID staff, YCST and partner staff, local leaders, members of the local authority, Ministry of Planning and International Cooperation (MoPIC) staff members, CEC/ACC members, and YCST service delivery subcontractors. Table 1 summarizes the number and type of key informant interviews (KIIs) the team conducted. TABLE 1: SUMMARY OF DATA COLLECTION Respondent Type Number of Interviews Completed Community Members 292 ACC Members 40 CEC Members 51 Local Leaders – Non-YCST Communities 16 Service Delivery Projects Observations 16 Service Delivery Project Staff 14 Local Leaders – PACE 12 Local Leaders – Stabilizations 12 YCST Staff 11 Grantees 11 YCST Community Mobilizers 10 Local Authorities – PACE Communities 10 YCST Subcontractors 8 USAID Staff 8 YCST Governorate Coordinators 3 Global Communities HQ Team 3 Partners for Good Staff 2 Build Up Staff 2 13 | YCST FINAL EVALUATION REPORT USAID.GOV PACE Officers 2 Donors 1 MoPIC Staff 2 Total 510 interviews and 16 observations At the community level, the evaluation team used purposive criteria to select a combination of ten PACE and stabilization communities. The evaluation team selected a geographically dispersed mix of communities that included four stabilization and six PACE communities, including four Phase 1 and two Phase 2 communities. The selected communities included urban and rural communities and included the different types of projects implemented under PACE and stabilization objectives (see Annex 3 for more detail on the selected communities). The team’s local data collection partner, Yemen Polling Center (YPC), conducted a mix of in-person and phone interviews as well as observations at the YCST community level. YPC interviewed community members, formal and informal leaders, CEC members, and local leaders. In addition to conducting interviews, the evaluation team conducted semi-structured direct observations at PACE and stabilization project sites including rehabilitated facilities such as schools, water supply stations, health centers and health facilities, parks, and recreational facilities. The primary purpose of the observation was to assess facility use and overall condition and maintenance. The evaluation team complemented facility observations by interviewing relevant facility staff using tailored discussion guides. Instruments included KII guides, semi-structured direct observation checklists, and a community member survey that included both close-ended and semi-structured questions. See Annex 4 for the instruments developed and used by the team. DATA COLLECTION The evaluation team conducted most of its interviews with key informants not at the community level remotely using Microsoft Teams, WhatsApp, and phone calls. Interviews with USAID staff and most YCST and partner staff were conducted in English, while the remainder were conducted in Arabic. Interviewers recorded most of the interviews and transcribed the recordings for analysis. Within each of the 10 sampled communities, YPC conducted 30 in-person interviews with YCST community members using a random sample design. The team selected community member respondents using a household interval approach to ensure the sample was distributed evenly within each community. Additional details on the methodology YPC used for community level data collection can be found in Annex 5. The evaluation team tried to conduct the same number of interviews within each community, irrespective of the population of the community. The sampling approach, while not representative in nature, ensured that the interviewers conduct interviews across the community and with male and female respondents. Each community was divided into roughly four equal sections, and the data collection team conducted between seven to eight interviews in each section, half with males and half with females. The implemented approach ensured that the evaluation team could gauge “awareness, participation, and access” to activities and services for community members across the community. The USAID.GOV YCST FINAL EVALUATION REPORT | 14 final achieved sample also indicated that the sampling approach was able to capture key demographic groups as summarized in Table 2. TABLE 2: SAMPLE BREAKDOWN Sample Composition Sample Size Percentage Total Sample 292 100% Male 153 52% Female 139 48% Under 30 92 32% Over 30 200 68% Primary or Less 146 50% More than Primary 146 50% All interview participants provided their verbal informed consent. The evaluation team stored the data collected from the reviewed secondary data and the interviews on MSI’s secured server. The files were only accessible to the evaluation team. DATA ANALYSIS The team organized and coded interview and secondary data in ATLAS.ti, a software package for qualitative data analysis. The evaluation team conducted a content analysis of qualitative data by evaluation question, respondent type, and YCST objective. The team also generated descriptive statistics for quantitative, close-ended questions from CEC and ACC members, local leaders, and community members. It is important to note that the community data included in this report is unweighted. LIMITATIONS The primary limitations of the data stem from difficulties accessing the community level respondents and meeting targets in the evaluation’s sampling approach. The team faced challenges accessing two of the originally sampled 10 communities, one in Lahj and another in Abyan governorate. In Lahj governorate the team was unable to access the Al-Nabyah community due to safety and security concerns around the presence of extremist group militants in the area. The team substituted a community of a similar population size, YCST phase, and type of service delivery also in Lahj in its place. Similarly, in Jayshan community in Abyan governorate, members of an extremist group stopped the YPC team at a checkpoint, and while the team passed this checkpoint and accessed Jayshan, community members hesitated to participate in data collection and no women participated in interviews. 15 | YCST FINAL EVALUATION REPORT USAID.GOV YCST COMMUNITY CONTEXT Prior to presenting findings by question, this section summarizes some contextual data based on community interviews conducted in the ten communities. For most community members, the overall situation, illustrated in Figure 1, has worsened because close to 60 percent (176/292) of the respondents mentioned that things are moving in the wrong direction. Close to a quarter (23 percent or 68/292) reported that things are moving in the right direction. FIGURE 1: OVERALL SITUATION IN THE COMMUNITY The economic situation, highlighted in Figure 2, has worsened for most respondents over the last 12 months. FIGURE 2: OVERALL ECONOMIC SITUATION 60% 23% 14% 3% 0% 10% 20% 30% 40% 50% 60% 70% Moving in the wrong direction Moving in the right direction Some are moving in the right direction, and some are moving in the wrong direction Don’t know/No response Overall Situation Are things moving in the right direction or in the wrong direction? 66% 25% 6% 3% 0% 10% 20% 30% 40% 50% 60% 70% Worsened Remained the same Don't know/No response Improved Economic Situation In your opinion, has the overall economic situation in your community improved, remained the same, or worsened over the last 12 months? USAID.GOV YCST FINAL EVALUATION REPORT | 16 Government is the primary provider of basic services in the communities including education, health services, electricity, and water. Electricity and water are not available to a considerable number of community members. While much smaller than the government, international nongovernmental organizations (NGOs) also play a role in providing basic services in the communities, shown in Figure 3. FIGURE 3: SERVICE PROVIDERS OF BASIC SERVICES Community members identified education service providers as the least responsive to their community needs, followed by electricity, health, and water service providers, shown in Figure 4. FIGURE 4: RESPONSIVENESS OF SERVICE PROVIDERS 39% 44% 57% 74% 13% 45% 5% 3% 1% 2% 6% 11% 5% 1% 3% 1% 1% 19% 3% 2% 1% 18% 8% 22% 16% Water Electricity Health Education Provision of Basic Services Who is primarily reponsible for providing basic services in your community? Government Service is not available Local organizations/CSOs International NGOs Private sector Community projects Other Don’t know 71% 63% 53% 53% 26% 31% 42% 38% 3% 6% 5% 9% 0% 10% 20% 30% 40% 50% 60% 70% 80% Water Health Education Electricty Provision of Basic Services – Responsiveness to Community Needs How responsive are the service providers to your community needs? Responsive Unresponsive Don’t know 17 | YCST FINAL EVALUATION REPORT USAID.GOV Community members identified water service providers as having become the most responsive to community needs over time, followed by health, electricity, and education service providers, illustrated in Figure 5. FIGURE 5: RESPONSIVENESS OF SERVICE PROVIDERS OVER TIME FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS This section provides evaluation findings by question. This is followed by a summary of the evaluation team’s conclusions and recommendations. EVALUATION QUESTION ONE: YCST’S CONTRIBUTIONS TO INTENDED OUTCOMES To what extent, and how, has the YCST award contributed to its intended outcomes, especially community and inter-community conflict mitigation and social cohesion, in PACE and stabilization intervention areas? This section presents whether and to what extent YCST contributed to its intended outcomes by objective/approaches. Key outcomes/results of interest by YCST approaches are summarized in Table 3. 31% 22% 15% 10% 30% 37% 41% 35% 36% 34% 37% 48% 4% 7% 7% 6% 0% 10% 20% 30% 40% 50% 60% Water Health Electricty Education Provision of Basic Services – Responsiveness to Community Needs Have the service providers become more or less responsive to community needs, over time? More responsive Less responsive Remained the same Don’t know USAID.GOV YCST FINAL EVALUATION REPORT | 18 TABLE 3: YCST APPROACHES AND INTENDED OUTCOMES YCST Approaches Intended Outcomes/Results 1. PACE approach (Objective 1) a. Increase in social cohesion b. Enhanced capacity of the community to resolve/mitigate conflicts c. Mitigation of conflict resulting from service delivery activities 2. Capacity-building approach (Objective 2) a. Improved organizational capacity for conflict mitigation and resolution 3. Stabilization approach (Objective 3) a. Enhanced capacity of local leaders to be responsive to community needs b. Mitigation of conflict resulting from service delivery activities 4. Adaptive learning approach (Objective 4) a. Activity adaptations based on M&E and contextual data Next we summarize key findings focused on how the YCST project contributed to these outcomes for each of the four objectives. OBJECTIVE 1: ENGAGING COMMUNITIES The PACE approach involves inclusively engaging communities to identify and implement service delivery and conflict mitigation projects. The approach intends to address drivers of and impacts of conflict and build social capital and skills within the community. The key intended outcomes/results for the PACE approach include: a. An increase in social cohesion b. Enhanced capacity of the community to resolve/mitigate conflicts c. Mitigation of conflict resulting from service delivery activities OUTCOME: INCREASE IN SOCIAL COHESION According to key stakeholders, YCST activities improved relationships between community members. Members of the local authority reported that the YCST activities improved relationships between community members by bringing them together to prioritize needs, identify stressors, resolve existing conflicts, and proactively seek solutions to existing problems. “The project improved relationships between community members, restoring the innovation spirit, and led to creating a team spirit.” –Member of the Local Authority 19 | YCST FINAL EVALUATION REPORT USAID.GOV According to key stakeholders YCST activities contributed to an increase in community members’ level of trust in local authorities. All ten community mobilizers reported that the PACE approach enhanced relationships between CECs and local authorities. They also explained that local authorities gained credibility and the YCST activities contributed to an increase in community members’ trust in local authorities. “We have a good relationship with them [CECs] as we work together, and we facilitate their activities and provide them the required support.” –Member of the Local Authority Notably, unlike under the Objective 3 outcomes, the increase in community members’ level of trust in local authorities was an unintended outcome of activities under the PACE approach. Most community members, however, reported that the level of trust between community members had not changed. For most community members (58 percent or 96/167), the level of trust between community members has remained the same over the last year. Around 19 percent (31/167) reported that it had improved while around 15 percent (25/167) reported that it had worsened. Close to a third of the community members were dissatisfied with their local leaders and had less trust in them than a year ago. Slightly more than a third of the respondents (37 percent or 61/167) were dissatisfied with the performance of their local leaders. For close to half of all the respondents (48 percent or 80/167), their level of trust in local authorities and government had remained the same over the last 12 months. Close to a third of the respondents (30 percent or 45/167) reported that their level of trust had decreased. Only 6 percent (10/167) reported that they trust the local authorities more. The same percentage of community members (27 percent or 45/167) described the relationship between their community and local authorities as neither good or bad. Those who were critical of local authority mentioned that the local authority does not respond to community needs and does not provide services in the community. Community members also mentioned that local authorities are absent and that they do not ensure security in the community. OUTCOME: MITIGATION OF CONFLICT RESULTING FROM SERVICE DELIVERY ACTIVITIES Most community members reported that their access to basic services supported by YCST had either remained the same (39 percent) or improved (36 percent) over the last year. Slightly less than a quarter of the respondents (23 percent or 25/107) reported that it had worsened. Community members in Phase 2 communities were very optimistic that their access will improve once YCST projects have been completed. While the plurality of community members were satisfied with the quality of YCST services, close to a third were dissatisfied. The plurality of respondents (43 percent or 46/107) were satisfied with the quality of services provided by YCST, while close to a third (32 percent or 34/107) were dissatisfied because services from the rehabilitated facilities, such as water or electricity, did not reach them or they had not experienced an improvement with access to electricity or water. Some respondents also mentioned that the services USAID.GOV YCST FINAL EVALUATION REPORT | 20 should have also included additional assistance, such as provision of medicine in health facilities or employment of additional teachers in school. Community members shared a mix of responses when asked if the overall peace and security situation and conflict mitigation mechanisms have improved. The YCST team selected communities that did not have any ongoing conflicts according to secondary data. Also, only a small percentage (16 percent or 26/167) of the community members mentioned the presence of conflict or conflicts impacting their household and their community. Almost an equal number of community members reported that the overall peace and security situation in their community had improved (31 percent or 52/167), remained the same (34 percent or 57/167), or worsened (32 percent or 54/167) over the last year. Those who felt that the security situation had improved explained that security had stabilized in their community. Some also reported that the government authorities and security institutions are playing their role to prevent issues of peace and security from escalating. Around 35 percent (19/54) of the respondents who reported a worsening security situation mentioned an increase in thefts and murders in their community because of the deteriorating economic situation and lack of access to basic services. According to key stakeholders, by addressing critical service delivery gaps, YCST activities contributed to mitigating conflicts. CEC members and community mobilizers noted that the projects implemented through YCST addressed pre-existing conflicts and issues in their community relating to access to basic services. Most CEC members (48/51) reported that the service delivery projects implemented by YCST reduced conflicts in their communities by improving access to basic services. Local authority members also reported that the YCST activities reduced the causes of conflicts resulting from lack of access to basic services such as water, electricity, and health. Most local leaders in PACE communities believed that YCST had helped reduce conflict (8/12) or helped reduce conflict to some extent (3/12) by addressing service delivery gaps through projects that helped bring people together in their communities and by targeting the most important issues behind disputes. Almost all service delivery project staff (9/10) in PACE communities reported that the projects contributed to reducing conflicts and improving relationships between community members. Subcontractors engaged by YCST reported that the service delivery projects contributed to reducing conflicts and improving relationships between community members. According to some key stakeholders, YCST activities contributed to or created conflicts. Some of the local leaders in PACE communities (6/12) were aware of instances in which YCST support contributed to creating conflicts. Leaders described that the assistance resulted in disagreements around how to distribute the activities and support. “It resolved a problem in one place and caused a conflict in another place because the water distribution network does not reach everyone.” –Local Leader 21 | YCST FINAL EVALUATION REPORT USAID.GOV Eight facility staff in the PACE communities indicated that projects sometimes created issues between communities that received services from YCST and those that did not. The Dar Sa’ad Electricity Rehabilitation Project staff reported that there were several complaints from neighboring communities who have not benefited from the project and who also demanded to be supported. Similarly, the Ras Al Aarah water enhancement facility staff reported that neighboring Al Harba community complained about not receiving water as they were not one of the selected communities. “During the implementation of the street lighting initiative, some people pointed their weapons at us because we did not install lights in their neighborhoods.” –CEC Member OUTCOME: ENHANCED CAPACITY OF THE COMMUNITY TO RESOLVE/MITIGATE CONFLICTS Key stakeholders discussed how the mechanisms (e.g., CECs) set up by YCST enhanced the capacity of communities to resolve/mitigate conflicts. When it came to their ability to come together and resolve conflicts, however, the CEC members’ opinions were split. Most CEC members (63 percent or 32/51) viewed the CECs as a new and effective mechanism for addressing conflicts impacting their communities and that with the support the project provided CECs can address urgent needs in their communities by providing an opportunity for community members to come together. “It gave the opportunity to the community members themselves to identify and select their needs and interventions. The project did not come to our community with readymade interventions or projects that has no relation the peoples’ needs.” –CEC Member Local authority members believed that the YCST activities improved the existing conflict resolution mechanisms in the communities, including traditional ones. For example, two of ten local authority members noted that local leaders (e.g., Sheikhs) are the main individuals involved in conflict resolution in their communities, and that through the training provided to them by YCST, they were able to further improve their capacity in conflict resolution. Local authorities viewed the role of CECs in their community as a new channel for community engagement, explaining that CECs are effective because they are made up of and were selected by community members. Almost all the local leaders (11/12) and members of the local authority (9/10) in PACE communities confirmed that the YCST project had created new channels for community engagement in their communities. Several leaders and local authority members highlighted that the committees within the community were crucial to the project, as they involved community members at all stages of the project design and delivery. Around half of the CEC members (23/51) believed that their communities can get together and address their problems. However, most of the rest (27/51) believed that their community is not capable of getting together to address their problems due to limited government presence to address the root causes of the problems and lack of financial support from international organizations that would allow them to solve these issues. Most community members reported that their community is not able to address the key problems they face. USAID.GOV YCST FINAL EVALUATION REPORT | 22 Close to a third of all the community members (34 percent or 56/167) reported that the existing conflict resolution mechanisms within their community had improved, while around 15 percent (25/167) reported that they had worsened. Despite this, around two thirds of the respondents (66 percent or 97/146) reported that their community is not able to address the key problems they face. A plurality of community members reported that local community (40 percent or 67/167), local leaders (44 percent or 73/167), and local organizations (42 percent or 71/167) were not at all involved in addressing community conflicts. Last, close to two thirds of the respondents (60 percent or 100/167) reported that their ability to participate in the political decision-making process has remained the same over the last year, and only 3 percent reported that it had increased. OBJECTIVE 2: BUILDING CAPACITY YCST Objective 2 activities focused on building the capacity of organizations, networks, and local leaders to identify and address unresolved grievances that can lead to violent conflict. Capacity-building activities were implemented in PACE and stabilization communities and were supported through a Peacebuilding Grants Fund; Women, Peace, and Security Sub-fund, and an Innovation Grant Sub-fund. Capacity-building activities also aimed to enhance the participation of women and youth in the conflict resolution and mitigation efforts. OUTCOME: IMPROVED ORGANIZATIONAL CAPACITY FOR CONFLICT MITIGATION AND RESOLUTION Grantees appreciated the skills they had gained while working with YCST. All grantees felt that they had gained from their engagement and partnership with YCST. Some mentioned learning administrative and planning skills, while others reported learning technical skills related to project implementation. Several grantees referred to strengthened organizational capacity to communicate with community members and leaders and to effectively engage them in project activities. Similarly, grantees referenced strengthened relationships and capacity to work with local authorities. Engagement in YCST allowed some grantees to work in more remote areas and on culturally sensitive issues. Several grantees also mentioned that they gained experience in working in rural, hard-to-reach, and culturally sensitive communities, which they were not able to access before their partnership with YCST. Grantees also mentioned that their participation in YCST had helped them work on sensitive issues (e.g., child marriage) they were not able to work on previously. OBJECTIVE 3: PROMOTING PEACE Objective 3 activities focused on assisting local authorities in understanding the priorities of their constituents and developing plans and implementing projects (infrastructure or other investments) to respond to these priorities. The YCST team also provided capacity-building support to local authorities to communicate with and be accountable to their constituents. 23 | YCST FINAL EVALUATION REPORT USAID.GOV OUTCOME: ENHANCED CAPACITY OF LOCAL LEADERS TO BE RESPONSIVE TO COMMUNITY NEEDS Key stakeholders viewed YCST as offering an effective new conflict mitigation mechanism and improving the relationship between community members and local authorities. Most ACC members (34/40) reported that the ACCs effectively solved issues in their communities by mitigating conflicts when they arose. Of those who had received training, most ACCs (25/26) found the training useful and reported that training had enhanced their knowledge of conflict management and strengthened their capacity to identify and respond to their community needs. More than half of the ACC members (23/40) reported that the project improved the relationship between members of the community and local authorities and strengthened the role of local authorities through YCST activities. Local leaders described that YCST activities had positively impacted the relationships between members of the community (8/12) and between members of the community and local authorities (10/12) and addressed causes of conflict in the community (9/12). Most (8/12) leaders confirmed that the YCST project had created new channels for community engagement in their communities. A plurality of community members were dissatisfied with the performance of their local leaders. Slightly more than a third of the respondents (39 percent or 49/125) were dissatisfied with the performance of their local leaders. Close to a quarter of the respondents (27 percent or 34/125) reported that their level of trust had decreased. Only seven percent reported that they trusted the local authorities more. Almost the same percentage of community members described the relationship between their community and local authorities as either good (26 percent or 32/125) or bad (24 percent or 30/125). Reasons for dissatisfaction with the local leaders and authorities included lack of responsiveness to community needs and the leaders and members of the local authority not playing their role in providing services and ensuring security in their communities. OUTCOME: MITIGATION OF CONFLICT RESULTING FROM SERVICE DELIVERY ACTIVITIES Key stakeholders mentioned that YCST activities had contributed to mitigating conflict. Most ACC members (35/40) reported that the YCST service delivery activities contributed to mitigating conflict through the implementation of service delivery projects focused on critical community needs. All but one local leader (11/12) believed that YCST activities had helped reduce conflict in their communities. Most facility staff (17/20) reported that the service delivery projects contributed to reducing conflicts. The Ja’ar park staff in Khanfar reported that the project created a space for families which strengthened social relations and cohesion between community members and reduced conflicts in their community. School staff noted that the rehabilitation work increased the capacity and enrollment in schools and improved safety, thus reducing conflicts between students’ parents and school staff. Access to YCST-supported basic services improved for a considerable number of community members. Access to basic services supported by YCST had improved for half of the community members (50 percent or 62/125). Most respondents (52 percent or 65/125) were satisfied with the quality of services provided by YCST. Community members stated that there had been an increased capacity and USAID.GOV YCST FINAL EVALUATION REPORT | 24 attendance in schools due to the rehabilitation work; increased access to recreational sites (parks/football fields) among community members, especially youth and children, and improved services at the health facilities because of the overall improved condition. For a plurality of community members, the overall peace and security situation in their community had not changed. For a plurality of respondents (42 percent or 53/125), the overall peace and security situation in their community had remained the same, followed by close to a third of the respondents (30 percent or 38/125) who reported an improvement, and a quarter of the respondents (26 percent or 32/125) who reported that the peace and security situation had worsened over the last 12 months. A quarter of the community members reported being affected by conflict, and a plurality of community members mentioned that local authorities, local community members, and organizations had not played any role in addressing community conflicts. Close to a quarter of the respondents (26 percent or 32/125) mentioned that the presence of conflict or conflicts impacted their household and their community. Only 12 percent (15/125) of the respondents reported that their local government/authorities have done anything to address conflicts in their community. A plurality of community members reported that the local community (46 percent or 57/125), local leaders (58 percent or 72/125), and local organizations (42 percent or 52/125) had played no role in addressing community conflicts. Most community members’ ability to participate in the political process had not changed. Most community members (54 percent or 67/125) reported that their ability to participate in the political decision-making process has remained the same over the last year. Close to 12 percent (15/125) reported that it had decreased, and only 2 percent (3/125) reported that it had increased. A considerable number of community members did not see an improvement in the conflict resolution mechanisms in their community. A plurality of respondents (42 percent or 52/125) reported that the existing conflict resolution mechanisms within their community had remained the same. Close to a quarter of the respondents (23 percent or 29/125) reported that they had worsened, with many of them mentioning the absence or minimal presence of any entity (local authorities or organizations) to facilitate or support conflict resolution mechanisms. Around 18 percent (23/125) of the community members reported that the conflict resolution mechanisms had improved because of the support being provided by NGOs responding to community needs. Most community members believed their community is not capable of addressing the problems they face. Most community members (56 percent or 57/102) reported that their community was not capable of addressing the key problems they face. Close to a third (36 percent or 37/102) reported that their community had the ability to address the key problems they face. 25 | YCST FINAL EVALUATION REPORT USAID.GOV OBJECTIVE 4: ENHANCING OUTCOMES Activities under Objective 4 focused on learning and adaptive management and included a learning agenda, continuous mapping, analysis, and adaptation. The learning was also intended to develop community entry and sustainability models and for developing tools for assessing success or failure. The YCST team also developed community dashboards and conducted conflict analyses and real-time context monitoring to respond to community and context-specific considerations. OUTCOME: ACTIVITY ADAPTATIONS BASED ON M&E AND CONTEXTUAL DATA YCST took a very intentional and resourced approach to monitoring, evaluation, and learning (MEL) under Objective 4 of the project, documenting routine monitoring, collecting beneficiary feedback about the effectiveness of programming, ensuring the project mainstreamed gender and inclusion, and convening teams to discuss lessons learned. The learning team put together a considerable number of learning products and learning sessions to inform adaptations to the project, including an adaptation workshop, stabilizations lessons learned reports, perception and satisfaction surveys, baseline studies, a learning agenda, and a lesson learned tracker. In addition, the MEAL team collected routine monitoring data to track progress of its activities against performance indicators. ACTIVITY ADAPTATIONS This section summarizes key activity adaptations undertaken by YCST. Grants Component: Multiple YCST staff described the challenges they faced issuing the annual program statement. They explained that not only was the process time consuming, but also that in allowing applicants to propose innovative activities, it became clear that proposals targeted emergency response instead of development programming. Project staff emphasized how smoothly the grants component has run since they started releasing requests for applications that requested specific activities. This adaptation greatly reduced the time required for the solicitation and streamlined the entire grant-making process and resulted in receiving more, and better, proposals and the teams’ ability to carry out more activities. Development of Soft Activities: The MEAL team provided feedback from the community to other technical teams on how to improve the community engagement. Feedback included implementing soft activities to keep communities engaged with YCST while they waited for the service delivery project. Learning Systems: YCST also continued to refine and adapt its own MEAL systems and products to address feedback from project stakeholders as well as the project’s knowledge gaps. In Year 3 of the project, the MEAL team acted on existing project learning and accelerated its efforts in revising YCST’s learning agenda and making a concerted effort to ensure the project collected and acted on beneficiary feedback. Participants at the Year 1 Learning Workshop highlighted that the project lacked formal feedback mechanisms for communities. To address this deficiency, YCST developed and sensitized communities on a toll-free hotline in communities. The project also hired an accountability officer to collect beneficiary feedback on service delivery projects. USAID.GOV YCST FINAL EVALUATION REPORT | 26 YCST also improved its M&E systems and transitioned from a paper-based system to electronic platforms. One USAID staff member, although they appreciated the transition YCST made in their M&E data collection systems, also mentioned the slowness of the initial process. “If they had started with that at the beginning that might have helped reporting and community analyses.” –USAID Respondents The lessons learned tracker highlighted the interactive monitoring platform as significantly improving the team’s monitoring and reporting on performance indicators as well as overall program team coordination with the MEAL team. Integrating Gender and Inclusion Considerations: The project made a concerted effort to include women in the CECs (30 percent membership target) and during community engagement processes. Improving women’s engagement was more successful in some communities than others. The project MEAL team disaggregated data by gender and age at multiple administrative levels, and two of the seven learning agenda questions focused on women’s participation and inclusion. The team also conducted learning activities to assess if YCST’s community-based structures improved attitudes toward gender and social inclusion in community decision-making. One YCST project staff member emphasized that their existing guidelines to ensure women’s participation in activities extended to their grantees and partner organizations as well. “(They did) a really nice job of ensuring that some of the nontraditional groups were also included in the decision￾making processes … the number of women included was quite impressive.” –USAID Staff In the more conservative communities, the team adapted its approach to ensure that women’s voices were considered in the prioritization of community needs by carrying out household surveys with them in their homes and schools to include their input. CONCLUSIONS • Overall, stakeholders including local leaders were more aware of YCST activities and held a more positive appraisal of YCST activities contributing to their intended outcomes including increase in community cohesion, mitigation of conflict resulting from service delivery projects, and enhanced capacity of community members to resolve/mitigate conflict. This was not the case among the community members, as a very small number of community members were aware of YCST activities beyond the service delivery projects. As per community members, the level of trust among the community members had not significantly changed. A considerable number of community members did not see an improvement in the conflict resolution mechanisms or the community’s ability to address its own problems. • Most community members did not view the YCST activities from the lens of conflict mitigation, community cohesion, and enhancing the collective competence of the community to solve its own problems. Instead, these activities were viewed as improving access to services in their area. YCST’s community selection criteria also included selecting communities without any conflicts. In such a context, while the service delivery activities were resolving community-level stressors by supporting 27 | YCST FINAL EVALUATION REPORT USAID.GOV provision of essential services, it is not entirely likely that absent the YCST intervention, these service delivery needs would have led to significant violent conflicts within the community. This may also account for why only a small percentage of community members viewed the YCST intervention as a conflict mitigation activity. • For a three-year activity implemented during the peak of the COVID-19 pandemic and in a very challenging context, the YCST team accomplished significant results. YCST’s accomplishments included implementing 31 service delivery projects across 22 communities and four governorates. There was general acknowledgement by community leaders and members that the service delivery activities were relevant, enhanced access to essential services, and were of good quality. Grantees also appreciated the skills and experience they had gained while working with YCST. EVALUATION QUESTION TWO: YCST’S DEVELOPMENT APPROACH How, if at all, did YCST’s development approach contribute to, or detract from, the award’s effectiveness and potential for sustainable results? Findings under this question are presented by objective. Under each objective, this section first discusses the YCST approach including how the activities were implemented and how that contributed to or detracted from the award’s effectiveness. This is followed by a discussion of the likelihood of sustainability of YCST activities and outcomes. OBJECTIVE 1: ENGAGING COMMUNITIES To assess sustainability of PACE efforts and outcomes, the evaluation team examined aspects of the PACE approach focused on sustainability of PACE mechanisms and outcomes. The evaluation team assessed the collective competence of the CECs and other PACE stakeholders to come together to address/mitigate future conflicts. The team also assessed the potential sustainability of CECs and reviewed existing practices related to actual and planned maintenance of facilities, community cost sharing (resource mobilization), and community monitoring of projects. The evaluation team also conducted in-person visits to specific sites to interview facility staff and to conduct direct observations to assess the state of infrastructure and maintenance efforts. Key findings related to the YCST PACE approach are summarized next. According to most key stakeholders and community members who were aware of YCST, the YCST program did not undermine or negatively affect the local authority structure in the community. All ten local authority members interviewed reported that the YCST program did not undermine or negatively affect the local authority structure in the community. “The project supported the local authorities and gave them a role in implementing the project through working with local communities … In addition, local authority members received trainings that enhanced their capacities.” –Member of the Local Authority USAID.GOV YCST FINAL EVALUATION REPORT | 28 “The project provided trainings in resolving conflicts which empowered the local authority through increasing their knowledge of the PACE approach and common identification of needs.” –Member of the Local Authority One local authority member reported that YCST activities improved community perceptions of NGOs and international NGOs because community members saw concrete outputs from the projects implemented. None of the CEC members and most local leaders (9/12) in PACE communities stated that the project had not undermined or negatively affected local authorities or leadership structures in their communities. Those who believed leaders had been negatively impacted mentioned that people in the community believed that the local authority was not effective because YCST conducted the work. Only 6 percent (6/109) of the community members who were aware of YCST mentioned that they were aware of instances in which the YCST support to their community contributed to creating conflicts between community members or with other communities. Additionally, none of the leaders in non-YCST communities reported that YCST activities led to any conflicts with members of their communities. Key stakeholders shared a mix of views on whether the YCST intervention contributed to creating issues or tensions between beneficiary and nonbeneficiary communities. Some CEC members (16/51) reported that members of non-YCST assisted communities were unhappy that they were not selected for assistance, which caused tensions between beneficiary and nonbeneficiary community members. “The project solved the problem for half of the community, and there were people who did not have access to water, which caused problems among the community members.” –CEC Member, Ras Al Aarah “There were disputes between community members and members of the neighboring community that did not receive YCST assistance. If the organization has a future vision, and it intervenes in the neighboring community, the problem will be resolved.” –CEC Member, Dar Sa’ad However, none of the 16 local leaders from the nonassisted neighboring communities were aware of YCST activities leading to or creating any conflict in their community or between their community and YCST-assisted communities. CEC members perceived YCST training as useful but also suggested potential improvements. Almost all CEC members (49/51) received some type of training from YCST, and all found the training useful for their roles as CEC members. Most CEC members (36/51), however, reported that training can be improved by either adapting the training to the changing context or providing refreshers and additional trainings such as trainings on community management, sustainability, and implementing awareness programs. Staff at facilities supported by YCST were mostly satisfied with the YCST planning and implementation approach. 29 | YCST FINAL EVALUATION REPORT USAID.GOV Almost all facility staff (9/10) had positive perspectives about the planning, implementation approach, and the work implemented by YCST. All facility staff were satisfied with the rehabilitation work, and all reported that the planned outcomes were achieved. Facility staff explained that the facilities operate effectively because of the rehabilitation or enhancement work implemented by YCST. Most subcontractors engaged by YCST were satisfied with the YCST planning and implementation approach. Subcontractors had positive views of YCST’s planning and implementation approach. Most subcontractors (8/10) found the bidding process clear and transparent, and most reported that their experience with the bidding process was good. Three subcontractors mentioned challenges with technical specifications required during implementation, mainly indicating that the specifications in the contracts did not match what was needed at the facility level. One contractor mentioned challenges during implementation, including amending some of the specifications, while another contractor complained about a three month delay in payment. The PACE approach required a significant amount of time and level of effort. Some community stakeholders and USAID staff members mentioned that the PACE approach takes a lot of time and effort. “PACE approach is inefficient.” –CEC Member, Al Shouhada Community There was variation in the relationship between CEC members and local authorities. CEC members’ relationships with local authorities varied. Most CEC members learned about YCST from the local authorities and YCST staff. Some (34/50) found their relationship positive and based on collaboration, a few others (7/50) reported that their relationship was sometimes positive and sometimes negative, while the remainder (9/50) reported that it was weak or nonexistent. Perspectives of subcontractors on the role of CECs varied. Some subcontractors viewed the CECs as very supportive, while some viewed their role as an obstacle that caused delays during the rehabilitation work. “They only delayed our work. They could not solve any obstacle… There was no cooperation found from the CECs, community members, and local authority. My staff were threatened, and the materials were about to be stolen.” –Subcontractor, Abyan Subcontractors view of the role of local authorities during the planning and implementation of the YCST support projects also varied. Some subcontractors (3/8) viewed the local authorities as supportive while others (5/8) reported that their role was either nonexistent or not effective. Most of the subcontractors (4/5) who reported that the local authorities’ role was not effective or nonexistent were in PACE communities. One of the subcontractors reported that the local authorities caused some issues because they wanted additional equipment in the Haydera Football Field, but the issue was resolved by GC staff members. USAID.GOV YCST FINAL EVALUATION REPORT | 30 Most local leaders were aware of PACE activities, though only a few had participated in these activities. All but one local leader (11/12) in PACE communities were familiar with the YCST program. Most described the service delivery projects that had taken place in their communities, while four in PACE communities also spoke about the CECs and training courses provided to them. Most local leaders in PACE communities (9/12) stated that they had not participated in any YCST activity. Those that had participated in an activity described participating in focus group discussions and the needs assessment and prioritization process. Close to a quarter of the community members were aware of YCST activities in their community, but most of them had not participated in any of the activities. Close to a quarter of the respondents (27 percent or 45 of 167) were aware of YCST activities in their community. Most of those who were aware of YCST activities (76 percent or 34/45) were aware of service delivery projects, followed by community engagement processes (9 percent or 4/45). Most of those who were aware of YCST activities within their communities (64 percent or 29/45) had not participated in any activity. A small percentage had participated in the service delivery activities (18 percent or 8/45) and other community engagement activities (4 percent; 2/45). While a small percentage of community members were aware of YCST activities, those who were aware shared a very positive appraisal of these activities and their effectiveness. The one exception was the dissatisfaction with CEC’s performance related to representing youth and women. Most of those who were aware of YCST activities within their communities (76 percent or 34/45) were satisfied with the activities. Most of those who were aware of YCST activities within their communities (73 percent or 33/45) believed that YCST activities addressed important community needs. Only slightly less than a fifth of the community members (17 percent or 29/167) were aware of CECs in their community. Of those who were aware of CECs, 66 percent (19/29) had participated in the selection of CEC members and around 41 percent (12/29) had participated in identifying and addressing stressors in their community. Most of those who were aware of CECs (93 percent or 27/29) were satisfied with the performance of CECs and mentioned that their local CEC was active in their community (86 percent or 25/29). Most of those who were aware of CECs were satisfied with the CECs’ performance representing community interests, selecting community projects, representing youth and women, addressing conflicts, and strengthening relations between community members. However, a meaningful number of respondents were dissatisfied with their CEC’s performance when it came to representing youth and women. SUSTAINABILITY Key findings related to sustainability are summarized in the following. Most CEC members and YCST staff were optimistic about the prospects of the CECs continuing to operate after YCST ends. 31 | YCST FINAL EVALUATION REPORT USAID.GOV Most of the CEC members believed that the CEC is somewhat likely or likely to continue to meet and function after the YCST program ends (Figure 6). The four CEC members that believed the CECs would not continue noted a lack of financial means as well as space for the committee to meet. When asked about the likelihood that CECs will continue beyond the project, YCST staff largely agreed that they were sustainable. Several staff noted that in some communities where YCST activities had been completed, CECs have already been sustained and continue to be functional when the YCST team conducts follow ups to see if they were still engaged. Primary and secondary data highlighted potential challenges to sustaining PACE activities and outcomes, including CECs. While most subcontractors reported that the facilities will continue to operate effectively in the future if they are well-maintained, seven of the ten supported facility staff mentioned that they do not have the resources needed to maintain the facilities. Ten of the 46 CEC members who were optimistic about CECs sustaining beyond YCST also mentioned that it would require additional resources that they themselves are not able to generate. Field observations conducted by the evaluation data collection team at a sample of facilities also provided evidence of several rehabilitated sites not being properly maintained. For specific observations and details, please reference the following chart summarizing the observation findings. Secondary data also suggests that in some communities CECs may not operate beyond the life of the project. The Year 1 annual learning workshop report mentioned low participation in CECs due to lack of financial incentives and a lack of willingness to accept the concept of volunteerism. Additionally, one third party monitoring (TPM) report signalled there could be issues around sustainability in certain communities where projects required community members to maintain the facilities without being compensated. In one PACE community, the TPM team noted that four out of five CEC members did not attend the training to take over the solar power water station project when they learned they would not receive per diem for the training. FIGURE 6: CEC PERCEPTIONS OF CEC SUSTAINABILITY 29 (57%) 17 (33%) 4, (8%) 1 (2%) Very likely Somewhat likely Very unlikely No response Likelihood that CECs will Meet and Function after YCST Program Ends (n = 51) USAID.GOV YCST FINAL EVALUATION REPORT | 32 OBSERVATION FINDINGS The YCST team provided additional information relevant to the evaluation team’s observation findings and had already followed up on several of the findings presented after the evaluation team’s visit had taken place. These updates are included where applicable. Community Service Delivery Project Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Aden/Dar Sa’ad/Abd Albari Dar Sa’ad Electricity Rehabilitation Project √ √ √ • The doors of the generators were open with no insulation fences around them. • Some cables on the electricity pole were tangled, and some were installed too low. Lahj/Al-Madraiba Wa Al A’ara/Al￾Suraih Al A’jabah School Rehabilitation Projects √ X X • Solar panel batteries were placed inside classrooms, which could be a potential risk to students. YCST confirmed that they had agreed with the school management to build a wooden cabinet for the solar panels, but this was not yet implemented at the time of the evaluation teams’ visit to the school. • One of the solar panels was broken due to the wind. YCST provided additional panels to the contractor to replace the ones with defects. YCST confirmed that 33 | YCST FINAL EVALUATION REPORT USAID.GOV Community Service Delivery Project Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes the broken panels were replaced by the contractor after the evaluation team’s visit. • YCST team was responsible for rehabilitating the roof of the school. According to the facility staff there were problems with the way some of the initial rehabilitation work was done by the YCST contractor, leading to part of the roof collapsing. As a result, the YCST contractor had to redo some of the rehabilitation work, leading to delays in completion. The facility staff did not share any safety concerns as the rehabilitation work was completed during the period when the school was closed. Lahj/Ras Al-Arah Ras Al-Aarah Water Enhancement Project √ √ √ Although YCST did not install water tanks as part of their intervention in this facility, the water tanks were leaking. USAID.GOV YCST FINAL EVALUATION REPORT | 34 Community Service Delivery Project Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Lahj/Al-Madraiba Wa Al A’ara/Al￾Suraih Al Sharaj School Rehabilitation Project √ X √ Some solar panels were broken due to strong wind. YCST provided additional panel for the contractor for any defects. YCST confirmed that the broken panels were replaced by the contractor after the evaluation team’s visit. 35 | YCST FINAL EVALUATION REPORT USAID.GOV Community Service Delivery Project Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Lahj/Al-Madraiba Wa Al A’ara/Al￾Suraih Al Suraih Health Center Rehabilitation Project √ √ √ Although it is not within the scope of YCST, the solar energy system and the water tanks were not well maintained (e.g., water tanks not clean) at the facility. Abyan/Jayshan Jayshan Water Supply Enhancement Project √ √ √ USAID.GOV YCST FINAL EVALUATION REPORT | 36 OBJECTIVE 2: BUILDING CAPACITY The evaluation team relied on primary and secondary data to assess the sustainability of capacity-building efforts on conflict mitigation. The team also examined if and how the grantees continued to apply their learning. The team also explored potential challenges to sustaining capacity-building successes, for example, resource and funding constraints or lack of community or political authority buy-in. Key findings related to YCST’s capacity-building approach are summarized in the following. Grantees found the application process to be clear and transparent. All grantees were satisfied with the grant application process. Most referred to it as clear and transparent. The tenders were advertised on YemenHR website with clear guidelines on the application process. None of the grantees reported facing any issues or challenges with the application process. There was no clear capacity-building plan for grantees, as most grantees did not receive any formal trainings. YCST had no plan to build the capacity of grantees. Although most grantees noted that their partnership with YCST strengthened their organizational capacity, most did not receive formal trainings from YCST. The few grantees that did receive training, e.g., on governance and data collection or from Build Up, had ongoing projects. Previous grantees from Phase I had not received any type of training. The second round of grantees had no role in designing activities. The second round of grantees had no role in designing YCST activities. All of them reported that they only implemented the activities that they had agreed on with YCST. Innovation sub-fund grantees were reduced from seven to three after being provided with the technical training from Build Up. One innovation sub-fund grantee mentioned that of the seven organizations that received training and guidance from Build Up, only three obtained a YCST grant. Build Up reported that this modification was due to the need for closeout of Build Up’s mentorship program of the seven organizations. “In general, the teams are still in the dark about their fate; they still have ideas and are waiting to see what they can do with them. Build Up could help them fundraise and absorb them into other work, but it is a missed opportunity and led to a lot of tension between members of these communities and YCST, as there were promises that in the end were not fulfilled. Updates being given to them were not clear enough, not frequent enough … this impacts the trust between a big actor/donor and small teams and communities in Yemen.” – Build Up Staff There was little or no linkage between the grantee activities (Objective 2) and PACE (Objective 1), and stabilization (Objective 3) activities. Data showed that there was no clear link between YCST and partners activities. For example, not all grantees engaged CEC/ACC members in their activities, and some were not aware of the YCST￾established committees. Similarly, grantees interviewed were not engaged in any YCST activities, and when asked if they believe YCST-supported activities reduced conflict, most were only able to discuss their own activities and their contributions to peace building and conflict resolution. None of the 37 | YCST FINAL EVALUATION REPORT USAID.GOV grantees were able to reference other YCST activities such as CECs/ACCs and service delivery projects, and whether and how they contributed to conflict mitigation. Grantees shared several recommendations to improve the YCST approach. Grantees recommended several changes to the YCST program including: • Two grantees mentioned the need for a network of YCST partners to share knowledge and experiences. • A couple of grantees underscored the importance of avoiding delays in financial payments as they had caused activities to be delayed. • Two grantees suggested engaging grantees in designing grant activities because they may have innovative ideas that can be effective in the communities. • One grantee suggested that YCST should orient grantees to the broader project. SUSTAINABILITY Key findings related to sustainability are summarized in the following. Grantees expect to continue to apply what they have learned through their engagement in the YCST project. Almost all interviewed grantees (8/11) confirmed that they plan to continue using what they learned after YCST activities ended. Grantees referenced that they would continue applying M&E systems, and that they planned to use the skills they gained for future peace-building projects. Grantees also mentioned that the partnerships and improved relationships they now have in their communities will also be useful in the future. A considerable number of grantees lack the resources to continue to apply what they have learned. Grantees provided more mixed responses when asked if they had the resources they needed to continue to apply the approaches they had learned. While a few confirmed they did have the resources to continue, five others stated that they lacked resources or needed additional funding to continue implementing these activities. “It all depends on funding opportunities.”–Grantee OBJECTIVE 3: PROMOTING PEACE To assess the likely sustainability of stabilization efforts and outcomes, the evaluation team gauged community perceptions related to the responsiveness of government. The evaluation team also assessed the collective competence of the ACCs and other stakeholders to come together to address/mitigate future conflicts. The evaluation team also examined whether the support provided by the project was sufficient to ensure that local authorities continue to effectively communicate with and remain accountable to their constituents. The evaluation team also reviewed existing practices related to actual and planned maintenance of facilities, soft skill trainings, and evidence of resource/revenue generation for maintenance of facilities. Key findings related to YCST’s stabilization approach are summarized in the following. USAID.GOV YCST FINAL EVALUATION REPORT | 38 Most stakeholders and beneficiaries who were aware of YCST mentioned that the YCST activities had not contributed to any conflicts in the community. Most local leaders in stabilization communities (9/12) were not aware of instances in which the YCST support contributed to creating conflicts. All but one local leader (11/12) stated that the project had not undermined or negatively affected local authorities or leadership structures in their communities. Additionally, almost all local leaders (9/12) noted that the project’s activities had not caused negative impacts in their communities. The three who described negative impacts stated that it caused tension with areas that had not been targeted. None of the facility staff in the stabilization communities reported instances or situations where the project contributed to creating conflicts between community members or other communities that did not receive assistance. “The school is located in a central location surrounded by 15 areas. This leads to some sort of cohesion to exist between people because all people gather in this area..” –School Staff None of the community members who were aware of YCST were aware of any instances in which the YCST support to their community contributed to creating conflicts between community members or with other communities. Most ACC members found the YCST activities to be relevant to the community needs. Most ACCs (33/40) found the YCST activities to be effective and relevant to the community needs. However, YCST activities did not always address communities most pressing needs. One ACC in Khanfar reported that the park rehabilitation project YCST implemented in their community was not as important as rehabilitating the sewage networks. Some ACC members had more engaged and involved roles than others. ACC members had different level of engagements within the committees. For example, seven female ACC members (4 in Khanfar and 3 in Zinjibar) reported that they did not have an active role in the committees. Most YCST-supported facility staff were satisfied with the YCST implementation approach and rehabilitation work. Facility staff had positive views about the planning and implementation approach. They explained that YCST conducted effective needs assessments at the outset and conducted follow up and expert supervision. All facility staff were also satisfied with the rehabilitation work, and all reported that the planned outcomes were achieved. Despite their overall satisfaction, facility staff also highlighted several issues and challenges. Some facility staff identified issues with the rehabilitation work, including poor quality work. For example, Al Khair school staff reported that there were cracks in the ceiling and Ja’ar park staff reported that the welding work was not of good quality. In Al Khawlah school, staff reported that the ramp’s steel guard broke, and in Al Ma’afer hospital staff reported that the main cesspit was overflowing. 39 | YCST FINAL EVALUATION REPORT USAID.GOV Five facility staff reported facing challenges during the implementation process including delays in implementation or technical specification issues. While most subcontractors were satisfied with the ACCs, one highlighted ACC interference and delays. One subcontractor mentioned that the ACC members intervened in technical work without having the qualification. “In Al-Suraih school, they all of a sudden decided that we should stop the work without any justifications. Although the engineer [GC engineer] decided we should continue, as we had implemented in accordance with the designs and specifications. This caused some delay.” –Subcontractor Only 17 percent of community members were aware of YCST activities in their community, and most of them had not participated in any of the activities. Only 17 percent (21/125) of the respondents were aware of YCST activities in their community. Most of those who were aware of YCST activities (62 percent or 13/21) were aware of service delivery projects. Close to 10 percent (2/21) were also aware of community engagement activities and ACC formation and activities. Most of those who were aware of YCST activities within their communities (71 percent or 15/21) had not participated in any activity. A small percentage had participated in the community meetings (14 percent or 3/21) and identification and selection of ACC members (less than 5 percent or 1/21). While a small percentage of community members were aware of YCST activities, those who were aware shared a very positive appraisal of these activities and their effectiveness. The one exception to this was the dissatisfaction with ACC’s performance related to representing youth and women. Most of those who were aware of YCST activities within their communities (71 percent or 15/21) were satisfied with the activities. Only 14 percent (3/21) reported being dissatisfied. Most of those who were aware of YCST activities within their communities (76 percent or 16/21) believed that YCST activities addressed important community needs. Ten percent (2/21) of the respondents disagreed. Only 11 percent (14/125) of the community members were aware of ACCs in their community. Of those who were aware of ACCs, 21 percent (3/14) had participated in selecting ACC members, and around a third (36 percent or 5/14) had participated in identifying and addressing stressors in their community. Most of those who were aware of ACCs (86 percent or 12/14) believed that ACCs effectively captured community feedback. Only 14 percent (2/14) described the ACCs as being ineffective at capturing community feedback. Most of those who were aware of ACCs were satisfied with their performance (86 percent or 12/14), and most (57 percent or 8/14) noted that their local ACC was active in their community (see Figure 7). Satisfaction with ACC performance varied across ACC functions: 57 percent were satisfied that the ACC represented community interests, and 64 percent were satisfied that the ACC had strengthened relationships between community members. About 50 percent (7/14) of the respondents were satisfied with the ACCs ability to represent women, and about 43 percent (6/14) were dissatisfied. Opinions were split when it came to ACC’s performance selecting community projects. USAID.GOV YCST FINAL EVALUATION REPORT | 40 Notably, more respondents were dissatisfied than satisfied when it came to ACCs representing youth and addressing conflicts. While most local leaders were aware of YCST, they had not participated in any YCST activities. All but one local leader in stabilization communities were familiar with the YCST program, and most of them were able to describe the service delivery projects that had taken place in their communities. Most local leaders in the stabilization (8/12) communities stated that they had not participated in any YCST activity, while the rest mentioned participating in focus group discussions and the needs assessment and prioritization process. SUSTAINABILITY Key findings related to sustainability are summarized in the following. ACC members and YCST staff held mixed views of the likelihood of ACC’s sustaining on their own. Half of the ACC members (20/40) reported that the ACC mechanism is likely or somewhat likely to sustain while 16 reported that it is not likely to sustain without financial support and additional capacity building for ACC members (see Figure 8). 86%,12 64%, 9 57%, 8 50%, 7 43%, 6 36%, 5 29%, 4 7%, 1 14%, 2 7%, 1 14%, 2 14%, 2 14%, 2 7%, 1 22%, 3 29%, 4 43%, 6 43%, 6 43%, 6 43%, 6 7%, 1 7%, 1 7%, 1 14%, 2 Overall satisfaction with ACC performance Strengthening relationships between community members Representing community interests Representing the interests of women Selecting important community projects Addressing conflicts Representing the interests of youth Satisfaction with ACC Functions Satisfied Neither satisfied nor dissatisfied Dissatisfied Don’t know FIGURE 7: SATISFACTION WITH ACC FUNCTIONS 41 | YCST FINAL EVALUATION REPORT USAID.GOV YCST project staff presented mixed opinions around whether ACCs will remain functional after the YCST project. The respondents noted that limited financial resources will continue to cause challenges related to sustainability in communities, even if there is a will or commitment from ACC members to address community needs. ACC members identified capacity-building and financial resources as key needs. A significant number of ACC members (14/40) did not receive any type of training, and eleven reported the need for additional capacity-building trainings to increase their knowledge in conflict resolution and other relevant topics to respond to community needs. ACC members also reported requiring additional financial resources to help them respond to community needs. Most facility staff indicated that they lack the resources to maintain the facilities. Most facility staff indicated that they have little or no financial resources to maintain the facilities and explained that the government does not provide any support to allow them to maintain the facilities. The few who reported that they do have resources indicated relying on support from organizations such as the World Food Programme and UN International Children’s Emergency Fund. Field observations also highlighted instances of the supported facilities not being properly maintained. Field observations conducted by the evaluation data collection team at a sample of facilities also provided evidence of several rehabilitated works not being properly maintained. For specific observations and details, please reference the following chart summarizing the observation findings. 16, 40% 14, 35% 6,15% 4, 10% Unlikely Likely Somewhat likely I don't know/No answer Likelihood that ACCs will Continue to Meet and Function after the YCST Program Ends (n = 40) FIGURE 8: ACC PERCEPTIONS OF ACC SUSTAINABILITY USAID.GOV YCST FINAL EVALUATION REPORT | 42 OBSERVATION FINDINGS Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Abyan/Khanfar/Ja’ar Ja'ar Park (Green Zone) Rehabilitation Project √ X X • There were some broken playground items such as the swings’ seats. • Some playground items were welded to work again. • A part of the old fence was still in place, which could be a safety issue as it could fall. • Latrines were out of order, the sewage pipes were clogged, and the sink tap was not working. YCST confirmed that maintenance works were implemented after the visit to ensure functionality of the facilities Abyan/Khanfar/Ja’ar Haydera Football Field Rehabilitation Project – Haydera Football Field √ X X • The steel fence light type and some of the sides were damaged. • The playing field was uneven. 43 | YCST FINAL EVALUATION REPORT USAID.GOV Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes • Although not within the rehabilitation works implemented by YCST, the stairs were high, and there were some cracks in them. Abyan/Zanjibar Khalwa School Rehabilitation Project √ √ √ - USAID.GOV YCST FINAL EVALUATION REPORT | 44 Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Abyan/Zanjibar Sahat Al Shohada Park (Green Zone) Rehabilitation Project √ √ √ There were some damaged playground items such as the plastic chairs and swings. Some items were completely broken including the swings. Abyan/Zanjibar Khadija School Rehabilitation Project √ √ √ There were cracks in the concrete observed. 45 | YCST FINAL EVALUATION REPORT USAID.GOV Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Taiz/Al-Ma’afer/Al￾Suwa’a Al Ma'afer Rural Hospital Sanitation System Rehabilitation Project √ X X • The solar energy system was not working, although the panels were in good condition and in a suitable place. YCST confirmed that they have followed up on this to ensure that the system is fully functioning. • There was a staircase at the hospital’s entrance that does not allow the patients’ beds or stretchers to be moved to and from the ambulance and hospital rooms. • The new septic tank has been built inside the hospital’s courtyard as approved by the District Health Officer and hospital management according to YCST, though it should be far from there. USAID.GOV YCST FINAL EVALUATION REPORT | 46 Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Taiz/Ash Shamayatain/Al￾Azaez Al Khair School Rehabilitation Project √ X √ • Lights and light switches were broken as maintenance work is conducted during the summer break while the visit took place during the ongoing school term. • Some classroom walls had cracks, scratches, and paint blistering. YCST confirmed that maintenance, painting works take place during the summer break. 47 | YCST FINAL EVALUATION REPORT USAID.GOV Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes Taiz/Ash Shamayatain/Al￾Azaez Abdulaziz School Rehabilitation Project √ √ √ - Taiz/Ash Shamayatain/Al￾Azaez Al Mansoora Health Center Rehabilitation Project This project was cancelled due to quality issues with the contractor. Taiz/Ash Shamayatain/Al￾Azaez Al Shaheed Yassin School Rehabilitation Project √ X X The facility lacked access to water and the solar energy system was not working. USAID.GOV YCST FINAL EVALUATION REPORT | 48 Community Service Delivery Project Name Functioning Facility Rehabilitation Work in Good Shape Signs of Safety Measures Observation Notes 49 | YCST FINAL EVALUATION REPORT USAID.GOV OBJECTIVE 4: ENHANCING OUTCOMES Key findings related to YCST’s learning and adapting approach are summarized in the following. While YCST staff discussed how the PACE team adapted its approach to the Yemen context, the extent of the adaptation was less clear. One member of YCST’s management team noted that in Jordan, the PACE approach operated as a purely development program targeting social cohesion. YCST adapted the approach in Yemen to focus on conflict mitigation instead of delivering development interventions. Another YCST staff member mentioned that the PACE approach and methodology was also adapted to include training on conflict resolution, analysis, and mitigation. At the same time, project staff also mentioned that the main criteria for site selection included selecting areas where there were no active conflicts. The PfG staff also mentioned that they had held discussions with the YCST team to ensure that the new PACE manual would fit the context. As per the PfG staff, while the PACE manual was “slightly adapted” to address Yemen’s conflicts and instability, the ten steps of the PACE process remained the same. The project did not make any significant adaptations in its approach to working in Phase 2 communities. Several YCST staff members, including community mobilizers, a governorate coordinator, PACE officers, and members of the MEL team as well as PfG staff noted that they did not necessarily change their approach for PACE between Phase 1 and Phase 2. Other than remotely conducting training, PfG staff explained that the training approach did not change between phases. However, as per secondary data, the YCST team did make some changes to the PACE manual based on lessons identified during a lessons learned workshop and the project staff mentioned that they had a better understanding of the dynamics within targeted communities. A YCST staff member highlighted that in some instance the YCST team streamlined the PACE steps by identifying the priorities prior to training the CECs. “Normally we do the CEC training first and then identify projects to be implemented. In the second phase we identified projects and submitted them for approval and then we did the training because the approval process takes months and CECs get frustrated by waiting.” –Community Mobilizer A senior YCST staff member also described that in Phase 2 they chose to conduct the PACE conflict assessment and needs assessments at the same time instead of sequentially, saving the team three to four weeks. The YCST team identified several opportunities to improve the community selection process, though the project was not always able to apply this learning to the selection of subsequent communities. Participants at the learning workshop from Year 1 suggested several changes for community selection in Year 2. YCST tried to use the community profiling process as well as closely coordinate with the Social Welfare Fund to specify communities; however, the YCST learning tracker (in August 2021) mentions that USAID.GOV YCST FINAL EVALUATION REPORT | 50 selection criteria should be developed based on a more detailed knowledge and understanding of context, demographic, and geographical characteristic of communities, and focused on a small community size where the project has learned it can have a deeper, more sustainable impact. CONCLUSIONS • While community engagement was a key focus of the YCST development approach, YCST activity implementation approach did not result in widespread community awareness and engagement. Most of the community members were not aware of YCST activities in their community, and most of them had not participated in any of the activities. In contrast, most of the stakeholders engaged by YCST were satisfied with the activity implementation approach. • While there were some exceptions, for the most part, YCST applied a conflict sensitive approach to activity implementation that ensured that they were not undermining or negatively impacting the local authority structures and not contributing to conflict or tension between beneficiaries and nonbeneficiaries. • YCST activities lacked a clear sustainability plan, which has impacted the likelihood of sustaining the activities (e.g., CECs, ACCs, maintenance of facilities supported by YCST) and overall project outcomes. Even among those who reported that YCST-supported CECs and ACCs will continue to meet or the grantees will continue to apply what they have learned, a significant number suggested that they will be able to do so if they have additional resources including financial support. Visits to YCST-built and -rehabilitated facilities also indicated that the community and YCST mechanisms are not actively engaged in the upkeep and maintenance of these facilities. 51 | YCST FINAL EVALUATION REPORT USAID.GOV CROSS-CUTTING FINDINGS—SERVICE DELIVERY PROJECTS’ USE, ACCESS, AND INCLUSION CONSIDERATIONS AND ACHIEVEMENTS This section includes cross-cutting findings related to service delivery projects’ use, access, and inclusion considerations and achievements. SERVICE DELIVERY PROJECTS—USE AND ACCESS A total of 59 percent (137/232) of community members were aware of at least one YCST facility in their community (see Figure 9). Of those who were aware of a facility, 60 percent (82/137) had used at least one of the facilities supported by the YCST project (see Figure 10). Satisfaction with the quality of services provided by YCST varied across communities (see Figure 11). Most community members (73 percent, 100/137) reported that facilities supported by YCST had a positive impact on their communities (see Figure 12). FIGURE 9: AWARENESS OF YCST FACILITIES FIGURE 10: USE OF YCST FACILITIES 59% 41% 0% 10% 20% 30% 40% 50% 60% 70% Aware of a facility Not aware of any facility Awareness of YCST Facilities n = 232 60% 40% 0% 10% 20% 30% 40% 50% 60% 70% Used a facility Did not use any facility Use of YCST Facilities by Community Members n = 137 USAID.GOV YCST FINAL EVALUATION REPORT | 52 FIGURE 11: SATISFACTION WITH QUALITY OF SERVICES PROVIDED BY YCST FIGURE 12: PERCEIVED IMPACT OF YCST FACILITIES ON COMMUNITIES Around 43 percent (100/232) of the community members reported that their access to services (YCST￾supported) had improved over the last year, while 12 percent (27/232) reported that it had worsened (see Figure 13). Close to a third (67/232) reported that it had remained the same. Access to YCST￾supported services varied within and across communities. 93% 66% 56% 50% 47% 46% 23% 23% 7% 9% 25% 7% 3% 6% 37% 47% 25% 10% 9% 3% 10% 19% 43% 40% 39% 37% 20% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Jayshan Al Azaez Al Suraih Ja'ar Al Suwa'a Zinjibar Abudbari Ras Al Aarah Satisfaction with the Quality of Services Provided by YCST per Community n = 232 Satisfied Dissatisfied Neither satisfied nor dissatisfied Don’t know 73% 9% 8% 6% 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% Had a positive impact Had no impact Had a negative impact Don’t know Some had positive/negative and some had no impact Impact of YCST Facilities on Their Communities n = 137 53 | YCST FINAL EVALUATION REPORT USAID.GOV FIGURE 13: ACCESS TO YCST SERVICES PER COMMUNITY YCST APPROACH AND OUTCOMES—INCLUSION CONSIDERATIONS AND ACHIEVEMENTS Despite the challenging context, the YCST team tried to include women across all its CECs and ACCs. While this was not always possible, YCST was still able to include female members in 15 out of 19 CECs in PACE communities and in all four ACCs set up in the stabilization communities (see Figures 14 and 15). 60% 57% 55% 47% 44% 40% 27% 23% 13% 17% 15% 22% 44% 27% 53% 34% 27% 30% 6% 6% 20% 43% 26% 25% 6% 33% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Jayshan Ja'ar Zinjibar Al Azaez Al Suraih Al Suwa'a Abudbari Ras Al Aarah Access to YCST Services per Community n = 292 Improved Remained the same Worsened Don’t know 10 14 17 11 10 12 15 15 9 8 15 10 7 5 9 4 7 7 6 10 6 5 6 4 6 7 5 7 9 5 8 4 3 3 Breakdown of Male and Female CEC Members per Community Males Females FIGURE 14: MALE/FEMALE BREAKDOWN OF CECS BY COMMUNITY USAID.GOV YCST FINAL EVALUATION REPORT | 54 FIGURE 15: MALE/FEMALE BREAKDOWN IN ACC MEMBERS BY COMMUNITY In general, males and old (>30 years old) community members were more aware of YCST and had a greater rate of participation in and satisfaction with activities and outcomes (see Figures 16 and 17). 37 31 32 23 14 11 8 7 Ja'ar Zinjibar Al Suwa'a Al Azaez Breakdown of Male and Female ACC Members per Community Male Female 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Awareness of YCST activities Awareness of YCST service delivery projects Awareness of CECs/ACCs Participation in YCST activities View that YCST addressed important community needs YCST service delivery project had a positive impact Improved level of trust among community members Improved relationships between community and their leaders Improved existing conflicts resolution mechanisms Community members can address their own problems YCST Activities—Male and Female Response Comparison Male Female FIGURE 16: MALE AND FEMALE COMPARISON OF RESPONSES 55 | YCST FINAL EVALUATION REPORT USAID.GOV The more educated community members (secondary or more) were more aware of YCST activities and viewed them as having more positive impact. The less educated or illiterate community members (primary or less) were more aware of the service delivery projects and were also considerably more likely to view these projects as addressing important community needs (see Figure 18). FIGURE 17: AGE COMPARISON 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Awareness of YCST activities Awareness of YCST service delivery projects Awareness of CECs/ACCs Participation in YCST activities View that YCST addressed important community needs YCST service delivery project had a positive impact Improved level of trust among community members Improved relationships between community and their leaders Improved existing conflicts resolution mechanisms Community members can address their own problems YCST Activities—Level of Education Response Comparison Primary or less Secondary or more FIGURE 18: LEVEL OF EDUCATION COMPARISON 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Awareness of YCST activities Awareness of YCST service delivery projects Awareness of CECs/ACCs Participation in YCST activities View that YCST addressed important community needs YCST service delivery project had a positive impact Improved level of trust among community members Improved relationships between community and their leaders Improved existing conflicts resolution mechanisms Community members can address their own problems YCST Activities—Under and Over 30 Years Response Comparison Under 30 Over 30 USAID.GOV YCST FINAL EVALUATION REPORT | 56 A slightly higher percentage of community members in the urban communities reported improved access to YCST services than in the rural communities (see Figure 19). Similarly, a greater percentage of urban community members reported that these facilities had a positive impact on their community (see Figure 20). FIGURE 19: ACCESS TO YCST SERVICES—URBAN/RURAL COMMUNITIES FIGURE 20: YCST SERVICE DELIVERY IMPACT - URBAN/RURAL COMMUNITIES Compared to community members in urban communities, community members in the rural communities were more aware of the YCST-supported facilities and had used or benefitted from these facilities (see Figures 21 and 22). 46% 41% 28% 30% 7% 15% 19% 14% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Urban Rural Access to YCST Services—Urban/Rural Communities Rural (n = 139); Urban (n = 93) Improved Remained the same Worsened Don’t know 81% 70% 10% 9% 5% 9% 2% 5% 2% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Urban Rural Impact on the Community—Urban/Rural Communities Rural (n = 95); Urban (n = 42) All facilities had a positive impact It had no impact It had a negative impact Some facilities had positive/negative and some had no impact Don’t know 57 | YCST FINAL EVALUATION REPORT USAID.GOV FIGURE 21: AWARENESS OF YCST FACILITIES—URBAN/RURAL COMMUNITIES FIGURE 22: USE OF YCST FACILITIES—URBAN/RURAL COMMUNITIES CONCLUSIONS • While most community members (59 percent) were aware of one or more YCST-supported facilities, many of them (41 percent) were not aware of any of the YCST-supported facilities. Access to YCST-supported services varied within and across communities, and satisfaction with the quality of services provided by YCST also varied across communities. • While the YCST activity made a very conscious effort to make its activities inclusive, female, and younger community members were less aware, less engaged in, and less satisfied with the YCST activities. However, compared to the urban communities, community members in rural communities were more aware and had used or benefitted from these facilities. 45% 68% 55% 32% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Urban Rural Aware of YCST Facilities—Urban/Rural Communities Rural (n = 139); Urban (n = 93) Yes No 50% 64% 50% 36% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Urban Rural Use of YCST Facilities—Urban/Rural Communities Rural (n = 95); Urban (n = 42) Yes Not aware of any facility USAID.GOV YCST FINAL EVALUATION REPORT | 58 EVALUATION QUESTION THREE: EFFECTS OF CHALLENGES OR CONSTRAINTS FACED BY YCST What operational challenges or constraints did YCST face, and how, if at all, did these challenges affect outcomes and prospects for sustaining results? Next we share operational challenges and constraints faced by YCST and how they affected YCST activities, outcomes, and prospects for sustaining results. COVID-19 The COVID-19 pandemic and its impact on YCST’s implementation was by far the most frequently referenced challenge across all respondent types, who described it as the biggest challenge impacting the project. GC, USAID, and YCST project and partner staff consistently discussed the negative impact the pandemic had on implementing the program as planned, the challenges of conducting community-level programming remotely in the Yemeni context, and the measures that the project took to adapt. At the beginning of the pandemic, in-person activities were suspended, including any kind of meetings, trainings, and service delivery in communities. Respondents from both PfG and Build Up noted that the poor internet connectivity in Yemen limited the effectiveness of remote trainings and calls at first. Additionally, the ability of project staff to engage with PACE and stabilization communities virtually was very limited, as one USAID staff noted, Community mobilizers noted that due to their absence “the community members started to lose faith in the organizations for the long delay.” Eventually, one YCST staff highlighted that FMF was able to send its staff to the field, as they had fewer restrictions as a local partner, and many of their staff were hired from the project’s communities. At the community level, both ACC and CEC members described how COVID-19 interrupted the work in their communities for a long period of time and discussed the protection measures that project staff were required to take to resume meetings and other community engagement activities. One CEC member stated that the period during which activities were paused “the target communities lost the trust towards us and the CECs,” while another stated that while activities were paused for over a year it created anger against the CECs and the program team. Local authorities echoed this view, describing communication gaps from YCST staff that occurred during the first phase of the pandemic and the delays that ensued. A second wave of COVID-19 hit in March of 2021, which resulted in the project suspending Phase 2 entry activities in communities, including a significant delay to the Phase 2 baseline study, as well as the direct and co-implementation of soft activities in Phase 1 communities. CHANGES TO THE PROJECT SCOPE The evaluation team spoke to respondents from both GC and USAID who highlighted that the changes to the project scope limited some of YCST’s gains and undercut prospects for sustainability. Several respondents described that there was uncertainty from USAID around their plans for development programming in Yemen and subsequently the project’s funding level, which became an operational challenge in the last year of the project. While the team had projected its spending on programming to 59 | YCST FINAL EVALUATION REPORT USAID.GOV the ceiling of their contract and moved forward with expanding Phase 2 entry activities in Hadramout, in the fall of 2021 USAID informed them of a significantly reduced budget for the remainder of the project. SECURITY CHALLENGES Respondents at all levels of engagement on the project consistently referenced security as a key challenge to implementing the YCST program. Over the life of the YCST project, insecurity worsened in Yemen, which impacted programming for long stretches of time in certain areas or daily in other communities. For service delivery projects, the dynamic security situation often impacted timelines, as often the YCST technical team would not be able to travel to communities to conduct quality control visits, requiring subcontractors to wait before they could move forward with their work plans. The complex operating environment in Yemen included significant security challenges that caused the suspension of field activities in Abyan and Aden in the fall of 2019, causing delays to activities. In Abyan this caused a reduction in access for two months, and lesser delays to PACE and stabilization activities in Aden, Lahj, and Taiz. FINANCIAL CHALLENGES The limited budgets allocated to service delivery projects within communities were a commonly referenced challenge by project staff and stakeholders at the community levels as well as by the USAID and GC HQ team and in lessons learned reports. For many PACE and stabilization communities, the projects proposed by CECs and ACCs to address their most prioritized need required significant financial resources for water, sanitation, or electricity services that were not matched by the project’s budget. As a result, communities were instructed to move down the list of their priorities until they identified a project that the budget could accommodate. STAFFING AND CAPACITY CHALLENGES Staffing issues related to frequent changes in YCST’s leadership and local staff recruitment and turnover in general were also widely acknowledged as a constraint to the project’s effectiveness. Respondents from USAID, GC HQ, and YCST highlighted that the project’s four different chiefs of party brought inherent problems to how the project was implemented and the pace of programming, while citing the welcome change the current chief of party brought in steadying the project operationally. Respondents also described the challenges they faced in programming because of local staff turnover and capacity issues, specifically in stabilization programming. CHALLENGES RELATED TO OBTAINING PERMISSIONS FROM AUTHORITIES The process of obtaining permissions from national, governorate, and local authorities to implement the YCST project in Yemen impeded the team’s progress considerably and was cited as a challenge by the project’s senior leadership and USAID staff. One respondent from USAID mentioned that the project and USAID made assumptions around Yemen’s governing context that were flawed, and ultimately wasted YCST’s time in negotiating to gain permissions to operate in the north to no effect: USAID.GOV YCST FINAL EVALUATION REPORT | 60 Project leadership discussed facing challenges and subsequent implementation delays gaining approvals in the south as well. GENDER AND INCLUSION CHALLENGES While the YCST project was very intentional in ensuring its programming was inclusive of gender and youth, requiring 30 percent of CEC members be women for example, several people emphasized the challenges they faced encouraging women’s participation in project activities. Several project staff noted that involving women and youth in decision-making for communities was opposed by traditional and religious authorities. Members of YCST’s MEAL team explained that the team encountered the greatest resistance to engaging women in rural areas, particularly in Abyan. Quarterly and annual reporting consistently highlighted that programmatically YCST faced challenges related to earning the buy-in of local communities around participation of women and youth in CECs, particularly in PACE communities in Abyan. INTERNAL CHALLENGES AND CONSTRAINTS In addition to external challenges faced by YCST, stakeholders also highlighted challenges internal to the YCST implementation approach including challenges related to communication, coordination, and delays. COMMUNICATION AND COORDINATION Respondents from USAID, GC HQ, YCST staff, its partners, and those at the community level described the impact communication challenges had on programming and the YCST’s team cohesion. A USAID staff member described that for at least the first year of the project the team operated in siloes, and it was not until the pause and reflect event that they understood how to work as a cohesive team. One GC staff attributed certain coordination challenges to the lack of day-to-day supervision in the country from senior leadership in Yemen, noting that “when the senior leaders are not close by, it can create problems.” In terms of partner collaboration, both PfG and Build Up staff described slow responsiveness from YCST staff that limited how much they were able to accomplish. One respondent from Build Up relayed that communication with grantees was not consistent and that “updates being given to them were not clear enough, not frequent enough.” One person explained they were under the impression that several grantees were still in the dark about the fact that they would not be moving forward and receiving a grant. The lessons learned tracker also cited challenges related to coordination and management, specifically in bringing different YCST teams out of their silos to work together as one team. GENERAL DELAYS Respondents across all levels of the project, including numerous USAID and YCST staff as well as grantees, cited delays caused by administrative and procurement processes and the long community entry processes as key challenges. Respondents cited the unusually long community entry process and 61 | YCST FINAL EVALUATION REPORT USAID.GOV subsequently long approval and procurement processes for the service delivery projects that caused community members to become impatient. One YCST staff highlighted, “After asking ACCs to do the rapid assessments on tight timelines, it took almost 2 years to start implementation in some cases because of approvals and internal processes, then implementation itself took only 6 months. It was difficult to explain to community members that they needed to do a rapid assessment, but then it would take 1 year plus to get through all of the USAID/YCST processes and approvals. This created tension with the communities and local authorities.” Respondents also highlighted the significant delays the project had in getting the grants component off the ground and fully implementing it, describing the vetting and other administrative procedures as very time consuming. In describing this, one grantee told the evaluation team that the grants training they attended was in 2020, but the project was not launched until 2022. The midterm performance evaluation highlighted delays that occurred due to the protracted start-up time frame. The Year 1 Learning Workshop report described those delays in obtaining local authority and USAID approvals as well as in carrying out stabilization assessments negatively impacted the program in moving forward to key milestones. CONCLUSIONS • When assessing the overall performance of the YCST activity, it is very important to consider the significant operational challenges and external factors that the project encountered. The COVID-19 pandemic, the worsening security context in Yemen, and the challenges the project team faced in obtaining permissions to enter communities all seriously impacted the project’s ability to implement its activities as anticipated and were largely beyond the team’s control. While the project maintained a flexible approach and adapted its activities, these and challenges related to the change in program scope likely impacted the activity’s potential performance. EVALUATION QUESTION FOUR: LESSONS LEARNED FOR FUTURE INTERVENTIONS As a result of this evaluation, what “lessons learned” or successful models should USAID consider for follow-on community-level interventions? Next we summarize key lessons learned and successful models to be considered for future community￾level interventions. YCST BEST PRACTICES TO BE CONSIDERED IN FUTURE PROGRAMMING PARTICIPATORY APPROACH TO PRIORITIZING COMMUNITY NEEDS WAS WIDELY APPRECIATED YCST’s participatory approach to allowing members of the community to identify their own priorities was cited frequently by respondents at all levels of engagement with the project as a key strength. Most CEC members (46/51) and six ACC members spoke about their satisfaction with this approach, noting that community members were happy to have their voices heard. USAID.GOV YCST FINAL EVALUATION REPORT | 62 DEVELOPMENT PROGRAMMING WAS VIEWED AS A WELCOME CHANGE BY SEVERAL RESPONDENTS Another key strength raised by multiple YCST staff as well as by GC and a grantee was that the project had a development focus as opposed to the humanitarian assistance programming that is much more prevalent in Yemen. Three respondents cited the newness of YCST’s development approach. Two people specifically highlighted the learning opportunity that the project presented in documenting the process and feasibility of carrying out peace-building activities in a humanitarian context. FOCUS ON PROJECT LEARNING WAS VALUABLE Stakeholders from USAID, GC HQ, and the YCST team spoke positively about the fact that learning and adaptation was built into the project design under objective four, highlighting its usefulness for programming. A few respondents cited that more could be done to ensure that lessons learned from implementation informed substantive program adaptations. SERVICE DELIVERY PROJECTS WERE RELEVANT TO YCST COMMUNITY NEEDS The service delivery projects delivered to YCST communities were viewed as relevant and appreciated by a wide range of stakeholders interviewed and as seen in project reporting. While the service delivery component appears to have been the most prominent feature of the project, more so than conflict mitigation and cohesion, stakeholders expressed that across Yemen people are in serious need of basic services and that YCST projects contributed to addressing priority service gaps in the project’s communities. LESSONS LEARNED PROJECT TIME FRAME WAS TOO AMBITIOUS FOR THE PROJECT SCOPE A key lesson learned cited by individuals from USAID, GC HQ, YCST senior management, and the YCST MEL team was that the project needed a longer time frame. One respondent from GC acknowledged that the project design’s time frame was unrealistic given Yemen’s complex operating context and the extended community entry process needed to implement PACE and stabilization programming: USAID staff also weighed in on the time frame, indicating that the project needed a five-year window to really show results. One USAID stakeholder noted that while the project achieved visible results in communities, the short time frame of the programming in communities potentially limited the long-term sustainability of project outcomes. SERVICE DELIVERY FOCUS DETRACTED FROM CONFLICT MITIGATION AND SOCIAL COHESION OUTCOMES At USAID, staff suggested that the service provision was more of a focus for the project than the peace￾building outcomes, and that connecting small infrastructure projects to conflict mitigation was a design flaw. Another agreed, 63 | YCST FINAL EVALUATION REPORT USAID.GOV “I felt as though I didn’t understand YCST. That somehow, I had missed some crucial bit of info about how this methodology connected to track three, connected to stabilization, connected to mitigating conflict at the local level … that linkage was not clear, and it seemed very much that the program was more focused on small infrastructure projects—building a well here, electrification projects, community park building, but it failed to make actual connection between the service they were providing and peace building and conflict stabilization.” The lessons learned tracker also acknowledged this challenge, noting that YCST communities and project staff were very focused on the “hard” component of service delivery projects rather than the “soft” interventions around conflict alleviation and social cohesion. THERE WAS LITTLE OR NO SUBSTANTIVE COLLABORATION AND COORDINATION BETWEEN YCST AND OTHER USAID OR DONOR ACTIVITIES There appeared to be interaction between YCST and some other USAID activities, but not much in terms of substantive collaboration and little to no coordination around stabilization work at the donor level. Several YCST and USAID staff explained that at the beginning of the project the YCST team conducted a mapping exercise during the community selection process to identify where USAID implementing partners (IPs) were working and to ensure that they did not duplicate any efforts as well as identify any peace-building and stabilization actors. Several respondents described that USAID explored the concept of an integrated approach as having potential to facilitate greater coordination between YCST and other partners, but it ultimately did not move forward. One USAID stakeholder described that the key aspect of the approach was that USAID would have fewer programs, which would operate across multiple sectors. While USAID eventually decided not to use this approach, one staff explained that as the development of the concept evolved, other USAID-funded health and education IPs were resistant to working with YCST because they were unwilling to make any changes to what they were doing or expand to include PACE communities. They noted “YCST was open to the idea, they were willing to do some coordination, sharing information and resources, but I can’t say that other programs were truly open to the idea.” Despite YCST efforts to collaborate with other NGOs in project communities, there was little coordination with local organizations outside of the grants component, and USAID staff acknowledged that coordination at the donor level is not something the development community does well. One USAID stakeholder cited failure of coordination at the donor level and an overall hesitancy on USAID’s part to put YCST on a pedestal and make connections because of their policy on stabilization, noting, “On the stabilization side none of the other donors were aware of our program or footprint, that we had a peace-building project. I do believe they touched base with the UNDP SIRI program, as well as I think a World Bank program, and then I believe they coordinated with Social Fund for Development … there were missed opportunities in the donor realm.” This finding was confirmed by the evaluation team when it reached out to other donors working in stabilization programming in Yemen, as a contact it reached out to from the German Embassy for Yemen stated they had never heard of YCST; this was also echoed in a discussion held with an EU Stabilization and Governance program manager. USAID.GOV YCST FINAL EVALUATION REPORT | 64 PACE AND STABILIZATION APPROACHES, AS IMPLEMENTED, WERE NOT VERY DIFFERENT FROM EACH OTHER Findings suggest that while there were certainly differences in the approach to ACC and CEC formation and site selection, the distinction between the activities that were carried out under the PACE and stabilization activities was less clear. One USAID stakeholder explained that the two approaches differed depending on the type of authority present in the community, “communities that have legitimate authorities tied into MoPIC or ROYG, those were stabilization. The ones that were informal without a structure or legitimate authority, those were community resilience cohesion programs … that was PACE.” YCST senior management detailed this further, describing that they chose to implement stabilization activities in the main cities or very populated communities where “legitimate” authorities existed, while PACE was implemented in rural areas where traditional authorities were present. For both types of communities YCST staff noted they coordinated and negotiated with MoPIC during the community selection process. Despite the differences in community selection, the differences in terms of the ACC and CEC composition were less clear, as both are composed of leaders, and PACE programming also has a significant component focused on engaging local authorities. One GC respondent acknowledged that between the approaches “there is a convergence. Community people also participate in the ACC. They are not versus; they can go together.” THERE WERE SEVERAL POTENTIAL FLAWS AND/OR LIMITATIONS IN THE YCST ACTIVITY DESIGN PROCESS THAT IMPACTED THE PROJECT AND ITS ACTIVITIES Several key lessons emerged from the challenges the team faced during implementation that can be traced back to potential flaws in the activity design process. 1. Local input was not adequately integrated into the activity design. USAID stakeholders discussed that due to funding pressures their original design process took place so quickly that it did not include any input from local foreign service national staff in Yemen until it was too late to substantively change anything. This was referenced as a missed opportunity for ensuring that the design of the project was more nuanced and reflected the realities of Yemen’s operating context. 2. Stabilization component was different from the overall activity scope and was added later. In reflecting on the activity design process, several USAID stakeholders referenced the internal politics around adding a stabilization component to the YCST project. Two respondents discussed the fact that the original design of the solicitation was not about stabilization, but instead about community cohesion and addressing community-level grievances. However, changes in USAID and U.S. State Department leadership as well as senior development advisors, and the added presence of a stabilization advisor at USAID, brought about an interest in piloting stabilization. One person noted that the addition of the stabilization component was a “political decision” due to the fact that the U.S. Government required USAID/Yemen to show what they were doing and how they were implementing against the stabilization assistance review framework. The addition of this component was viewed as interference by some and as complicating the scope of the project, including diverting and diluting its original focus. 65 | YCST FINAL EVALUATION REPORT USAID.GOV 3. The operating context in Yemen was not fully factored in the design. Several respondents also voiced that YCST’s design had not fully factored in the larger operating context in Yemen. A respondent from GC highlighted that the TOC and project design in general were a good way of looking at conflict resolution and peace building, but for this project they did not account for the bigger picture that war in Yemen was intensifying. Similarly, a USAID stakeholder questioned whether the PACE approach specifically was too much of a cookie cutter, off-the-shelf approach that did not account for the complexities of operating in Yemen. Respondents also noted that USAID staff had differing understandings of the project design, and this ultimately influenced attitudes toward the project. One USAID staff described that the design was built on the premise of “green spots” where the project could implement in communities with less conflict and improve cohesion. However, a respondent from GC noted that despite the fact that the project design indicated YCST would take a bottom-up approach to building cohesion and conflict mitigation, there appeared to be different opinions from within USAID around the type of conflict the project should have been addressing, whether local drivers of conflict or the larger effects of the national-level conflict. 4. Key stakeholders raised several concerns about the YCST TOC. Key stakeholders raised several concerns about the YCST TOC. A USAID staff questioned the validity of the TOC, indicating that it was convoluted, too ambitious given Yemen’s context, and that ultimately the project focused and reported on primarily the progress of service delivery projects rather than on its outcomes. While one USAID staff acknowledged that the service provision is a tremendous need and that gaps in services do cause conflicts, the project did not produce evidence to confidently show that the TOC holds. Among USAID, GC, and YCST senior management stakeholders, there was some agreement that at an individual level the TOC held for PACE communities where people were brought together to identify stressors and work toward a common goal. However, the linkages were less clear to the latter part of the TOC that references contributions to the national peace-building process. Project documents and reporting also suggest that YCST’s approach and the PACE model specifically focused on identifying and addressing community-level stressors that could lead to conflict, which has less of a connection to the larger context of the conflict in Yemen that is referenced in the second half of the TOC. Additionally, while the stabilization component involved local authorities in increasing their accountability to their communities, learning documents and ongoing project reporting did not connect or capture any increased capacity to respond to community needs to an ability to participate in national peace-building efforts, as suggested in the TOC. YCST Theory of Change (TOC) “If local capacity to collaboratively identify and address conflict stressors and threats to social cohesion is built, and if tangible community-level results are demonstrated, then the resilience of Yemeni communities and institutions will increase, local authorities who participate in systems of accountability strengthened by YCST will be better prepared to lead their communities’ participation in national peace efforts, and the effectiveness of models that can be replicated across the country will be demonstrated, laying the foundation for a peaceful and effective transition process.” USAID.GOV YCST FINAL EVALUATION REPORT | 66 CONCLUSIONS • Three-year activity duration was too short a time frame to apply the PACE model across several communities and governorates. COVID-19 complicated this further, as did the changes in activity scope, including the inclusion of the stabilization component and changes in the activity budget. This also impacted the community selection process, the ability to learn and adapt, and the prospects of sustainability of activities, mechanisms, and outcomes supported by YCST. • Dynamics within and guidance from USAID impacted the scope of the project, including the addition of a stabilization component after the award, and significant budget cuts/revisions. There were also differences of opinion within USAID related to whether YCST was set up to help community members and leaders mitigate more local conflicts or was it set up to prepare them for the larger peace process (as stipulated in the activity TOC). RECOMMENDATIONS Key recommendations and considerations for USAID and USAID Implementing Partners (IPs) are summarized below. INTEGRATING LOCAL PERSPECTIVES. • It is important that USAID engage local perspectives and expertise in activity design and in adapting activity designs to suit the local context. This requires actively engaging Foreign Service Nationals (FSNs) and other local stakeholders and allowing sufficient time to incorporate their inputs prior to finalizing the activity design. DEVELOPING A SHARED UNDERSTANDING OF ACTIVITY OBJECTIVES AND OUTCOMES. • It is important to ensure that USAID staff develop a shared understanding of the activity objectives. Interviews with USAID staff indicated considerable difference in understanding of the overall purpose of the activity, including the types of conflict it was designed to address. A shared understanding of objectives will facilitate more effective tracking of activity progress and results, and provision of consistent feedback to the IP. This will also facilitate more effective activity handover within USAID during staff transitions. ALIGNING ACTIVITY DURATION AND SCOPE. • It is important to allow more time (e.g., 5-years vs. 3-years) for implementation of activities that operate across multiple communities and involve a multi-stage implementation model within each community. Where it is only feasible to implement activities over a shorter duration, it is important to limit the activity to fewer communities to ensure effective implementation. Implementation can also be simplified by having activities focus on fewer objectives, for example solely focusing on PACE, or a combination of PACE and capacity building. It is also important to note that since the PACE model needs to be applied within each community, there are only limited economies of scale when implementing it across multiple communities in a short period. 67 | YCST FINAL EVALUATION REPORT USAID.GOV ACTIVITY DESIGN AND IMPLEMENTATION MODEL FLEXIBILITY. • At the activity design stage, it is important to consider what adapting the activity may look like and to assess whether the activity design and/or the implementation model allows for adaptation. Effective implementation of the PACE model for example requires several steps and time. Adapting the PACE model by simultaneously conducting some activities can potentially undermine the model, for example, selecting the priorities prior to community engagement and discussions. Understanding the importance of having flexibility within activity design and implementation models will ensure realistic expectations around learning and adapting. PERIODICALLY REVIEWING THE ACTIVITY THEORY OF CHANGE (TOC). • It is important that USAID and the IP periodically review the activity TOC and its assumptions to ensure that it aligns with the activity objectives and outcomes. For example, while the YCST TOC references the larger peace process, most of the community engagement work primarily focused on local level stressors and service delivery needs. Regularly examining the TOC against the project progress will help ensure that necessary adjustments to the activity implementation approach and/or TOC can be made in a timely manner. UNDERSTANDING THE NATURE OF THE COMMUNITY ENGAGEMENT PROCESS AND ITS IMPLICATIONS. • It is also important for USAID and IPs to consider the particulars of a community engagement model to determine and track activity outcomes. Community engagement can take several forms and it is important to consider how different community members engage with the activity. YCST’s two￾pronged community engagement approach—selecting CEC members to represent different segments of the community while capturing the perspectives of the broader community through surveys—may account for the different levels of awareness, participation and satisfaction between stakeholders and community members. Participation in a survey to prioritize community service delivery needs may be viewed as a light touch intervention by the community members. Additionally, given the scale of service delivery needs in Yemen a community survey may highlight several needs out of which the activity is only able to address one. While community members are likely to appreciate any service delivery support, they may not connect this support to the community engagement process or to contributing to conflict mitigation. Activity budget limitations can also limit the type of interventions that can be supported. This can also lead to some community members thinking that the needs they had identified were not addressed, and as a result to them questioning the overall effectiveness of the community engagement process. NEED FOR A BROADER AND DEEPER COMMUNITY ENGAGEMENT APPROACH TO ENHANCE INCLUSION. • Although the YCST team made a conscious attempt to engage women and youth in the CECs and ACCs, this did not result in wider awareness, engagement and participation of women and youth. Increasing the engagement of women and youth will likely require a community engagement model that goes beyond service delivery, priority surveys, and representation in the CECs and ACCs. It is also useful to consider approaches designed specifically to engage women and youth, for example by creating women or youth only committees or structures. USAID.GOV YCST FINAL EVALUATION REPORT | 68 CONNECTING THE COMMUNITY SELECTION PROCESS TO ACTIVITY OBJECTIVES. • It is important that the community selection process is connected to the activity objectives. Activities focused on conflict mitigation should be implemented in communities that are experiencing or impacted by conflicts. Additionally, if part of the aim (as specified in the TOC) of YCST was to support the larger peace process, the YCST team might have considered selecting communities that are more directly impacted by the larger conflict in Yemen. COMMUNITY SELECTION PROCESS AND THREATS POSED BY EXTREMIST GROUPS. • The presence and influence of extremist groups in two out of the ten communities selected by the evaluation team points to the risks and challenges of operating in the Yemeni context. USAID and IPs need to consider these types of risks as part of the community selection process. For example, interventions designed to improve services in areas controlled by extremist groups, may inadvertently enhance the influence of local leaders associated with extremist groups or result in undermining the outcomes and gains achieved by the activities. BUILDING SYNERGIES ACROSS OBJECTIVES AND ACTIVITIES. • It is important for USAID and the IPs to regularly assess how different activity components/objectives are linked to and reinforce each other. YCST was not able to take advantage of opportunities to link the capacity building activities to PACE and stabilization activities, such as connecting the work of grantees with the work being undertaken by community mobilizers and/or the ACCs and CECs. GRANTS COMPONENT - CAPACITY BUILDING, ENGAGEMENT AND NETWORKING. • For future activities that include a grants component, it is important to take a formal approach and develop a plan to increase networking, learning, and knowledge sharing opportunities among grantees. This includes taking a formal approach to capacity building of the grantees, including how their work can support and reinforce the work being undertaken by other activities; and creating opportunities for local grantees to contribute to the design and implementation of new activities and adapting on￾going activities. IDENTIFYING AND ADDRESSING THREATS TO SUSTAINABILITY OF ACTIVITIES AND OUTCOMES • It is important for USAID and Implementing Partners to assess the likely sustainability of mechanisms, activities, and outcomes from an early stage, including identifying threats or challenges to sustainability such as lack of resources and the potential for and willingness of community members and stakeholders to come together to sustain activity outcomes. 69 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 1: EVALUATION DESIGN Background and Purpose Introduction The Yemen Communities Stronger Together (YCST) evaluation will examine the effectiveness of the YCST activity in contributing to its intended outcomes around mitigating conflict and increasing social cohesion in targeted communities. The evaluation will explore the relevance of YCST’s four development approaches, the quality of implementation, and if and how operational challenges influenced the activity’s achievement of intended outcomes. The evaluation’s findings, conclusions, and recommendations will help the United States Agency for International Development (USAID) design a new governance, peace, and stability activity focused on community resilience, making lessons learned from YCST’s community engagement model immediately applicable. Project Description The YCST award is a four-year (including a no-cost extension), USAID-funded, $22 million activity implemented by Global Communities (GC) together with Yemeni partner Field Medical Foundation (FMF) and specialized international partners Build Up and Partners for Good (PfG). The award is scheduled to end in September 2022. Since its inception in 2018, YCST has focused on supporting local-level initiatives that mitigate conflict, strengthen social cohesion, promote the peaceful resolution of differences, and, where feasible, capitalize on stabilization opportunities. It currently targets communities in the four governorates of Abyan, Aden, Lahj, and Taiz in southern Yemen. The YCST Theory of Change (TOC) posits: “If local capacity to collaboratively identify and address conflict stressors and threats to social cohesion is built, and if tangible community-level results are demonstrated, then the resilience of Yemeni communities and institutions will increase, local authorities who participate in systems of accountability strengthened by YCST will be better prepared to lead their communities’ participation in national peace efforts, and the effectiveness of models that can be replicated across the country will be demonstrated, laying the foundation for a peaceful and effective transition process.” The award has four objectives: Objective 1: Engage communities in a participatory action for community enhancement (PACE) approach to build the capacity of community members and local leaders to collaboratively identify and address the drivers and impacts of conflict through conflict mitigation and service delivery projects. Objective 2: Through small grants, build the capacity of formal and informal local leaders to identify and address unresolved grievances that can spark violent conflict at the district level and within their communities. Objective 3: In partnership with credible and local authorities and systems, pilot stabilization activities that reduce conflict and improve social cohesion. USAID.GOV YCST FINAL EVALUATION REPORT | 70 Objective 4: Carry out regular and action-oriented learning and contextual analysis across YCST interventions to support adaptive management for enhanced outcomes. Project Activities In this section we briefly review the YCST implementation approach and activities by objective. The YCST team relied on a sequenced implementation approach with selection of an initial list of communities (Phase 1) and implementation of PACE, capacity-building, and stabilization activities in these communities. Following these activities and lessons learned from the Phase 1 communities, YCST implemented its activities in a new group of communities (Phase 2). PACE (Objective 1) and stabilization (Objective 3) activities were implemented in separate communities. The stabilization pilot activities were implemented in four communities across two governorates. Capacity-building activities under Objective 2 were implemented across Phase 1 and Phase 2 communities and in communities where PACE or stabilization activities were implemented. Objective 1 Activities: Objective 1 PACE activities are summarized in the following figure and include identifying and training PACE mobilizers and Community Enhancement Committees (CECs), mapping community needs and priorities, and developing community-specific action plans. The CECs help identify community needs, build community consensus on priority needs, and oversee the implementation of the service delivery projects. Working in coordination with local stakeholders and CECs, the YCST staff directly manage implementation of infrastructure activities and procurement. Following the completion of the service delivery projects, the YCST team support “soft” activities (involving behavior change and skills building) to ensure that the infrastructure or equipment are used and properly maintained to ensure sustaining service delivery improvements. 71 | YCST FINAL EVALUATION REPORT USAID.GOV Objective 2 Activities: YCST Objective 2 activities focus on building the capacity of organizations, networks, and local leaders to identify and address unresolved grievances that can lead to violent conflict. Capacity-building activities are implemented in PACE and stabilization communities and are supported through a Peace Building Grants Fund; Women, Peace, and Security Sub-fund; and an Innovation Grant Sub-fund. Capacity-building activities also aim to enhance the participation of women and youth in the conflict resolution and mitigation efforts. Objective 3 Activities: Objective 3 activities focus on assisting local authorities in understanding the priorities of their constituents and developing plans and implementing projects (infrastructure or other investments) to respond to these priorities. The YCST team also provides capacity-building support to local authorities to communicate with and be accountable to their constituents. Local authorities are provided with support tailored to the activity they are implementing. Objective 4 Activities: Activities under Objective 4 are focused on learning and adaptive management and include a learning agenda, continuous mapping, analysis, and adaptation. The learning agenda is intended to examine assumptions underpinning the YCST TOC and technical evidence gaps. The learning activities also include learning from activities implemented under Phase 1 communities and applying the key lessons learned in implementing the activities in Phase 2 communities. The learning is also intended to develop community entry and sustainability models and for developing tools for assessing success or failure. The YCST team also develops community dashboards and conducts conflict analyses and real-time context monitoring to respond to community and context-specific considerations. Evaluation Purpose This evaluation will examine the effectiveness of the YCST award in contributing to its intended outcomes. It will also examine positive and negative unintended outcomes. To contextualize findings on effectiveness, the evaluation will examine if and how the relevance of the YCST development approach, the quality of implementation, and operational challenges influence the activity’s effectiveness. The evaluation’s findings, conclusions, and recommendations will help USAID design a new governance, peace, and stability activity, expected to be awarded in 2023. The new design will focus on community resilience, making lessons learned from YCST’s community engagement model immediately applicable. Audiences and Intended Uses The primary audience for this evaluation is the USAID/Yemen Mission, as well as future implementing partners (IPs) of community resilience initiatives in Yemen. Evaluation Questions This evaluation will examine the effectiveness of the YCST activity in the context of its TOC. The relevance of an award’s design, the fidelity of implementation to the design, the quality of implementation, and the extent to which the risks and assumptions that underpin the TOC materialized all contribute to an award’s effectiveness. The questions thus examine the award’s effectiveness, the relevance of the design, the fidelity and quality of implementation, and operational challenges the award faced during implementation. USAID.GOV YCST FINAL EVALUATION REPORT | 72 1. To what extent, and how, has the YCST award contributed to its intended outcomes, especially community and inter-community conflict mitigation and social cohesion, in PACE and stabilization intervention areas? 2. How, if at all, did YCST’s development approach contribute to, or detract from, the award’s effectiveness and potential for sustainable results? 3. What operational challenges or constraints did YCST face, and how, if at all, did these challenges affect outcomes and prospects for sustaining results? 4. As a result of this evaluation, what “lessons learned” or successful models should USAID consider for follow-on community-level interventions? Overall Evaluation Approach Understanding the Evaluation Questions Question 1: This question asks whether YCST has contributed to its intended outcomes as specified in its TOC. Key outcomes/results of interest by YCST approaches are summarized in Table 1. Table 1: YCST Approaches and Intended Outcomes YCST Approaches Intended Outcomes/Results 1. PACE approach (Objective 1) a. Increase in social cohesion b. Enhanced capacity of the community to resolve/mitigate conflicts c. Mitigation of conflict resulting from service delivery activities 2. Capacity-building approach (Objective 2) a. Improved organizational capacity for conflict mitigation and resolution 3. Stabilization approach (Objective 3) a. Enhanced capacity of local leaders to be responsive to community needs b. Mitigation of conflict resulting from service delivery activities 4. Adaptive learning approach (Objective 4) a. Activity adaptations based on monitoring and evaluation (M&E) and contextual data The answer to this question will also examine unintended outcomes, such as whether assistance provided to a beneficiary community contributes to conflicts with neighboring communities that do not benefit directly from YCST interventions. Question 2: This question focuses on the relevance and effectiveness of YCST’s community-based approach with respect to contributing to intended outcomes and prospects for sustainable results. The evaluation will consider the PACE methodology (including the formation of CECs), stabilization pilots, small peace-building grants, and geographic distribution. This question will examine both the award’s development approach and how the approach was implemented. Question 3: This question will examine the operational challenges YCST faced in implementing its interventions, and how these challenges affected achieving outcomes or sustainability. 73 | YCST FINAL EVALUATION REPORT USAID.GOV Question 4: To answer this question, the evaluation will document lessons learned about how to design and implement follow-on community-level interventions. Approach to Answering Evaluation Questions For data collection for this evaluation, the evaluation team will use purposive criteria to select a combination of Phase 1 and Phase 2 PACE and stabilization communities (8 to 10 communities). The evaluation team will also consider USAID feedback related to specific communities of interest in the community selection process. The evaluation team will finalize the number and list of communities following the initial set of interviews with USAID and the key YCST staff. Data collection at the community level will include interviews with key informants (10–15 per community) including with formal and informal leaders, community mobilizers, CEC members, and staff working at the service delivery projects. The evaluation team will also conduct between 20 and 35 interviews with community members. Next, we describe our approach to answering the four evaluation questions. To avoid repetition, we present our approach for answering question 1 (focused on outcomes) and 2 (focused on implementation approach) under one section. This discussion is followed by our approach to answering question 3 (focused on operational challenges) and 4 (focused on lessons learned). Question 1 and Question 2 We start by discussing our approach to understanding the YCST development approach and how it contributed to or detracted from the award’s success and potential for sustainable results. We also describe how we will measure the extent to which this award contributed to its intended outcomes under the four objectives. YCST Development and Implementation Approach The evaluation will separately assess the four approaches being implemented by YCST, including: 1. PACE approach, 2. YCST capacity-building approach, 3. Stabilization approach, and 4. Adaptive learning approach. In addition to examining how these distinct approaches contributed to or detracted from the award’s effectiveness and potential for sustainable results, the evaluation team will examine whether and how the activities complemented each other. For example, assessing whether and how the capacity-building and adaptive learning approaches supported the PACE and stabilization approaches. The following summarizes our approach for separately assessing the different YCST approaches, and the complementarities, in case any, between them. a. PACE Approach The PACE approach involves inclusively engaging communities to identify and implement service delivery and conflict mitigation projects. The approach intends to address drivers of and impacts of conflict and build social capital and skills within the community. USAID.GOV YCST FINAL EVALUATION REPORT | 74 To understand the community engagement process and approach, the evaluation team will interview relevant members of IP staff, including past and present senior leadership, the YCST capacity-building manager, PACE advisors in Yemen, Jordan, and GC headquarters (HQ), and community mobilizers and relevant GC partners responsible for adapting and delivering the PACE approach, including partners and staff who conducted trainings in Jordan. The sample of community mobilizers will be selected purposively to ensure that the evaluation team interviews male and female mobilizers, as well as mobilizers belonging to different types of communities (e.g., urban/rural, YCST categorization related to level of conflict and degree of cohesion, etc.). In addition to collecting community mobilizer feedback on the PACE process and their training and experiences, the evaluation team will also review the content of trainings provided to the community mobilizers. The evaluation team will closely examine how the PACE approach was introduced to the local communities, as well as the approach used to identify community assets and key stakeholders. The evaluation team will also interview local leaders (formal and traditional) who were part of the community engagement process. In examining the introduction of the PACE approach, the evaluation team will also explore whether and how the YCST team engaged marginalized groups during the community engagement process. The team will also review the relevant secondary data including the baseline household survey and other assessments of needs, assets, conflict drivers and impacts, and the nature of political economy captured in the community dashboards and other relevant products generated by the YCST monitoring, evaluation, and learning (MEL) team, for example mapping of youth and gender dynamics. Through a combination of secondary data review and interviews, the evaluation team will review the processes used to select CEC members and to understand how these committees operate. The evaluation team will assess the extent to which the CECs are inclusive by looking at their actual composition, as well as by gauging perceptions regarding the CEC formation process. The evaluation team will also interview the CEC members and the general population living in the communities. The evaluation team will use a purposive selection approach to identify CEC and community members for interviews that represent male and female, older and younger, and potentially more vulnerable respondents. CEC members will also be asked about the support they received from the YCST award, including training and mentoring. The evaluation team will closely examine how the CECs have operated, including the processes used for identifying community needs, building consensus on priorities within the community, and oversight functions including for service delivery and peace-building projects, as well as strategies to ensure gender equity and principles of conflict sensitivity. The evaluation team will also assess the extent to which a focus on service delivery projects was relevant to addressing local conflicts. The evaluation team will closely review the process of selecting priority projects and how the selected projects addressed common causes of conflict within the communities. As part of this analysis, the evaluation team will also reference the participatory conflict analyses and project prioritization, community action planning documents, and project data collection sheets that summarize why a specific project was selected. Interviews with the community residents will provide useful data on community perceptions related to the performance of the CECs and the overall effectiveness of the activities undertaken by the CECs. 75 | YCST FINAL EVALUATION REPORT USAID.GOV The evaluation team will interview local leaders and stakeholders who were engaged in identifying and implementing projects. The evaluation team will also attempt to identify and interview stakeholders and key informants who were not part of the YCST community engagement efforts, but are aware of these efforts, to capture their perspectives on the relevance and effectiveness of the work undertaken by the CECs. The evaluation team will also gauge the relationship between the CECs and other existing mechanisms and whether and how they complement or challenge each other. Through interviews and secondary data review, the evaluation team will also assess how the PACE activities contributed to building social skills and cohesion among the members of the communities, and how these skills are valued and used by community members. To measure an increase in social skills and cohesion, the evaluation team will develop questions corresponding to the key aspects of the GC Social Capital Index, including questions that measure changes in participation in community groups and activities, attitudes toward participation, willingness to trust others, norms of accountability and inclusion, institutional trust, and strength and breadth of social networks. The evaluation team will also aim to understand the extent to which the community members believe that YCST-funded community projects address critical community needs and contribute to improved conflict resolution. Through interviews and review of secondary data, the evaluation team will aim to determine whether the community-specific causes/drivers of conflict were addressed or reduced by the PACE activities. Where relevant, the evaluation team will also look into whether and how the PACE activities contributed to mitigating inter-community conflicts. Measuring PACE Sustainability The evaluation team will examine aspects of the PACE approach focused on sustainability of PACE mechanisms and outcomes. The team will assess the potential sustainability of CECs and will also review existing practices related to actual and planned maintenance of facilities, community cost sharing (resource mobilization), community monitoring of projects, soft skill trainings, and capacity-building efforts for members of sustainability groups. The evaluation team will conduct in-person visits to specific sites to interview facility staff and to conduct direct observations to assess the state of infrastructure and maintenance efforts. In instances where PACE activities are clearly associated with addressing and mitigating existing conflicts, the evaluation team will also assess to what extent these outcomes are sustainable. This will be assessed by looking at the collective competence of the CECs and other PACE stakeholders to come together to address/mitigate future conflicts. b. Objective 2: Capacity-Building To assess the effectiveness of activities under Objective 2, the evaluation team will review the relevant secondary data (e.g., list of beneficiary organizations and individuals, training materials, and implementation approaches, etc.) and conduct interviews with individuals and organizations who were part of these capacity-building efforts. The evaluation team will also speak with the YCST grants fund manager and review the data generated by the supported grants, including from grants primarily focused on generating contextual data. The team will also review grant-specific monitoring plans. The document reviews and interviews will try to gauge the overall relevance and quality of these capacity-building efforts, as well as whether and how the trainees applied these trainings for identifying and addressing unresolved grievances through arbitration and conflict mitigation efforts to respond to the needs of the communities they serve. To assess how the capacity-building activities contribute to improved capacity for conflict mitigation, the evaluation team will also review activities designed to USAID.GOV YCST FINAL EVALUATION REPORT | 76 address community-specific conflicts, including in communities where YCST implemented the PACE or the stabilization approach. The evaluation team will also consider community members’ feedback related to trust and overall effectiveness of the types of mechanisms and organizations supported under Objective 2. The evaluation team will review the community mapping and participatory conflict analyses conducted by the YCST team for an understanding of existing conflict resolution mechanisms, leaders, and government stakeholders identified by the YCST team. The evaluation team will analyze interview respondents’ feedback on whether the most relevant mechanisms and beneficiaries (individuals and organizations) were selected as part of the capacity-building efforts under Objective 2. The evaluation team will also examine the extent to which these capacity-building efforts focused on increasing the participation of underrepresented groups (women and youth) in peace building, conflict mitigation, and civic life. The team will also interview key informants from organizations such as UN Women, Yemeni Women’s Pact for Peace and Security, and Yemeni Women’s Union to gauge whether and how they were consulted and/or engaged by the YCST team in identifying and supporting initiatives and grant priorities that reduce marginalization from decision-making and peace processes. Under Objective 2, the evaluation team will evaluate the effectiveness of the capacity-building efforts focused on YCST partner organizations (FMF and National Foundation for Development and Humanitarian Relief) to implement the PACE process and whether and how these organizations are applying what they have learned in their work outside of YCST. Additionally, the evaluation team will review the process used for selection of grants under the Peace Building Grant Funds. The team will also interview the beneficiary nongovernmental organizations (NGOs) and civil society organizations to gauge their feedback on the quality, relevance, and use of the technical assistance and learning opportunities they participated in. The evaluation team will review the activities under the innovation sub-fund and assess how these activities contributed to other more significant PACE and stabilization activities, and whether any of these activities were successfully scaled up or have the potential to be scaled up in the future. Measuring Objective 2 Sustainability The evaluation team will rely on primary and secondary data to assess the sustainability of capacity￾building efforts on conflict mitigation including by examining how the beneficiaries of the capacity￾building efforts continue to apply their learning. The team will also capture potential challenges to sustaining capacity-building successes, for example, resource and funding constraints or lack of community or political authority buy-in. c. Objective 3: Stabilization The evaluation team will examine YCST activities focused on assisting local authorities to understand and address their constituents’ priorities. As part of this process, the evaluation team will interview the relevant IP staff including the stabilization senior officer stabilization advisors and key informants including members of the local authority (e.g., governor’s office, district council, district managers, local councils, relevant ministry, and facility staff), as well as formal and informal leaders. 77 | YCST FINAL EVALUATION REPORT USAID.GOV The evaluation team will review the process/steps through which the YCST team facilitates local authorities to effectively respond to constituent concerns and/or priorities, including through service delivery initiatives and improvements. This will include reviewing capacity-building and mentoring efforts focused on local authorities, as well as capturing the perspectives of members of the local authorities on how useful these capacity-building activities were. The evaluation team will examine and inquire about all the key stages including the process of consulting constituents, identifying priorities, developing a response plan, and communicating progress and results to citizens. The evaluation team will also review the tailored assistance provided to local authorities. The evaluation team will focus on both small-scale pilot activities initiated in the early phase of YCST and stabilization activities implemented during the subsequent phase. The interviews with members of the general population will be used to gauge citizen perceptions regarding the stabilization activities including whether the citizens viewed these activities as relevant, timely, and responsive to their needs, whether these activities contributed to conflict mitigation and reduction in violence, and whether and how the success of these activities contributed to an increase in the legitimacy and greater credibility of authorities. The evaluation team will also review the community needs and opportunities captured in the stabilization assessments conducted by YCST. To assess citizen satisfaction, the evaluation team will also review the satisfaction data from the YCST Citizen Score Cards. Measuring Objective 3 Sustainability The evaluation team will also examine aspects of the stabilization approach focused on sustainability of outcomes. The evaluation team will assess whether the support provided by the activity is sufficient and will ensure that local authorities continue to effectively communicate with and remain accountable to their constituents. In communities where a stabilization intervention is clearly linked with mitigating specific conflict/s, the evaluation team will also assess the extent to which these peace-building gains are sustainable. The evaluation team will also review existing practices related to actual and planned maintenance of facilities, soft skill trainings, and evidence of resource/revenue generation for maintenance of facilities. The evaluation team will conduct in￾person visits to specific sites to interview facility staff and to conduct direct observations to assess the state of infrastructure and maintenance efforts. The team will also gauge community perceptions related to the responsiveness of government. Examining this in the initial YCST communities where the stabilization activities occurred earlier will Community Selection Process/Approach For both PACE and stabilization efforts, the evaluation team will also collect data on the community selection process, including how the communities were selected and the prevailing context in the selected communities, including the nature, frequency, and prevalence of conflict. The evaluation team will also review the relevant YCST documents (e.g., community-specific stabilization documents with information against the seven parameters) that detail stabilization challenges and opportunities within specific communities. This will help the team assess community-specific needs, the YCST community-specific approach, the conflicts YCST activities were designed to address and to tailor the evaluation instruments and data collection approach. k USAID.GOV YCST FINAL EVALUATION REPORT | 78 provide a good sense of whether and how community perceptions regarding responsiveness of government have evolved over time. d. Objective 4: Learning and Adaptive Management The evaluation team will review the learning agenda, including the evidence base for community cohesion, resiliency, and stabilization. The evaluation team will review the learning products generated by the YCST team and rely on interviews with the YCST staff, including senior leadership, technical, and MEAL unit staff to assess how these products supported and informed adaptive management and enhanced outcomes. The evaluation team will also review whether and how the YCST team leveraged learning to effectively integrate gender and inclusion considerations across their programming. The evaluation team will look for instances in which the YCST team scaled up or replicated activities based on findings from earlier pilots and activities. The team will also review the data related to the activity TOC. The team will also assess whether and how the YCST team utilized the contextual data generated by the activity itself including adapting to changes in the context. The evaluation will also look at the learning dissemination efforts and gauge perspectives of the stakeholders involved in these efforts to assess how these efforts contributed to adaptive learning. Question 3 The evaluation team will rely on a review of secondary data (e.g., synthesizing the relevant activity data captured in monthly, quarterly, and annual reports; contextual data generated by the activity or other entities/activities; etc.), in addition to interviews across respondents to identify operational challenges and constraints faced by YCST and how these impacted the activity outcomes. Beyond the technical staff, the evaluation team will also interview the YCST security and liaison manager and director of finance and administration to understand operational challenges and how they varied by location/s or objectives. Gender and Inclusion Considerations—Measuring Outcomes As part of assessing the extent to which YCST contributed to its intended outcomes, the evaluation team will also factor gender and inclusion considerations, including whether and how these considerations informed the YCST approaches, and how inclusive the outcomes/results achieved by YCST were. In addition to interviewing the YCST gender and social inclusion expert, the evaluation team will also review the gender analyses conducted by YCST. Relevant questions will also be included in interviews with YCST senior and objective￾specific leadership. Last, sampling of community stakeholders and residents will include purposive quotas to ensure that the perspectives of females, youth, and other marginalized groups are effectively captured as part of the evaluation data collection. Impact of COVID-19 The evaluation team will pay specific attention to the overall impact of COVID-19 on the YCST activities, including activities related to capacity￾building, community mobilization, and engagement activities. In addition to assessing the constraints and challenges, the team will also review the COVID-19 adaptations made by the YCST team. 79 | YCST FINAL EVALUATION REPORT USAID.GOV The evaluation team will identify crosscutting as well as objective-specific challenges, for example challenges related to community entry (Objective 1), grant mechanisms (Objective 2), engaging the authorities (Objective 3), and adaptive learning (Objective 4). Challenges and constraints related to integrating gender and inclusion considerations will also be examined. The evaluation team will also identify potential challenges and constraints that can impact the likely sustainability of successful outcomes. In addition to understanding these challenges and constraints, the evaluation team will also inquire about how some of these challenges were mitigated. Question 4 Like question 3, question 4 is a cross-cutting question that will be asked across different types of respondents. The evaluation team will also review the lessons learned captured in the activity documents, for example the lessons learned tracker. The evaluation team will also review and synthesize the findings generated by YCST after action reviews and YCST learning agenda outputs (e.g., tools for mapping changes in conflict triggers and tracking conflict mitigation outcomes; Yemen-specific indicators of social cohesion, stabilization, and community resilience; and findings related to effects of PACE and stabilization efforts on conflict mitigation and peace building). The evaluation team will also interview the YCST accountability unit staff and request anonymized data based on the feedback shared over the award hotline and/or during the feedback sessions. Interviews with community members and local and regional leaders and authorities will also include questions on lessons learned and key considerations for implementing community-level interventions. Last, in addition to synthesizing responses from direct questions about lessons learned, the evaluation team will also identify lessons learned during its analysis of data and based on the expertise of the evaluation team members. USAID.GOV YCST FINAL EVALUATION REPORT | 80 ANNEX 2: LIST OF SECONDARY DATA SOURCES REVIEWED # Document Type Document Title 1 Cooperative Agreement Cooperative Agreement 72027918CA00001 – Yemen Communities Stronger Together (Global Communities) (1) 2 Assessments Stabilization Programming Assessment Report_2.1.19 3 Baseline and Annual Reports Y1 APR_11.1.19 4 Y1 APR_11.18.20 5 Y3 APR_10.29.21 6 YCST Baseline Report Phase i 7 YCST Baseline Report_2.1.19 8 YCST Phase II Baseline_Final Presentation_20210919 9 Community Dashboard and Social Capital Index Data Community Dashboard and Social Capital Index Data 10 Learning Agenda YCST Year 1 Learning Workshop Report_January 2020 11 Adaptation Workshop Report 12 After Action Review of the Stabilization Report No. 2 Agenda 13 200930 YCST Stabilization 2 AAR Live Notes 14 YCST GIS Report – Phase 1 Approved Projects 15 YCST Learning Agenda Reflection Workshop Report 16 YCST LearningAgenda_CONSOLIDATED Final Version 2306 17 YCST LessonsLearnedTracker YCST LessonsLearnedTracker_Updated Sept 2021 18 YCST SOW _Learning Agenda Data Collection Activities _Nov 14_21 19 Mid-Term Evaluation 9. YMEP II Mid-Term Performance Evaluation of the YCST Project (1) 20 Stabilization Reports Stabilization Report Abyan and Taiz_10.20 21 Stabilization Report Al Azaez_5.19 22 Stabilization Report Al-Suwa'a_5.19 23 Stabilization Report Khanfir_5.19 24 Stabilization Report Zenjubar_5.19 25 8.3. Stabilization Second Lessons Learned Report _ FINAL 26 USAID YCST Lessons Learned – Using Stabilization Assessment Framework in Yemen 10.18.19 27 YCST SOW _Stabilization Lessons Learned Draft_Final_121221 28 YCST Stabilization Lessons Learned #3 29 USAID Approach ICS-Yemen_UNCLASS_508 30 Work Plan USAID YCST Bi Annual and NCE Workplan Schedule FY22_Oct 27 2021 31 USAID YCST Bi-Annual Implementation Plan and NCE Revision FY22_ 5 November 2021 (1) 32 MEAL Plan ANNEX B_USAID YCST PIRS FINAL 81 | YCST FINAL EVALUATION REPORT USAID.GOV # Document Type Document Title 33 YCST MEAL Plan Update_02 December 2021 (2) 34 YCST Organogram YCST Organogram-Sept 2021 35 YCST Surveys YCST Perception Survey Raw Data+Phase 1&2 36 YCST Phase II Communities_ Data Tabs_Gender and Age_20210919 37 YCST Phase II Communities_ Data Tabs_20210816_All 38 YCST Phase II Household Survey Data_Raw_Clean Final 39 Quarterly Reports QPR Y1Q1_1.30.19 40 QPR Y1Q2_5.6.19 41 QPR Y1Q3_7.1.19 42 QPR Y2Q1_1.1.20 43 QPR Y2Q2_5.9.20 44 QPR Y2Q3_8.10.20 45 QPR Y3Q1_1.30.21 46 QPR Y3Q2_4.30.21 47 QPR Y3Q3_7.30.21 48 YCST Quarterly Program Performance Report Y4Q1-Final Draft - V3 49 TPM and V Reports YCLE -20-54 YCLE TPM Site Visit Report_YCST Ja'ar Park_Draft for IP and USAID review V2 50 YCLE -21-219 TPM Site Visit Report_YCST RSDSRP_USAID and IP Review_IP Feedback YCLE Response 51 YCLE-21-89 TPM Site Visit Report_YCST SHCRP_Draft for USAID and IP Review 52 YCLE-21-91 TPM Site Visit Report_YCST ASRP_IP feedback and YCLE response 53 YCLE-21-96 TPM Site Visit Report_YCST Haydara Football Rehabilitation Project_IP Review YCLE Response 54 YCLE-21-123 TPM Site Visit Report_YCST NWSEP_IP Review and YCLE Response 55 YCLE-21-148 TPM Site Visit Report_YCST_ Al Soqaya Water Supply Enhancement SWSEP_IP Reviewed_YCLE Response 56 YCLE-21-186 TPM Site Visit Report_YCST SYSRP_USAID and IP_Reviewed_YCLE Response 57 YCLE-21-186 TPM Site Visit Report_YCST SYSRP_USAID and IP_Reviewed_YCLE ResponseV2 58 YCLE-21-192 TPM Site Visit Report_YCST KSRP_USAID and IP Reviewed _YCLE response 59 YCLE-21-206 TPM Site Visit Report_YCST Al-Shaheed Abdulaziz School AASRP_USAID and IP Reviewed_YCLE Response 60 YCLE-21-234 TPM Site Visit Report_YCST MWRP_ IP Reviewed YCLE Response USAID.GOV YCST FINAL EVALUATION REPORT | 82 # Document Type Document Title 61 YCLE-22-252 TMP Site Visit Report_ YCST Enhancing the Role of Women Training_IP Feedback YCLE Response 62 Build Up Report Build Up Final Results Report GC Yemen 83 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 3: YCST COMMUNITIES SAMPLED PACE or Stabilization Governorate District Community Phase Pop. Size Project (s) 1 PACE Abyan Jayshan Jayshan 1 9,060 Water Supply Enhancement Project 2 PACE Abyan Khanfir Al-Dergag 2 4,716 Water Supply Enhancement Project 3 PACE Aden Dar Sa’ad Abd Albari 1 22,93 7 Electricity Rehabilitation Project 4 PACE Aden Al Buraiqeh Al-Shouhada 2 23,15 3 Electricity Rehabilitation Project 5 PACE Lahj Al￾Madraiba Wa Al A’ara Al-Suraih 1 1,577 Al Suraih Schools Rehabilitation Projects Al Suraih Health Center Rehabilitation Project 6 PACE Lahj Al￾Madraiba Wa Al A’ara Al-Nabyah 1 3,283 Water Supply Enhancement Project 7 Stabilization Abyan Khanfar Ja’ar Ja'ar Park (Green Zone) Rehabilitation Project Haydera Football Field Rehabilitation Project— Haydera Football Field USAID.GOV YCST FINAL EVALUATION REPORT | 84 PACE or Stabilization Governorate District Community Phase Pop. Size Project (s) 8 Stabilization Abyan Zinjibar Zinjibar Khalwa School Rehabilitation Project Sahat Al Shohada Park (Green Zone) Rehabilitation Project Khadija School Rehabilitation Project 9 Stabilization Taiz Al￾Ma’afer Al-Suwa’a Al Ma’afer Rural Hospital Sanitation System Rehabilitation Project 10 Stabilization Taiz Ash Shamayta in Al-Azaez Al Khair School Rehabilitation Project Abdulaziz School Rehabilitation Project Al Mansoora Health Center Rehabilitation Project Al Shaheed Yassin School Rehabilitation Project 85 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 4: COMMUNITY SURVEY FINDINGS PACE COMMUNITIES 19 58 15 9 0 20 40 60 80 Trust More About the Same Trust Less Don't Know Level of Trust among Community Members - PACE Communities Do you trust other members of your community more, about the same, or less than you did 12 months ago? Level of Trust among Community Members 50 25 2 1 9 10 3 0 10 20 30 40 50 60 Lack of Services Poverty and Living Conditions Security Situation Absence of Government Other No Problems / Challenges Don't Know Community Problems - PACE Communities In your opinion, what is the most important problem facing your community/area? Problems Faced by the Community 16 84 0 20 40 60 80 100 Yes No Presence of Conflict - PACE Communities Is there an ongoing conflict or conflicts impacting your household and community? Presence of Conflict USAID.GOV YCST FINAL EVALUATION REPORT | 86 70 1 8 1 0 1 4 16 30 1 10 1 16 4 23 16 49 1 17 2 0 3 7 22 46 0 0 0 0 0 46 8 0 10 20 30 40 50 60 70 80 Government Private Sector INGOs Local Organizations/ CSOs Community Project Other Service is not available Don't Know Provision of Basic Services - PACE Communities Who is primarily reponsible for providing basic services in your community? Electricity Health Services Water Education 36 39 23 2 0 10 20 30 40 50 Improved Remained the Same Worsened Don't Know Access to Basic Services - PACE Communities Has your access to services (YCST-supported) improved, remained the same, or worsened over the last year? Access to Services 87 | YCST FINAL EVALUATION REPORT USAID.GOV 83 2 15 0 20 40 60 80 100 Will Improve Will Worsen Don't Know Access to Basic Services - PACE Communities Will your access to services (YCST-supported) improve, remain the same, or worsen once the YCST-supported projects are completed? (Two Phase 2 Communities) Access to Services 43 4 32 22 0 10 20 30 40 50 Satisfied Neither Satisfied Nor Dissatisfied Dissatisfied Don't Know Satisfaction with the Quality of Services provided by YCST - PACE Communities Satisfaction with the Quality of YCST Services 26 9 62 3 0 10 20 30 40 50 60 70 Moving in the Right Direction Some are moving in the Right Direction and some are moving the Wrong Direction Moving in the Wrong Direction Don't Know / No Response Overall Situation - PACE Communities Are things moving in the right direction or in the wrong direction? Overall Situation USAID.GOV YCST FINAL EVALUATION REPORT | 88 31 34 32 3 0 5 10 15 20 25 30 35 40 Improved Remained the Same Worsened Don't Know / No Response Peace and Security Situation - PACE Communities In your opinion, has the overall peace and security situation in your community improved, remained the same, or worsened over the last 12 months? Peace and Security Situation 30 66 4 0 10 20 30 40 50 60 70 Yes No Don't Know Community Ability to Address Key Problems - PACE Communities Is you community capable of addressing the key problem impacting your community? Community's Ability to Address its Problems 18 70 13 0 10 20 30 40 50 60 70 80 Yes No Don't Know Local Government and Authorities' Conflict Mitigation Role - PACE Communities Over the last 12 months have your local government/authorities done anything to address conflicts in your commuity? Local Government and Authorities' Conflict Mitigation Role 89 | YCST FINAL EVALUATION REPORT USAID.GOV 14 19 40 26 18 10 44 29 23 10 42 24 0 10 20 30 40 50 Very Involved Somewhat Involved Not at all Involved Don't Know Efforts to Address Conflicts - PACE Communities Over the past 12 months, have your local community, leaders, or organizations been involved in addressing community conflicts? Local Community Local Community Leaders Local Organizations 34 35 15 16 0 5 10 15 20 25 30 35 40 Improved Remained the Same Worsened Don't Know / No Response Conflict Resolution Mechanisms - PACE Communities In your opinion, have the existing conflict resolution mechanisms within your community improved, remained the same, or worsened? Peace and Security Situation 25 13 37 26 0 5 10 15 20 25 30 35 40 Satisfied Neither Satisfied Nor Dissatisfied Dissatisfied Don't Know / No Response Satisfaction with Local Leaders'/Authority Performance - PACE Communities Satisfaction with Local Leaders' Performance USAID.GOV YCST FINAL EVALUATION REPORT | 90 6 48 30 19 0 10 20 30 40 50 60 Trust More About the Same Trust Less Don't Know Level of Trust of Local Authorities and Government - PACE Communities Do you trust local authorities and government more, about the same, or less than you did 12 months ago? Level of Trust of Local Authorities and Government 27 27 14 32 0 5 10 15 20 25 30 35 Good Bad Neither Good Nor Bad Don't Know Community and Local Authority Relationship - PACE Communities How would you describe the relationship between your community and local authorities? Community and Local Authority Relationship 33 38 29 0 5 10 15 20 25 30 35 40 Yes No Don't Know Community and Local Authority Relations - PACE Communities Is there a way to improve the relationship between your community and local authorites? 91 | YCST FINAL EVALUATION REPORT USAID.GOV 27 67 6 0 10 20 30 40 50 60 70 80 Yes No Don't Know Awareness of YCST Activities - PACE Communities 9 76 2 2 4 18 0 10 20 30 40 50 60 70 80 Community Engagement Process Service Delivery Projects Initiatives Implemented by CECs Aware of CEC formulation, roles and responsibilities Other Don't Know Awareness of Specific YCST Activities (n = 45) - PACE Communities Awareness of Specific YCST Activities 75 2 13 9 0 20 40 60 80 Satisfied Neither Satisfied Nor Dissatisfied Dissatisfied Don't Know Level of Satisfaction with YCST Activities (n = 45) - PACE Communities Level of satisfaction with how YCST organized activities (N = 45) USAID.GOV YCST FINAL EVALUATION REPORT | 92 17 75 7 0 10 20 30 40 50 60 70 80 Yes No Don't Know Awareness of Community Enhancement Committees (n = 167) - PACE Communities 66 41 34 59 0 10 20 30 40 50 60 70 Selection of CEC Members Selecting and Addressing Stressors Engagement/Participation in CEC Activities (n = 29) - PACE Communities Yes No 100 0 0 0 20 40 60 80 100 120 Effective Ineffective Don't Know CEC Effectiveness (n = 29) - PACE Communities Capturing Community Feedback Capturing Community Feedback 93 | YCST FINAL EVALUATION REPORT USAID.GOV 93 7 0 0 20 40 60 80 100 Satisfied Dissatsifed Don't Know Satisfaction with CECs Performance (n = 29) - PACE Communities Satisfaction with CECs Performance 86 14 0 0 20 40 60 80 100 Yes No Don't Know CEC Activities (n = 29) - PACE Communities Is your local CEC active in the community? CEC Activities 86 93 72 55 83 89 3 7 17 31 10 7 7 0 0 7 3 3 0 10 20 30 40 50 60 70 80 90 100 Representing Community Interests Selecting Community Projects Representing Youth Representing Women Addressing Conflicts Strengthening Relationships between Community Members Satisfaction with CECs Performance (n = 29) - PACE Communities Satisfied Dissatisfied Neither Satisfied Nor Dissatisfied Don't Know USAID.GOV YCST FINAL EVALUATION REPORT | 94 STABILIZATION COMMUNITIES 6 74 20 0 20 40 60 80 Yes No Don't Know PACE Communities Are you aware of any instances in which the YCST support to your community contributed to creating conflicts betweeen community members or with other communities? 12 71 17 0 10 20 30 40 50 60 70 80 Yes No Don't Know Local Government and Authorities' Conflict Mitigation Role - Stabilization Communities Over the last 12 months have your local government/authorities done anything to address conflicts in your community? Local Government and Authorities' Conflict Mitigation Role 4 22 46 28 10 10 58 22 15 17 42 26 0 10 20 30 40 50 60 70 Large Role Small Role No Role Don't Know Efforts to Address Conflicts - Stabilization Communities Over the past 12 months, have your local community, leaders, or organizations played a role in addressing community conflicts? Local Community Local Community Leaders Local Organizations 95 | YCST FINAL EVALUATION REPORT USAID.GOV 18 42 23 17 0 5 10 15 20 25 30 35 40 45 Improved Remained the Same Worsened Don't Know / No Response Conflict Resolution Mechanisms - Stabilization Communities In your opinion, have the existing conflict resolution mechanisms within your community improved, remained the same, or worsened? Conflict Resolution Mechanisms 18 14 39 29 0 10 20 30 40 50 Satisfied Neither Satisfied Nor Dissatisfied Dissatisfied Don't Know / No Response Satisfaction with Local Leaders' Performance - Stabilization Communities Satisfaction with Local Leaders' Performance 7 42 27 24 0 10 20 30 40 50 Trust More About the Same Trust Less Don't Know Level of Trust of Local Authorities and Government - Stabilization Communities Do you trust local authorities and government more, about the same, or less than you did 12 months ago? Level of Trust of Local Authorities and Government USAID.GOV YCST FINAL EVALUATION REPORT | 96 26 24 13 37 0 10 20 30 40 Good Bad Neither Good Nor Bad Don't Know Community and Local Authority Relationship - Stabilization Communities How would you describe the relationship between your community and local authorities? Community and Local Authority Relationship 39 21 39 0 10 20 30 40 50 Yes No Don't Know Community and Local Authority Relations - Stabilization Communities Is there a way to improve the relationship between your community and local authorites? 2 54 12 32 0 10 20 30 40 50 60 Increased Remained the Same Decreased Don't Know Ability to Participate in the Political Decision-Making Process over the Last Year - Stabilization Communities Ability to Participate in Political Decision-Making Process 97 | YCST FINAL EVALUATION REPORT USAID.GOV 26 40 2 5 9 10 8 0 10 20 30 40 50 Lack of Services Poverty and Living Conditions Security Situation Absence of Government Other No Problems / Challenges Don't Know Community Problems In your opinion, what is the most important problem facing your community/area? Problems Faced by the Community 36 56 8 0 10 20 30 40 50 60 Yes No Don't Know Community's Ability to Address Problems - Stabilization Communities Is you community capable of addressing the key problem impacting your community? Community's Ability to Address its Problems 26 70 4 0 20 40 60 80 Yes No Don't Know Presence of Conflict - Stabilization Communities Is there an ongoing conflict or conflicts impacting your household and community? Presence of Conflict USAID.GOV YCST FINAL EVALUATION REPORT | 98 80 0 0 0 1 0 2 17 51 2 1 1 23 2 0 21 67 2 4 2 0 0 4 22 42 6 0 0 0 0 43 9 0 10 20 30 40 50 60 70 80 90 Government Private Sector INGOs Local Organizations/ CSOs Community Project Other Service is not available Don't Know Provision of Basic Services - Stabilization Communities Who is primarily reponsible for providing basic services in your community? Electricity Health Services Water Education 43 73 53 50 49 22 35 33 7 5 12 18 0 20 40 60 80 Education Water Health Electricity Provision of Basic Services – Responsiveness to Community Needs - Stabilization Communities How responsive are the services providers to your community needs? Responsive Unresponsive Don't Know 99 | YCST FINAL EVALUATION REPORT USAID.GOV 50 20 2 29 0 10 20 30 40 50 60 Improved Remained the Same Worsened Don't Know Access to Basic Services - Stabilization Communities Has your access to services (YCT-supported) improved, remained the same, or worsened over the last year? Access to Services 14 46 26 13 0 10 20 30 40 50 Trust More About the Same Trust Less Don't Know Level of Trust Among Community Members - Stabilization Communities Do you trust other members of your community more, about the same, or less than you did 12 months ago? Level of Trust among Community Members 20 20 58 2 0 10 20 30 40 50 60 70 Moving in the Right Direction Some are moving in the Right Direction and some are moving the Wrong Direction Moving in the Wrong Direction Don't Know / No Response Overall Situation - Stabilization Communities Are things moving in the right direction or in the wrong direction? Overall Situation USAID.GOV YCST FINAL EVALUATION REPORT | 100 30 42 25 3 0 10 20 30 40 50 Improved Remained the Same Worsened Don't Know / No Response Peace and Security Situation - Stabilization Communities In your opinion, has the overall peace and security situation in your community improved, remained the same, or worsened over the last 12 months? Peace and Security Situation 17 70 13 0 10 20 30 40 50 60 70 80 Yes No Don't Know Awareness of YCST Activities - Stabilization Communities 71 14 14 0 0 10 20 30 40 50 60 70 80 Satisfied Neither Satisfied Nor Dissatisfied Dissatisfied Don't Know Level of Satisfaction with YCST Activities (n = 21) - Stabilization Communities Level of satisfaction with how YCST organized activities (N = 21) 101 | YCST FINAL EVALUATION REPORT USAID.GOV 11 72 17 0 20 40 60 80 Yes No Don't Know Awareness of Authority Community Committees (n = 125) - Stabilization Communities 86 14 0 0 20 40 60 80 100 Effective Ineffective Don't Know ACC Effectiveness (n = 14) - Stabilization Communities Capturing Community Feedback Capturing Community Feedback 86 14 0 0 20 40 60 80 100 Satisfied Dissatsifed Don't Know Satisfaction with ACCs Performance (n = 14) - Stabilization Communities Satisfaction with ACCs Performance USAID.GOV YCST FINAL EVALUATION REPORT | 102 57 36 7 0 10 20 30 40 50 60 Yes No Don't Know ACC Activities (n = 14) - Stabilization Communities Is your local ACC active in the community? CEC Activities 57 43 29 50 36 64 29 43 43 43 43 21 14 14 14 7 14 7 0 10 20 30 40 50 60 70 Representing Community Interests Selecting Community Projects Representing Youth Representing Women Addressing Conflicts Strengthening Relationships between Community Members Satisfaction with ACCs Performance (n = 14) - Stabilization Communities Satisfied Dissatisfied Neither Satisfied Nor Dissatisfied Don't Know 0 80 20 0 20 40 60 80 100 Yes No Don't Know Stabilization Communities Are you aware of any instances in which the YCST support to your community contributed to creating conflicts betweeen community members or with other communities? 103 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 5: CEC MEMBERS’ FINDINGS 24 18 7 2 0 5 10 15 20 25 30 Moving in the right direction Moving in the wrong direction Some are moving in the right direction, and some are moving in the wrong direction Don't know / No response Overall Situation in the Area (n = 51) 35 12 3 1 0 5 10 15 20 25 30 35 40 Worsened Remained the same Improved Don't know / No response Overall Economic Situation (n = 51) 23 14 13 1 0 5 10 15 20 25 Improved Remained the same Worsened Don't know / No response Peace and Security Situation (n = 51) USAID.GOV YCST FINAL EVALUATION REPORT | 104 27 23 1 0 5 10 15 20 25 30 No Yes No response Community Ability to Address Problems (n = 51) 23 17 8 3 0 5 10 15 20 25 Not at all involved in ending conflicts Somewhat involved in ending conflicts Very involved in ending conflicts Don't know / No response Local Authorities Involvement in Mitigation or Ending Conflicts (n = 51) 34 32 24 21 16 12 0 5 10 15 20 25 30 35 40 NGOs CECs Sheikhs Local organizations Local leaders Local government authorities Organizations/Institutions/Individuals that are Most Effective to Address Conflicts (Multiple Responses Allowed) 105 | YCST FINAL EVALUATION REPORT USAID.GOV 49 2 0 10 20 30 40 50 60 Received at least one trainings Did not receive any training Trainings Received as Members of the CEC (n = 51) 41 8 2 0 5 10 15 20 25 30 35 40 45 Very useful Somewhat useful Don't know Usefulness of Trainings Provided (n = 51) 5 10 14 4 9 3 1 5 0 2 4 6 8 10 12 14 16 More than once a week About once per week About once per month About twice a month About once every two months About once every three months About once every six months Has not met Frequency of CEC Meetings Over the Last Six Months (n = 51) USAID.GOV YCST FINAL EVALUATION REPORT | 106 46 2 2 1 0 10 20 30 40 50 Satisfied Dissatisfied Don't know / No response Neither satisfied nor dissatisfied CEC Members Satisfaction with the PACE Approach (n = 51) 48 2 1 0 10 20 30 40 50 60 Yes No No Response YCST Service Delivery Projects Contribution to Reducing Conflicts (n = 51) 107 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 6: EVALUATION INSTRUMENTS 1. YCST SENIOR MANAGEMENT DISCUSSION GUIDE............................108 2. USAID DISCUSSION GUIDE........................................................................110 3. YCST PROJECT STAFF DISCUSSION GUIDE – PACE............................112 4. YCST PROJECT STAFF DISCUSSION GUIDE – CAPACITY BUILDING 115 5. YCST PROJECT STAFF DISCUSSION GUIDE – STABILIZATION .......118 6. YCST PARTNERS DISCUSSION GUIDE ....................................................121 7. YCST COMMUNITY MOBILIZER DISCUSSION GUIDE.........................122 8. YCST MONITORING, EVALUATION, AND LEARNING TEAM DISCUSSION GUIDE .........................................................................................125 9. COMMUNITY ENHANCEMENT COMMITTEE MEMBER KEY INFORMANT INTERVIEW GUIDE..................................................................127 10. COMMUNITY MEMBERS (PACE AND STABILIZATION COMMUNITIES)..................................................................................................133 11. LOCAL AUTHORITY KEY INFORMANT INTERVIEW GUIDE – PACE COMMUNITIES...................................................................................................141 12. LOCAL AUTHORITY KEY INFORMANT INTERVIEW GUIDE – STABILIZATION COMMUNITIES...................................................................143 13. GRANTEE KEY INFORMANT INTERVIEW GUIDE ...............................148 USAID.GOV YCST FINAL EVALUATION REPORT | 108 1. YCST SENIOR MANAGEMENT DISCUSSION GUIDE 1. How long have you been working on or worked on the YCST project? Please tell us about the nature of your role/s and responsibilities? 2. What key outcomes was this project designed to achieve? a. What types of conflict was the project trying to address? b. What aspect of community cohesion was it trying to strengthen? c. What type of capacity gaps was it trying to address? 3. How did you select the communities in which to work? a. What criteria did you use to select the initial set of communities? [Probe: access, security, USAID interest, frequency/nature of intra and inter-communal conflict] b. How did the project assess the existing level of conflict in communities when determining which communities to select? [Probe: what type of data sources informed the understanding of conflict levels in potential communities] c. Are YCST activities designed to address specific types of conflict within communities? d. How did the selection process for phase 2 communities differ from the first phase? 4. What was the rationale for implementing the PACE approach in some communities and stabilization approaches in other communities? a. What criteria did you use to decide whether to implement the PACE or stabilization approach in a community? b. Why were the two approaches (participatory action for community enhancement (PACE) and stabilization) applied in different communities? 5. Did your implementation approach vary based on the nature of conflict, and the existing levels of cohesion in the community? If so, please share how? 6. What is your appraisal of the overall success of this project, and why? a. To what extent has this project achieved its intended outcomes? i. Key outcomes achieved. 1. How sustainable are these outcomes and why? 2. What are the key threats to the sustainability of these outcomes? ii. Was the project more successful in some communities than in others? What factors contributed to greater success in some communities than in others? iii. Were there certain communities where YCST struggled to implement activities? Which communities and why? 7. Was the PACE or the stabilization approach most effective in achieving intended outcomes? What factors influenced the relative effectiveness of the two approaches? 8. Can you describe the project’s Theory of Change? a. What aspects of the Theory of Change for this activity have held true based on your experience and based on your learning and M&E data? Why? 109 | YCST FINAL EVALUATION REPORT USAID.GOV b. Did the key assumptions of the Theory of Change hold? In case not, please describe. c. Are there aspects that need to be rethought? If so, which ones and why? 9. Do the technical teams supporting different objectives coordinate their activities? If so, please share details. 10. In your view, what are the key challenges this project faced in achieving its planned outcomes at the intended scale and within the planned time frame? 11. What aspects of implementing this project worked well, and which did not? What would you do differently if you implemented this type of project again? [Probe: how suitable was the PACE model for the context in Yemen?] 12. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. 13. Did the YCST activity interact (coordinate, collaborate, etc.) with other USAID, donor, or local activities? Please elaborate (probe: nature of interaction, how well the collaboration worked, challenges with coordination). 14. Given that a key focus of this project was on learning and adapting, can you please share the key modifications and adaptations? How, if at all, did the project adapt its approach based on what it learned during implementation? Please give specific examples. 15. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 110 2. USAID DISCUSSION GUIDE 1. How long have you been involved in or engaged with the YCST project? Please tell us about the nature of your involvement. a. Were you involved in the YCST design? If so, please share the role you played and your overall experience? b. Were you involved in managing YCST? If so, please share the role you played and your overall experience? 2. What key outcomes was this project designed to achieve? a. What types of conflict was the project trying to address? b. What aspect of community cohesion was it trying to strengthen? c. What type of capacity gaps was it trying to address? 3. What was the rationale for implementing the PACE approach in some communities and stabilization approaches in other communities? a. What criteria did you use to decide whether to implement the PACE or stabilization approach in a community? b. Why were the two approaches (PACE and stabilization) applied in different communities? 4. What is your appraisal of the overall success of this project, and why? a. To what extent has this project achieved its intended outcomes? i. Key outcomes achieved. 1. How sustainable are these outcomes and why? 2. What are the key threats to the sustainability of these outcomes? ii. Were there certain communities where YCST was more successful than others? Please elaborate. iii. Were there certain communities where YCST struggled to implement activities? Which communities and why? 5. Was the PACE or the stabilization approach most effective in achieving intended outcomes? What factors influenced the relative effectiveness of the two approaches? 6. What is your assessment of how effectively the technical teams supporting different objectives within YCST coordinated their activities and collaborated with each other? Please elaborate. 7. Can you describe the project’s Theory of Change? a. What aspects of the Theory of Change for this activity have held true based on your experience and understanding of the learning and M&E data? Why? b. Did the key assumptions of the Theory of Change hold? In case not, please describe. c. Are there aspects that need to be rethought? If so, which ones and why? 8. In your view, what are the key challenges this project faced, and how did they affect the project’s ability to achieve its outcomes at the intended scale and within the planned time frame? 111 | YCST FINAL EVALUATION REPORT USAID.GOV 9. In your view, and based on your experience with the YCST project, what aspects of the project worked well, and which did not? What should be done differently on future awards of this nature? 10. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. 11. Are you aware of how the YCST activity interacted with (coordination, collaboration, etc.) with other USAID, donor, or local activities? Please elaborate (probe: nature of interaction, how well the collaboration worked, challenges with coordination). 12. Given that a key focus of this project was on learning and adapting, what is your assessment of how effectively the project adapted its approach based on what it learned during implementation? 13. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 112 3. YCST PROJECT STAFF DISCUSSION GUIDE – PACE 1. Please tell me about how long you have been involved with the YCST project? What is your role? Community Selection Process 2. How did the project team select the communities in which to work? a. What criteria did you use to select the initial set of communities? [Probe: access, security, USAID interest, frequency/nature of intra and inter-communal conflict] b. How did the project assess the existing level of conflict in communities when determining which communities to select? [Probe: what type of data sources informed the understanding of conflict levels in potential communities?] c. Are YCST activities designed to address specific types of conflict within communities? d. How did the selection process for phase 2 communities differ from the first phase? 3. Did the implementation approach you selected for a community depend on the nature of conflict and the existing levels of cohesion in the community? If so, please share how? Community Engagement Process 4. Please tell us about the initial training you received on the PACE process. a. What did you learn? [Probe: community-based facilitation skills, community profiling, conflict analysis, and mitigation, forming and building the capacity of the Community Enhancement Teams, Do No Harm principles, and applications] b. Did you find the training relevant to your role? How could it have been improved? 5. How did you introduce the PACE approach to the selected communities? Who was involved in these introductions and over what period did they take place? 6. Please tell us about the community engagement process in the PACE communities. a. How did the project team identify community assets and key stakeholders? b. Did the project team engage women, youth, and other marginalized groups during this process? If so, how? c. How did the project team identify CEC members in the PACE communities? d. Did the household surveys affect—positively or negatively—the community engagement process? How did the community view the surveys? How did the CECs view the surveys? 7. Are CECs involved in operating, overseeing, or maintaining the facilities delivered to their communities? If yes, do you think they will continue to monitor or be involved in this maintenance or facility operation when the YCST program ends? Why? 8. In your view, how effective have the CECs been in supporting and overseeing the service delivery projects in their communities? 113 | YCST FINAL EVALUATION REPORT USAID.GOV Engaging Leaders and Local Authorities 9. Can you describe your approach for engaging local leaders and authorities in the areas where you implemented PACE activities? Social Cohesion 10. Have the YCST PACE activities—for example, working with CECs to identify and prioritize community needs—affected the relationship between community members? Which aspect/s of the relationship? How? Please share examples. 11. Have the YCST PACE activities affected—positively or negatively—the relationship between community members and local government entities and civil society organizations? Which aspect/s of the relationship? How? Please share examples. Conflict and Conflict Mitigation 12. What type of conflicts were/are you trying to address through the YCST PACE approach? 13. Do you think that the services/projects delivered by YCST in the PACE communities have mitigated conflict in the communities? If yes, how? If no, why not? Please provide examples. 14. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. Sustainability 15. Have you coordinated with organizations or community groups that support your objectives in the YCST communities? If so, how? 16. Do you think CECs will continue to meet/function after the YCST program ends? Do you think this mechanism or approach of community-driven projects will continue? a. If yes, how do you think CECs can be sustained? What are challenges to sustaining the CECs? b. Are CECs capable of mobilizing resources independently for future service delivery projects? c. Do you think CECs have the capacity to identify and address future conflicts that arise? Please explain. 17. Do you expect the outcomes achieved by your activities to sustain over time? [Probe: focus on outcomes around social cohesion and conflict mitigation] a. Why do you say that? b. What are the key threats to sustaining these outcomes? USAID.GOV YCST FINAL EVALUATION REPORT | 114 Challenges 18. What have been the biggest challenges you have faced in implementing the YCST activities? [Probe: government approvals, budget/resource constraints, security/access, challenges with vendors during service delivery projects, capacity issues] a. Have the challenges been different based on the different types of YCST communities? How so? b. How have challenges evolved over the life of the project? c. How did you address the challenges? 19. How, if at all, has the COVID-19 pandemic affected how you implement YCST activities? a. What have been the main challenges in working with PACE communities during the COVID-19 pandemic? b. How have you adapted the way you implemented activities to address these challenges? [Probe for frequency of visits, meetings with CEC members/grantees/local authorities, feasibility/impact of virtual trainings/sessions] Adaptive Learning and Management 20. Did you adapt project implementation to learning from YCST’s activity monitoring and assessment work? a. For example, did your team use the findings from the baseline study and other data generated by your M&E and technical teams to guide implementation? Explain with specific examples. b. If not, why not. 21. How did you track gender and inclusion considerations across the activities? 22. Have some monitoring, evaluation, and learning products—for example, the community dashboard and lessons learned reports—generated by the team been more effective than others at supporting adaptive management? If yes, which ones? 23. Has the monitoring, evaluation, and learning team effectively worked with different members of the YCST project? If yes, how? If not entirely, why not? 24. How, if at all, has implementation of the PACE model changed between Phase 1 and Phase 2 communities? [Probe: approach to community engagement, training content, stakeholder selection, formation/training delivery, approach to working with vendors] 25. Have any findings from the monitoring, evaluation, and learning efforts led to changes in the PACE programming, in terms of scaling up, eliminating, or replicating activities? If yes, which? 26. Is there anything else that you would like to share with us? 115 | YCST FINAL EVALUATION REPORT USAID.GOV 4. YCST PROJECT STAFF DISCUSSION GUIDE – CAPACITY BUILDING 1. Please tell me about how long you have been involved with the YCST project? What is your role? Community Selection Process 2. How did the project team select the communities for Phase 1 and Phase 2 activities? Capacity Building 3. Please describe your grant-making process. a. How are grantees selected? b. How do you track their performance? 4. Did your implementation approach vary based on the nature of conflict and the existing levels of cohesion in the community? If so, please share how? 5. How do you ensure that the capacity-building grants contribute to other YCST objectives including PACE and stabilization? a. Are there specific successes of your capacity-building efforts that you would like to highlight? What accounted for this success? 6. What is your approach for coordinating with the YCST teams working on PACE and stabilization activities? Has either the PACE or stabilization activities been more successful than the other in achieving expected outcomes? If so, please share more details and examples. 7. Have you experienced any challenges related to capacity-building efforts? If yes, how did they affect your work? a. Have the challenges been different based on the different types of YCST communities? How so? b. How have challenges evolved over the life of the project? c. How did you address the challenges? 8. What aspects of capacity-building efforts worked well, and which did not? What would you do differently if you implemented this type of project again? Social Cohesion 9. Have the YCST capacity-building activities affected relationships among community members? Which aspect/s of the relationship? How? Please share examples of any positive and negative (unintended) effects. 10. Have the YCST capacity-building activities affected—positively or negatively—the relationship between community members and local government entities and civil society organizations? USAID.GOV YCST FINAL EVALUATION REPORT | 116 Which aspect/s of the relationship? How? Please share examples of any positive and negative (unintended) effects. Conflict and Conflict Mitigation 11. What type of conflicts were/are you trying to address through the YCST capacity-building efforts? 12. Do you think that the capacity-building efforts have mitigated conflicts in communities? If yes, how? If no, why not? 13. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. Sustainability 14. Have you coordinated with other organizations or community groups that support your objectives in the YCST communities? If so, how? a. How, if at all, do YCST activities complement other ongoing activities in the community? 15. Do you think the grantees you have supported will continue to function and apply what you have trained them on? Why or why not? 16. Do you expect the outcomes YCST achieved will last beyond the end of the project? [Probe: focus on outcomes around social cohesion and conflict mitigation] a. Why do you say that? b. What are the key threats to sustaining these outcomes? Challenges 17. What are the greatest challenges you faced in implementing YCST capacity-building activities? a. Have the challenges differed across communities and/or the types of organizations you supported? If so, how? b. How have challenges evolved over the life of the project? c. How did you address the challenges? 18. How, if at all, has the COVID-19 pandemic affected the way you implemented the YCST capacity-building activities? a. What have been the main challenges in working with communities during the COVID￾19 pandemic? b. How have you adapted the way you implemented activities to address these challenges? [Probe for frequency of visits, meetings with community members/grantees/local authorities, feasibility/impact of virtual trainings/sessions] Adaptive Learning and Management 19. Did you adapt project implementation to learning from YCST’s activity monitoring and assessment work? 117 | YCST FINAL EVALUATION REPORT USAID.GOV a. For example, did your team use the findings from the baseline study and other data generated by your M&E and technical teams to guide implementation? Explain with specific examples. b. If not, why not. 20. How, if at all, did you track gender and inclusion considerations across the activities? 21. Have some monitoring, evaluation, and learning products—for example, the community dashboard and lessons learned reports—been more effective than others at supporting learning and adaptive management? If yes, which ones? 22. Has the monitoring, evaluation, and learning team effectively worked with different members of the YCST project? If yes, how? If not entirely, why not? 23. How, if at all, has the implementation of the capacity-building model changed between Phase 1 and Phase 2 communities? [Probe: approach to community engagement, training content, stakeholder selection, formation/training delivery, approach to working with vendors] 24. Have any findings from the monitoring, evaluation, and learning efforts led to changes in YCST programming, in terms of scaling up, eliminating, or replicating activities? If yes, which? 25. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 118 5. YCST PROJECT STAFF DISCUSSION GUIDE – STABILIZATION 1. Please tell me about how long you have been involved with the YCST project? What is your role? Community Selection Process 2. How did you select the communities in which to work? a. What criteria did you use to select the initial set of communities? [Probe: access, security, USAID interest, frequency/nature of intra and inter-communal conflict] b. How did the project assess the existing level of conflict in communities when determining which communities to select? [Probe: what type of data sources informed the understanding of conflict levels in potential communities?] c. Are YCST activities designed to address specific types of conflict within communities? d. How did the selection process for Phase 2 communities differ from the first phase? 3. Did your implementation approach vary based on the nature of conflict, and the existing levels of cohesion in the community? If so, please share how? Engaging Leaders and Local Authorities 4. How did you engage local leaders and authorities in your stabilization activities? 5. How did you create the Authority Community Committees (ACC)? 6. Whether and how are the ACCs different from the existing local leadership and authority structures? 7. What kind of support did/do you provide as part of the stabilization activities? 8. Do you have a formal plan for sustaining your stabilization activities and outcomes? Explain. Social Cohesion 9. Have the YCST stabilization activities affected the relationship between community members? Which aspect/s of the relationship? How? Please share examples. 10. Have the YCST stabilization activities affected—positively or negatively—the relationship between community members and local government entities and civil society organizations? Which aspect/s of the relationship? How? Please share examples. Conflict and Conflict Mitigation 11. What type of conflicts were/are you trying to address through the YCST stabilization approaches? 119 | YCST FINAL EVALUATION REPORT USAID.GOV 12. Do you think that the services/projects delivered by YCST in the stabilization communities have mitigated the conflicts in the communities? If yes, how? If no, why not? 13. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. Government Responsiveness and Credibility 14. How successful, if at all, have stabilization activities been at enhancing government credibility and community perceptions related to overall government responsiveness? a. Have they been more successful in some communities than others? If so, please share examples and the possible reasons why. Sustainability 15. Have you coordinated with organizations or community groups that support your objectives in the YCST communities? If so, how? 16. Do you expect the outcomes achieved by your activities to sustain over time? [Probe: focus on outcomes around social cohesion and conflict mitigation] a. Why do you say that? b. What are the key threats to sustaining these outcomes? Challenges 17. What are the greatest challenges you faced implementing YCST activities [probe: government approvals, budget/resource constraints, security/access, challenges with vendors during service delivery projects, capacity issues]? a. Have you faced different types of challenges in different types of YCST communities? How so? b. How have challenges evolved over the life of the project? c. How did you address the challenges? 18. How, if at all, has the COVID-19 pandemic impacted the way that you have implemented the YCST activities? a. What have been the main challenges in working with stabilization communities during the COVID-19 pandemic? b. How have you adapted the way you implemented activities to address these challenges? [probe: for frequency of visits, meetings with local authorities, feasibility/impact of virtual trainings/sessions] Adaptive Learning and Management 19. Did you adapt project implementation to learning from YCST’s activity monitoring and assessment work? a. For example, did your team use the findings from the baseline study and other data generated by your M&E and technical teams to guide implementation? Explain with specific examples. USAID.GOV YCST FINAL EVALUATION REPORT | 120 b. If not, why not? 20. How did you track gender and inclusion considerations across your activities? 21. Have certain monitoring, evaluation, and learning products—for example, the community dashboard and lessons learned reports—generated by the YCST M&E team been more effective than others at supporting adaptive management? If yes, which ones? 22. Has the monitoring, evaluation, and learning team effectively worked with different members of the YCST project? If yes, how? If not entirely, why not? 23. How, if at all, has implementation of the stabilization model changed between Phase 1 and Phase 2 communities? [probe: approach to community engagement, training content, stakeholder selection, formation/training delivery, approach to working with vendors] 24. Have any findings from the monitoring, evaluation, and learning efforts led to changes in your programming, in terms of scaling up, eliminating, or replicating activities? If yes, which? 25. Is there anything else that you would like to share with us? 121 | YCST FINAL EVALUATION REPORT USAID.GOV 6. YCST PARTNERS DISCUSSION GUIDE Below we provide a preliminary list of questions for the YCST partner interviews. These will be updated following interviews with the core members of the YCST team. Partners for Good 1. Did you have to adapt the PACE trainings to fit the Yemen context? Explain. 2. Can you briefly describe the training you delivered to the YCST and local partner staff? 3. What, if any, challenges did you face delivering the trainings? [Probe: challenges that were operational, context-specific, thoroughness of training content] 4. How, if at all, has the COVID-19 pandemic affected how you trained and worked with the Global Communities team? a. How, if at all, have you adapted the ongoing technical assistance you provide to community mobilization staff because of COVID-19? [Probe for frequency of visits, meetings with community mobilizers, feasibility/impact of virtual trainings/sessions, any change in the level of interaction with YCST staff] 5. Did you learn anything while delivering PACE training to Phase 1 communities that caused you to change your training approach for Phase 2 communities? If yes, what types of changes? [Probe: training approach, virtual platforms, training content, communication with staff and mobilizers] 6. How could the training and ongoing technical assistance be improved for future programs that use this model? 7. Is there anyone else that you think we should speak to about the YCST program? Field Medical Foundation and Build Up 8. What kind of training and ongoing capacity-building have you received from YCST staff? [IN CASE NOT MENTIONED PROBE TRAINING TOPICS: USAID regulations on ethics, procurement, finance, and staffing; community-based facilitation skills, community profiling, conflict analysis, and mitigation, forming and building the capacity of the Community Enhancement Teams, Do-No-Harm principles, and applications] a. Did you find the capacity-building you received useful in preparing you for supporting this project? Explain. b. How could the capacity-building have been improved? 9. Do you think you will continue to use the skills you learned in YCST trainings when the project ends? If yes, how will you apply these skills outside of the project context? 10. Is there anyone else that you think we should speak to about the YCST program? USAID.GOV YCST FINAL EVALUATION REPORT | 122 7. YCST COMMUNITY MOBILIZER DISCUSSION GUIDE 1. Please tell me about how long you have been involved with the YCST project? What is your role? 2. Please tell me about the initial training you received on the PACE process. a. What topics did you learn about [Probe: community-based facilitation skills, community profiling, conflict analysis, and mitigation, forming and building the capacity of the Community Enhancement Teams, Do-No-Harm principles, and applications] b. Did you find the training useful in preparing you for your role as a community mobilizer? How could it have been improved? 3. How did you introduce the PACE approach to the communities you work in? How does the community see your role as a community mobilizer? 4. Please tell me about the community engagement process in the PACE communities. a. How often did you meet with community members? b. How did you identify community assets and key stakeholders? c. Did you engage women, youth, and other marginalized groups during this process? If so, how? d. Did the household surveys affect—positively or negatively—the community engagement process? How did the community view the surveys? How did the CECs view the surveys? 5. How did YCST identify CEC members in the PACE communities? What was your role in this selection process? 6. What kind of training did you deliver to the CEC members? How often did you typically meet with the CEC in a PACE community? [Probe training topics: community engagement principles, gender equity, conflict sensitivity] a. Which trainings do you think were the most needed and useful for CEC members? b. In your opinion, were there any trainings that CEC members needed but did not receive? If so, what were they? 7. Did you work with the CEC members to hold community action planning meetings, community surveys, and participatory conflict analysis? Please describe this process. 8. How, if at all, has the COVID-19 pandemic impacted the way that you trained and worked with the CECs? a. What have been the main challenges in implementing the PACE model in communities during the COVID-19 pandemic? b. How did you adapt the way you conducted capacity-building/training? How have you adapted community engagement activities in general? [Probe for frequency of visits, meetings with CEC members, feasibility/impact of virtual trainings/sessions, any change in the level of interaction with YCST staff] 123 | YCST FINAL EVALUATION REPORT USAID.GOV 9. How effective have the CECs been in supporting and overseeing the service delivery projects in their communities? Social Cohesion 10. Have the YCST PACE activities affected the relationship between community members? Which aspect/s of the relationship? How? Please share examples. 11. Have the YCST PACE activities affected—positively or negatively—the relationship between community members and local government entities and civil society organizations? Which aspect/s of the relationship? How? Please share examples. Conflict and Conflict Mitigation 12. What type of conflicts were/are you trying to address through the PACE approach? 13. Do you think that the services/projects delivered by YCST through PACE activities mitigated the conflicts in your community? If yes, how? If no, why not? Please provide examples. 14. Have you received any negative feedback regarding the PACE activities in your communities? If so, please share the feedback you have received. 15. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. 16. Are you satisfied with how the PACE model has worked in the communities where you are a YCST mobilizer? a. What are the key outcomes these activities achieved or contributed to? Challenges 17. What have been the biggest challenges you have faced in implementing the PACE model in your communities? [Probe: government approvals, budget/resource constraints, security/access, challenges with vendors during service delivery projects, capacity issues] b. How have challenges evolved over the life of the project? c. How did you address the challenges? Sustainability/Learning 18. Do you think CECs will continue to meet/function after the YCST program ends? Why do you say that? 19. Has the implementation of the PACE model changed between Phase 1 and Phase 2 communities? If so, how did the team decide on the changes? [Probe: approach to community engagement, training content, CEC selection, CEC formation/training delivery] 20. If you were to recommend changes to the PACE process, what would you propose and why? 21. Is there anyone else that you think we should speak to about the YCST program? USAID.GOV YCST FINAL EVALUATION REPORT | 124 22. Is there anything else that you would like to share with us? 125 | YCST FINAL EVALUATION REPORT USAID.GOV 8. YCST MONITORING, EVALUATION, AND LEARNING TEAM DISCUSSION GUIDE 1. Please tell me about how long you have been involved with the YCST project? What is your role? 2. How did the ongoing activity monitoring and assessment work support adaptive learning on the project? For example, did the project team use the findings from the baseline study and other data generated by your monitoring, evaluation, and learning team to guide implementation? Please explain with specific examples. 3. How did you track gender and inclusion considerations across the activities? 4. Have some monitoring, evaluation, and learning products—for example, the community dashboard and lessons learned reports—generated by your team been more effective than others at supporting adaptive management? If yes, which ones? How were they more effective? 5. How did your monitoring, evaluation, and learning team ensure that it worked effectively with other YCST teams to collect data and share learning? 6. Did you face any challenges implementing best monitoring, evaluation, and learning practices? Can you provide some examples? 7. How, if at all, has the learning that came out of project M&E informed the implementation approaches in Phase 1 and Phase 2 communities? Please provide examples. [IN CASE NOT MENTIONED PROBE: approach to community engagement, training content, stakeholder selection, formation/training delivery, approach to working with vendors] 8. Has any learning from the monitoring, evaluation, and learning efforts led to changes in the YCST activities, in terms of scaling up, eliminating, or replicating activities? If yes, give examples? 9. What, if anything, do your M&E data and activities suggest about the effects YCST has had on: a. Social cohesion b. Mitigating conflict c. Capacity of local organizations and leaders to mitigate conflicts d. Mitigation of conflict through service delivery activities e. Other outcomes (Please Specify) 10. What, if anything, do your M&E data and activities suggest about the likelihood that communities will be able to sustain these outcomes after YCST ends? Give examples. [Probe: focus on outcomes around social cohesion and conflict mitigation] 11. Are you aware of any instances in which the YCST support to specific communities contributed to creating conflicts, whether between the community members or between communities receiving assistance and those who did not? Please explain. USAID.GOV YCST FINAL EVALUATION REPORT | 126 12. What are the most important challenges to sustaining outcomes? [Probe: focus on outcomes around social cohesion and conflict mitigation] 13. Based on the contextual data generated by your team, what, if any, challenges did YCST face that impacted the activities and outcomes of the YCST project? How did they impact the project in achieving its development objectives at the intended scale and on time? 14. Based on the learning data generated by your team, what are the key lessons learned by the project? a. What aspects of the project worked particularly well for YCST communities? Why? b. What would you do differently if implementing this kind of program in a similar context? Why? 15. Is there anything else that you would like to share with us? 127 | YCST FINAL EVALUATION REPORT USAID.GOV 9. COMMUNITY ENHANCEMENT COMMITTEE MEMBER KEY INFORMANT INTERVIEW GUIDE Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent organization (MSI/Data Collection Firm Reference) that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community, your experience as a CEC member. The interview is estimated to last 30 to 40 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” NOTE FOR INTERVIEWERS: THE RESPONSE OPTIONS ARE PRE-CODES AND SHOULD NOT BE READ TO THE RESPONDENTS. IF A RESPONSE MATCHES ONE OF THE EXISTING PRECODES, PLEASE SELECT IT, AND WHERE RELEVANT ASK THE FOLLOW-UP QUESTION/S. Demographics 1. Respondent sex [To be answered by the enumerator: Do not ask] a. Male b. Female 2. What is your current age? (NUMERIC ANSWER (18–100)) a. __________ b. Don’t know (DO NOT READ) 3. What is the highest level of education (grade) you have completed? a. No education b. Elementary c. Primary/Basic d. Secondary e. Vocational f. Diploma g. University degree (BA) h. Other advanced degree, specify: 4. How long have you been living in this community? a. Less than 6 months b. Between 6 months and 2 years USAID.GOV YCST FINAL EVALUATION REPORT | 128 c. Between 2 and 10 years d. More than 10 years Background and Contextual Questions 5. In your opinion, are things generally moving in the right direction or the wrong direction in your area over the last 6 months? Select one: 1 Moving in the right direction Why do you say that? What has improved? 2 Moving in the wrong direction Why do you say that? What has gotten worse? 3 No opinion 4 Don’t know (DO NOT READ) 6. When it comes to the overall peace and security situation in your community, has the situation improved, worsened, or remained the same, over the last year? Select one: Why do you say that? 1 Improved 2 Remained the same 3 Worsened 4 Don’t know 7. When it comes to the overall economic situation in your community, has the situation improved, worsened, or remained the same, over the last year? Select one: Why do you say that? 1 Improved 2 Remained the same 3 Worsened 4 Don’t know 8. In your opinion, what is the most important problem facing your community/area? 9. In your view, is your community capable of getting together and addressing this problem? Select one: 1 Yes How? 2 No Why not? 129 | YCST FINAL EVALUATION REPORT USAID.GOV 10. What type of support do you and your community members need to be able to resolve this problem? 11. Is presence of conflict a concern for you, your family, and members of your community? Select one: 1 Yes Please explain, including • What type of conflict/s? • What are the reasons/causes/sources/drivers of these conflicts? 2 No 12. In your view, is your community capable of getting together and addressing these conflicts? Select one: 1 Yes How? 2 No Why not? 13. What type of support do you and your community need to be able to resolve the conflicts affecting your community? 14. Over the last year, what role, in case of any, have your local government/authorities played in mitigating or ending conflicts in or around your community? 15. Over the last year, what role, in case of any, has the local community played in mitigating or ending conflicts in or around your community? 16. Over the last year, what role, in case of any, have local organizations played in mitigating or ending conflicts in or around your community? 17. In your view, which organization, group, individual, or institution offers the most effective mechanism for addressing conflicts impacting your community? Selection and Community Engagement 18. How did you learn about the YCST program and PACE process? 19. How did you become a member of the CEC? 20. What is your role as a CEC member? How does the community see your role? 21. How does your role as a CEC member differ from that of the local authorities or civil society organizations? How are the roles different, and how do they overlap? 22. What type of training/s did you receive as a CEC member? Select all that apply: a. Community engagement principles b. Gender equity USAID.GOV YCST FINAL EVALUATION REPORT | 130 c. Conflict sensitivity d. Grievance mechanisms e. No training f. Other, specify 23. How useful were the trainings you received as a CEC member? Select one: Please share the reasons for your response: 1 Very useful 2 Somewhat useful 4 Not at all useful 24. How, if at all, have you applied the skills you gained from the training(s) in your work in the community? 25. In your opinion, can these trainings be improved? Select one: 1 Yes How? 2 No PACE Process 26. Please tell me about the activities conducted by the CEC in your community? 27. During the past 6 months, about how often has the CEC met? a. More than once a week b. About once per week c. About once per month d. About once every 3 months e. About once every 6 months f. Has not met g. Other, specify 28. Who organizes and leads CEC meetings? a. Community mobilizer b. Head of CEC c. YCST staff d. Other, specify: 29. How did you introduce the PACE approach to the communities you work in? 30. How did you identify needs in your community? 31. What activities did you undertake as a member of the CEC? Select all that apply: a. Held community-wide action planning meetings b. Conducted community surveys 131 | YCST FINAL EVALUATION REPORT USAID.GOV c. Conducted participatory conflict analysis d. Engaged women and youth during the PACE process? 32. How did you build consensus on which needs in your community should be prioritized? 33. How, if at all, was the CEC involved in the project(s) implemented by YCST in your community? 34. How, if at all, did your CEC address and respond to the needs of women and youth? Challenges 35. How, if at all, did the COVID-19 pandemic impact the CEC activities? 36. What have been the biggest challenges you have faced in your role as a CEC member? Conflict Mitigation 37. Which sources of conflict were your activities trying to address/mitigate? 38. Do you think that the services/projects delivered by YCST reduced these conflicts? Select one: 1 Yes How? 2 No Why not? 39. Are you aware of any instances in which the YCST support to your community contributed to creating conflicts, whether between the community members or between your community and those communities who did not receive any assistance? Please explain. 40. Are you satisfied, dissatisfied, or neither satisfied nor dissatisfied with how the PACE process has worked in your community? Select one: Why do you say that? 1 Satisfied 2 Dissatisfied 3 Neither satisfied nor dissatisfied 4 Don’t know (DO NOT READ) 41. How would you describe the relationship between the CEC and local authorities/government? 42. Did YCST undermine or negatively affect any pre-existing conflict mitigation practices in the community? If yes, how? Sustainability USAID.GOV YCST FINAL EVALUATION REPORT | 132 43. Have you coordinated with other organizations or community groups in the PACE communities engaged in community engagement work? Select one: 1 Yes How? 2 No 44. Do you think it is very likely, somewhat likely, somewhat unlikely, or very unlikely that the CEC will continue to meet and function after the YCST program ends? Select one: Why do you say that? 1 Very likely 2 Somewhat likely 3 Somewhat unlikely 4 Very unlikely 5 Don’t know (DO NOT READ) 45. Do you have any recommendations related to how the CECs and YCST project can be further improved? 46. Is there anyone else that you think we should speak to about the YCST program? 47. Is there anything else that you would like to share with us? 133 | YCST FINAL EVALUATION REPORT USAID.GOV 10. COMMUNITY MEMBERS (PACE AND STABILIZATION COMMUNITIES) Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent organization (MSI/Data Collection Firm Reference) that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community and to better understand any service delivery challenges. The interview is estimated to last 30 to 40 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” NOTE FOR INTERVIEWERS: THE RESPONSE OPTIONS ARE PRE-CODES AND SHOULD NOT BE READ TO THE RESPONDENTS. IF A RESPONSE MATCHES ONE OF THE EXISTING PRECODES, PLEASE SELECT IT, AND WHERE RELEVANT ASK THE FOLLOW-UP QUESTION/S. Demographics 1. Respondent sex [To be answered by the enumerator: Do not ask] a. Male b. Female 2. What is your current age? (NUMERIC ANSWER (18–100)) a. __________ b. Don’t know (DO NOT READ) 3. What is the highest level of education (grade) that you have completed? a. No education b. Elementary c. Primary/Basic d. Secondary e. Vocational f. Diploma g. University degree (BA) h. Other advanced degree, specify: 4. How long have you been living in this community? a. Less than 6 months b. Between 6 months and 2 years USAID.GOV YCST FINAL EVALUATION REPORT | 134 c. Between 2 and 10 years d. More than 10 years Background and Contextual Questions 5. In your opinion, are things generally moving in the right direction or the wrong direction in your area over the last 6 months? Select one: 1 Moving in the right direction Why do you say that? What has improved? 2 Moving in the wrong direction Why do you say that? What has gotten worse? 3 No opinion 4 Don’t know (DO NOT READ) 6. In your opinion, has the overall peace and security situation in your community improved, remained the same, or worsened over the last 12 months? Select one: Why do you say that? 1 Improved 2 Remained the same 3 Worsened 4 Don’t know 7. When it comes to the overall economic situation in your community, has the situation improved, worsened, or remained the same, over the last 12 months? Select one: Why do you say that? 1 Improved 2 Remained the same 3 Worsened 4 Don’t know 8. In your opinion, what is the most important problem facing your community/area? 9. In your view, is your community capable of addressing this problem? Select one: 1 Yes How? 2 No Why not? 10. What type of support, if any, does your community need to be able to resolve this problem? 135 | YCST FINAL EVALUATION REPORT USAID.GOV 11. Is there an ongoing conflict or conflicts impacting your household and community? Select one: 1 Yes Please explain, including • What type of conflict/s? • What are the reasons/causes/sources/drivers of these conflicts? 2 No 12. In your view, is your community capable of addressing these conflicts? Select one: 1 Yes How? 2 No Why not? 13. What type of support do you and your community need to be able to resolve the conflicts affecting your community? 14. Over the past 12 months, have your local government/authorities done anything to address conflicts in or around your community? Explain. 15. Over the last year, what role, in case of any, has the local community played in mitigating or ending conflicts in or around your community? 16. What role have local community leaders played in resolving conflicts in or around your community? 17. Over the last year, what role, in case of any, have local organizations played in mitigating or ending conflicts in or around your community? 18. In your view, which organization, group, individual, or institution has been most effective in addressing conflicts impacting your community? YCST Awareness 19. Are you aware of the YCST program and its activities in your community? a. Yes b. No c. Don’t know 20. If Q19 = Yes, what specific YCST activities are you aware of or engaged in? Select all that apply: a. Aware of community engagement process and presence of project in community b. Participated in community meetings around PACE c. Participated in service delivery project implementation d. Uses service/facility from YCST project in community e. Other (describe): 21. If Q19 = Yes, please describe your awareness and/or involvement with these activities USAID.GOV YCST FINAL EVALUATION REPORT | 136 a. Probe: awareness and/or involvement with community engagement, sensitization, community meetings, service delivery projects b. Are you satisfied, dissatisfied, or neither satisfied nor dissatisfied with how these activities were organized, and why? Select one: 1 Satisfied Why? 2 Dissatisfied Why? 3 Neither satisfied nor dissatisfied Why and for which services? 4 Don’t know 22. If Q19 = Yes, did you find the YCST activities addressed important needs in your community? Select one: 1 Yes Why, how? 2 No Why not? 4 Don’t know 23. If Q19 = Yes, in your opinion, how did these activities most affect you or your community? 24. If Q19 = Yes, how do these activities compare to other activities/support provided to your community? Please share examples. 25. Are you aware of INSERT YCST-FUNDED FACILITY NAME? a. Yes b. No 26. Do you or your family members use INSERT YCST-FUNDED FACILITY NAME? a. Yes b. No 27. How has this facility impacted your community? 28. Are you aware of any instances in which the YCST support to your community contributed to creating conflicts, whether between the community members or between your community and those communities who did not receive any assistance? Please explain. PACE Approach [Q29–Q31 PACE Communities Only] 29. Are you aware of the CECs set up by the YCST project? a. Yes b. No 137 | YCST FINAL EVALUATION REPORT USAID.GOV 30. Is your local CEC active in the community? Select one: 1 Yes How? 2 No 3 Don’t know (DO NOT READ) 31. If aware of the CECs (If Q20 = A), are you satisfied, neither satisfied nor dissatisfied, or dissatisfied with the CECs when it comes to: Satisfied Neither satisfied nor dissatisfied Dissatisfied Why? a. Representing community interests 1 2 3 b. Selecting important community projects 1 2 3 c. Representing the interests of youth 1 2 3 d. Representing the interests of women 1 2 3 e. Addressing conflicts 1 2 3 f. Strengthening relationships between community members 1 2 3 Stabilization Approach [Q32–Q34 Stabilization Communities Only] 32. Are you aware of the ACCs set up by the YCST project? a. Yes b. No 33. Is your local ACC active in the community? Select one: 1 Yes How? 2 No 3 Don’t know (DO NOT READ) USAID.GOV YCST FINAL EVALUATION REPORT | 138 34. If aware of the ACC (If Q32= A), are you satisfied, neither satisfied nor dissatisfied, or dissatisfied with the ACC when it comes to: Satisfied Neither satisfied nor dissatisfied Dissatisfied Why? a. Representing community interests 1 2 3 b. Selecting important community projects 1 2 3 c. Representing the interests of youth 1 2 3 d. Representing the interests of women 1 2 3 e. Addressing conflicts 1 2 3 f. Strengthening relationships between community members 1 2 3 Service Delivery 35. Who is primarily responsible for providing education, health, utilities, and other services in your community? [Probe who provides the various types of services if the service providers are different] 36. In your opinion, how responsive is INSERT THE RESPONSE FROM THE PREVIOUS QUESTION to the needs of the community? 37. When it comes to being responsive to community needs including service delivery needs, have the INSERT THE RESPONSE FROM THE PREVIOUS QUESTION become [Select one:] 1 More responsive over time How? 2 Less responsive over time How? 3 Remained the same 139 | YCST FINAL EVALUATION REPORT USAID.GOV 38. When it comes to your access to (INSERT COMMUNITY-SPECIFIC SERVICES— SEPARATELY ASK ABOUT DIFFERENT SERVICES SUPPORTED BY YCST), has it improved, remained the same, or worsened over the last year? Select one: 1 Improved How and why? 2 Remained the Same 3 Worsened How and why? 4 Don’t know [DO NOT READ] 39. Are you satisfied, dissatisfied, or neither satisfied nor dissatisfied with the quality of services delivered to your community? Select one: 1 Satisfied Why? 2 Dissatisfied Why and for which services? 3 Neither satisfied nor dissatisfied Why? 4 Don’t know [DO NOT READ] 40. How would you describe the relationship between your community and the local authorities? Select one: 1 Good Why? 2 Not good Why? 3 Neither good nor bad Why do you say that? 4 Don’t know [DO NOT READ] 41. Is there a way to further improve the relationship between your community and local authorities? Select one: 1 Yes Please explain how? 2 No Why not? 42. In your opinion, what is the most effective mechanism/approach to ensuring that the government is responsive to your community needs? 43. Do you trust other members of your community more, about the same, or less than you did 12 months ago? Select one: USAID.GOV YCST FINAL EVALUATION REPORT | 140 1 Trust more How and why? 2 Trust remained the same 3 Trust less Why? 4 Don’t know 44. Do you trust local authorities and government more, about the same, or less than you did 12 months ago? Select one: 1 Trust more Why? 2 Trust remained the same Why? 3 Trust less 4 Don’t know 45. Has your ability to participate in the political decision-making process, for example, your access to decision-makers and their responsiveness, increased, reduced, or remained the same over the last year? Select one: 1 Increased Why? 2 Reduced Why? 3 Remained the same 4 Don’t know 46. Is there anyone else that you think we should speak to about the YCST program? 47. Is there anything else that you would like to share with us? 141 | YCST FINAL EVALUATION REPORT USAID.GOV 11. LOCAL AUTHORITY KEY INFORMANT INTERVIEW GUIDE – PACE COMMUNITIES Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent organization (MSI/Data Collection Firm Reference) that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community and your experience as a local authority. The interview is estimated to last 30 to 40 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” Introduction of PACE Methodology to the Community/Awareness and Participation in the Community Engagement Process 1. What is your role in the community? 2. How did you learn about the YCST program and the PACE approach? 3. How did you participate in the community engagement process? a. How often? Who did you engage with? Conflict and Drivers of Conflict 4. What are the most important causes of conflict in your community? Explain. 5. Do you think that these causes of conflict have become more or less important in the community over the last two to four years? Please explain. 6. What do you think are the best approaches to addressing these causes of conflict? YCST Impact on Mitigating Conflict 7. Did YCST/PACE activities help reduce conflict? Please explain, with examples. 8. Are you aware of any instances in which the YCST support to your community contributed to creating conflicts, whether between the community members or between your community and those communities who did not receive any assistance? Please explain. PACE Impact on Social Cohesion/Social Cohesion Impact on Conflict USAID.GOV YCST FINAL EVALUATION REPORT | 142 9. What are some of the effective interventions/activities that the program has implemented in your community? 10. Please describe any positive and negative effects that these activities had on your community when it comes to: a. Relationships between members of the community. b. Relationship between members of the community and local authorities. c. The capacity of nongovernmental organizations such as [give some examples of NGOs]. d. Causes of conflict in your community. e. Other effects (please specify): 11. How would you describe your relationship with the CECs? 12. Did the program undermine or negatively affect local authority or leadership structures in the community? If yes, how? 13. In your opinion, could the PACE teams/YCST have done things better? If so, please share how and why? Constraints/Challenges 14. Did the program’s activities cause any negative effects in your community? If so, what were they? 15. Did the project have any positive effects beyond those we have discussed? If so, what were they? 16. Did YCST undermine or negatively affect any pre-existing conflict mitigation practices in the community? If yes, how? 17. Has the COVID-19 pandemic affected the way you work with the YCST program? If so, how? Sustainability/Lessons Learned 18. Has the program created any new channels/processes for community engagement in your community? If yes, what are they? 19. Which of the project’s approaches worked well to help the community resolve conflicts? What could have been done better? Please describe how and why? 20. If you were to recommend changes to the program, what would you propose? 21. Is there anyone else that you think we should speak to about the YCST program? 22. Is there anything else that you would like to share with us? 143 | YCST FINAL EVALUATION REPORT USAID.GOV 12. LOCAL AUTHORITY KEY INFORMANT INTERVIEW GUIDE – STABILIZATION COMMUNITIES Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent organization (MSI/Data Collection Firm Reference) that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community and your experience as a local authority. The interview is estimated to last 30 to 40 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” Awareness and Participation in YCST Stabilization Activities 1. What is your role in the community? How does the community see your role? 2. How did you learn about the YCST program and activities? 3. How did you participate in the design and implementation of YCST activities in your community? Please explain. Conflict and Drivers of Conflict 4. What are the most important causes of conflict in your community? Explain. 5. Do you think that these causes of conflict have become more or less important in the community over the last two to four years? Please explain. 6. What do you think are the best approaches to addressing these causes of conflict? 7. How effective were YCST service delivery activities in addressing these causes of conflict? Please explain. 8. Are you aware of any instances in which the YCST support to your community contributed to creating conflicts, whether between the community members or between your community and those communities who did not receive any assistance? Please explain. Appraisal of Trainings and Mentoring Efforts (including Relevance and Application) 9. Are you satisfied, dissatisfied, or neither satisfied nor dissatisfied with the quality and usefulness of training and mentoring activities provided by the YCST team? Select one: USAID.GOV YCST FINAL EVALUATION REPORT | 144 1 Satisfied Why? 2 Dissatisfied Why? 3 Neither satisfied nor dissatisfied Why? 4 Don’t know [DO NOT READ] 5 Not applicable (did not participate in the activities) 10. How well did these trainings prepare you to: a. Respond to the concerns of your constituents and community. i. Please share the reasons for your response. b. Select activities, including improving services, that were most important to your community i. Please share the reasons for your response. 11. Can these trainings and mentoring efforts be improved? a. Yes i. How b. No i. Why not c. Don’t Know Satisfaction with Stabilization Approach and Outcomes 12. What interventions/activities that YCST implemented in your community were the best and why? 13. Please describe any positive and negative effects that the YCST activities had on your community when it comes to: a. Relationship between members of the community and local authorities. b. Relationship between the local authorities and local NGOs c. Your capacity to respond to community needs including providing services d. The capacity of NGOs e. Conflict in your community. f. Other impact (please specify): 14. Did the program undermine or negatively affect local authority or leadership structures in the community? If yes, how? Constraints/Challenges 15. Did the program’s activities cause any negative effects in your community? If so, what were they? 145 | YCST FINAL EVALUATION REPORT USAID.GOV 16. Did the project have any positive effects beyond those we have discussed? If so, what were they? 17. Did YCST undermine or negatively affect any pre-existing conflict mitigation practices in the community? If yes, how? 18. Has the COVID-19 pandemic affected the way you work with the YCST program? If so, how? Sustainability/Lessons Learned 19. Has the program created any new channels/processes for community engagement in your community? If yes, what are they? 20. Which of the project’s approaches worked well to help the community resolve conflicts? What could have been done better? Please describe how and why? 21. If you were to recommend changes to the program, what would you propose? 22. Is there anyone else that you think we should speak to about the YCST program? 23. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 146 13. SUBCONTRACTORS INTERVIEW GUIDE Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent company, YPC, that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community and your experience as a local authority. The interview is estimated to last 20 to 30 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” General Information 1. Specify Project Name 2. What is your role in this company? 3. Can you provide the name of the contractor/company you represent? 4. When was this YCST project [name project] implemented? 5. Can you briefly explain the works that you have done in the facility as part of the YCST project? 6. Are all works related to the YCST project completed? If no, when do you anticipate the work to be completed? 7. Can you briefly explain your experience with the bidding process? Project Planning, Implementation, and Sustainability 8. Can you please share your appraisal of the planning and implementation approach used by YCST? Your level of satisfaction with this approach? Any potential areas of improvement? 9. What role, if any, did CEC members/ACC members play in the planning and implementation of this YCST support project (e.g., supervision, maintenance)? Please explain. 10. What role, if any, did the local authorities play in the planning and implementation of this YCST support project? 11. What role, if any, did local experts or field monitors play in planning and implementation of this YCST support project? 147 | YCST FINAL EVALUATION REPORT USAID.GOV 12. Do you think that the YCST support facility will continue to operate effectively in the future? If no, why not? If yes, please explain. a. Do you think the facility will face any challenges in maintaining and sustaining the works completed? If yes, please explain. 13. In your opinion, does this project supported by YCST contribute to reducing conflicts or improving relationships between community members? If so, please explain. Challenges/Constraints 14. What challenges, if any, did you face during the planning and implementation of this YCST support project (e.g., delays in implementation, delays in payments, technical issues, lack of availability of material)? a. If challenges were faced, how did you and YCST handle these challenges? 15. What challenges, if any, did you experience working with: a. Global Communities b. Community members c. Local leaders d. Any other stakeholder 16. Did the assistance provided by YCST address all the facility needs? Please explain. 17. In your opinion, are there still needs that remain to be addressed in this facility (supported by YCST)? Please share additional details, if possible. 18. Are there any additional needs or areas of improvement to improve the sustainability of this YCST-supported project and what this project has achieved? Please explain. 19. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 148 14. GRANTEE KEY INFORMANT INTERVIEW GUIDE Introduction and Informed Consent: “It is a pleasure to meet you. My name is [Insert Name]. We are here to collect feedback on the assistance being provided by the Yemen Communities Stronger Together (YCST) program. We are part of an independent organization (MSI/Data Collection Firm Reference) that is responsible for obtaining feedback for donors supporting this assistance. The purpose of this interview is to gather information on the activities that YCST implemented in this community and your experience as a grantee. The interview is estimated to last 30 to 40 minutes. There are no right or wrong answers, and you can refuse to answer any question and can terminate this interview at any time. Your responses to this survey will be kept confidential. While the information that you provide may be used in our reports, your name or personally identifiable information will not be. However, please note that even if your feedback is used in the report, this does not mean that the issues raised will lead to immediate changes in the future. Thank you. Do you have any questions before we begin?” 1. Can you please tell us about your organization and your specific role within the organization? 2. Which areas, communities, and populations does your organization serve? 3. How did you learn about the YCST program and activities? 4. What was your experience with the grant application process? 5. How, if at all, did YCST support build your organization’s capacity? Explain with examples. 6. How, if at all, did you participate in designing and implementing YCST activities in your community? Please explain. 7. Did you learn anything through your interaction with YCST that helped you improve your work? Please explain. 8. Do you plan to continue using what you learned after YCST activities end? Why or why not? 9. Do you have the resources you need to continue to apply these approaches even after the YCST support ends? 10. What would you say was the most significant benefit of the YCST support? Was there anything that you were able to achieve that you would not have been able to achieve without the YCST support? 11. Did the YCST-supported activities provide beneficiaries with skills to resolve or help reduce conflict in their communities? Please explain, with examples. 149 | YCST FINAL EVALUATION REPORT USAID.GOV 12. If you were to recommend changes to the YCST program, what would you propose? 13. Is there anyone else that you think we should speak to about the YCST program? 14. Is there anything else that you would like to share with us? USAID.GOV YCST FINAL EVALUATION REPORT | 150 ANNEX 7: YEMEN POLLING CENTER COMMUNITY-LEVEL METHODOLOGY INTRODUCTION Management Systems International (MSI) contracted with Yemen Polling Center (YPC) for this assessment. YPC was established in 2004 as the first and only polling center in Yemen. YPC designs and implements opinion polls, household and other surveys, and provides other services that fulfill research needs of national and international institutions, nonprofit organizations, corporations, government agencies, and professional associations. The headquarters (HQ) for Yemen Polling Center is located in Taiz, Yemen. The Yemen Communities Stronger Together (YCST) program uses the participatory action for community enhancement (PACE) model to identify and support local-level initiatives that mitigate conflict, strengthen social cohesion, and promote the peaceful resolution of differences. PACE utilizes a step-by-step framework for building the resilience of community members, civil society organizations, the private sector, traditional leaders, and local government officials to collaboratively identify and address local issues through conflict mitigation and service delivery projects. YCST also trains formal and informal leaders, organizations, and networks to serve as neutral arbitrators and peace builders to meet community and constituent needs. Executive Summary This assessment consists of qualitative and quantitative approaches: 1- 292 household interviews with adults (people 18 and older) living in the YCST communities of the four governorates, Abyan, Aden, Lahj, and Taiz, covered by the program were interviewed 2- 107 key informant interviews (KIIs) with a. 36 Community Enhancement Committee (CEC) Members b. 30 YCST facility staff c. 24 YCST community local leaders d. 16 Non-YCST communities local leaders e. One interview with Ministry of Planning and International Cooperation (MoPIC). A multi-stage random sample design was implemented that combined stratification at governorate level and purposive sampling for respondents for the KIIs. The average length of the completed household interviews was 31:64 minutes. Interview length ranged from 14 minutes to 181 minutes. Fieldwork was conducted between June 6 and June 26, 2022. The field team consisted of 36 Arabic￾speaking interviewers under the management of YPC. Table 1: Project Timeline Project Phases Start Date End Date Training May 17 May 19 Fieldwork June 6 June 26 Data Entry June 6 July 20 Quality Control June 7 July 19 Data Delivery June 8 July 27 151 | YCST FINAL EVALUATION REPORT USAID.GOV SAMPLE DESIGN The target sample for this study was a stratified, multi-stage cluster combined with purposive selection of respondents—CEC members, YCST staff, local leaders, women, and men in 20 communities in the four targeted Yemeni governorates. A. Community Members Table 2: Sample by Governorate and Community Governorate Community Households Abyan Jaar 30 Ad-Dergag 30 Zinjubar 33 Jyshan 15 Taiz Al-Azaez 32 As-Sawa 30 Aden Abd Albari 30 Al-Shouhada 30 Lahj Ras al-Arah 30 As-Suraih 32 Total 292 Sampling Methodology Target Population: Adults (people 18 and older) from the 10 communities in the four targeted governorates (Abyan, Aden, Lahj, and Taiz). Target Sample Size: 292 The sampling methodology is as follows: Stage 1: Distribution of interviews by communities to distribute the sample evenly between the 10 targeted communities with 30 interviews for each. Stage 2: Distribution of interviews within each community. Once the team reached the community, based on the sample information sheet, they should move to the YCST facility named in the sample. The supervisor followed this process: ▪ Collected all necessary information about all areas that benefited from the YCST facility services (villages/neighborhoods/population and households’ numbers) ▪ Divided the community areas that benefitted of facility services into four roughly equal sections based on geographic distance. ▪ Distributed the 30 community members interviews to each section equally with 7–8 interviews, half of them with female respondents. ▪ Distributed interviewers to the four sections and accompany them to each section to select the right starting point and direct them how to start moving. ▪ Specify the number of intervals by dividing the number of total households in the section by the total number of interviews. For example, if there are 100 households in the section, then the interval will be 12 (100 / 8 = 12). USAID.GOV YCST FINAL EVALUATION REPORT | 152 Stage 3: Selection of household and respondents. The interviewer introduces himself/herself, and the organization that he/she works for as well as of the study to the household members by reading the paragraph in the tool script. If the interviewer has the household’s consent to participate then he/she follows the CAPI script electronic mechanism to select one respondent from the household members. He/she should fill the household roster for those who are above 18 years old and by gender—match rule is applied as male interviewers should only interview male respondents while female interviewers should only interview female respondents. Thus, half of the interviews in each community will be completed with female respondents. Once the respondent is selected, interviewer should read the informed consent, confidentiality, and data privacy, to obtain respondent agreement to participate in the study. By the end of each day of data collection, all completed interviews will be uploaded to a project server once interviewers have internet access to synchronize their devices. The collected data are reviewed and validated by the YPC quality control officer on a daily basis. Stage 4: Respondent substitution (if necessary). Respondent selection at the household-level was not permitted. Interviewers continued on the route if three attempts to interview the respondent designated by the Kish grid were unsuccessful. 1. Selection of starting points for the walk pattern. Supervisors are taught during the training sessions how to select an appropriate starting point following the right method. In each section of community, supervisor should start moving from the landmark such as a mosque, school, hospital, etc. In most cases, the YCST project facility was selected to be the starting point for one of the community sections. 2. Household selection. Once the starting point is identified by the supervisor, he/she accompanies the team of enumerators (2 or 4 enumerators in the team) to the sampling starting point and identifies the first household1 where interviewing begins by using the right-hand rule and a set interval between households (excluding the starting point). The interval between households will be households. The interval of two households will ensure there is greater variation of households within the community (i.e., greater area is covered such that all respondents are not likely from a row of homes on the same street). The field supervisor gives instructions regarding the specific households that every enumerator in the team should visit. The process of distributing households to team members is controlled by the supervisor while conducting the interviews. Enumerators maintain contact with the supervisors during the interviews regarding substitutions or other issues. Enumerators make up to three attempts in each selected household to obtain an interview. If the attempt fails in a refusal or no one is available in that household after three attempts, the enumerator will count two households past the last main household along the walking path to identify the first replacement household. The replacement household can be a replacement for any of the failed attempts at a main household. In each replacement household, up to three attempts will be made using the same rules described for obtaining an interview with a main household. If attempts at the first replacement also result in failure, interviewer will continue to use the right-hand rule and the interval of 1 Household is a dwelling in which members live permanently in that dwelling and have a cooking facility (such as a stove, a fire in the courtyard, etc.). In the case of communal living where multiple families live in an apartment, each family sharing one room and a common cooking facility, it would be important to use the notion of a cooking pot to determine households. That is, members who share food cooked from a common pot or shared from the same budget. 153 | YCST FINAL EVALUATION REPORT USAID.GOV two to determine the next replacement household. This process will be repeated until the required number of completes is obtained in the selected PSU. 3. Respondent selection. The final step prior to commencing the interview is respondent selection. To ensure random selection of participants who meet the sample criteria within the household, YPC employs the automatic selection by CAPI software for interviews. Once the household is chosen, the interviewers list details of each household member into the tablet. Gender matching of interviewers and respondents is applied. Male interviewers should only list all male members of the household aged 18 and older, while the female interviewers should list all female members of the household, and the CAPI program randomly selects the individual to be interviewed. All eligible household members have an equal chance of being selected for the interview. Within each household, up to three attempts will be made to select and complete an interview with an eligible respondent. These attempts will be made at different times of day and days of the week. If the household contact results in a refusal, then the household is substituted with one adjacent to the original sampled household. If the refusal continues or contact didn’t succeed, the household is substituted with the first household after the last main household that has not been visited by any interviewer of the team. B. KIIs Target Groups2: ▪ Community enhancement members ▪ Project facility staff ▪ YCST community local leaders ▪ Non-YCST community local leaders Completed Interviews: 107 KIIs Community Enhancement Members Simultaneously with community member interviews, the team conducting CEC member interviews conducted interviews based on the coordination with CEC members and appointments that had been set the day before the visit. The team completed 36 interviews in six PACE communities as follows: 2 One additional KII was completed with a representative of the MoPIC. x MH1 MH2 R1 MH3 MH10 MH4 MH9 MH5 MH8 MH7 MH6 USAID.GOV YCST FINAL EVALUATION REPORT | 154 Table 3: Sample by Governorate and PACE Community Governorate Community # of Interviews Abyan Ad-Dergag 6 Jyshan 6 Aden Abd Albari 6 Al-Shouhada 6 Lahj Ras al-Arah 6 As-Suraih 6 Total 36 Participants Selection MSI provided the list of CEC members to be used to identify the respondents of interviews. During data collection, additional lists were provided by MSI to complete the necessary number of interviews in each sampled YCST project community. Data Collection All interviews were completed face-to-face (F2F) during the field visits to the YCST communities by supervisors. Supervisors contacted the CEC member a few days before the field visit to the community and arranged the best time and day for the interview that suited the respondent. Data Entry All interviews were entered using CAPI script Kobo. Completed interviews were uploaded to the server directly or once internet access was available. The data were exported and translated in SPSS and xsl formats. Staff Facility and Observations Simultaneously with community members interviews, and during the visit to the YCST site, the team of interviewers accompanied by photographers conducted staff facility interviews and observations. The team completed 30 interviews and 16 observations in 16 facilities as follows. Table 4: Sample by Governorate, Community, and Facility Governorate Community Facility Interviews Observations Abyan Zinjibar Khadija School 6 1 Khalwa School 1 Sahat Al Shohada Park 1 Jaar Ja’ar Park 4 1 Haydera Football Field 1 JaFyshan Jayshan Water Supply 2 1 Aden Abd Albari Dar Sa’ad Electricity 2 1 Lahj Ras al-Arah Ras al-Arah Water Supply 2 1 As-Suraih Al Suraih Schools 2 2 Al Suraih Health Center 2 1 Taiz Al-Azaez Abdulaziz School 2 1 Al Khair School 2 1 Al Shaheed Yassin School 2 1 155 | YCST FINAL EVALUATION REPORT USAID.GOV Al Mansoora Health Center 2 1 As-Sawa Al Ma'afer Rural Hospital Sanitation System 2 1 Total 30 16 Participants Selection The list of YCST facilities was provided by MSI that included the information of locations and other contact details. Once the supervisor selected the starting point and gave interviewers the necessary directions for walking route and intervals, he/she should return back to the facility site to conduct the interviews with 1–2 staff who have sufficient knowledge about services provided by the facility. Data Collection All interviews were completed F2F during the field visits to the YCST facility by supervisors. Supervisors, after accompanying interviewers on community members interviews, visited the YCST facility to complete 1 or 2 interviews. A paper and pencil method was used to complete the staff interviews and observations. A professional photographer accompanied the supervisor during the facility visit and took photos that supported the observation notes. Data Entry All interviews were completed using PAPI instrument and then entered to docx template and delivered to YPC HQ along with the audio recordings. YPC quality control officers at the office reviewed the transcripts and listened to the recording to verify the accuracy and ensure there is no missing data. The approved interviews by YPC quality control officer were shared with MSI team through a Dropbox secure folder. YCST Local Leaders Interviewers recruited local leaders while conducting community members and CEC interviews. In each of the six PACE communities, two interviews were completed with local leaders while three interviews were completed in each of the four stabilization communities. The team completed 24 interviews in 10 YCST communities. Table 5: Sample by Governorate, Community, and Facility Governorate Community # of Interviews Abyan Ad-Dergag 2 Zinjubar 3 Jaar 3 Jyshan 2 Aden Al-Shouhada 2 Abd Albari 2 Lahj Ras al-Arah 2 As-Suraih 2 Taiz Al-Azaez 3 As-Sawa 3 Total 24 Participants Selection During the team’s visit to the community, the supervisor gathered information about the most influential leaders in the communities, either formal or informal, to identify and conduct interviews with them. The USAID.GOV YCST FINAL EVALUATION REPORT | 156 supervisor gave priority to interview those leaders who had no involvement in the YCST activities in the community. Interviewers who conducted interviews with CEC members also asked respondents about the community leaders with influence in the community. A list of local leaders were composed that the supervisor used to select the most eligible names who met requirements for local leader interviews: • Those who had no involvement in YCST activities. • Where it was available, some female local leaders were selected to be interviewed. • Where it was available, both formal and informal leaders were selected to be interviewed. Data Collection All interviews were completed F2F during the field visits to the YCST communities by supervisors. Supervisors after accompanying interviewers on community members interviews and the visit to the YCST facility completed 2–3 interviews with local leaders in each community. A paper and pencil method was used to complete the YCST local leaders’ interviews. Data Entry All interviews were completed using the PAPI instrument and were then entered to a docx template and delivered to YPC HQ along with the audio recordings. YPC quality control officers at the office reviewed the transcripts and listened to the recording to verify the accuracy and ensure there is no missing data. The approved interviews by YPC quality control officer were shared with the MSI team through a Dropbox secure folder. Non-YCST Local Leaders After completion of the team visit to the YCST community, supervisors visited one non-YCST community that is in the same or nearby district. The team completed 16 interviews in eight non-YCST communities. Table 6: Sample by Governorate, Community, and Facility Governorate Community # of Interviews Abyan Zinjubar 2 Jaar 2 Jyshan 2 Aden Abd Albari 2 Lahj Ras al-Arah 2 As-Suraih 2 Taiz Al-Azaez 2 As-Sawa 2 Total 16 Participants Selection The team of interviewers and supervisors gathered information about the most influential leaders in the communities, either formal or informal, to identify and conduct interviews with them. A list of local leaders was composed in each community that the supervisor used to select the most eligible names who met requirements for local leaders interviews: • Recommendations of some local leaders as a result of the supervisor network in each of these communities. • When the field team passed by these communities, interviewers had a short visit to ask for some names of local leaders who have influence in the community and collected their contact details such as name, address, mobile, etc. 157 | YCST FINAL EVALUATION REPORT USAID.GOV • Interviewers who conducted CEC interviews and after they completed all CEC interviews, they attempted to complete interviews with non-YCST community local leaders by phone calls using the lists that been provided and approved by the supervisor. • In each community, 1 or 2 interviews were completed with local leaders by phone call. • Non-YCST interviews were completed at a YPC call center in Taiz at the end of the data collection of PAPI and CAPI interviews. Data Collection All interviews were completed through phone calls from a YPC call center after the completion of the main data collection. Supervisors delivered the approved lists of local leaders that had been recruited to YPC HQ in Taiz. The YPC quality control officer reviewed the lists and distributed them to the call center enumerators who made five calls during different times of a day in an attempt to reach the respondents. If after all attempts, the interview was not successful, enumerator replaced that name with the third name in the list. The process was repeated until all required interviews were successfully completed in each community. Data Entry All interviews were completed through phone calls and have been audio recorded; enumerators took detailed notes of respondents’ responses. A transcript of each interview was written by enumerators after listening to the audio recordings or from the notes that had been written during the phone call. All data was entered into a docx template submitted to YPC quality control officers at the office, who reviewed the transcripts and listened to the recording to verify the accuracy, ensuring there is no missing data. The interviews approved by YPC quality control officers were shared with the MSI team through a Dropbox secure folder. Replacements Due to security concerns, one of the YCST communities was substituted after being approved by MSI. The information of the substitution is in the below table: Table 7: Community Replacement by Governorate Old Community Substitutions Governorate District name Governorate District Reasons Lahj Al-Nabyah Lahj Ras al-Aarah Security Concerns Field Teams Training The YPC conducted a training workshop for fieldwork teams for three days between May 17 and May 19 at YPC HQ in Taiz. The training was led by the YPC fieldwork manager, YPC quality control manager, and YPC qualitative officer. Interviewers from the four targeted governorates gathered in the training room. Table 8: Interviewers by Gender Governorate Male Female Total Aden 1 6 7 Abyan 4 3 7 Taiz 4 3 7 Lahj 3 4 7 Total 12 16 28 During the three-day training session, the following topics were covered: USAID.GOV YCST FINAL EVALUATION REPORT | 158 • Introduction to the YCST study • Research ethics • Sampling plan • Data collection methods • KII skills • Data security • Conduct mock interviews Tools Pretest The YPC conducted 10 interviews for testing the community members tool that had been completed F2F in al-Maafer district in Taiz. A team of two male and female interviewers completed the interviews between May 11 and May 14, 2022. The sample for the pretest was selected randomly from the targeted community in al-Maafer district following the same method of household and respondent selection for survey interviews. The interviews were distributed by gender, age, and educational levels to represent different population demographics. Table 9: Pretest Interviews by Respondent Demographics Interview # Governorate Int. Duration Gender Age Education 1 Taiz 42 Female 26 University 2 Taiz 38 Female 35 Secondary 3 Taiz 22 Female 45 Illiterate 4 Taiz 29 Female 20 Primary 5 Taiz 19 Female 55 Illiterate 6 Taiz 54 Male 53 University 7 Taiz 30 Male 24 Secondary 8 Taiz 21 Male 40 Primary 9 Taiz 37 Male 34 Primary 10 Taiz 31 Male 20 Illiterate Field Team The YPC has field teams in all of Yemen’s governorates. For this project, every team consists of one supervisor and 2–4 interviewers (more than half of them are female). Table 10: Interviewer Distribution by Governorate and Gender Governorate Male Female Total Taiz 2 3 5 Abyan 4 4 8 Aden 0 6 6 Lahj 3 2 5 Total 9 15 24 159 | YCST FINAL EVALUATION REPORT USAID.GOV Table 11: Interviewers Education Governorate Bachelor Diploma Taiz 4 1 Abyan 7 1 Aden 6 0 Lahj 4 1 Total 21 3 I. Quality Control Procedures The process of reviewing and processing collected data started from the very beginning at receiving the first completed survey/interview. The process continues during the data collection phase and after completion of the last survey and interview. During data collection, any missing or discrepancy cases were reported by the YPC data officer to the related enumerator/interviewer to address and resolve. This process includes the following actions: - Uploaded data of F2F surveys to project secured server were checked and processed on an ongoing basis to report any errors, missing data, or inconsistencies to enumerators so they addressed and resolved them by calling back respondents and making the necessary corrections. CAPI script ensures that no missing information or inconsistencies will be left without being addressed, since every instrument is programed using Kobo software in a way that prevented skipping errors or missing questions, alerting interviewers with such errors immediately during the interviews. However, this daily review is necessary to report any inconsistencies or ambiguous responses that don’t reflect respondents’ actual answers. - The similar process of reviewing F2F surveys has been followed for reviewing phone surveys in addition to listening to audio records. The phone calls will be recorded to help in checking uploaded data against the respondent’s answers in the audio records. - KIIs transcripts were subject to a first check from the YPC supervisor immediately after completion. The supervisor then delivered the reviewed transcripts to the YPC quality control officer who reviewed and verified the collected data to report anything missing or discrepancies before accepting it. Table 12: Implementation of Quality Control Procedures Quality Control Procedure Interviews Subject to Procedure % Accompanied interview (Full or partial interview) 72 24.7 Re-visited/re-called by person 8 2.7 Not validated 212 72.6 Total 292 100% Fieldwork Obstacles and Challenges The data collection phase took place between June and July 2022. Several challenges and difficulties were recorded by the interviewers and reported to the YPC office. Below are the challenges and issues recorded during fieldwork. i. Aden The filed team in Aden reported the following challenges and issues during data collection: USAID.GOV YCST FINAL EVALUATION REPORT | 160 • Some CEC members included in the lists provided by MSI were not cooperative and responsive to YPC coordination efforts. As a result, YPC had to ask for additional names to complete the necessary number of interviews in each community. • Interview appointments that had been agreed on with the CEC members were not met by some CEC members, which caused delays for the team during data collection. • For the local leaders in YCST communities, interviewers reported difficulties in finding candidates that met the selection requirements, as most leaders were involved in or had been members of YCST committees. This issue required the team to extend their efforts and the timeline to select those who were not involved in the YCST CEC. • Due to interviewer withdrawals several times before or at the midpoint of data collection, YPC had to re-train new interviewers to replace the old ones. The reasons of withdrawals mostly were security concerns. ii. Lahj • The field team in Lahj faced significant issues and challenges before accessing the target community of Al Nabyah. Due to the presence of al-Qaeda militants, YPC made several efforts and coordination trying to ensure the safety of field team coordinating with some local leaders and sheikhs in the area, but these were not sufficient because most of these leaders reported that all areas of the district are not safe at all for the organization’s work. One of the most influential leaders, Sheikh Shu'eeb as-Sobehi, leads the campaign against any activities or project funded by international organizations, accusing them of spreading immoral behaviors and corruption in the Muslim community. YPC reported this issue to MSI along with suggestions of some replacement options. In close collaboration with the MSI team, the community was substituted, and Ras Alarah district was the new target community where it was safe to complete the data collection. iii. Abyan • In Jayshan Abyan, due to the presence of al-Qaeda militants in the area, the team faced several difficulties and security risks. The team was on the road to the area when a checkpoint that belongs to al-Qaeda stopped them and started investigating their work. The supervisor, with help from his contacts with community leaders in the area, ensured the militants allowed them to continue their travel to the area safely. When the team arrived in the community and started conducting interviews with community members, they noticed that all female women refused to participate in any kind of activities supported by any organizations. They discovered that there are campaigns supported by some in the area mosques against all organizations working in Yemeni communities, accusing them of spreading corruption targeting women specifically. It was not only women who refused to participate, but also a high number of men from the community. We asked the team to attempt having phone contacts for some female and male participants who agree to participate through mobile calls later from the office, but unfortunately, they received the same refusals from female members of the community. Finally, we asked the team to keep working until late in the day to complete as many interviews as possible, even if it means to complete all 15 interviews with only men, and leave the area early in the morning the next day. The next day, the team left the area travelling back to the Abyan capital where they accessed the internet and uploaded the completed interviews. The team completed 10 interviews in Jayshan; all of them are with men members, and they successfully completed all KIIs. 161 | YCST FINAL EVALUATION REPORT USAID.GOV ANNEX 8: YPC COMMUNITY SURVEY CROSS TABS MSI Survey 2022 Preliminary findings(By Variables)_08_07_2022.xlsx