MID-TERM EVALUATION OF THE PROMOTING YOUTH FOR CIVIC ENGAGEMENT PROJECT (PYCE) FINAL REPORT July 14, 2012 This publication was produced for review by the United States Agency for International Development. It was prepared under the Yemen Monitoring & Evaluation Project by International Business and Technical Consultants, Inc. (IBTCI) Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) i MID-TERM EVALUATION OF THE PROMOTING YOUTH FOR CIVIC ENGAGEMENT PROJECT (PYCE) JULY 14, 2012 International Business & Technical Consultants, Inc. 8618 Westwood Center Drive Suite 220 Vienna, VA 22182 USA Contracted under RAN-I-00-09-00016, Order Number 1 Yemen Monitoring and Evaluation Project DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Author: Leyla Moubayed Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) ii Table of Contents Acronyms..................................................................................................................................................... iii EXECUTIVE SUMMARY ................................................................................................................................. iv I.  INTRODUCTION – BACKGROUND.............................................................................................................. 1 A. INTRODUCTION........................................................................................................................................... 1 B. YEMEN POPULAR UPRISING........................................................................................................................ 2 C. PROJECT SUMMARY.................................................................................................................................... 3 II.  EVALUATION METHODOLOGY ................................................................................................................. 7 A. PYCE PROJECT STAKEHOLDERS................................................................................................................... 7 B. PYCE MIDTERM EVALUATION INSTRUMENTS ............................................................................................ 8 C. IMPLEMENTATION OF PYCE MIDTERM EVALUATION ................................................................................ 9 III.  EVALUATION FINDINGS......................................................................................................................... 11 A. PROJECT’S EFFECTIVENESS ....................................................................................................................... 11 B. PROJECT OUTCOME ON STABILIZATION................................................................................................... 20 C. PROJECT’S REPORTING & M&E SYSTEMS ................................................................................................. 30 D. APPRAISAL OF AMIDEAST REQUEST FOR PYCE EXTENSION..................................................................... 32 IV.  SUMMARY CONCLUSION & RECOMMENDATIONS............................................................................... 36 ANNEXES ..................................................................................................................................................... 38 ANNEX A ‐ Interview & Focus Group List...................................................................................................... 39 ANNEX B ‐ Desk Review Documentation ...................................................................................................... 43 ANENX C ‐ SURVEY ‐ Translated Copy ........................................................................................................... 45 ANNEX D – PYCE Cumulative F Indicators List .............................................................................................. 47 ANNEX E: AMIDEAST PYCE Organizational Chart.......................................................................................... 48 ANNEX F: Training Plan ................................................................................................................................. 49 ANNEX G: USAID Yemen, PYCE Mid-term Evaluation Final Scope of Work .................................. 51 Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) iii Acronyms AO Assistance Objective AOTR Assistance Objective Technical Representative CMM Conflict Management & Mitigation CoAg Cooperative Agreement CoP Chief of Party EYSY Engaging Youth for a Stable Yemen FG Focus Group G-PO Governorate Project Officer G-SC Governorate Steering Committee IBTCI International Business & Technical Consultants, Inc. IR Intermediate Result MC Mercy Corps MoEG Ministry of Endowments and Guidance MoYS Ministry of Youth and Sports PACA Participatory Analysis for Community Action PDC PMP Partners for Democratic Change Performance Management Plan PN Peer Network PPI PeacePlayers International PYCE Promoting Youth and Civic Engagement ROYG Republic of Yemen Government SSI Semi Structured Interviews ToT Training of Trainers USAID United States Agency for international Development YMEP Yemen Monitoring and Evaluation Project Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) iv EXECUTIVE SUMMARY The Promoting Youth for Civic Engagement (PYCE) project is a two-year project (October 2010 – September 2012) implemented by AMIDEAST in Yemen and designed to “improve the livelihoods of Yemeni youth in vulnerable areas by supporting USAID’s strategy to engage dynamic, responsive and credible religious leaders.” The PYCE project falls under the USAID Conflict Management and Mitigation (CMM) initiatives and is also expected to resonate with and positively feed into the 2010 – 2012 USAID Yemen stabilization strategy. The implementation of the PYCE project was impeded by Yemen’s popular uprising that began in late January 2011. By the end of March 2012, PYCE had been operating for eighteen months or the equivalent of 75% of its originally contracted Cooperative Agreement (CoAg) timeframe. However, project expenditures by this date amounted to only 23% of the project’s total program budget. The purpose of this mid-term performance evaluation is to assess the PYCE project’s implementation, effectiveness, and suitability of the project’s approach to achieving the project’s objectives. The assessment results will inform USAID with regard to AMIDEAST’s request for a one year, no-cost extension of the project beyond its current September 2012 closing date. The evaluation methodology targeted PYCE’s stakeholders’ with semi structured interviews, focus group meetings and surveys designed to elicit their feedback on the evaluation questions. This executive summary presents the main evaluation findings. Project Effectiveness: The evaluation desk review and field assessment revealed that the PYCE project has diverged to a certain degree in approach, targeting strategy, and implementation methodology from the technical approach prescribed in the CoAg. These modifications to the project’s approach and implementation methodology have to a certain extent foregone communication with and involvement of the concerned Ministries, undermined the role of the Governorate Steering Committee and neglected the peer education model in favor of a direct implementation of activities that involve youth in sports and recreational initiatives and engage youth with community work. This approach has defeated the ‘stabilization’ purpose recommended by USAID to improve trust and perception of the Yemeni youth with their government officials and local leaders and will be unlikely to create positive linkages and increase trust and communication between the various stakeholders through sports, recreational programs and community work. Furthermore, the direct implementation approach that has been favored by PYCE staff in the second half of the project will negatively impact the sustainability of some of the project’s initiatives. It is normal that projects experience certain changes during implementation, changes that are often necessary to accommodate the realities “on the ground” which often vary from the original program design assumptions. Nevertheless, these changes should not have diverted the project from its conflict mitigation framework. Of note is that when the project undertook the ‘correct’ outreach and engagement strategy of Republic of Yemen Government (ROYG) officials and community leaders (launch meetings, Steering Committee), the project managed to successfully engage these actors and to create communication and trust linkages between these actors and youth. On the effectiveness questions, the evaluation findings are summarized as follows: Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) v What progress has been achieved in each programmatic area to date? Has the program achieved all of the expected results by this program midpoint of March 2012? If not, why not? It is difficult to assess the exact progress achieved by PYCE in each programmatic area due to the changes made in project’s approach, methodology, targeting, and indicator definitions. Under the first objective of “strengthening the role of religious actors in positively influencing Yemeni youth,” the project has trained certain youth to prepare for their engagement with other youth and their community, but has missed out on the overall objective of engaging religious actors and establishing lines of communication between the various project stakeholders to ‘positively influence Yemeni youth.’ Under the second objective of “establishing and supporting youth sports and recreational programs,” PYCE has supported and promoted sports and recreational activities but is still behind in renovating sports and community centers and in the formation of new recreational sports teams and league competitions. The project’s progress is delayed compared with its original and amended plan, especially in Sana’a and Marib governorates, and the events in Yemen do not account for all of the delay of PYCE from its initial implementation schedule. Is the PYCE design and approach an effective strategy for achieving program results? The original program design and approach can be an effective strategy for achieving program results when properly implemented. The PYCE project’s approach in Aden governorate was successful in ensuring community buy-in and in gaining the trust of the local leaderships and engagement with key stakeholders. The project was successful in this region up to the point where the project stopped delivering on its promises, especially with the youth Peer Network (PN) members. Project Outcome on Stabilization: It was difficult to assess the degree or measures of outcome-impact achieved by the project since PYCE did not report on the stabilization outcome indicators prescribed in the Performance Management Plan (PMP). However, various assessments with the project’s stakeholders provided the following feedback to the evaluation questions: Is the implementation methodology appropriate in achieving USAID’s strategy of positively impacting stabilization by targeting youth groups? To what extent has the project contributed to accomplishing mission’s stabilization strategy objectives? The PYCE project activities implemented so far have not yet created a ‘critical mass’ or a large enough momentum to assess the outcome on youth targeting and stabilization. Nevertheless, ministries’ officials and local leadership believe that the project has good potential (when properly implemented in terms of leadership involvement) to positively impact stabilization. Is the PYCE original plan and implementation mechanism still appropriate to meet the needs of youth and target communities? Interviews with various community stakeholders including the youth indicated that the ‘original plan’ and ‘implementation mechanism’ would be suitable to meet youth needs and target communities when properly implemented. A review of certain activities, approaches and targeting, including engagement with the Ministry of Education, should take place, but overall the needs of these youth and target communities have not changed substantially from the time of the project design. Hence, as per the evaluation findings, the PYCE original design is still appropriate to meet those needs. Is the project inclusive and building consensus within target communities? Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) vi Interviews with various stakeholders’ groups clearly demonstrate that the project was not inclusive and did not build consensus except in some cases, such as in Aden and with the Aden leadership, (officials of the Ministry of Youth and Sport and the Ministry of Endowments and Guidance in Aden and the Aden steering committee). Is the project achieving any level of sustainability of the PYCE activities within targeted communities? The answer is ‘yes’ for the PYCE activities that focused on youth and around centers—whether sports, educational or recreational, and the leadership of these centers or recreational events. However, the PYCE initiatives that do not revolve around such structures (a center and leadership) are not and cannot possibly be sustainable. An example of unsustainable initiatives would be some of the Ramadan activities and the training of youth, who never played basketball and who are not affiliated with sports clubs or schools, in basketball coaching. Project’s Reporting & M&E (Monitoring & Evaluation) Systems: The PYCE project management does not have in place appropriate M&E systems to capture data to assess whether or not implementation is on track toward achieving the project’s objectives and their projected PYCE outcomes. PYCE Cost Effectiveness: This task proved to be unrealistic since the project’s financial reporting does not provide detailed financial cost information for activities and tasks that can be correlated with the project’s outputs to undertake this analysis. This issue needs to be addressed by USAID in the future to keep track of where the project might be experiencing spending difficulties. Appraisal of AMIDEAST Request for PYCE Extension: AMIDEAST’s claim that security risks were the major impediments to a timely delivery of the project is not fully credible, as other similar USAID programs were able to operate under the same circumstances in similarly restive and insecure areas of Yemen. Based on the project’s best scenario in past burn rates, PYCE management’s capacity and ability to implement PYCE and deliver on its performance requirements during the extended period is doubtful. Summary Conclusion & Recommendations: In summary, the issues confronting the PYCE project go far beyond its low burn rates, the Yemeni youth uprising and other security issues. The project has changed its focus from a stabilization initiative targeting youth to improve their livelihood to a project promoting sports initiatives and training youth on ‘life skills.’ It is the evaluators’ assessment that the original project design, implementation methodology, stabilization intent and youth targeting are still valid during this transitional period even though USAID is reconsidering its stabilization strategy in light of the country’s “Arab Spring.” The AMIDEAST management, at present staff levels, is inadequate and unable to deliver on the PYCE project’s objectives. The evaluation appraisal of PYCE burn rates over the past project reporting periods is not encouraging with regard to consideration of an extension of the project on the present terms. Should USAID favor an extension, it is recommended that AMIDEAST present the following to USAID for review and possible approval: 1- An amended and feasible project work plan with a detailed activity plan and a related cost budget. 2- A detailed training plan for the community actors, PN and Yemeni youth that clearly identifies the training topics and explains the expected impact of those topics on the final objective of the project. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) vii 3- An amended M&E plan that reflects the revised work plan in terms of output and projected impact, and is clearly linked with the project objectives. It is also recommended that USAID, possibly through the Yemen Monitoring & Evaluation Project (YMEP), undertake a much closer monitoring of the project both financially and programmatically in the field to assess that the project is on target and is achieving its ‘stabilization’ objectives. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 1 I. INTRODUCTION – BACKGROUND A. INTRODUCTION At the request of the USAID Mission in Yemen, the Yemen Monitoring and Evaluation Project (YMEP) implemented by International Business & Technical Consultants, Inc. (IBTCI) mobilized an international evaluator to work with a local data collector. This evaluation report on the main assessment findings of the ‘Promoting Youth for Civic Engagement’ (PYCE) project mid-term evaluation. Implemented by AMIDEAST in Yemen, PYCE is a two-year project with a budget of US$3,578,594 funded by the United States Agency for International Development (USAID) under Cooperative Agreement #279-A-00-10- 00060-00. The PYCE project falls under the USAID Conflict Management and Mitigation (CMM) initiatives designed to improve community capacity to mitigate conflict and to address some of Yemen’s root causes of instability. There is wide recognition that one of the main drivers/root causes of instability in Yemen is a large youth bulge: more than half (54%) of the population is younger than 18 years, with youth unemployment identified as a major problem that contributes to Yemen youth susceptibility to extremist messages.1 Other related drivers of instability are unequal development, political marginalization and repression of the disaffected, widespread corruption, weak state institutions, declining government revenues, growing natural resource scarcity, and the spread of violent Islamist extremism.2 In addition to improving community capacity to mitigate conflict, the CMM initiatives, inclusive of the PYCE project, are also expected to resonate with and positively feed into the 2010 – 2012 USAID Yemen stabilization strategy. The USAID stabilization strategy aims to mitigate sources of what has been referred to as ‘drivers’ of instability “while simultaneously introducing stabilization factors through improving the livelihoods of citizens in disadvantaged communities and improving governance capacities to provide services and to respond to citizen’s needs. The stabilization / development hypothesis is that improved community and individual perceptions that government is enhancing its services delivery and demonstrating legitimate state presence in response to their needs reduces perceptions and behaviors that lead to instability.” The original geographic focus of the USAID Yemen stabilization strategy was on districts and governorates (regions) that are vulnerable and which pose serious risk to Yemen’s overall stability. The ten USAID identified priority governorates include the five target governorates of the PYCE project namely: Sana’a, Marib, Aden, Abyan and Amran. USAID’s overall goal to increase Yemen’s stability through targeted interventions in vulnerable areas was planned to be realized though two Assistance Objectives (AO): 1- Livelihoods in targeted communities improved; and 2- Governance capacities improved to mitigate drivers of instability. The PYCE project responds to USAID Yemen AO2, specifically the project falls under Intermediate Result (IR) 2.3: ‘Community-based institutions and mechanisms to ensure active participation in governance and locally-driven solutions strengthened’.                                                              1 USAID Yemen Performance Management Plan (PMP) FY 2011 - FY 2013 2 USAID 2010 – 2012 Yemen Country Strategy Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 2 B. YEMEN POPULAR UPRISING The achievement of the Yemen stabilization strategy objectives was impeded by Yemen’s popular uprising that first began in late January 2011. A turning point in the Yemeni “Arab Spring” came on March 18, 2011 when security forces and government supporters opened fire on demonstrators in Sana’a in a failed and bloody attempt to break the protests. The day of El Karameh, as it was later termed, resulted in the death of 52 people and injury to more than 100. On March 21, 2011 five army commanders and one of the country’s most important tribal leaders threw their support behind the protesters. This event was followed by a stream of Yemeni officials’ resignation from the government, including the mayor of the restive southern city of Aden, a provincial governor and at least one of the country’s ambassadors. Secession within the army and shifts in Salah’s tribal support ultimately led to fighting within the capital city Sana’a between Sheikh Al Ahmar’s supporters and government security forces loyal to President Saleh. Despite numerous mediation agreements for a cease, fire heavy artillery clashes continued for several weeks in Sana’a causing heavy damages to infrastructure and households and the displacement of hundreds of families. Demonstrations and aggressive government repression were even more pronounced in the south where many southerners feel marginalized politically, economically and socially by the northern government and many groups are calling for secession. To complicate matters even further, a security vacuum in the country allowed an Al Qaeda affiliate, Ansar Al Shariah, to grow stronger and to take over large areas in the southern governorate of Abyan. Lastly, the failed attempt on the life of President Saleh, with what became known as Jame’a El Nahreyne,3 forced a breakthrough in the downward spiraling chain of events. This incident ultimately led to President Saleh signing the Gulf Cooperation Council agreement in November 2011, and his agreement to the handover of power in return for legal immunity from prosecution. In February 2012, Abed Rabbo Mansour Hadi, the country’s former Vice President, was elected as Yemen’s interim President for two years. Yemen is just beginning to assess and deal with the damage to the economy and the social fabric after a nearly a year-long public uprising against Mr. Saleh, who governed the country for more than three decades. It is against this backdrop of political, social and security upheaval that the PYCE project has been implemented since late 2010. The country context is reported from the perspective which shows how it affected PYCE and delayed its implementation.                                                              3 The Mosque of the Two Rivers. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 3 C. PROJECT SUMMARY PYCE is a two-year project (October 2010 – September 2012) designed to “improve the livelihoods of Yemeni youth in vulnerable areas by supporting USAID’s strategy to engage dynamic, responsive and credible religious leaders”. To achieve this goal, PYCE strives to deliver on two main project objectives: 1- Strengthening the role of religious actors in positively influencing Yemeni youth. 2- Establishing and supporting youth sports and recreational programs. a. PYCE Design and Implementation Methodology According to the program design plan (Cooperative Agreement (CoAg) - technical approach), PYCE should operate through steering committees comprised of educators, religious leaders and other local actors. This network of influential leaders reaches out to young people and their communities by offering training and encouraging sports and recreational events in community youth centers. PYCE steering committees (SCs) are selected during governorate-level launch meetings that invites moderate religious leaders, community leaders, sports managers and enthusiasts nominated by the Ministries of Endowments and Guidance (MoEG) and Youth and Sports (MoYS). The objective of the launch meetings is to inform, to ensure local buy in, to select the steering committee members, and to identify at-risks area with large number of vulnerable youth to be targeted by the program. The SC’s role is pivotal to PYCE project design because these committees are entrusted with maintaining communication with the project stakeholders to ensure continued community buy-in to PYCE activities, to nominate PYCE Peer Network members, to assess rapid response grants and finally to secure PYCE project sustainability over the long term. Selected by the governorate-level steering committee, the Peer Network (PN) comprises young religious and community actors, and youth members of local sports teams who have demonstrated an aptitude in influencing their peers. PN members participate in trainings on participatory assessment for community appraisal (PACA). PACA results should inform the design of the PN training curriculum and refine each Steering Committee’s governorate work plan. Following Training of Trainers (ToT) training and other related capacity building activities, members of the PN are encouraged and supported in reaching out to their less-advantaged peers with awareness training and community engagement undertakings. The PYCE project has designed an additional instrument to engage youth with their communities and extend communication with and to local leaderships: The Rapid Response Grants are awards of US$500 each for small projects aimed at community awareness and development, and the projects are implemented by youth in their communities. PACA assessment results help PN members identify and propose projects based on community involvement. The steering committees review these applications for small grants and provide recommendations based on local knowledge. The second PYCE objective, namely “establishing and supporting youth sports and recreational programs” is designed to be realized through an integrated series of activities, including renovating and equipping selected sports clubs and recreational centers and training coaches and referees nominated by MoYS. Once trained, coaches are encouraged to form PYCE youth sports teams. In addition, teams are supported through the provision of equipment and transport grants to form leagues so as to be able to engage in competitions. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 4 Overall, the project seeks “to reach out to uninvolved or alienated youth and facilitate relationships of mutual respect and communication between young people and moderate religious leaders through steering committees, workshops and trainings as well as catalyzing community participation and responsibility in young people through peers education and sports”.4 b. PYCE Status – March 31, 2012 Through the end of March 2012, the program had been contractually in operation for one and a half years, and had spent, according to AMIDEAST financial reports, only US$842,460 or 23% of its total CoAg budget. The Yemen youth revolution and the ensuing demonstrations and security events limited the PYCE project activities almost from the inception of the project. In June 2011, AMIDEAST submitted a revised ‘strategy’ to adjust to the field challenges encountered during implementation. The proposed revisions involved delaying investments in equipment and major rehabilitation works till the end of 2011, and the suggestion to implement the project’s activities in only three of the five originally targeted governorates. Additional minor changes were also proposed, such as supporting some youth initiatives during Ramadan, and targeting school-aged youth which entailed working with educators and physical education professionals. In July 2011, USAID approved the proposed changes which they decided would not necessitate an amendment to the project’s original cooperative agreement. The following is a summary of the project’s main achievements in each of the targeted governorates of Aden, Sana’a and Marib as at the end of March 2012: b.1: Aden Governorate: PYCE was first launched in Aden and the project achievements in this governorate5 are more numerous than in Sana’a and Marib governorates. This progress was facilitated by the already-established presence of AMIDEAST in Aden, according to the PYCE Chief of Party (COP). In accordance with the project design methodology, PYCE initiated contacts with the various relevant ministries and local leaders, launched the project in a community meeting-workshop, and successfully formed the governorate level steering committee. The Aden Steering Committee has effectively participated in the selection of youth for the PN, has facilitated community relations and has recommended vulnerable target areas for project activities. Activities in center renovation have been undertaken at the Al Rawdah Youth and Sports club, PN members and other youths have been trained in PACA, first aid, self-defense, photography, calligraphy and various other topics, the most important of which is the ToT to develop a community of peers’ trainers. Following the ToT, a select number of PN members have trained youth in various life skills such as self-confidence, positive messaging, and communication skills. Additionally, small mini-grants of US$100 or less have been awarded to select youth to encourage and support community engagement in various awareness and beautification campaigns.                                                              4 PYCE Cooperative Agreement signed on 30 September 2010. 5 Unfortunately, the actual project “dollar investment” amounts in each governorate are not available for comparison purposes. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 5 To promote youth sports and recreational activities, PYCE Aden provided training in basketball coaching and one-day clinics in other sports, as well as the rehabilitation of the Aden medical college basketball and volleyball courts. PYCE also supported various ‘street children’ teams and sports competitions. b.2: Sana’a Governorate: PYCE operations in Sana’a started later, in March and April 2011, and followed a different approach from that of Aden. The project hired a consultant to form the Sana’a steering committee and this committee was later suspended following an incident with the PYCE consultant.6 As a consequence, PYCE activities which were planned to strengthen the Azal Men’s Club and the Nuqum Women’s Training Center as relevant community institutions in the Musaiq neighborhood, were suspended. In Sana’a the youth Peer Network members were directly and almost solely recruited from AMIDEAST present or past English language students, and were not clustered around youth sports and recreational center(s) in a vulnerable area. Still, PYCE directly implemented project activities in Sana’a and offered similar training courses and life skills development as in Aden. These included PACA, first aid, self-defense, photography and media skills, chess competitions, handball drills and basketball coach training. Community engagement mini￾grants were also awarded to youth projects in community awareness and engagement and the project undertook the renovation of an outdoor basketball court for the 22nd May Sport Club. It should be noted that the list of PYCE project activities by governorate clearly demonstrates that operations (total project’s activities) in Sana’a have progressed much less than for Aden. b.3: Marib Governorate: Contacts in Marib were begun by PYCE in 2011 and then put on hold due to political events. PYCE operations in this governorate resumed in February 2012. Consequently, PYCE activities in this region are still more limited in scope than in Sana’a.7 PYCE undertook the same approach as in Sana’a, with the recruiting of a consultant to be the governorate coordinator who advised the project forming the steering committee (in progress) and activities to be undertaken. The PN has not yet been selected. Still, PYCE undertook to implement activities directly in Marib in the El Juba district with the Naseem Club. PYCE provided coach training to a select number of youth and assisted the Club with renovations of their basketball courts as well as providing sports clothing (t-shirts, shorts and basketball shoes), formation of teams and supporting team sports competitions. The project also undertook training in PACA for a select number of youth, preliminary to launching the community engagement component through mini grants awards. Additionally, PYCE carried out some training in volleyball for girls from Mohamed Hae’l Girls’ School in Marib. In conclusion, by the end of March 2012, PYCE had been operating for eighteen months or the equivalent of 75% of its originally contracted CoAg timeframe. However, project expenditures by this date amounted to only 23% of the project’s total program budget. AMIDEAST has requested an extension of the project for 12 months beyond its current CoAg closing date (30 September 2012). “USAID needs to                                                              6 This incident has been reported differently by different groups. 7 Refer to PYCE project Quarterly Reports for lists of activities by governorate. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 6 analyze PYCE from the technical side to assess the continued appropriateness of the project design and implementation methodology to meet the needs of youth and target communities and to ensure that the extension of the project will improved burn rates with effective expenditures”.8 This document reports on the main evaluation findings. Following this introductory background section, the evaluation methodology is presented, followed by the evaluation findings in the main body of the report. Finally, the report concludes with summary conclusions and final recommendations to USAID.                                                              8 PYCE midterm evaluation Terms of Reference. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 7 II. EVALUATION METHODOLOGY The PYCE evaluation methodology was designed to address the evaluation questions and issues covered in the terms of reference. The project’s desk review documentation has identified the stakeholders’ groups listed below. A representative sample of these groups was targeted with interviews, focus groups (FG) meetings and surveys relevant to their participation and their level of engagement with the project. A. PYCE PROJECT STAKEHOLDERS The assessment methodology clustered PYCE project stakeholders into the following main category groups based on their role and level of engagement with the project: A.1: PYCE Program Management and Staff: Interviews and FG meetings were undertaken with PYCE CoP and staff of both the Sana’a and Aden offices. The assessment discussions with PYCE project senior management elicited their views and perspectives on the project design and implementation strategies, the project M&E systems, in-country operating challenges, and contingency measures undertaken to overcome these challenges. A.2: Governorate Steering Committees (G-SC): The AMIDEAST PYCE project is coordinated through governorate steering committees (G-SC) comprising religious leaders, educators and other local actors. The project’s Quarterly Reports document the formation of three steering committees, one for each of the three governorates of Aden, Sana’a (G-SC not functioning) and Marib. The evaluation team undertook interviews and FG meetings with representatives of the three G-SCs, with the objective of discussing their role in the project and their level of engagement and commitment to the project’s objectives, along with their assessment of PYCE’s impact on youth and stabilization. A.3: PYCE Program Participants: The Governorate Steering Committee nominates a PYCE PN comprised of youth, religious and community actors who reach out to peers and communities, conduct training and encourage sports and recreation events in youth centers. PN members in Aden and Sana’a (the PN is not formed in Marib yet) have been invited to participate in focus group meetings where their experience with and benefit from the project was discussed. A short survey was distributed at the end of the meeting to collect objective and quantifiable data that measures the PYCE project’s outcomes and results. A.4: PYCE Direct Beneficiaries: PYCE conducted trainings on PACA, community engagement, basketball coaching and referee training, and awarded community engagement grants to youth and PN members. Trainees and grant beneficiaries from the three governorates (Aden, Sana’a and Marib) were invited to attend an FG meeting to discuss their experience with and benefits from participating in the project. A short survey was distributed at the end of the meeting to collect objective and quantifiable data that measures PYCE project’s outcomes and results. A.5: Renovated Sports Clubs: PYCE undertook the renovation and refurbishment of youth and sports clubs. The evaluation team carried out site visits to the renovated clubs and met with the managers of these facilities to assess the benefits and impact of the renovations and discuss their participation in the project. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 8 A.6: Sub Grantee: PeacePlayers International (PPI) was invited to share their views of the PYCE project strategy and effectiveness in achieving expected results. PPI staff was interviewed by email as they are not presently in Yemen. A.7: Other USAID CMM and or Youth Initiatives which are (or have been) active in the same areas and governorates, such as Mercy Corps’ (MC) ‘Engaging Youth for a Stable Yemen’ (EYSY) project and Partners for Democratic Change’s (PDC) “Yemen Community-Based Conflict Mitigation” project. The objective of meeting with this group was to assess the impact of the country’s environment on other project’s implementation and the effectiveness of their contingency measures undertaken to counter Yemen’s security challenges. A.8: Republic of Yemen Government (ROYG) Stakeholders: The primary Yemeni government stakeholders for the PYCE project are the Ministries of Endowments and Guidance, and Youth and Sports. Interviews were conducted with representatives of these ministries in the three PYCE “active” governorates of Aden, Sana’a and Marib. Meetings with this group aimed at assessing their level of endorsement, involvement and awareness of the project’s objectives, as well as their assessment of PYCE benefits and project impact on youth and community stabilization. A.9: USAID: Meetings with the USAID Technical Director and USAID AOTR for PYCE to discuss project concerns and anticipated changes to USAID strategy for Yemen. B. PYCE MIDTERM EVALUATION INSTRUMENTS To address PYCE evaluation questions, the assessment methodology involved the following qualitative as well as quantitative assessment tools and instruments: B.1: Desk Review: The desk review documentation facilitated the development of the evaluation approach and tools. The documents made available for the purpose of this evaluation consisted of: AMIDEAST Cooperative Agreement, PYCE Stabilization and Performance Monitoring Plan, PYCE Quarterly Reports up to March 2012, PYCE activities’ lists, and training interventions for each of the three governorates of Aden, Sana’a and Marib, Federal Financial Reports up to March 2012, and draft burn rate forms, as well as the USAID Yemen 2010-2012 Yemen Country Strategy and the USAID Performance Management Plan for FY 2011-FY 2013. Additional documents were requested directly from the project management, and included a PYCE organization chart and a cumulative indicators’ list. A detailed list of the documents consulted is attached in the report annex section B.2: Semi-Structured Interviews (SSIs): The semi-structured interview form was designed with mainly open-ended questions, and interviews were conducted with the project staff, ROYG officials, the management of other CMM funded projects, PPI sub-grantee, the managers of renovated sites, the USAID Technical director and the PYCE AOTR. A total of 21 PYCE stakeholders and beneficiaries, listed in Table 1, below, were interviewed in the course of the evaluation. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 9 B3: Focus Group Meetings: FG meetings of two to three hours each were conducted with members of the PYCE Steering Committees, the PN, and with project participants and beneficiaries. A total of 709 participants of the three governorates of Aden, Sana’a and Marib participated in the 10 FGs that were implemented during the course of the evaluation. B.4: Short Survey: A short survey was completed by the participants of the FGs at the end of the meeting to measure quantitatively the impact of their participation and involvement with the project. A total of 54 PYCE participants and beneficiaries participated in the survey. A copy of the translated survey is included in Annex C of this report. C. IMPLEMENTATION OF PYCE MIDTERM EVALUATION Following USAID’s approval, the PYCE midterm evaluation began at the end May, 2012. The field assessment in Yemen commenced on May 29 and was completed by June 15th 2012. The evaluation team interviewed 37 key informants, conducted 10 FG meetings, and administered 54 short surveys. In summary, the PYCE midterm evaluation involved interviewing and surveying a total of 91 concerned individuals as per Table 1 below: Table 1: Evaluation Stakeholders Groups and Assessment Instruments Assessment Tool SSI (FG) Short Stakeholder / Beneficiary Aden Sana’a Marib Survey A.1 PYCE Management and Staff for Aden and Sana’a offices. 4 2 -- (2) -- A.2 Governorate Steering Committee: Aden, Sana’a and Marib. 6 8 2 (3) -- A.3 Project Participants: Peer Network of Aden and Sana’a. -- (2) 23 A.4 PYCE Direct Beneficiaries (Aden, Sana’a, and Marib): - Coach basketball trainees - PACA trainees - Small Grants beneficiaries -- (3) 31 A.5 Site visits: Renovated clubs 2 1 1 -- -- A.6 Sub grantee: PeacePlayer International (PPI) 1 A.7 Other USAID-funded CMM initiatives active in the same governorates 1 1 -- A.8 ROYG Stakeholders 2 2 2                                                              9 Steering committee members’ input consisted of information from the interviews alone, since they did not complete a survey at the end of the FG meeting. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 10 Assessment Tool SSI (FG) Short Stakeholder / Beneficiary Aden Sana’a Marib Survey A.9 USAID Technical Director and PYCE AOTR - 2 - Total 37 (10) 54 Information collected from desk reviews, assessment interviews, FG meetings and short surveys has been triangulated across the evaluation issues to substantiate and/or identify measures of concurrence (or the lack thereof) among the PYCE results and outcomes. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 11 III. EVALUATION FINDINGS The purpose of this mid-term performance evaluation is to assess PYCE project’s implementation, effectiveness, and suitability of the project’s approach to achieving the project’s objectives. The assessment results will also inform USAID with regard to AMIDEAST’s request for a one year, no-cost extension of the project beyond its current September 2012 closing date. The midterm evaluation findings are presented in the subsequent sections of this report following the evaluation criteria and assessment issues: A- Project effectiveness; B- Project outcome on stabilization; C- Project reporting and M&E systems; and, D- Appraisal of AMIDEAST’s request for a PYCE project extension. The evaluation report will end with summary conclusions and recommendations to USAID. A. PROJECT’S EFFECTIVENESS The PYCE design and programmatic approach involves strengthening community actors, i.e., community leaders, local authorities, and religious leaders, to positively influence Yemeni youth, and to support the establishment and expansion of youth sports and recreational opportunities through Governorate Steering Committees involving educators, religious leaders and other local actors. To assess measures of PYCE effectiveness, defined as “the extent to which the project has attained its stated objectives or produced its desired results”, the midterm evaluation will provide answers to the following evaluation questions:  What progress has been achieved in each programmatic area to date?  Has the program achieved all of the expected results by this program midpoint of March 2012? If not, why not?  Is the PYCE design and approach an effective strategy for achieving program results? A.1: Modifications to Project Methodology Before starting an analysis of the PYCE ‘measures of effectiveness’, it is important to state that the project has diverged to a certain degree in approach and in implementation methodology from the technical approach prescribed in the CoAg. The alterations made to the PYCE ‘conflict mitigation’ model would explicate the project’s minimal outcome on stabilization. This will be detailed in section (B) of this report and will shed some light on the issues of concern that confronted the program during implementation. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 12 To achieve PYCE project’s goal of “improving the livelihoods of Yemeni youth in vulnerable areas by supporting USAID’s strategy to engage dynamic, responsive and credible religious leaders”, the PYCE project’s technical approach designed a set of activities that sought to realize two chief objectives: 1- Strengthening the role of religious actors in positively influencing Yemeni youth. 2- Establishing and supporting youth sports and recreational programs. The evaluation desk review and field assessment noted the following modifications to the project’s approach, targeting strategy, and implementation methodology as presented in the original project description: Strengthen Religious Leaders: The first objective of the PYCE project is “to strengthen the role of moderate religious actors in positively influencing Yemeni youth”. The assessment uncovered the fact that only two persons out of the numerous stakeholders involved with the PYCE are “religious actors”: one is an Imam and the second is a religious studies teacher. The question should be: How is PYCE achieving its objective of strengthening the role of moderate religious leaders when only two religious actors are involved in the project as part of the Aden Steering Committee? To substantiate this finding, it is noted that AMIDEAST amended its reporting under the first objective to:  “promoting and strengthening the role of influential community leaders, including non-traditional groups, to inspire Yemeni youth to better themselves and their communities.”10 Launch Meetings and Formation of Steering Committees: The project’s methodology involved launching the project in each governorate through ‘launch meetings’ with participants nominated by the concerned ministries of MoEG and MoYS. The aim of the launch meetings was to inform and engage all projects stakeholders, especially the RoYG ministries. The primary output of the launch meetings was the formation of the Governorate Steering Committee whose role in the project is considered vital to maintaining communication with the various ministries and project stakeholders, to ensuring community buy-in and the project’s future sustainability, and for nominating the PN members as well as advising on the selection of the rapid response grants. The evaluation desk review and field assessment revealed the following:  Only one launch meeting was carried out instead of the initially planned five. This meeting took place in the Aden governorate. Unfortunately, for reasons that were not made clear to the YMEP evaluator, the Aden Governorate Steering Committee that resulted from this meeting ended up being a neighborhood committee for Kalu’a (one of the targeted area in Aden). This change from ‘governorate’ to ‘neighborhood’ devalued the SC ‘governorate umbrella’ function to a much lower level. To validate this finding, it is noted that project reports and interviews with staff mention that the PYCE was in the process of forming another ‘neighborhood’ committee for the second neighborhood of Sheikh Ishaq to be targeted by the project in Aden.  In Sana’a (as well as in Marib governorate), PYCE hired a consultant to form the committee. An incident took place during one of the PYCE events and different versions of this incident have                                                              10 This change is first reported in the project Quarter 3 Progress Report dated July 31, 2011. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 13 been reported by different stakeholders. The PYCE staff had a different version of this conflict than the one reported by the Sana’a Steering Committee. The evaluation will not elaborate further on the incident per se (ironically a conflict) but rather state the finding that the consequence of this conflict was that the PYCE management ignored the role of the steering committee (instead of attempting to resolve the conflict) and began to implement PYCE project activities directly without any relationship to the community.  Our interviews and meetings with the Marib Steering Committee and the Ministries’ representatives confirmed that PYCE staff is directly selecting members of the Steering Committee in Marib without any reference to or endorsement of the relevant Ministries. Furthermore, Marib committee members interviewed felt that they have no real input in the PYCE project and that their role is “on paper” only.  One role of the Steering Committee is to review and advise on youth applications for small grants based on their knowledge of the local context. Again, it is only in Aden that this approach was implemented most of the time (but not always). In Sana’a as in Marib, applications for grant awards were reviewed and decided upon by AMIDEAST staff.  During the evaluation interview with the PYCE project leadership, PYCE management affirmed that they do not want to have the steering committees “drive the project”. It is important to note that the project’s ‘new’ approach has thus foregone communication and involvement of the primary Ministries and other local stakeholders concerned with the project. This lack of Yemeni stakeholder involvement has defeated the purpose of the stabilization approach recommended by USAID to improve trust and perception of the Yemeni youth with their government officials and local leaders. Except for Aden’s ‘neighborhood’ steering committee, local leaders’ involvement with PYCE in the other two governorates is practically non-existent. Steering Committee and Peer Network: The original PYCE project concept was built around peers role model development and peers education and outreach to other peers. The Steering Committee would advise on the selection of the PN members based on leadership aptitudes and ability to influence others. The youth selected for the PN would receive life skills training to develop self-confidence, leadership, teamwork skills, critical thinking, and volunteerism to foster civic responsibility. Trained PN youth would reach out to other less advantaged Yemeni youth (peers) with awareness raising, and serve as role models in positive messaging, community engagement and trainings through sports and recreational initiatives. The final objective was to create communication channels and trust relationships between alienated youth and community leaders. Field assessment and interviews with the project’s stakeholders and the peers themselves noted the following divergence in the project conceptual approach:  It is only in Aden that PN members were selected by the Steering Committee. In Sana’a, following the conflict incident mentioned earlier, the project staff selected the PN members directly through AMIDEAST outreach without any input or relationship to the ‘suspended’ Steering Committee. Marib Peer Network selection has not taken place to date (three months before the end of the project). Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 14  On only a few occasions, trained PN members (and trained basketball coaches) reached out to other peers in their community through implementation of the small grants for community engagement or the formation of sports teams. In most of the PYCE project activities, the youth participants are invited directly for trainings and participation in events by the project staff without PN or Steering Committee involvement. The changes that were implemented to the project implementation methodology have undermined the role of the Governorate Steering Committee and neglected the peers education model in favor of a direct implementation of activities that involve youth in sports and recreational initiatives and engage youth with community work. This approach will hardly create positive linkages and increase trust and communication between the various stakeholders through sports, recreational programs and community work. Furthermore, the direct implementation approach that has been favored by PYCE staff in the second half of the project will negatively impact on the sustainability of some of the project’s initiatives. Promoting Youth Sports: Under the second objective of “establishing and supporting youth sports and recreational programs” some of the project activities were designed to promote sports as a recreational activity for vulnerable Yemeni youth through training of a core group of coaches and referees, the majority of them to be nominated by MoYS. The project would then encourage the newly-trained coaches in each governorate to form PYCE sports teams, to engage in competitions with other youth and teams, and finally, to launch PYCE sports leagues and formal competitions. The ultimate objective is to encourage sports for leadership skills, team building, as an outreach to community members and vulnerable youth and for the constructive and healthy use of youth leisure time. “Due to security events”11, the initial targeting for this activity was changed. The evaluation interviews with PYCE management, PPI, a subcontractor to AMIDEAST, and FG meetings with the youth trainees themselves resulted in the following findings:  The criteria for participant selection changed significantly from the project’s original design and intent. Initially, the goal was to train community-minded basketball coaches to use new tools to reach out to their communities. Instead, the project ended up training community-minded young people as basketball coaches, according to PPI.  None of the youth that were trained by the project in ‘coaching’ was nominated by the MoYS or were sports coaches to start with.  More than half of the youth basketball coach trainees that were interviewed had never played the game prior to their coach training. This is especially true for women.  Though the youth trainees appreciated the training in its approach and content, very few of the trainees that were interviewed had managed to form teams following the training or were playing the game regularly. This is due to the PYCE change in target audience from those already invested in basketball to those only interested in basketball according to our interview with PPI. The above-noted change in the project target audience has led to a sizable reduction in the expected activity outreach, and in the outcome of “a network of basketball teams and leagues throughout Yemen                                                              11 PeacePlayers International (PPI) was informed by AMIDEAST of this information. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 15 grounded in the same approach of using sport as a tool for engaging young people”. The change in the target audience has also negatively impacted the sustainability of the training outcome. Summary Conclusion: It is usual that projects experience certain changes during implementation, changes that are often necessary to accommodate the realities “on the ground” which often vary from the original program design assumptions. The evaluation found that AMIDEAST was able to operate under difficult country circumstances. Nevertheless, the changes that occurred in the project’s conceptual approach have certainly affected the project’s outcome on stabilization and minimized its potential outreach to and positive influence on Yemeni youth. The evaluation findings note that the project leadership should have kept in mind the conflict mitigation framework of PYCE when undertaking changes to the initial project design for security concerns. Other options or choices would have been more appropriate to achieve the project’s objectives. When the project undertook the ‘correct’ outreach and engagement strategy of ROYG officials and community leaders (launch meetings, Steering Committee), the project managed to successfully engage these actors. The evaluation interviews and FG meetings with the representatives of the MoEG and MoYS offices in Aden, as well as with members of the Aden Steering Committee, found that initial suspicions about the project and the implementing organization were put to rest and that the project managed to gain the trust, support and full endorsement of Aden stakeholders. For these leaders, the project’s targeting of youth through training, sports and recreational activities is relevant to providing alternatives for youth to detract from the appeal of extremist movements. Finally, it is the evaluator’s opinion that the changes undergone by the PYCE project methodology and targeting has shifted its perspective from a conflict mitigation initiative to a project that now promotes sports and youth training and engagement in community awareness. A.2: Appraisal of PYCE Progress to End of Project Results PYCE is a two-year project that was launched in October 2010. As of late March 2012, the project has been in operation for 18 months or 75% of the life of the project. To assess the PYCE progress, the following Table 2 contains comparative presentation of PYCE outputs achieved under each objective and activity to the end of March 2012, with the planned end of project outputs in the USAID-approved Performance Monitoring Plan. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 16 Table 2: PYCE Comparative Performance Indicator Table Assistance Objective 2, Intermediate Result 2.3: Community-based institutions and mechanisms to ensure active participation in governance and locally-driven solutions strengthened Activity 1: Strengthening the role of moderate religious actors (F Indicator 1.6.2-5: Number of people attending facilitated events that are geared towards strengthening understanding among conflict-affected groups that were supported by USG assistance) Achieved - % of Target Target Task 1.1. Number of religious leaders engaged in governorate-level meetings in selected governorates 208 83% 250 Task 1.2. Number of Governorate Steering Committees established in selected governorates 3 60% 5 Task 1.3. Number of Peer Network members trained to conduct PACA 113 75% 150 Task 1.4. Number of sustainability meetings held at selected youth centers for renovation 3 11% 28 Task 1.5. Peer Network Training curriculum developed 1 100 % 1 Task 1.6. Number of religious and community actors received TOT for Peer Network 12 10% 125 Task 1.7. Number of participants trained by the PYCE Peer Network (TOT graduates) in selected governorates 179 7% 2700 Task 1.8. Number of grants awarded in selected governorates 13 26% 50 Activity 2: Establish and support youth and recreation programs(F Indicator 1.6.2-5: Number of people attending facilitated events that are geared towards strengthening understanding among conflict-affected groups that were supported by USG assistance) Task 2.1. Identify current and coming initiatives that develop individual sports, recreation and community service among youth in targeted governorates 32 320 % 10 Task 2.2. Strengthen current initiatives to support PYCE Peer Network and an array of individual sports and recreational and other community service activities 9 90% 10 Task 2.3. Number of sites renovated in selected governorates 2 28% 7 Task 2.4. Number of coaches and referees trained in selected governorates 65 217 % 30 Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 17 Assistance Objective 2, Intermediate Result 2.3: Community-based institutions and mechanisms to ensure active participation in governance and locally-driven solutions strengthened Activity 1: Strengthening the role of moderate religious actors (F Indicator 1.6.2-5: Number of people attending facilitated events that are geared towards strengthening understanding among conflict-affected groups that were supported by USG assistance) Achieved - % of Target Target Task 2.5. Number of teams/youth groups formed in selected governorates 17 113 % 15 teams - 150 playe rs Task 2.6. Number of youth clubs renovated by equipment and grants provided 4 57% 7 grant s - 7 sites Task 2.7. Number of communication channels engaged with the PYCE Peer Network to disseminate information, success stories, announcements, and press releases 11 55% 20 coach es/ 20PN mem bers Task 2.8. Number of leagues established and launched in selected governorates 2 7% 30 comp etitio n Note: Percentage numbers have been rounded to the next whole number. It should be noted that the information provided in the above table in its cumulative form was not made available through the project quarterly reports.12 It was provided to the evaluation team following our direct request to PYCE project management. A general review of the project’s achieved results to 30 March 2012 versus end-of-project targets might indicate that the PYCE has managed to achieve many of its target results, while sometimes exceeding PMP targets. However if this is actually the case, what would then account for the fact that the project burn rates up to 30 March is only 23% of the project’s obligated amount? To assess the validity of the project’s reported output indicators; the evaluation team compared the reported output indicators in Table 2 with the “definition of the indicator’s description” in the approved                                                              12 A discussion of issues regarding the project M&E data and reporting is to be found in section C Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 18 Performance Monitoring Plan. The review and comparison of both documents with the Quarterly Report resulted in the following findings: Task 1.1: “Number of religious leaders engaged in governorate-level meetings in selected governorates”. “This indicator counts the number of religious and community leaders contacted through the MoEG and who participated in the governorate level meetings to discuss the project objectives (50 participants x 5 meetings)”.13 According to the PYCE Quarterly Reports, the figure of 208 (83%) reported religious leaders is actually the number of participants in the Aden Launch Meeting (160) plus 48 community members who participated in activity events such as a photo exhibit and an environmental discussion in Aden – at Al Rawdha club. As reported earlier, no governorate launch event was undertaken in the other two governorates. Therefore, the correct number should be only 160. In addition, not all of the participants at the Aden Launch Meeting were religious or community leaders nor were they all nominated by the MoEG. Task 1.2: “Number of Governorate Steering Committees established in selected governorates”. “This indicator counts the number of governorate steering committees in each of the selected governorates (1 committee x 5 governorates). PYCE reported three steering committees (60%): one Steering Committee in Aden, one committee in Sana’a which is ‘suspended’, and one in Marib. However, this Steering Committee is under formation with only four members involved at present. It would have been more accurate to report only two steering committees at best. Task 1.3: “Number of Peer Network members trained to conduct PACA”. “This indicator counts the number of network youth community actors trained to conduct PACA and to train youth”. The output indicator table above states 113 or 75% of the project target of 150. Reviewing the total number of youth members of the PN, the evaluators note that the total number of PN members cannot by any account exceed 70 (40 for Aden, of which only 25 remain, plus three to four members added recently plus 25 for Sana’a. Marib does not yet have a PN). AMIDEAST has provided some PACA training to 113 youths14 but not all of them are members of a PN and thus should not be added to the cumulative number under this indicator. Task 1.8: “Number of grants awarded in selected governorates”. “This indicator counts the number of grants awarded to members of the PYCE Peer Networks (average of US$500)”. Table 2 reports 13 grants or 23% of the 50 end of project target. It should be noted under this indicator that the average amount of a grant was US$ 100 and less, rather than US$500, as per our interviews with staff and PYCE grants beneficiaries. This smaller amount of the individual grant, which is less than what was originally budgeted would, consequently account for much more than 50 as an end of project target. Also, the grants were provided to youth in general and not only to Peer Network members. Task 2.4: “Number of coaches and referees trained in selected governorates”. “This indicator counts the number of coaches and referees trained in the selected governorates”. As mentioned previously, no actual                                                              13 Definition of the indicator in the USAID approved Performance Monitoring Plan. 14 AMIDEAST provided some short PACA training to youth as a way of engaging them with their community and for preparing applications for grants awards. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 19 coach or referees were trained and the numbers reported under this indicator of 65 or 217% of the end-of￾project target of 30 refer to youth that have been trained on the basics of basketball coaching. Task 2.5: “Number of teams/youth groups formed in selected governorates”. “This indicator counts the number of teams formed by the trained coaches and referees in selected governorates”. Under this indicator, the above table reports 17 teams or 113% of the end-of-project target of 15. The YMEP evaluators noted the discrepancy of the October – December 2011 Quarterly Report which mentions the formation of nine teams. However, the coaches’ training had not yet taken place, according to the same report. The evaluation interviews and YMEP monitoring of the PYCE project report that AMIDEADT recruited a consultant in Aden to promote sports’ games and team competitions. This consultant has been recruiting teams of youth or ‘street teams’, supporting them with some supplies and equipment (balls, T-shirts) and organizing competitions. Those informal activities might account for the number of teams reported in this Quarterly Report, whereas this indicator should be reporting on the impact of training coaches and formation of new teams. Finally, it is noted that the above analysis is not a data validation assessment. The analysis has been included in the evaluation report to highlight the fact that the numbers reported do not accurately reflect the actual PYCE project’s output as defined in the project’s Performance Monitoring Plan. Also, that the project’s comparatively low burn rate is in discrepancy with these numbers. We would recommend that YMEP undertake a data validation analysis to address this issue further and possibly also a training of PYCE staff on data collection and validation. Furthermore, PYCE has implemented various activities in sports training, Ramadan recreational events, and community awareness. The output of some these activities is not reported nor is it slated to be reported in the PYCE performance plan as these activities were initiated apart from the original project description and work plan. To recapitulate and respond to the evaluation questions indicated in the evaluation terms of reference: What progress has been achieved in each programmatic area to date? Has the program achieved all of the expected results by this program midpoint of March 2012? If not, why not? It is difficult, to assess the progress achieved by PYCE in each programmatic area. It also seems nonproductive to compare the project’s initial work plan and M&E plans with the project’s present achievements in numbers or activities due to the changes made in project’s approach, methodology, targeting, and indicator definitions. Under the first objective of “strengthening the role of religious actors in positively influencing Yemeni youth”, the project has trained certain youth to prepare for their engagement with other youth and their community, but has missed out on the overall objective of engaging religious actors and establishing lines of communication between the various project stakeholders to ‘positively influence Yemeni youth’. Under the second objective of “establishing and supporting youth sports and recreational programs”, PYCE has supported and promoted sports and recreational activities but is still behind in renovating Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 20 sports and community centers and in the formation of new recreational sports teams and league competitions. Finally, the project’s progress is delayed compared with its original and amended plan especially in Sana’a and Marib governorates. Although USAID in July 2011 approved AMIDEAST’s revised PYCE ‘strategy’ changes15, some of the proposed activities slated in the document for the end of 2011, such as center renovations, have yet to take place. It should be noted that overall, Yemen has been ‘relatively stable’ since the signing of the GCC agreement in November of 2011 and basically no security events should have halted or delayed the implementation of these activities. As to the reason the project has not achieved its expected results, according to AMIDEAST leadership, the main challenge that accounts for most of the delay in the PYCE project’s implementation schedule is the security issue in Yemen. It is the evaluator’s opinion that the events in Yemen might account for a certain delay, but cannot possibly explain the extent of divergence of PYCE from its initial implementation schedule. This will be discussed in Section D with data from other similar CMM-funded initiatives targeting youth, such as the EYSY project of Mercy Corps, which is well on its way to achieving the project objectives even though faced with similar country challenges as PYCE. The data discussed previously leads the evaluators to the conclusion that PYCE might be mismanaged and also understaffed16. “Mismanagement” refers to the project’s capacity to involve and engage with multiple stakeholders especially ROYG ministries and officials, and to put in place a ‘feasible’ project plan and contingency measures that take the risk realities of an insecure environment into account and put in place measures to address them. Is the PYCE design and approach an effective strategy for achieving program results? The original program design and approach can be an effective strategy for achieving program results when properly implemented. As mentioned earlier, the PYCE project’s approach in Aden governorate was successful in insuring community buy-in and in gaining the trust of the local leaderships and engagement with key stakeholders. The project was successful in this region up to the point where the project stopped delivering on its promises, especially with the youth Peer Network members. The survey results in the following Section B: Project Outcome on Stabilization will shed further light on this issue. B. PROJECT OUTCOME ON STABILIZATION The overall goal of the PYCE project is “to improve the livelihoods of Yemeni youth in vulnerable areas by supporting USAID’s strategy to engage dynamic, responsive and credible religious leaders…” To assess project outcomes, the evaluation efforts sought answers to the following evaluation questions.  Is the implementation methodology appropriate in achieving USAID’s strategy of positively impacting stabilization by targeting youth groups?                                                              15 USAID requested a revised work plan for Year 1 and 2 but this plan, if available, was not included in the desk review documents. 16 Refer to project organization chart attached in the annex section of this report. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 21  Is the PYCE original plan and implementation mechanism still appropriate to meet the needs of youth and target communities?  Is the project inclusive and building consensus within target communities?  Is the project achieving any level of sustainability of the PYCE activities within targeted communities?  To what extent has the project contributed to accomplishing mission’s stabilization strategy objectives? The evaluation team noted after reviewing the PYCE Quarterly Progress Reports and PMP plan that PYCE has not been reporting on the stabilization outcome indicators prescribed in the Performance Management Plan.17 It was difficult (if not impossible) to assess the degree or measures of outcome￾impact achieved by the project in the absence of these long term (LT) and short term (ST) stabilization outcome indicators. Thus, the evaluation undertook to collect some ‘midterm’ indirect outcome information through interviews, FG meetings and survey data with the various stakeholders’ groups as detailed in the methodology section earlier. A project’s outcomes are usually assessed at the end of the project, or following project’s closure. As PYCE is yet in a ‘midterm phase’, the evaluation undertook to appraise the possibility of these outcomes being achieved by the end of the project. Survey Results: In each governorate, the evaluators arranged meetings with members of the steering committee, PN and youth trainees in basketball coaching, PACA and grants’ beneficiaries. A short survey was completed at the end of the meeting with the PN and youth trainees. The survey first inquired of the participants’ involvement in PYCE activities, assessed their views on the appropriateness of the training to their needs and communities and finally, asked their opinion regarding the training impact on  self;  relations with others; and  relations with community and leaders. B.1: Aden Governorate Following are the tabulated results of the surveys completed in Aden by groups of PYCE participant￾beneficiaries. a. Aden Peer Network: A total of 15 PN members (10 male and 5 female) attended the FG meeting and shared in the discussions. The PN members participated in the following training activity: First Aid (13); Calligraphy (11); CV- Resume (10); Photography (6); Training of Trainers (6); Creative Thinking (2); Presentation Skills (1); Self Defense (1); Chess (1); Management of Small Projects (1).                                                              17 M&E issues are discussed in a subsequent section of this report Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 22 Fourteen out of 15 PN members thought that the training and awareness raising topics were not appropriate to their and community needs; mostly because either the training topics were unrelated to personal and community needs or when related, the training quality (trainers and materials) was substandard; also because the training topics did not respond to their suggestions nor to the recommendations of the community assessment that they undertook. The training did not have an impact! Table 3: Aden – Peer Network Survey Results Aden: Peer Network (FG2) Program Impact/Outcome on Self – Development Yes Yes, but Minor No - Not Selected 18 Total Increased self-confidence 4 5 - 6 15 A broader vision 2 1 5 - 7 15 A more positive attitude 1 5 - 9 15 I acquired new Information 8 1 2 - 4 15 I acquired new skills 4 3 4 - 4 15 A change in behavior 1 2 5 - 7 15 Self Development 1 1 6 -7 15 Others, specify -- -- - -- Program Impact/Outcome on Relations with Others Improved understanding of others 5 -- 4-6 15 Readiness to cooperate with others 4 2 2-7 15 Belief in group work 5 2 2-6 15 Positive relations with others 5 2 2-6 15 Others, specify -- -- -- -- Program Impact/Outcome on Relations with Community & Community Leaders Acquaintance with community leaders 1 1 7-6 15 Constructive cooperation with some leaders 1 -- 6-8 15 Constructive interaction with community -- 3 4-8 15 Improved appreciation of community 3 5 3-4 15 Increased motivation for advocacy & engagement with community 2 2 4-7 15 Others, specify -- -- -- -- The results in the Table 3 above show the relatively small number of PN who admitted to the training impact on the issues listed, specifically on relations with community and community leaders. The highest positive selection is for “acquiring new information”. The youth were actually very frustrated, with the management of the project. The extent of their frustration was in direct proportion to the expectations that they had at the beginning of the program. The number of selected PN members was originally 40 and it quickly dropped to 25 because many youth left the project. Most of those who stayed did so because they wanted to participate in the ToT program and thus, acquire some skill which later on might become a source of income for them. One of their chief complaints was the quality of the training and the trainers in                                                              18 Some participants did not select an item i.e. it does not apply or another way of saying ‘No’. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 23 some of the training events and also the fact the project did not respond to their suggestions and recommendations regarding training topics and other issues. b. Aden PACA Trainees, Basketball Coaches and Grant Beneficiaries: A total of 18 youths, participants in various PYCE trainings, attended the FG meeting and shared in the discussions. The youth had participated in the following training activities: Basketball Coaching (13); PACA (6); Small Grants (3); Boxing (1); and Photography (1). Nine out of 18 youths (50%) thought that the training and awareness-raising topics were not appropriate to their individual needs or to community needs. Most of those that responded negatively had participated in the basketball coach training and were female. Three participants in PACA also gave a negative answer, one withdrew from the training early because the objective was not clear, another because he did not benefit, and the third because some of the recommended topics were not offered in future trainings. Table 4: Aden - Youth Participants / Beneficiaries Survey Results Aden: PACA, Basketball, Grantees (FG3) Program Impact/Outcome on Self - Development Yes Not Selected19 Total Increased self confidence 15 3 18 A broader vision 7 11 18 A more positive attitude 7 11 18 I acquired new Information 8 10 18 I acquired new skills 14 4 18 A change in behavior 4 14 18 Self Development 4 14 18 Others, specify -- -- -- Program Impact/Outcome on Relations with Others Improved understanding of others 12 6 18 Readiness to cooperate with others 10 8 18 Belief in group work 16 2 18 Positive relations with others 8 10 18 Others, specify -- -- -- Program Impact/Outcome on relations with Community & Community Leaders Acquaintance with community leaders 5 13 18 Constructive cooperation with some leaders 1 17 18 Constructive interaction with community 3 15 18 Improved appreciation of community 3 15 18 Increased motivation for advocacy & engagement with community 5 13 18 Others, specify -- -- --                                                              19 Some participants did not select an item i.e. it does not apply or another way of saying ‘No’ Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 24 Respondents in this FG (Table 4 above) had more positive feedback about the trainings that they received and the outcome in terms of increased self-confidence, acquired skills, belief in group work, than the PN members. This is due to the fact that PN youth already have a certain level of leadership and ‘readiness’ prior to their engagement with the program in addition to the experienced frustrations that we mentioned earlier. Still, the impact of the project’s activities on relations with community and leaders is comparatively low especially considering the fact that this group of respondents includes youth that participated in PACA trainings and implemented community engagement activities with the grants awards B.2: Sana’a Governorate Following are the tabulated results of the surveys completed in Sana’a by groups of PYCE participant￾beneficiaries. a. Sana’a Peer Network: A total of eight (seven male and one female) PN members attended the focus group meeting and shared in the discussions. The PN members participated in the following training activity: First Aid (8), Self Defense (7), PACA (5), Basketball Coaching (5), Presentation Skills (5), Photography (2), Life Skills (1). Five out of eight PN members thought that the training and awareness raising topics was appropriate to their and community needs; those that responded negatively said that the training was not sufficient and they did not get to put the training into action, that is, practice the activities. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 25 Table 5: Sana’a- Peer Network Survey Results The PN respondents in Sana’a gave relatively higher positive ratings as compared with the PN of Aden; PYCE launched PN recruiting and training in Sana’a at the end of 2011. So the program is still relatively new in Sana’a and the PN members are still awaiting additional trainings. The PN in Sana’a, and similarly to their counterparts in Aden noted that the program presented lots of gaps in time where no activity had taken place. Additionally, they recently heard that the project was undertaking activities directly with other youths without their knowledge or involvement which posed some questions about their ‘supposed’ role in the PN project. b. Sana’a: PACA Trainees, Small Grants, Basketball Coaching: A total of nine (seven male and two females) youth participants in various PYCE trainings, attended the focus group meeting and shared in the discussions. The youth participated in the following training activity: PACA (2); Basket Ball coaching (4); Small Grants (3); Presentation Skills (1); Leadership Training (1); Participation in Events (2); First Aid (1); Grants (1); Management of Small Projects (1); PYCE workshop (1)                                                              20 Some participants did not select an item i.e. it does not apply or another way of saying ‘No’ Sana’a: Peer Network (FG5) Program Impact/Outcome on Self - Development Yes Yes, but minor Not Selected20 Total Increased self confidence 6 1 1 8 A broader vision 5 1 2 8 A more positive attitude 6 1 1 8 I acquired new Information 7 1 8 I acquired new skills 8 -- 8 A change in behavior 4 4 8 Self Development 4 1 3 8 Others, specify -- -- -- -- Program Impact/Outcome on Relations with Others Improved understanding of others 8 8 Readiness to cooperate with others 7 1 8 Belief in group work 7 1 8 Positive relations with others 5 3 8 Others, specify Program Impact/Outcome on relations with Community & Community Leaders Acquaintance with community leaders 2 6 8 Constructive cooperation with some leaders 1 7 8 Constructive interaction with community 2 6 8 Improved appreciation of community 2 6 8 Increased motivation for advocacy & engagement with community 4 4 8 Others, specify -- -- -- -- Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 26 All nine youths responded positively reference the appropriateness of the training and awareness-raising topics to their individual needs; Table 6: Sana’a - Youth Participants / Beneficiaries Survey Results Sana’a: PACA Trainees, Small Grants, Basket Ball Coaching (FG6) Program Impact/Outcome on Self – Development Yes Not Selected21 Total Increased self confidence 8 1 9 A broader vision 2 7 9 A more positive attitude 5 4 9 I acquired new Information 7 2 9 I acquired new skills 7 2 9 A change in behavior 4 5 9 Self Development 5 4 9 Others, specify -- -- -- Program Impact/Outcome on Relations with Others Improved understanding of others 7 2 9 Readiness to cooperate with others 8 1 9 Belief in group work 8 1 9 Positive relations with others 6 3 9 Others, specify Program Impact/Outcome on relations with Community & Community Leaders Acquaintance with community leaders 4 5 9 Constructive cooperation with some leaders 2 7 9 Constructive interaction with community 2 7 9 Improved appreciation of community 4 5 9 Increased motivation for advocacy & engagement with community 4 5 9 Others, specify -- -- -- Again, the survey results show a relative difference between the PN members’ assessment of the training benefits as compared with the more positive appraisal of other ‘less advantaged’ youth that participated with the project. The program impact on relations with community and community leadership is relatively highest for this group. Our discussions with this group confirmed that implementing PACA, small grants projects and basketball and coach trainings led to relationships with other youths, community awareness, and interaction with community members. B.3: Marib Governorate Following is the tabulated results of the surveys completed in Marib with groups of PYCE participant￾beneficiaries.                                                              21 Some participants did not select an item i.e. it does not apply or another way of saying ‘No’ Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 27 a. Marib: Basketball Coaching: A total of four (two males and two females) youth participants in basketball coaching attended the focus group meeting and shared in the discussions. 50% of the trainees confirmed the appropriateness of the training and awareness raising topics to their individual needs and to community needs; Table 7: Marib - Youth Participants / Beneficiaries Survey Results Marib – Basketball coach trainees (FG8) Program Impact/Outcome on Self - Development Yes Not Selected22 Total Increased self confidence 4 4 A broader vision 2 2 4 A more positive attitude 1 3 4 I acquired new Information 4 -- 4 I acquired new skills 4 -- 4 A change in behavior 2 2 4 Self Development 2 2 4 Others, specify -- -- -- Program Impact/Outcome on Relations with Others Improved understanding of others 4 4 Readiness to cooperate with others 4 4 Belief in group work 4 4 Positive relations with others 4 4 Others, specify Program Impact/Outcome on relations with Community & Community Leaders Acquaintance with community leaders 1 3 4 Constructive cooperation with some leaders 1 3 4 Constructive interaction with community 1 3 4 Improved appreciation of community -- 4 4 Increased motivation for advocacy & engagement with community 2 2 4 Others, specify -- -- -- PYCE activities in Marib are recent and basketball training is one of the few activities that has been implemented so far. The male basketball trainees were nominated by the Naseem Youth sports club which provides for an improved selection of youth already invested in sports and thus the training is much more appreciated. B.4: Summary, Concluding Remarks of the evaluation assessment with all stakeholders groups:  Youth Training Plan: A review of the training topics provided by the project shows the difference between the life skills and civic education topics proposed in the original project description, and                                                              22 Some participants did not select an item i.e. it does not apply or another way of saying ‘No’ Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 28 the topics that have actually been presented so far. How does a training in calligraphy ‘improve the livelihoods of Yemeni youth’? There does not seem to be a well-defined training plan with integrated topics and a curriculum with clear learning outcomes that ultimately lead to a peers ‘education’ objective. The trainings provided so far, and in most cases, are not linked to each other with a clear and ultimate training impact in mind. This assessment has been confirmed by reviewing the “training plan” that was provided by AMIDEAST upon request. A copy of this document is attached in Annex E.  Gap Periods: A common complaint of youth PN in Aden and Sana’a is that the program experienced many ‘gap’ periods where no activity or training was taking place. These gap periods, according to the youth interviewed, cannot be explained simply by pointing to security issues. Survey results (one question references the amount of time that youth invest in participation with PYCE) confirm that youth leisure time engagement with the project’s activities was minimal, which does not correspond to the project’s initial intent. Only a few of the survey participants (three or four) indicated a relatively high and regular investment of time as they formed sports teams and played or practiced the activity regularly.  Unrealistic Expectations: The fact that AMIDEAST is implementing the PYCE project led numerous youth to form unrealistic expectations of the benefits that they would gain from their participation in PYCE program activities, such as attending English and computer courses, educational grants...etc. The location of the PYCE project’s offices in the same premises as AMIDEAST or close by did not help in clarifying that PYCE is different from the traditional AMIDEAST programs in Yemen.  Unclear Project Objectives: Most of the beneficiaries and stakeholders that were interviewed during the course of this evaluation did not have a clear understanding of the PYCE project objectives. This fact helped in raising unrealistic expectations and consequent frustrations.  Targeting of Vulnerable – Disadvantaged Youth: The majority of youth that were targeted by the project in Sana’a are (or have been) AMIDEAST students which does not really serve the USAID project purpose of targeting vulnerable youth. Additionally, the last survey question asked what the youth surveyed (all groups) would have done with their time if they had not participated in PYCE activities. Around 95% of the respondents said they would be following up on their studies or getting involved with other organizations or possibly reading, writing, watching TV, working on the internet or playing basketball. Only 2% of the youth surveyed said they would be chewing qat or would be unemployed. It is questionable whether the PYCE youth beneficiaries targeted by AMIDEAST are those suitable to the project objectives and impact.  Local Leaders Support and Endorsement of PYCE: The evaluation meetings and various interviews with the local leaderships in Aden, inclusive of the ROYG officials of the MoYS and MoEG, confirm that the project has received their support and endorsement. The recommendations of these stakeholders will be presented in the last section of this report. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 29 To recapitulate and respond to the PYCE evaluation questions indicated in the evaluation terms of reference: Is the implementation methodology appropriate in achieving USAID’s strategy of positively impacting stabilization by targeting youth groups? To what extent has the project contributed to accomplishing mission’s stabilization strategy objectives? Again, it is not possible to provide a clear answer to this question since the initial project implementation methodology has diverged from the original plan, thus affecting the potential stabilization outcome. Furthermore, the PYCE project activities implemented so far have not yet created a ‘critical mass’ or a large enough momentum to assess the outcome on youth targeting and stabilization. On the other hand, the Ministries and local leadership interviewed in Aden, as well as the ‘suspended’ Steering Committee in Sana’a believe that the project has a good potential (when properly implemented in terms of leadership involvement) to positively impact stabilization. However, no tangible or measurable indication can be provided at this time specifically because the project did not report on the M&E stabilization indicators. M&E issues will be discussed in more depth in the following section of this report. Is the PYCE original plan and implementation mechanism still appropriate to meet the needs of youth and target communities? The original plan and implementation mechanism was properly implemented only in Aden and only up to certain extent. But it failed to engage religious leaders, to properly train PN members and to empower these peers to engage with other less-advantaged youth. The interviews with the various community stakeholders including the youth, indicated that the original plan and implementation mechanism would be suitable to meet youth needs and target communities. A review of certain activities, approaches and targeting, including the necessity of PYCE project’s engagement with the Ministry of Education should take place. It is also necessary to take into account the Yemeni cultural environment when targeting women, but overall the needs of these youth and target communities did not change from the time of the project design. The transitional period did not greatly affect the previously existing needs, but only made them more critical. Is the project inclusive and building consensus within target communities? Based on earlier analysis and the feedback collected from the various stakeholders’ groups interviewed in the course of this evaluation, the data shows that the project was not inclusive and did not build consensus except in some cases, such as in Aden and with the Aden leadership23. The project did not even manage, nor attempt, to resolve the conflict that arose with the Steering Committee in Sana’a although the members of this Committee were more than willing to resume their engagement with the project. Is the project achieving any level of sustainability of the PYCE activities within targeted communities? The answer is ‘yes’ for the PYCE activities that focused on youth and around centers—whether sports, educational or recreational, and the leadership of these centers or recreational events. An example would be the youth nominated by the Naseem club in Juba (Marib) for participation in the basketball coach training who later helped by assisting in the formation of sports teams within the overall structure of the club with the teams participating in competitions. However, the PYCE initiatives that do not revolve around such structures (a center and leadership) are not and cannot possibly be sustainable. This does not                                                              23 Aden leadership refers to the officials of the MoYS, MoEG and Aden steering committee mentioned earlier. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 30 mean that the activity per se is not beneficial or does not empower youth, just that these types of activities will not be sustained in the future after the project ends. An example of unsustainable activities would be some of the Ramadan initiatives, the organization of bike riding events, the training youth in basketball coaching who never played basketball and who are not affiliated with sports clubs or schools. C. PROJECT’S REPORTING & M&E SYSTEMS The terms of reference of the PYCE midterm evaluation includes a review of the project’s M&E systems and assessment “whether program reporting has met USAID standards and whether sufficient baseline data was collected”. C.1: Review of PYCE Performance Monitoring Plans Following the CoAg clauses, PYCE developed its internal performance monitoring and evaluation plan in alignment with the PYCE project objectives and planned activities. This performance plan was later replaced with the ‘Stabilization and Performance Monitoring Plan’ which utilizes USAID F and stabilization strategy indicators and follows the Mission’s PMP. The stabilization plan was approved by USAID in February 2011. The project’s second quarter report dated April 25, 2011 institutes reporting following the stabilization indicators. Reviewing this report and subsequent PYCE project reports, the following can be noted:  Each quarterly report provides a number for each ‘output indicator’ achieved during this quarter without any comparative reference to the end-of-project ‘target’ or to the cumulative numbers achieved to the date of the reporting period. This fact made it difficult for the project staff or anyone else to note the difference between achieved versus planned. This lack of reference between ‘achieved to date’ and ‘target’ makes tracking actual progress of the project quite difficult.  When requested, the PYCE staff prepared and made available to the evaluation team a table listing the project’s output indicators along with the end-of-project targets and cumulative data up to March 2012 (see a copy of this table in the evaluation report annexes section). However, the evaluation field assessment and review of the project’s documentation has triggered concerns with reference to the quality and validation of the reported data. For example, under Task 1.2. ‘Number of Governorate Steering Committees Established in selected governorates’, the data reports three steering committees including one in Sana’a. However interviews with the Sana’a Steering Committee provided evidence that the Committee has been suspended and non￾operational almost since its establishment.  The project quarterly reporting has provided M&E data on the PMP F Objective of “Peace and Security: Peace and Reconciliation Processes” and related F output indicators. However, the project did not collect nor reported data on the outcome stability effect indicators such as “% of targeted imams consistently providing stability-supporting messages (LT)” or “number of person days imams are engaged by the program (ST). This fact created a gap between the activity output and the projected stability outcome, as became clear during the data analysis (discussed in the previous sections of the evaluation report). Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 31  During interviews, PYCE staff reported some confusion with YMEP instructions on how to report and how to input the project data into the Clearinghouse database and consequently PYCE staff decided to suspend inputting their project data into the USAID/YMEP database. Inadequate reporting on the PYCE project’s performance and stabilization indicators has concealed the fact that the project has changed considerably from its original conflict mitigation end purposes and is not only ‘late in implementation’ as noted by the USAID Mission. C.2: Assessment of M&E Systems The PYCE project management does not have in place appropriate M&E systems to capture data to assess whether implementation is on track or not toward achieving the project’s objectives. Additionally, there is not a system in place to show that the activities being implemented are delivering on their planned outputs and on the projected PYCE outcomes. Interviews and meetings with the PYCE staff and project’s stakeholders confirmed the following:  A lack of M&E staff dedicated to monitoring and evaluating the project’s activities despite the fact that the CoAg Technical Approach clearly states that one M&E staff position was planned in the original project document. This fact was confirmed during the interviews with the PYCE management and also made clear in the project’s organization chart (a copy of the PYCE organization chart is attached in Annex F of this report).  Interviews and meetings with the project’s participants (grant beneficiaries, PN members, youth trainees) confirm that most, if not all, of the activities and trainings were carried out, either directly by the PYCE staff or through the youth grantees and trainers, and did not incorporate a monitoring and evaluation component. Grantees merely report on the event that was carried out with the grant fund but no PYCE M&E staff has monitored the PYCE activities nor assessed activity outcomes. Additionally, the majority of the trainings did not conclude with an end-of￾training assessment eliciting the participants’ feedback on the training topics, trainers, learning objectives or level of satisfaction. During one of the FG meetings, some PYCE youth complained that their oral comments and feedback on trainers and training topics were belittled and not taken seriously by PYCE. Monitoring and timely evaluation of project activities are important elements in project management as they ensure that project activities are supporting achievement of the project’s objectives and delivering on its ultimate outcome goal. Alternatively, monitoring and evaluation also are critical to enabling an organization to identify problems and challenges, and to take corrective actions when needed. C.3: Cost Effectiveness The evaluation terms of reference requested an analysis of the project’s cost effectiveness. This task proved to be unrealistic in light of two facts: 1. The project’s financial reporting does not provide detailed financial cost information for activities and tasks that can be correlated with the project’s outputs so as to undertake a cost effectiveness analysis. It seems that the only financial reports required of AMIDEAST per the CoAg are the quarterly Federal Financial Reports. The issue of the lack of financial details needs to be addressed by USAID to ensure that the project provides detailed financial reports, not only on the main budget line items, but also on the Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 32 main activity and task expenditures. More detailed quarterly financial reports would have highlighted the line items and activity tasks where the project is experiencing spending difficulties. 2. As noted previously, the PYCE outputs data is not reliable and therefore cannot provide the basis for an accurate cost-effectiveness analysis. In addition, some activities are not reported in the output indicators as being added to the project’s original plan at a later date. D. APPRAISAL OF AMIDEAST REQUEST FOR PYCE EXTENSION One of the primary objectives of the PYCE midterm evaluation is to assist USAID in responding to AMIDEAST’s request for a one-year, no-cost extension of the PYCE project. To facilitate the USAID decision-making process, the focus of the analysis will be on objectively assessing, to the extent possible, two key issues: 1. AMIDEAST’s claim that Yemen’s political and security events account for the extensive delay in the PYCE implementation schedule; and 2. a one-year extension with a revised implementation plan will enable PYCE management to improve burn rates with more effective expenditures in order to ensure delivery on the PYCE performance plans. To assess the effects and measure the impact of Yemen’s volatile security environment on PYCE implementation, the evaluation team also met with and interviewed the management of two comparatively similar projects funded under the USAID Conflict Mitigation Initiatives (CMM) in Yemen namely: Mercy Corps’ EYSY program which is a two-year project for US$1.29 million being implemented in two southern governorates of Yemen. The EYSY project is a conflict-mitigation program that brings youth together and builds their capacity to create positive change by teaching them life skills and job skills, and by helping them engage in community service. The project also promotes participatory decision-making through dialogue between youth and local leaders in order to identify community needs, and supports the establishment of youth groups, and the engagement of youth in community service projects. Partners for Democratic Change (PDC) ‘Yemen Community-Based Conflict Mitigation’ Program (Y￾CCM) is a two-year project budgeted at US$590,000 that was implemented in eight targeted districts of the tribal governorates of Marib, Al Jawf, Shabwa and Al Baida until the end of March 2012. The Y￾CCM goal was to increase in-country capacity to manage conflict over natural resources and social services. Y-CCM built the capacity of local actors, local authorities and community-based organizations to establish sustainable systems and structures for interventions that address the root causes of conflict over natural resources, as well as address disputes between corporations and local communities. The Mercy Corps EYSY project is quite similar to PYCE in that it targets youth, trains them on life skills, supports their engagement with their communities and instigates dialogue between youth and local leaders. The EYSY project is also being implemented in southern governorates and this entails facing comparable security challenges as those faced by PYCE. These challenges have been more intense in the south with the Hirak El Janoubi and the threat of Al Qaeda in the Arabian Peninsula (AQAP) in Abyan. When faced with threats and environmental security constraints, MC undertook contingency measures Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 33 that proved viable alternatives and enabled the local and expatriate staff to maintain and/or resume operations throughout uncertain times.24 MC also approached USAID to authorize a change in the geographic targeting because certain districts became impractical on security grounds. MC’s rationale was that the project would still aim to achieve the same total number of participant-beneficiaries but in two districts, instead of the originally-planned four. According to the EYSY project leadership, USAID was flexible in allowing the change and was timely in their response, by approving this request in only three weeks. Similar to AMIDEAST, in September of 2011 Mercy Corps requested a no-cost time extension of two months compared with AMIDEAST’s request for a one-year extension. Both organizations received the same response from USAID, namely that it was too early in the life of the project to decide on an extension. During the evaluation interview with the EYSY program manager, he explained that MC managed to make up for the time delay that occurred during the popular uprisings of 2011 and is well on schedule to completing the EYSY project by the end of September 2012 as originally planned. PDC’s Y-CCM project was launched in April 2010 and they completed all project activities by the end of the allotted two-year implementation period, March 2012, despite operating in restive and insecure governorates and districts in Yemen. The evaluation interview with the Y-CCM project leadership confirmed that the project is similar to PYCE and EYSY, although operating in different geographic areas. Y-CCM also faced problems resulting from the security break-downs which occurred during the Yemen popular uprising. The Y-CCM Project Manager cited as an example the fact that at certain times there were as many as 15 road blocks managed by different factions on the road from Sana’a to Marib. In spite of this, the Y-CCM project managed to train 393 community leaders out of the project target of 400. Additional changes that occurred during implementation were working in different centers than those included in the original project plan, and a time delay of two to three months in the issuance of official decrees that provided for the creation of 10 commissions of community mediators along with the affiliation of these commissions to the district council administrations. Additionally, the Y-CCM forums brought together leaders and lower-level government officials than originally planned, as the presence of high-level officials during those times might have been politically misinterpreted and could have caused harm rather than bringing positive value to these conflict mitigation gatherings. According to PDC, the major factors that accounted for Y-CCM’s ability to implement in such difficult environments were PDC’s previously-established relationships with the local leaderships of the tribal areas and the credibility gained throughout implementing activities and programs that had a positive impact on development and on peoples’ lives. In summary, the political and security turmoil experienced in Yemen since February 2011 forced certain changes to USAID programs and target areas, but not to the extent of seriously affecting project objectives or activities, even the implementation of conflict mitigation initiatives being carried out in times of actual conflict.                                                              24 YMEP monitoring reports of Mercy Corps EYSY project. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 34 Significantly, it should be noted that delays incurred by the Y-CCM and EYSY projects amounted to around three months. These small delays and minor program adjustments cannot be compared to the PYCE project which has expended only slightly more than 23% of the total budget amount after 18 months of a 24-month project or 75% of the project total time period (to March 2012). The success of other similar project operations during the security crises in Yemen serves to show that the issues affecting PYCE project implementation and burn rate are likely attributable to other internal PYCE issues and go well beyond PYCE’s explanation of delayed implementation due to security risks. 2. A one-year time extension will enable PYCE management to improve burn rates with effective expenditures and will ensure delivery on the program’s performance plans. AMIDEAST has requested a one-year no-cost time extension to deliver on its contractual obligations under the PYCE project agreement. A review of the program’s financial performance during the past implementation period shows the following: Table 8: PYCE Quarterly Financial Expenditures Period Quarterly Expenditures Total Expended Remaining Balance US$ US$ 3,578,594 start Oct – Dec 2010 57,295 57,295 3,521,299 Jan-March 2011 176,994 234,289 3,344,305 April – June 2011 97,830.54 332,119.54 3,246,474.46 July – Sept 2011 153, 058.73 485,178.27 3, 093,415.73 Oct – Dec 2011 140, 481.07 625,659.34 2,952,934.66 Jan – March 2012 216,801.10 842,460.44 2,736,133.56  Up to March 2012, PYCE had expended only US$842, 460 or around 23% of its total program budget of US$2,736,133 with a 77% remaining balance.  Expenditures per quarter have averaged US$140,410 with a peak expenditure period of US$216,801 during January – March 2012.  Assuming that during the remaining two reporting periods, the PYCE project maintains the same high expenditures level of the January- March report (US$ 216,000), it can be assumed that the project would expend a total of US$1,274,460 ($216,000 x 2= $432,000 + $842,460) by the end of its original two-year period, with a remaining balance for the extension year of US$2,304,134.  To expend all the PYCE budgeted project funds25, AMIDEAST would have to plan, execute and disburse project implementation with an average expenditure level of US$576,033 per quarter throughout the requested four quarters of the requested extension. PYCE management claims that such plans exist and that they are ready to fast-track project implementation to meet the project performance requirements within the requested one-year period. Despite numerous and direct requests by the evaluation team to present such plans to USAID and also to                                                              25 As well as delivering on the program’s planned output and performance indicators. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 35 make them available to the YMEP evaluators for consideration during the mid-project evaluation, no plans were produced or provided by AMIDEAST. It is therefore the evaluators’ assessment that even if AMIDEAST is granted a no–cost time extension, their management capacity and ability to implement PYCE and deliver on its performance requirements during the extended period is doubtful. AMIDEAST’s claim that security risks were the major impediments to a timely delivery of the project is not fully credible, as other similar USAID programs were able to operate under the same circumstances in similarly restive and insecure areas of Yemen, albeit with some changes to geographic areas and slightly lesser program performance achievements. The fact that both Y-CCM and EYSY had considerably less funding than PYCE is of minimal significance if proper management of the grant funds was put in place and contingency measures were implemented during times of high risks. The fact that a direct life threat was issued to the AMIDEAST’s American leadership26 should have encouraged the leadership to immediately put in place contingency measures with strong local leadership that would have enabled them to maintain operations in times of security risks either through delegating some management responsibility to senior Yemeni staff or by subcontracting some project activities to capable local Yemeni organizations.                                                              26 A life threat was issued to the American leadership of PYCE which forced this leadership to leave the country for some periods of time. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 36 IV. SUMMARY CONCLUSION & RECOMMENDATIONS In summary, the issues confronting the PYCE project go far beyond its low burn rates or the Yemeni youth uprising and other security issues. The project has changed its focus from a stabilization initiative targeting youth to improve their livelihood to a project promoting sports initiatives and training youth on ‘life skills.’ The life skills and technical / vocational skills are still very much needed by the Yemeni youth of this transitional period. The PYCE project management has clearly selected to implement the project’s planned activities—youth training and sports initiatives—without significant levels of involvement of the religious, government and community leaders with the Yemeni youth project participants. The political and security events in Yemen and ensuing changes in the political leadership do not adequately explain the extent of the changes made to neither the project design, nor the extensive delay in the PYCE implementation schedule and the very low project burn rate only six months before the end of the current contract end date. It is the evaluators’ assessment that the original project design, implementation methodology, stabilization intent and youth targeting are still valid during this transitional period even though USAID is reconsidering its stabilization strategy in light of the country’s “Arab Spring.” The AMIDEAST management, at present staff levels, is inadequate and unable to deliver on the PYCE project’s objectives. The large remaining amount of grant funds, its geographic targeting and the project’s stabilization background pose many more challenges than can be handled by the present team and require additional expertise and professional staff more experienced in project management, conflict mitigation and youth programming. The project planning and monitoring lacks a larger vision and a broader action plan. At present, the PYCE project focuses on activity details and not on strategic impact or outcomes. The evaluation appraisal of PYCE burn rates over the past project reporting periods is not encouraging with regard to consideration of an extension of the project on the present terms. Should USAID favor an extension, it is recommended that AMIDEAST present the following to USAID for review and possible approval: 1- An amended and feasible project work plan with a detailed activity plan and a related cost budget. The work plan must identify the exact activities to be implemented, specifically the centers to be renovated since these comprise a large portion of the remaining project budget. The implementing organization needs to secure the agreement of the centers prior to their inclusion in the work plan. 2- A detailed training plan for the community actors, PN and Yemeni youth that clearly identifies the training topics and explains the expected impact of those topics on the final objective of the project. 3- An amended M&E plan that reflects the revised work plan in terms of output and projected impact, and is clearly linked with the project objectives. The new M&E plan should show specifically how the activities will be measured to show achievement of the project’s objectives and describe reliable data collection methods and an effective monitoring system to be put in place immediately. It is also recommended that USAID, possibly through YMEP, undertake a much closer monitoring of the project both financially and programmatically in the field in order to assess that the project is on target and is achieving its ‘stabilization’ objectives. In addition, USAID will have to request more detailed Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 37 financial reports from AMIDEAST showing the amounts disbursed with the implementation of specific project activities and this financial report should be presented quarterly to USAID, along with the activity progress reports. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 38 PYCE EVALUATION REPORT ANNEXES Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 39 ANNEX A - Interview & Focus Group List # Name Organization Position Date & Location 1. Dr. Charles Swagman USAID Director – Technical Programs 5/29/2012 Sana’a 2. Afrah A. Al-Zouba USAID Democracy & Governance Specialist 5/29/2012 Sana’a PYCE Program Staff 1. Sabrina Faber AMIDEAST PYCE - Chief of Party 5/30/2012 Sana’a 2. Cassandra Filer AMIDEAST PYCE - Youth Sports & Recreation Coordinator 5/30/2012 Sana’a 3. Maher Faisal Saeed Al Medhagi AMIDEAST PYCE - Youth Community Development Coordinator 5/30/2012 Sana’a 4. Entisar Al Rudaini AMIDEAST Project Accountant 5/30/2012 Sana’a 5. Waddah Abdulla Khader AMIDEAST Programs Officer 6/3/2012 Aden 6. Omar Al Kaff AMIDEAST Admin& Finance Officer 6/3/2012 Aden Other Organizations 1. Raphael Velasquez Garcia Mercy Corps Program manager - EYSY 6/4/2012 Aden 2. Fahd AbdulMomein A. Saif Partners – Yemen Program Manager Y-CCM 6/10/2012 Sana’a ROYG Officials 1. Jamal Mohamed El Yemeni MoYS General Office Manager-Aden 6/4/2012 Aden 2. Foad Ahmed Al Burahee MoEG General Office Manager-Aden 6/ 5/2012 Aden 3. Jamil El Hitar MoEG Secretary-Prior Minister- Sana’a 6/15/2012 Sana’a 4. Fawziah Ahmed MoYS Secretary – Minister’s Office-Sana’a 6/17/2012 Sana’a 5. Ali Hashwan MoYS Office Manager-Marib 6/11/2012 Marib 6. Naji Bin Ali Alzaide ROYG Governor – was not able to secure an appointment at short notice Marib Focus Group 1 Kalua Steering Committee 6/3/2012 1. Yasmin Molhy Ali Member Principle- Abdulla Mohariz High School (Females) Aden 2. Mona Mohamed Nasser Member Physical Education Teacher - Azal School Aden Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 40 3. Najmee Abdulmajeed Member Author & Journalist in 14 October Foundation Aden 4. Mohamed Hamed Ba Hamish Member Manager of (Ebda’a) Center Aden 5. Osama Mansoor Mokred Member Imam Of the Main mosque in Kaluaa Aden 6. Omar Ali Hasan Member Soccer Coach - Al Rawda Club Aden Focus Group 2 Peer Network Aden 6/4/2012 1. Ayman Mohamed Abdallah Aldoosh 2. Ahmed Mohamed Salem Kasim 3. Mohamed Saeed Abdo Abdullah 4. Mohamed Ali Mohamed Hadee 5. Bakeel Abdulrhman Najee 6. Haneen Labeeb Khaid 7. Mohamed Nizar Mohamed Abdulkhader 8. Ebrahim Saeed Najee Abdo 9. Mohamed Saleh Husen Naser 10. Roshdi Abdulkawi Alsaeed Mohsen 11. Ahmed Yassen Ali 12. Ebtihal Salah Ahmed Mokbel 13. Nareman Nasser Mohamed Saleh Ahmed 14. Mea'ad Mohamed Abdo Omar 15. Hind Mohamed Saleh Abdullah Hules Focus Group 3 Basket Ball Coaching-Mini Grant-PACA Aden 6/5/2012 1. Zahra Nader Ahmed Mohamed 2. Dalia Mohamed Abdallah Seif 3. Rami Ahmed Seif Ali 4. Mohamed Nabil Hosein Abdel Hak Abed Allah 5. Amajed Jamil Kasem 6. Saleh Ali Ahmed Ali 7. Mohamed Said Mokbel Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 41 8. Housam Naser Mohamed AlBokey 9. Yassen Abed AlChader Rajeh Ali AlChabhi 10. Abdallah Mohamed Abdallah El Jaefari 11. Rami Riyad Abdo Fadea’ 12. Mohamed Abed AlRahmen Abdo Ahmed 13. Bassam Ebrahim Homeidan 14. Rawouya Mohamed Abdallah Seif 15. Mounyah Mokhtar Abbas Salem 16. Yasmine Saleh Thabet 17. Ahmed Khaled Seif AlShaybani 18. Zahra Abader Mohsen Focus Group 4 Steering Committee Sana’a 6/10/2012 1. Muna Mohammed Hamoud Al-Dilami 2. Hayat Othman Abdullah El Kabboudi 3. Muna Mohammed Ali Al-Marwalah 4. Muna Ali Ali Ahmed Al-Sofi 5. Gamal Musa'd Ahmed Al-Jabri 6. Abdulhafed Hizam Mohammed Saeed Al-Yusufi 7. Mohammed Hassan Saqeer Mohammed 8. Ra'ed Abdo Ahmed Gelan Focus Group 5 Peer Network Sana’a 6/11/2012 1. Khouloud Ahmed Houssein Koubas 2. Ahmed Ateeq Mohammed El-Haimi 3. Ayman Ahmed Mohammed Ismail 4. Hamid Abduljabar Abdulwasae Abdullah 5. Ismaeel Mohammed Al-Mutarreb 6. Muhammed Ebrahim Yahya Al-Razeqi 7. Murad Muhsen Saleh El-Huthifi 8. Nabeel Abdullah Ali Merfeq Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 42 Focus Group 6 Basketball Coaching-PACA –Mini Grants Sana’a 6/12/2012 1. Laith Lutfi Ali 2. Saleh Mohsin Saleh Al-Hodify 3. Wesam Sharaf Mohammed Al-Sarori 4. Naseem Adbulkarim Ahmed Yahya Al-Shami 5. Salwa Mohammed Ali Al-Fakih 6. Bilal Ameen Othman Al-Salehi 7. Abdulkarim Yahya Ibrahim 8. Saddam Abdulmalek Hamod Al-Radaee 9. Ali Abdallah Saleh Al-Qubaisi Focus Group 7 Marib Steering Committee 6/11/2012 1. Nabila Ali Mohammed Alhumati Member Director of Women's Sports Activities – Marib. Manager of Murshidat office Marib 2. Sharifa Mohammed Yahya Alshami Member Mothers Council at Alsaleh School Marib Focus Group 8 Marib Basketball Coaching 6/11/2012 1. Mohammed Mogahed Qaid Nmran 2 Musali Saif Ali Bahaibeh 3. Miada Saeed Abdorabh Alkadi 4. Soror Ahmed Mohammed Alkadi Site Visits Organization Date & Location Abdallah Hadi Mohammed Aden Medical College – Admin Director June 6, 2012 - Aden Salah Jaladi El Rawda Club- Club President June 6, 2012 – Aden Qaid Namran Nasim Club Manager June 11, 2012 - Marib – El Juba Samir Mohamed 22 May Club Sana’a Deputy Sana’a – was not available at the interview date Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 43 ANNEX B - Desk Review Documentation Title Date AMIDEAST Cooperative Agreement September 30, 2010 PYCE Quarterly Progress Report October 1- December 30, 2010 January 31, 2011 PYCE Quarterly Progress Report January 1- March 31, 2011 April 25, 2011 PYCE Quarterly Progress Report April 1-June 30, 2011 July 31, 2011 PYCE Quarterly Progress Report July 1- September 30, 2011 October 31, 2011 PYCE Quarterly Progress Report October 1 – December 31, 2011 January 28, 2012 PYCE Quarterly Progress Report January 1- March 31, 2012 April 17, 2012 Performance Monitoring & Evaluation Plan – Results Framework --- Stabilization & Performance Monitoring Plan October 2010 – September 2012 February 21, 2011 PYCE Year 1 Annual Work Plan October 1, 2010 – September 30, 2011 December 4, 2010 PYCE Cumulative Indicators QRs June 2, 2012 Strategy and Appraisal Current Draft June 2011 PYCE Aden Activities October 2010 – May 2012 May 19, 2012 PYCE Projects in Aden May 19, 2012 Training Interventions in Aden May 9, 2012 PYCE Sana’a Activities October 2010 - May 2012 May 21, 2012 Training Interventions – Sana’a May 21, 2012 PYCE Marib Activities October 2010 – May 2012 May 21, 2012 Training Interventions – Marib May 21, 2012 PYCE Projects-Aden-Marib-Sana’a May 23, 2012 Federal Financial Report October – December 2010 12/31/2010 Federal Financial Report January – March 2011 3/31/2011 Federal Financial Report April – June 2011 6/30 2011 Federal Financial Report July – September 2011 9/30/2011 Federal Financial Report October – December 2011 12/31/2011 Federal Financial Report January – March 2012 3/31/2012 Draft Burn Rate Form PYCE AMIDEAST 12/14/2011 Draft Burn Rate Form PYCE AMIDEAST 3/13/2012 AMIDEAST PYCE Organizational Chart (revised) 5/12/2012 PYCE Peer Network Training and TOT Curriculum 6/25/2012 2010 – 2012 Yemen Country Strategy -- USAID Assistance to Yemen – Performance Management Plan FY2011 - FY2013 February 2011 Yemen Cross-Sectoral Youth Assessment: Final Report (EDC – Equip3) November 2008 Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 44 Title Date Assessing Youth & Gender Programming in Yemen – Final Report (IBTCI￾YMEP) September 19, 2011 Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 45 ANENX C - SURVEY - Translated Copy Midterm Evaluation - PYCE PERSONAL INFORMATION Name: Gender: Male Female Age: 14-17 Yrs 18-21 Yrs 22 -29 Yrs 30Yrs & Above Residence Address Governorate: District: Region: Area: EDUCATION  Illiterate  Education technical / professional  Informal Education  Diploma two years  Primary Education  Bachelor degree  Secondary Education  Post-graduate degree 1- How and through whom did you hear about the PYCE program? _______________________________ 2- Participation with PYCE: Please list the project’s activities that you were involved with Activity Topic _________________________ ______________________________________ _________________________ ______________________________________ ________________________ ______________________________________ 3- In your opinion, are the PYCE training activities – topics appropriate to your personal and community’s needs?  Yes  No If no, please specify why ________________________________________________________________ 4- Did you participate with AMIDEAST in other programs – Training activities than PYCE?  Yes  No If yes, please list trainings/activities________________________________________________________ Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 46 5- Program impact: Please identify the impact of your participation with PYCE project in the following areas (check the appropriate option – you can select more than one option and add under others) Program Impact/Outcome on Self – Development Increased self-confidence X A broader vision X A more positive attitude X I acquired new Information X specify I acquired new skills X specify A change in behavior X identify Self Development X identify Others, X specify Program Impact/Outcome on Relations with Others Improved understanding of others X Readiness to cooperate with others X Belief in group work X Positive relations with others X Others, X specify Program Impact/Outcome on Relations with Community & Community Leaders Acquaintance with community leaders X Constructive cooperation with some leaders X Constructive interaction with community X Improved appreciation of community X Increased motivation for advocacy & engagement with community X Others, X specify 6- What is the estimated amount of time that you spend during a week with the project activities? ______ _____________________________________________________________________________________ 7- How would you be spending your leisure time if not for the project activities? ____________________ _____________________________________________________________________________________ 8- Please provide suggestions to improve the project’s appropriateness to your needs during this transitional period _____________________________________________________________________ _____________________________________________________________________________________ Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 47 ANNEX D – PYCE Cumulative F Indicators List Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Oct ‐ Dec 2012 Jan ‐ Mar 2011 Apr ‐ Jun 2011 Jul ‐ Sep 2011 Oct ‐ Dec 2011 Jan ‐ Mar 2011 Apr ‐ Jun 2011 Task 1.1. Number of community actors engaged in governorate‐ level meetings in selected governorates 160 48 208 250 Task 1.2. Number of Governorate Steering Committees established in selected governorates 1 al‐Kalua 1 Maala 1 Sana'a; 1 Marib 1 Marib 1 Marib 35 Q3: Maala SC formation not completed; Q5 and Q6 reporting repeated formation of Marib SC from Q4 Task 1.3. Number of Peer Network members trained to conduct PACA 39 5 5 19 45 113 150 Task 1.4. Number of sustainability meetings held at selected youth centers for renovation 1 1 1 3 28 Task 1.5. Peer Network Training curriculum developed 1 11 Task 1.6. Number of com munity actors received TOT for Peer Network 5 7 12 125 Task 1.7. Number of participants trained by the PYCE Peer Network (TOT graduates) in selected governorates 3 176 179 2,700 Task 1.8. Number of grants awarded in selected governorates 1 1 1 10 13 50 Task 2.1. Identify current and coming initiatives that develop individual sports, recreation and community service among youth ‎in targeted governorates 6 17 2 1 6 32 10 Task 2.2. Strengthen current initiatives to support PYCE Peer Network and an array of individual sports and recreational and other community service activities 2 1 1 3 2 9 10 Task 2.3. Number of sites renovated in selected governorates 1 22 7 Q3: al‐Rawda Youth Club (should be reported under 2.6) Task 2.4. Number of coaches and referees trained in selected governorates 65 65 30 Task 2.5. Number of teams/youth groups formed in selected governorates 98 17 15 teams / 150 players Task 2.6. Number of youth clubs renovated by equipment and grants provided 1 12 4 7 grants, 7 sites 4 grants, 2 sites (al‐Rawda and 22 May) Task 2.7. Number of communication channels engaged with the PYCE Peer Network to disseminate information, success stories, announcements, and press releases 2 1 5 2 4 611 20 coaches / 20 PN members * Had been reporting channels (facebook, blog, etc) rather than PN and coaches (in red) Task 2.8. Number of leagues established and launched in selected governorates 11 2 30 competitions TOTAL to Date Target Notes Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 48 ANNEX E: AMIDEAST PYCE Organizational Chart Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 49 ANNEX F: Training Plan PYCE Peer Network Training and TOT Curriculum: I. PACA I and II, Volunteerism Spirit (Civic Engagement), Community Presentation Skills, Positive Messaging/Visioning These workshops are given to all youth as training for the peer network. Upon completion of these, youth perform PACA in their community, and then give a 5 – 10 minute presentation detailing what they learned about their communities through PACA to PYCE steering committee members. SC members then make the final decision about PN membership. II. Additional PN member training is determined according to PN interests and trainer availability and PACA. The topics are also those that involve participation and actively doing, creating, or performing something, and thus serve as models for future potential trainings by PN trainers, as well as additional skills that those who may continue on to become PYCE peer trainers can share as trainers. Topics have included the following: a. First Aid (PN expressed interest in First Aid as a way to serve their communities) b. Calligraphy (PN expressed interest, requested a follow-up course, and PN trainers have offered calligraphy workshops to neighborhood youth). c. Community Mapping Through Photography (PN expressed interest; offers an alternative method of mind mapping positive activities and youth spaces in community, and mode of youth expression) d. Chess (PN expressed interest; very popular for males and females alike) III. TOT The original TOT outline is as follows: Unit 1: Introduction and Expectations Unit 2: Communication and Learning Session 1: Communication Puzzle Session 2: Effective Learning Session 3: Learning Styles Session 4: Facilitation and Feedback Session 5: Questions as Learning Tools Session 6: Eight-Minute Trainings Unit 3: Participatory Training Methods Session 1: Role Plays Session 2: Brainstorming Techniques Session 3: Case Studies Session 4: Mind-Mapping Session 5: Lecture Techniques (ways to incorporate participants; alternatives to standard lecture format) Session 6: Discussion Techniques Games and Experiential Learning Exercises Unit 4: Objectives and Assessments Session 1: Setting Objectives Session 2: Types of Assessment Session 3: Self-Assessment Session 4: Group Assessment Session 5: Trainer / Trainee Assessment Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 50 Unit 5: Practicum Session 1: Community Needs Assessment Session 2: Planning a Workshop Session 3: Portfolio PYCE opened a bid for designing the TOT curriculum last spring. However, upon a lack of interest by Yemeni organizations, PYCE staff adapted open-source materials from PathFinder International, UNESCAP, and the US Peace Corps. Feedback: After the first pilot of the TOT, trainers and participants alike found critical thinking aspects of the TOT challenging. Further they found the following theoretical units difficult to follow: Unit 2/ Session 5: Questions as Learning Tools; Unit 3/Session 3: Case Studies; and Unit 4/Session 3: Self-Assessment. They requested more examples for each unit. Thus, edits were made to the curriculum to clarify those sections, and trainers have responded that it was a lot clearer in the second TOT. In fact feedback collected noted that these sections were some of the best. PYCE Observations: Trainees also found difficulties recruiting practicum participants and identifying relevant topics. Trainees liked communication and facilitation skills and it is evident that these areas are essential to engagement with youth. *** Lessons Learned *** After using the materials listed above, PYCE has learned a number of lessons and thus has adapted the above sessions in the following manner: I. PACA I and II, Volunteerism Spirit (Civic Engagement), Community Presentation Skills, Positive Messaging are now all combined into the Community Youth Engagement workshop. II. The Community Youth Engagement Workshop has been combined with fundamental elements of the TOT curriculum for PN groups specified by the SC. This one-week intensive program shortens the TOT to focus on Units 2 and 3, Communication and Learning and Participatory Methodology. PYCE shortened the TOT upon receiving feedback that it took too long. PYCE also observed that youth are more likely to adapt previous materials found online or from friends or from other trainings they have attended than they are to create trainings from scratch. PYCE decided to concentrate on communications and classroom engagement techniques to make the TOT available to more participants. PYCE applies a time cost benefit analysis to maximize the benefits resulting from how much time beneficiaries can invest. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 51 ANNEX G: USAID Yemen, PYCE Mid-term Evaluation Final Scope of Work April 18, 2012 I. Evaluation Use and Purpose The Promoting Youth Civic Engagement (PYCE) Project is a two-year initiative, launched in January 2011, implemented by AMIDEAST. It is designed to increase opportunities for Yemeni youth by improving their skills and promoting the engagement of dynamic, responsive, and credible local leaders in target communities in Sana’a, Aden, Marib, and two other governorates. PYCE project approach and design was based on improving the livelihoods of Yemeni youth in vulnerable areas by supporting USAID strategy to engage dynamic, responsive and credible religious leaders in the governorates of Sana’a, Aden, Abyan, Amran and Marib. To achieve this, AMIDEAST PYCE project is coordinating, through steering committees populated primarily with religious leaders, to develop a PYCE Peer Network of youth, religious and community actors. This network will reach out to peers and communities conduct training and encourage sports and recreation events in youth centers. Additionally, AMIDEAST will capitalize on its extensive collaborative network of private sector partners in the US and across the region to leverage USAID funding of capital improvements, training resources and community outreach. PYCE’s two principal objectives are: (1) to strengthen the role of community actors in positively influencing Yemeni youth, and (2) to support the establishment and expansion of youth sports and recreational opportunities. PYCE operates through steering committees involving educators, religious leaders and other local actors. This network of influential leaders reaches out to young people and their communities by offering training and encouraging sports and recreation events in community youth centers. The original implementation period of this project is September 2010 to September 2012. For different reasons, the project implementation is late and behind the work plan. The implementing partner (AMIDEAST) is requesting a no cost extension for the project for about 12 months beyond the current closing date. To help USAID making a decision about the request for extending the project timeframe, input from a midterm evaluation is needed. Since early 2011, the PYCE project has had to face new realities that are different from when it was developed. These realities include the changing political situation due to 2011 Arab Spring revolution, the emergence of extreme movements in areas like Aden and Abyan, and the escalating activities of Hirak groups in south Yemen. For the PYCE project to be extended, it needs to be analyzed from the technical side as to appropriateness of its original plan and implementation mechanism to determine whether it is still able to meet the needs of youth and target communities. AMIDEAST has been requested to develop a new plan to be submitted with their request for no cost extension. It is not clear whether AMIDEAST’s work in the last year has been adequately assessed. Thus, this midterm evaluation will be helpful for USAID and AMDEAST to know better the project’s successes and lessons learned. This will input into the decision concerning an extension. Political instability and operational security challenges have resulted in delays in activity implementation which limited mobility of staff to in target governorates. USAID in 2011 agreed with AMIDEAST to limit its geographical area under PYCE to 2-3 governorates: Sana’a, Aden and possibly Marib. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 52 The burn rate of PYCE in the first year was behind the planned levels and USAID needs to ensure that the extension of the project and with revising the implementation plan will ensure an improved burn rate with effective expenditures. The purpose of this mid-term performance evaluation is to assess the implementation, effectiveness, and suitability of the project’s approach to achieving project objectives. The findings of this evaluation will inform USAID whether the project is achieving its objectives and targets. Is the implementation methodology appropriate in achieving USAID’s strategy of positively impacting stabilization by targeting youth groups. II. Evaluation Questions This mid-term performance evaluation will measure and analyze the accomplishments toward achieving PYCE goals. Given Mission concern with the PYCE burn rate, the evaluation will examine and analyze the internal and external challenges that have impacted on the program and finances. The evaluation will highlight program aspects that have proved successful and document why other aspects have not been successful. The primary evaluation questions are:  The PYCE implementation approach includes strengthening community actors’ i.e community leaders, local authorities, religious leaders to positively influence Yemeni youth, and support the establishment and expansion of youth sports and recreational opportunities through steering committees involving educators, religious leaders and other local actors. Is this an effective strategy for achieving program results? What progress has been achieved in each programmatic area to date?  Has the project continued to be efficient and appropriate to youth needs and their expected role in the transitional period?  To what extent has the project contributed to accomplishing mission’s stabilization strategy objectives? Is there a need to realign activities and/or budget?  Has the program achieved all of the expected results by this program midpoint of March 2012? If not, why not.  Is it inclusive and building consensus within target communities?  Is it cost effective? III. Data collection and analysis methods Evaluators are encouraged to use the following data collection and analysis methods: a review of data collected by the program; interviews with participants including beneficiaries, community leaders and the local authorities; interviews with major stakeholder; and interviews, surveys and focus groups from a broad sample of PYCE beneficiaries. A. Review of Data and M&E Systems Evaluators will put together an accurate historical narrative of the award from signing through to March 31, 2012. This includes challenges faced, stoppages of work, how problems were overcome, and what other steps were taken to correct or change the work flow. In sum, the evaluators should analyze the program design and approach vis-à-vis each objective to determine their effectiveness in completing outcomes and outputs to date compared to the work plan and the PMP, determine whether the PMP and work plan are effectively linked and whether the data they include is accurate and sufficient to establish causal links to the IRs. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 53 Taking into account quality and consistency, evaluators will determine whether program reporting has met USAID standards and whether sufficient baseline data was collected. B. Key Informant Interviews Evaluators will conduct key informant interviews to examine project impact and efficiency in achieving results. The key informants could be community leaders, parents and local authorities. The interviews will try to find out if the project’s approach was successful in supporting Yemen stabilization in targeted communities. C. Beneficiary Interviews, Surveys and/or Focus Groups Interviews, surveys and focus groups with project beneficiaries will explore how youth from different political backgrounds and gender have assisted in reaching program objectives. How aware are youth and sports offices of PYCE’s activities, how is the project perceived and valued by the stakeholders such as ROYG officials, beneficiaries, civil society and local communities. What recommendations does the Team present to improve performance and impact on local community stabilization, on consensus building and supporting youth. IV. The Evaluation Team 1. Team Leader: One senior-level evaluation expert with extensive experience designing and conducting evaluations in fragile states plus experience evaluating USAID youth programs. The senior level evaluation methodologist will be team leader and be responsible for managing the process of document review, field work, interviews, analysis, the draft and final evaluation reports, and all debriefs. 2. A local YMEP research assistants/evaluator. VI. Evaluation Timeline and Logistics Total of approximately 27 days (based on 6 day work week) – 3 days for preparation, 12 days in field (5 Sana’a, 5 Aden, 2 Marib), 4 travel international days, 5 days for first draft report writing, 3 days for revisions and final report preparation. USAID will supply comments on the draft evaluation within one week. The Contractor is responsible for providing required logistical support to undertake the evaluation. Prior to the launch of the evaluation, the Contractor will specify its main point of contact for the evaluation. Final Evaluation Tasks 1. Desk Review a. Documents USAID will provide for desk review include: i. Cooperative Agreement and modifications ii. PYCE PMP iii. Yemen Mission PMP iv. PYCE work plans v. PYCE Quarterly Program reports vi. USAID Yemen country strategy vii. Other relevant PYCE documents (success stories, articles, M&E procedures and protocols etc.) 2. Develop an appropriate methodology for the evaluation including data collection tools. Mid-Term Evaluation of the Promoting Youth for Civic Engagement Project (PYCE) 54 3. Prepare a field and HQ work plan, including interview plan (both current and former PYCE and USAID staff responsible for PYCE). 4. Field work with data gathering and analysis 5. Write a draft evaluation report with findings, lessons learned, conclusions, and recommendations VII. Evaluation Deliverables The contractor shall provide the following deliverables: 1. Brief outline of methodological approach for evaluation before departure for Yemen and a detailed evaluation budget. 2. In-brief with USAID upon arrival to discuss methodological approach and plan action for the field. 3. Weekly presentations to USAID on progress and problems. 4. Debrief USAID Yemen four working days prior to departure to allow for Mission feedback and additional field work, if needed 5. Draft of the evaluation report will be submitted to YMEP’s COP and IBTCI’s HQ two working days prior to departure from Yemen 6. USAID comments on the draft evaluation report within a week of USAID’s receipt of the draft 7. Final evaluation report deliverable no later than three days after receipt of USAID’s comments on the draft. The report must include recommendations specifically on what changes in the project ought to be considered to help youth become a positive means of change in their communities.