June 18, 2009 This publication was produced for review by the United States Agency for International Development. It was prepared by Harvey Herr, Mary F. Hayden, Ph.D., Rich Mason, and Alexandra Lenton. COMMUNITY STABILIZATION PROGRAM (CSP): AN EXAMINATION OF THE YOUTH ENGAGEMENT PROGRAM FINAL REPORT COMMUNITY STABILIZATION PROGRAM (CSP): AN EXAMINATION OF THE YOUTH ENGAGEMENT PROGRAM FINAL REPORT International Business & Technical Consultants, Inc. 8614 Westwood Center Drive Suite 400 Vienna, VA 22182 Contracted under 267-C-00-05-00508-00 Monitoring and Evaluation Performance Program, Phase II (MEPP II) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) i Report on Community Stabilization Program: An Examination of the Youth Engagement Program TABLE OF CONTENTS LIST OF TABLES............................................................................................................... I LIST OF QUOTES FROM KEY INFORMANT INTERVIEWS AND FOCUS GROUPS..... I LIST OF ACRONYMS...................................................................................................... III EXECUTIVE SUMMARY .................................................................................................IV INTRODUCTION.............................................................................................................................. 1 THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE....................................................... 1 PURPOSE OF THE EVALUATION .................................................................................................. 6 RESEARCH DESIGN AND EVALUATION METHODOLOGY......................................................... 7 STUDY FINDINGS........................................................................................................................... 9 RECOMMENDATIONS.................................................................................................................. 19 CONCLUSIONS............................................................................................................................. 20 ANNEX I: KEY INFORMANT INTERVIEW FORMS ....................................................... 23 ANNEX II. IRD AND USAID RESPONSES TO REPORT RECOMMENDATIONS ....... 37 LIST OF TABLES Table 1: CSP Rollout: Project Start Date by Type of CSP Component and Percent of Projects Rolled Out ......................................................................................................................4 Table 2: Cost Effectiveness of CSP Program Elements ............................................................13 Table 3: Average Daily Insurgent Attacks by Province and by Quarter .....................................18 LIST OF QUOTES FROM KEY INFORMANT INTERVIEWS AND FOCUS GROUPS Box A: Incentives for Youth to Participate in Violence ...............................................................10 Box B: Reduction in Violence.....................................................................................................10 Box C: Youth Engagement Reduced Violence ..........................................................................11 Box D: Provision of Conflict Mitigation .......................................................................................15 Box E: Lessons Learned: Implementing “Attitudinal Change” Activities in Violent Conflict Settings.......................................................................................................................................16 Box F: Sustainability of the Youth Engagement Program...........................................................17 Box G: Level of Youth and Community Participation .................................................................21 International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) ii Report on Community Stabilization Program: An Examination of the Youth Engagement Program International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) iii Report on Community Stabilization Program: An Examination of the Youth Engagement Program List of Acronyms AoR Area of Responsibility (of IPs) AOTR Agreement Officer Technical Representative BDP Business Development Program CDA Collaborative for Development Action CERP Commander’s Emergency Relief Program CIES Community Infrastructure and Essential Services COIN Counterinsurgency COP Chief of Party CSO Community Service Organizations COTR Contracting Officer’s Technical Representative CSP Community Stabilization Program ePRT Embedded Provincial Reconstruction Team FSO Focused Stabilization Office FY Fiscal Year GOI Government of Iraq IBTCI International Business & Technical Consultants, Inc. IDP Internally Displaced Person IR Intermediate Result LG Local Government LGP Local Governance Program M&E Monitoring and Evaluation MEPP II Monitoring and Evaluation Performance Program, Phase II MOU Memorandum of Understanding MOYS Ministry of Youth and Sports NGO Non-Governmental Organization OECD Organization for Economic Co-Operation and Development PMP Performance Management Plan POD Program Office Director POO Program Office Officer PRT Provincial Reconstruction Team Q Quarter RF Results Framework SO Strategic Objective SOW Scope of Work SPSS SPSS predictive analytics software www.spss.com TSS Transition Strategy Statement UNHCR United Nations High Commission for Refugees UNDP United Nations Development Programme US United States USAID United States Agency for International Development USG United States Government USM United States Military International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) iv Report on Community Stabilization Program: An Examination of the Youth Engagement Program EXECUTIVE SUMMARY In response to a Scope of Work (SOW) issued by United States Agency for International Development/Iraq’s Focused Stabilization Office (FSO), International Business and Technical Consultants, Inc. (IBTCI), the implementer of the Monitoring and Evaluation Performance Program, Phase II (MEPP II) studied the fourth component of the Community Stabilization Program (CSP), youth engagement. The purpose of the Youth Engagement Program was to “engage youth on life skills and conflict mitigation through sports, cultural events and public service campaigns.” Specifically, the Youth Engagement Program was to enable youth to 1) connect to their identity, culture, and community, 2) engage as community leaders on issues important to them, and 3) come together with other youth from different ethnic and religious backgrounds to learn coexistence and tolerance. The study was designed to consider the extent to which CSP achieved its intended results and helped to facilitate reconciliation, conflict mitigation, and change hearts and minds in Iraq society. Additionally, the study was to measure the attitudes of youth participating in the Youth Engagement Program to determine if there was any change in attitude towards the Government of Iraq (GoI), their community, and/or individuals of different tribes and religious beliefs. The study questions included the following: 1. How effective was the Youth Engagement Program in reducing incentives for participation in violent conflict? 2. How cost-effective was the Youth Engagement Program been in achieving results? 3. Did the Youth Engagement Program make use of prevailing methodologies and practices in the areas of youth and conflict mitigation? 4. While acknowledging that CSP was not a sustainable development program, how successful was the Youth Engagement Program been in transitioning youth to the GoI, the communities, civic groups, Community Service Organizations (CSOs), Non-Government Organizations (NGOs), and/or other local counterparts? 5. Where CSP was active, was there any apparent correlation between site￾by-site historical reported incidents of violence and CSP youth activities? 6. What were the key lessons learned by CSP’s Youth Engagement Program as a counterinsurgency (COIN) activity/strategy? The lack of attitude and behavior baseline and related program data limited the methodology options available to the study. Therefore, efforts to determine the impact of the Youth Engagement Program relied on: 1) a desk review of the literature related to youth engagement was conducted by a Youth Mitigation Expert, 2) a desk review of CSP documents and reports were conducted by IBTCI staff, 3) key informant interviews (KIIs) and focus groups (FGs) with Iraqi and ex-patriot stakeholders by IBTCI staff and an Iraqi research firm, and 4) secondary data analyses conducted by IBTCI staff. The study was conducted from 1 March 2009 to 15 May 2009. Desk Review of the Literature The literature review on youth conflict mitigation and violence reduction found there was no body of “best practices.” Engaging youth in conflict mitigation, violence reduction International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) v Report on Community Stabilization Program: An Examination of the Youth Engagement Program and most recently, COIN programming was gaining attention among the international aid community. Interest in “the youth factor” has risen because youth are seen both as potential threats to stability and as potential constituencies for peace. However, it is still an emerging field, and the evidence base for the efficacy of such programming was slim. The desk review found few methodologies that were used to successfully assess attitudinal change. The review found the following: • There is a relative paucity of knowledge on peace education and how effective a practice it was. Although there is proliferation of peace education activities, there is little empirical evidence to demonstrate the efficacy of this type of programming. • Research on the sports and youth development literature revealed that the field is still in its infancy, and there is still a lack of reliable valid measures of positive behaviors. • Many violence reduction or violence programs aim to have an impact on levels of violence through positive youth development. Therefore, it may be useful to identify methodologies for assessing positive youth development. • It is possible in developing indicators or lines of inquiry around attitude change, or resulting behavior change by borrowing from other models that might be applicable such as those developed for evaluating “social capital/social cohesion.” The following summary lists the programmatic lessons learned on youth and conflict mitigation programming: • Programs should be conflict sensitive. Programs working in conflict situations (and especially working on the conflict) should be based on a conflict analysis to understand conflict drivers and triggers, and how program interventions may impact the conflict dynamic, in order to avoid exacerbating tension and to achieve program objectives. • Programs should try to have explicit “theories of change” that are relevant to youth and their conflict context, be realistic in what they attempt to achieve and where possible, be based on evidence or growing evidence. • Programs should be designed in a way that ensures that their outcomes are relevant to their objectives. • Attention needs to be paid to ensure that the selection of beneficiaries matches the program rationale. • Programs should be meaningful for youth. • Programs should be holistic. Holistic programming that is strategic, integrated and cuts across sectors can better address the multiple challenges affecting youth. The desk review revealed the following lessons learned from programs in other countries: International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) vi Report on Community Stabilization Program: An Examination of the Youth Engagement Program • Youth face tremendous challenges in conflict settings and it is questionable whether programs based on changing attitudes can achieve their conflict mitigation or prevention objectives. • Sports and recreation activities are overwhelmingly popular but there is little evidence in the literature to suggest they achieve conflict mitigation goals. • The violence reduction and prevention field suffer from difficulties in demonstrating actual impacts on levels of violence. When conducting an evaluation that includes gathering information from youth participants, the following points should be taken into account with regard to monitoring and evaluation practices: • Questions need not change because they were being directed at youth. • Change in youth behavior and attitudes may only take place slowly. This needs to be taken into consideration when evaluating program impacts in these areas. Tracking change over time is advisable. • Youth were, by definition, in a state of transition, and were constantly changing. • Youth expectations can differ from program objectives. • Adults connected with the program can provide useful information on attitude change. If they have a long-term relationship with youth participants, they can see differences or changes, if they occur. • Attitude change evaluations in insecure conflict environments can reveal different results. In many contexts, what mattered were the collective opinion, and quite often, the leader’s opinion. These factors, among others, can affect results. KIIs, Focus Groups, and Secondary Analyses Findings included the following: • The CSP Youth Engagement Program was primarily focused on engaging vulnerable youth in organized sports, life skills, and cultural events, rather than deliberate efforts to mitigate conflict or bring about reconciliation. However, evidence showed that these activities may have had an indirect contribution to reconciliation which was concurrent with a reduction in violence in some areas. • The youth engagement was an effective component of the COIN strategy, providing popular and acceptable means for occupying youth in community￾supported normalizing activities. • The cost per engaged person was much less for the Youth Engagement Program than for the other CSP components such as Community Infrastructure and Essential Services (CIES), Business Development Program (BDP), and vocational education. • By a wide margin, the Youth Engagement Program touched more in the community than the other components within CSP. • Youth engagement was not uniform across the programs. Youth engagement can be a fleeting encounter perhaps lasting no longer than International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) vii Report on Community Stabilization Program: An Examination of the Youth Engagement Program one soccer match, whereas vocational education was a two to three month commitment, and a BDP grant engaged beneficiaries for more than a year. • Comparisons based on estimated cost per day of engagement ranks vocational education as the most cost effective with the Youth Engagement Program as last. • CSP’s Youth Engagement Program was clearly one of many factors that correlated to reductions in violence. • CSP’s Youth Engagement Program used prevailing methods and practices in the field of conflict resolution and violence reduction programs. • Local councils and the Ministry of Youth and Sports were mobilized effectively to implement youth programs but the removal of CSP funding led to a reduction in youth activities and engagement. • Sustainability through government budget mechanisms was a slow process and will need the advocacy of all stakeholders. There was no evidence from the focus groups or KIIs showing that youth groups were advocating/lobbying the government or private sector for funds. Lessons Learned • As reported by Baqubah CSP staff, youth activities can be the initial step toward building trust and, as a result, gain entry into a community. • Members from all of the communities indicated that an indirect outcome of youth activities was the engagement of adults. • The ability to respond to and adjust the program to current issues in the community is important. For example, Baqubah experienced a sudden increase of female suicide bombers. CSP staff immediately created activities that targeted female youth. The number of suicide bombs by female youth decreased from 19 in 2007 to 2 to 3 in 2008. • Several community leaders indicated that implementing programs in local schools improve the schools, provide additional opportunities to children who are currently attending school, provide a means for dropouts to re-enter the mainstream, and increase the number of re-enrollment among dropouts. • It is important to engage even younger children. • The quality of equipment and materials supplied and used by the program was dependent on the quality of the quality assurance system that was in place. • Media is a positive and efficient means to get messages to the general public. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) viii Report on Community Stabilization Program: An Examination of the Youth Engagement Program Recommendations Implementer 1) Develop indicators that show a clear link between program interventions and resulting changes in youth behavior and attitudes. 2) Conduct research that examines the relationship between the change in behavior and attitudes among youth with those of adults in their communities. 3) Employ a more comprehensive and systematic M&E of activities that continuously tracks participants over time and includes observation and interviews with spectators, audiences and parents. 4) Continue to take advantage of national events to enhance the impact of programs, e.g., when Iraq won the Asia Cup, CSP rolled-out soccer tournaments that capitalized on nationwide enthusiasm. 5) Pay attention to ensuring the selection of beneficiaries matches the program rationale. USAID 6) Conduct youth activity needs assessments in each target community to inform program planning prior to program rollout in target communities. 7) Encourage youth engagement activities that involve the wider community as spectators. 8) Establish youth-activity CAGs (e.g., as is done under CAP III) as advocates/lobbyists to MOYS and local governments, and to source greater community contribution and perform a secondary M&E and quality control of events and the resources used including: CSOs/clubs; venues; equipment; and materials. 9) Require the encouragement of public-private partnerships to sponsor teams and support other youth activities. 10) Require conflict mitigation programs take place regularly, and are attended by symmetrical (i.e. both sides) or close to symmetrical numbers of participants. 11) Use a “participant retention” model that ensures a program is responsive to the needs and interests of the participants so that they continue to take part, and are motivated to actively engage in it for its duration.1 12) Develop a transition strategy to ensure that local government agencies develop the organizational capacity and sources of funding to maintain and build upon the activities initiated by the program as it phases out. 1 This recommendation comes from the Literature Review (Annex II, pages 12 and 16) and refers to the Basic Functioning Approach and “good enough” Peace Education Model described by Ifat Mooz in “Conceptual Mapping and Evaluation of Peace Programs: The Case of Education for Co-existence through Inter-group Encounters between Jews and Arabs in Israel” in Gavriel Salomon and Baruch Nevo, Peace education: the concept, principles, and practices around the world, Lawrence Erlbaum Associates, 2002, p. 264 - 2 International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 1 Report on Community Stabilization Program: An Examination of the Youth Engagement Program INTRODUCTION International Relief & Development, Inc. (IRD) was awarded the Community Stabilization Program (CSP) contract (267-A-00-06-00503-00) under the RFA on 29 May 2006. CSP was a multi-faceted program designed to reduce the incentives for participation in violent conflict by developing and implementing activities that supported the social and economic stabilization of Iraqi communities. CSP was a non-traditional program for USAID in that it focused on short-term results in support of the broader United States Government (USG) Counterinsurgency (COIN) Strategy in Iraq, rather than focusing on long-term developmental impact. CSP targeted unemployed youths who were the most vulnerable to overtures from violent elements of Iraqi society. The program had four broad components: 1) short-term employment through the Community Infrastructure and Essential Services (CIES), 2) sustainable job creation through the Business Development Program (BDP), 3) training and employment placement through Vocational Training and Apprenticeships, and 4) youth engagement on life skills and conflict mitigation through sports, cultural events, and public service campaigns. Initially, CSP was limited to Baghdad, Mosul, Kirkuk, Fallujah, Ramadi, Al Qaim and Basrah. In August 2007, funding was increased and the project length extended through to September 2009. As a result, CSP expanded its Area of Responsibility (AoR) to include a total of 18 administrative offices located in urban areas across Iraq. At the start of 2009, CSP began drawing down its offices as part of a three-stage closeout. As offices were being closed out, the Agreement Officer Technical Representative (AOTR) for CSP in USAID/Iraq’s Focused Stabilization Office (FSO) determined that a special study was needed “to measure the results of CSP’s Youth Program component and establish its success in supporting the Mission’s counter￾insurgency goals.” The FSO wanted “to use the findings to answer questions on the effectiveness of CSP’s Youth Engagement Program in meeting its immediate objective of reducing incentives for young people to engage in violent activity through positive attitudinal change.” International Business & Technical Consultants, Inc. (IBTCI), the implementer of the USAID funded Monitoring and Evaluation Performance Program, Phase 2 (MEPP II), was asked to undertake the study on behalf of the USAID/Iraq FSO (See Annex 1: Scope of Work). This study focused on the non-labor-related youth activities performed at CSP’s remaining active offices in Sadr City, Beiji, Tikrit, Samarrah, Mosul, Kalak, Tal Afar, Baquba, and Basrah. THE DEVELOPMENT PROBLEM AND USAID’S RESPONSE Problem Statement Decades of war and the collapse of the Ba’athist government resulted in the near collapse of the entire Iraqi economy and social services, characterizing Iraq as an extremely fragile state. One factor that contributes to its fragile state is the large number of poor, unemployed, out-of-school, and disengaged youth who are at risk of becoming involved in violent practices and joining the insurgency. These youth are recruited to the International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 2 Report on Community Stabilization Program: An Examination of the Youth Engagement Program insurgency because they do not have alterative non-threatening opportunities to engage in positive, non-violent activities such as attending school, working, and participating in conflict mitigation through sports, cultural events, and public service campaigns. Theory of the Intervention In 2005, USAID/Iraq developed a three-year Strategic Plan (2006-2008) to lay the foundation for democratic governance and private sector led economic growth in Iraq. The USAID strategy was approved in January 2006. It supports peace, economic expansion, and democracy through four Strategic Objectives: 1. Reduce incentives for participation in violent conflict (SO 7). 2. Expand private sector economic opportunities (SO 8). 3. Strengthen responsive and effective local government (SO 9). 4. Improve capacity of national government institutions (SO 10). SO 7 was charged with implementing all activities related to reducing the incentives for youth to become involved in violent practices and to join the insurgency. The origin of SO 7 was in the Iraq Transition Strategy Statement (TSS) of November 2005. That document states, “[w]hile Coalition Forces and the GOI address the insurgency militarily and politically, USAID programs will continue to offer Iraqis hope and opportunities by providing…programs for vulnerable groups…which help diminish the influence of the insurgency.” In keeping with this COIN focus, the TSS states that, SO 7, “will focus on employment generation, infrastructure rehabilitation, youth programs, assistance to municipal governments and conflict mitigation” in cities of strategic interest to the USG. It was presumed that once stability was achieved, these “cities [would] be integrated into USAID’s longer-term development initiatives in health and education, agriculture, micro￾credit and building the capacity of communities and civil society organizations for advocacy, and the capacity of local government to provide basic services.” In addressing focused stabilization, the TSS notes that, “USAID’s approach recognizes that insurgent groups prey on disenfranchised populations left vulnerable by their government’s inability to meet their basic needs” and states that, “USAID will implement programs that enhance…social stability to communities that have been affected by insurgent violence.” The objectives to be achieved by USAID’s programs “include: [d]iminishing support for insurgent recruitment efforts…. [and]… [d]ecreasing tension among religious and ethnic groups.” The TSS also acknowledges youth as a vulnerable group, stating that it “recognizes the future impact of a significant and growing youth population, which….has tremendous implications for the future stability of Iraq,” and notes that, among other non-security interventions, “USAID will focus on…youth programs” that “include…[y]outh and sport activities and cultural shows.” It was hoped that this COIN strategy would “promote inter￾and intra-communal and ethnic dialogue and peace-building activities to diminish sectarian and ethnic cleavages exacerbated by ongoing political violence.” TSS Strategic Objective 1 became crystallized as SO 7 in USAID Iraq’s Performance Management Plan 2006-2008 (USAID PMP). That plan breaks SO 7 out into three intermediate results (IRs) based on three different areas of activity: employment; communal activities; and community infrastructure and essential services. The “aim [was] International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 3 Report on Community Stabilization Program: An Examination of the Youth Engagement Program that these linked activities [would] provide reduced incentives for violent conflict in the short term.” Each IR was designed to provide a short-term impact that would lay the groundwork for “longer-term efforts to advance stability, reform and institutional capacity.” As such, SO 7 and its related programs were clearly part of an integrated COIN strategy, first envisaged in the TSS, and designed to create the environment into which a development strategy could later be introduced. The Design of Community Stabilization Program (CSP) CSP was created to reduce the incentives for participation in violent conflict. It was initially implemented in May 2006 by USAID through a Cooperative Agreement (CoAg) with International Relief and Development (IRD). Cities were targeted due to high levels of insurgency activities and high unemployment and numbers of disenfranchised people which helped to fuel the insurgency. CSP focused on neighborhoods and districts in cities and semi-urban areas of greatest need. It provided a vital link between US military programs funded under Commander’s Emergency Response Program (CERP) and longer-term initiatives under the US Government’s (USG) Provincial Reconstruction Teams (PRTs), elected Provincial Councils (PCs), and the Government of Iraq (GoI). CSP was expected to jump-start the development of effective local government services by redirecting local energies toward productive economic and social opportunities, and away from insurgency activities. CSP contributed to the achievement of results under SO 7. As part of the CSP, the Youth Engagement Program supported IR 7.2, intended to mitigate conflict through increased civil society organization and community activities. The Youth Engagement Program collaborated with local government, community groups and leaders, and the Ministry of Youth and Sports (MoYS) to conduct activities that would enable Iraqi youth to 1) connect to their own identity, culture, and community, 2) engage as community leaders on issues important to them, and 3) come together with other youth from different ethnic and religious backgrounds to learn coexistence and tolerance. While USAID originally established 17-25 years of age as the target group age range, CSP recommended -- and USAID accepted -- expanding this range to 12-35 years of age to more accurately reflect the insurgent-aged youth of Iraq. Lowering the age range to 12 allowed the program’s youth activities to better interact with secondary schools, while raising it to 35 captured significant numbers of unemployed physically active young men and harmonized the program’s parameters with Iraqi labor laws that currently consider persons aged 17-35 years old as vocationally challenged. To date, the youth engagement program has engaged approximately 333,000 youth in a wide range of recreational and sports activities, informal education activities, arts, life skills, and other activities. Young men were the primary beneficiaries of the program, By May, 2009, 300,189 had participated in a range of activities including soccer, volleyball, swimming competitions, leadership seminars, environment awareness, t-wall painting, and language proficiency. By May 2009, 32,158 young women had participated in activities including forums for women’s issues, volleyball tournaments, fun runs, music, conflict mitigation camps, and sewing. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 4 Report on Community Stabilization Program: An Examination of the Youth Engagement Program In addition to sporting and cultural events, conflict mitigation was addressed in ways that complemented other CSP components. Examples of this integrated approach included: • Activities involving young girls, such as sewing classes, volleyball, and fun runs; • Developing more civic education focused activities, interwoven with the activities described above; • Holding larger scale youth oriented events in communities; • Revitalizing, rehabilitating and supplying youth centers with the essential needs to make them fully operational; • Conducting youth activities that fall within the following categories: • Enabling the Iraqi youth to connect to their identity, culture, and community; • Engaging Iraqi youth as community leaders on issues that are important to them; • Bringing together Iraqi youth from different ethnic and religious backgrounds to learn coexistence and tolerance; • Implementing the youth activities by working through the following channels: - Local government (LG) entities. - Ministry of Youth & Sports (MOYS), Ministry of Culture (MOC), Ministry of Environment (MOEN), and Ministry of Health (MOH). - CSP also works in close cooperation with local NGOs and civil society organizations (CSOs) to offer community-based activities. Implementation of the Youth Engagement Program As shown in Table 1, programming for youth engagement lagged behind the CIES and vocational education components.2 CIES projects were the first on the ground and were typically street cleaning or canal cleaning projects employing large numbers of unemployed youth. The Youth Engagement program and the BDP programs began later. Table 1: Project Start Date by Type of CSP Component and Percent of Projects Rolled Out Project Start Type of CSP Component and Percent of Projects Rolled Out Date CIES BDP Voc Ed Apprentice Youth 2006 3.0 1.4 3.0 0.0 1.6 Jan-Jun 2007 14.8 13.8 6.5 13.5 6.9 Jul-Dec 2007 23.4 15.6 14.9 6.1 15.4 Jan-Jun 2008 31.1 32.4 28.1 10.2 27.5 Jul-Dec 2008 13.5 23.7 44.6 65.2 32.8 Jan-May 2009 14.2 13.0 3.0 5.0 15.8 TOTAL 100.0 100.0 100.0 100.0 100.0 CIES = Community Infrastructure and Essential Services; BDP = Business Development Program; Voc Ed = Vocational Education; Apprentice = Apprenticeships; Youth = Youth Engagement Program 2 Based on an analysis of project start dates, recent IRD CSP project tracking sheets (as of 9 May 2009) were used. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 5 Report on Community Stabilization Program: An Examination of the Youth Engagement Program The monitoring of CSP’s Youth Engagement Program activities was faced with three major obstacles. First, there were data quality issues related to the indicator including a lack of evidence to support the causal linkage between engagement and conflict reduction. The CSP’s Performance Management Plan (PMP) indicated the following: The data set to measure this indicator was weakly valid to represent the intended results although the number of participants in youth activities has direct linkages with the information collected. This weak relationship was caused by the nature of the activities; duplicated counting may occur sometime and field offices and the IZ M&E office work together to minimize those limitations. In addition, the causal linkages between conflict reduction and increased youths participations in CSP activities were not clearly established; this emphasizes the weak relationship between the data and this indicator. The critical assumption made between participation in youth activities and attitude towards insurgency may partially hold but was not certainly supported by strong empirical evidence… Second, the meaning and purpose of the USAID PMP indicators to measure the effectiveness of programmatic efforts was complicated by different program expectations. CSP’s youth activities were intended to manage conflict through increased community activities (IR 7.2). The USAID PMP stated that IR 7.2 “focus [es] on developing activities targeting community mobilization and youth.” Examples of program activities were the establishment of “[s]ports clubs and youth associations.” It was also noted that “[o]pportunities for diverse groups to work together in communities will be taken advantage of’’ and that local government will be incorporated as a partner in the planning and implementation of these activities. The USAID PMP listed two indicators to measure the effectiveness of programmatic efforts under IR 7.2. They included the number of youth participating in non-formal education activities and the number of activities initiated through integrated decision￾making with local government. Non-formal education activities were defined as “consist[ing] of sports, theatre, youth associations, summer camps, internships, etc.” It also noted that by working with local leaders and government the program aimed to “build trust and foster a relationship between citizens and their local government.” In the CSP PMP, the Performance Indicator Reference Sheet (PIRS) for indicator 7.2.1 defined the rationale for the indicator as: “Non-formal education was a proven way to provide youths with alternatives to insurgent activities, which at the same time, links skill development to longer-term employment opportunities. The maintained hypothesis here was that increased youth participation in CSP activities would positively improve their attitudes towards violence and give them opportunities to develop their creativity while increasing their tolerance towards their mutual religious preferences.” Third, the CSP PMP measured the Youth Engagement Program’s strategy as a COIN strategy, rather than a development strategy. The CSP PMP tells us that “[d]ata for this indicator was recorded by program staff from the CSP/EGY component. It was based on attendance rosters and participant lists from CSP sponsored youth activities that were International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 6 Report on Community Stabilization Program: An Examination of the Youth Engagement Program part of the project documentation. The number of youth participating in such activities was reported every two weeks.”… “When an activity that has extended over several weeks was complete, the highest attendance number was then recorded and used towards the cumulative total.” The indicator selected was a reflection of the FSO view that the strategy was to engage as many people as possible thereby reducing their vulnerability to recruitment.3 Although CSP was funded and implemented by traditional development organizations it was a COIN program. As a result, information on outcomes (e.g., build hope and opportunity, provide for basic needs, build trust between citizens and government, and improve the attitudes of youth) was not collected or available to IBTCI. PURPOSE OF THE EVALUATION Through this study, the FSO sought to measure the results of CSP’s Youth Program component and establish its success in supporting the Mission’s COIN goals through USAID/Iraq’s SO 7. FSO intended to use the findings from the study to answer questions on the effectiveness of CSP’s Youth Engagement Program in meeting its immediate objective of reducing incentives for young people to engage in violent activity through positive attitudinal change. The study answered the following questions: 1. How effective was the CSP Youth Engagement Program in reducing incentives for participation in violent conflict? 2. How cost-effective has the Youth Engagement Program been in achieving results? 3. Did the Youth Engagement Program make use of prevailing methodologies and practices in the areas of youth and conflict mitigation? 4. While acknowledging that CSP is not a sustainable development program, how successful has the Youth Engagement Program been in transitioning youth to the Government of Iraq (GoI), the communities, civic groups, NGOs, and/or other local counterparts? 5. Where CSP was active, was there any apparent correlation between site-by￾sites historical reported incidents of violence and CSP’s youth activities? 6. What were the key lessons learned by CSP’s Youth Engagement Program as a COIN activity/strategy? 3 From the FSO comments on the MEPP II Community Stabilization Program (CSP) Vocational Education and Apprenticeship Special Study: “However, the primary measure of success for CSP has been whether or not it has had a COIN impact through engaging as many participants as possible to get them off the streets in the short-term, thus reducing their vulnerability to be recruited as insurgents.” International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 7 Report on Community Stabilization Program: An Examination of the Youth Engagement Program RESEARCH DESIGN AND EVALUATION METHODOLOGY Initially this study was to use multiple methods drawing on quantitative and qualitative techniques. The SOW asked to determine “if there was any change in attitude towards the GOI, their community and /or individuals of different tribes and religious beliefs.” Detecting attitude change hinged on having access to individual participants in the youth program. CSP only required youth program participants to register names and jensiya (national identification) numbers, and did not sustain a database of participants that could become a sample frame for a quantitative study. This effectively ruled out as intractable selecting a sample of youth program participants that would support a quantitative study. Therefore, it must be noted that the findings, conclusions and recommendations from this study are largely based on perceptions of adults who were stakeholders during the implementation of the Youth Engagement Program. These are completed with findings by the youth expert. IBTCI collected information from the following sources: Primary Analyses The following qualitative methods were employed to collect data related to the environment, fidelity, functionality, type of activities, and impact. 1. A desk review of the literature related to youth engagement (See Annex II). 2. A desk review of CSP documents and reports. 3. Key informant interviews (KIIs) and focus groups (FGs) with Iraqi and expatriate stakeholders. Desk Review of the Literature. A Youth Conflict Mitigation Expert conducted a literature review to; 1) identify best practices in youth conflict mitigation and violence reduction; 2) review youth programs in other countries to identify lessons learned and methodologies that have been used to successfully assess attitudinal change; and 3) identify best monitoring and evaluation practices. The Expert searched the Web for documents discussions with academicians and practitioners. Thirty-eight studies were reviewed, and three extensive interviews completed during the one-week study. Desk Review of CSP Documents and Reports. IBTCI staff requested and reviewed all relevant documents and reports from USAID and IRD. Key Informant Interviews and Focus Groups. Questionnaires were developed for the KIIs and the focus groups (See Annex III: Key Informant Interview and Focus Group Guides). Based on the desk review of the literature, the Youth Conflict Mitigation Expert recommended that IBTCI developed questions that would do the following: 1. Assess adult actor perceptions of the conflict and what variables could impact program outcomes, account for changes in youth attitudes, and help explain program contributions, if any, to conflict mitigation and stabilization. 2. As a substitute for trying to evaluate feelings and beliefs, the evaluation should draw on adult observations of any changes in youth behavior. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 8 Report on Community Stabilization Program: An Examination of the Youth Engagement Program 3. Evaluate the program in terms of youth engagement (skills, self esteem, and motivation), being in a supportive and safe environment, and potentially building positive relationships. 4. Assess how effective the Youth Engagement Program was in motivating behavior change in youths and to assess if youth were able to build new skills and new ways of engaging with one another. In addition to the expert’s recommendations, the questionnaires were based on basic functionality4 and good enough5 models as a basis to assess program impact and outcomes. Secondary Analyses Cost Effectiveness. IRD’s CSP tracking sheet data was the basis for analyzing cost effectiveness of program components in comparison to Youth Engagement. Tracking sheets were lists of projects funded under each of CSP’s four program components. Depending on the program data included were: project location, grant amounts, number of participants, start and end dates and program status. The tracking sheets were up to date as of May 9, 2009. These data were analyzed using SPSS to help define a suitable metric of comparing cost effectiveness between programs. The section on findings below has the results of the quantitative analysis. Correlation between CSP Projects and Daily Insurgent Attacks. IBTCI analyzed the Brookings Institution Iraq Index data on average daily insurgent attacks used by IRD to construct an analysis showing the statistical correlation of CSP programs to a reduction in insurgent attacks. IBTCI requested updated data from the Brookings Institution and repeated the regression analysis done by IRD. File handling and data analysis were done using SPSS. Procedures IBTCI staff conducted KIIs with ex-patriot key stakeholders and CSP/EGY staff.----, a local Iraqi firm, conducted KIIs and focus groups with Iraqi key stakeholders and CSP/EGY staff. These data were collected from March 2009 to 10 April 2009. KIIs with ex-patriot stakeholders were conducted at Tikrit, Mosul, Kalak, Baquba, Basra and Baghdad (Sadr City). The focus group team successfully conducted KIIs and focus groups with Iraqi stakeholders from Sadr City, Samarra, Tikrit, Beiji, Mosul, Baquba and Basra. Apart from dust storms delaying access to Baquba and the return from Basra, the fieldwork went as planned. 4 Basic functionality is defined as: “Effective programs would be, according to these criteria, programs that a) function regularly and b) were attended by symmetrical (i.e. both sides) or close to symmetrical numbers of participants.” Was there regular attendance and by both sides in the conflict situation. 5 Good enough models were: “based, at the minimum, on whether or not a program was responsive to the needs and interests of the participants so that they could take part, and would be motivated to actively engage in it.” International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 9 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Ninety people were involved in KIIs or focus groups. Participants were USAID personnel (n=2), PRTs (n= 7), Civil Military Affairs Officers (n=2), Ministry of Youth and Sports staff (n=6), implementer management and field staff (n=28), adult directors/referees/event coordinators (n=21), provincial council members (n=1), local government officials (n=7) and community services organizations (CSOs) and non-government organizations (NGOs) (n=25). The qualitative data from the KII’s and focus groups and other sources6 were synthesized using the following five-step process (See Annex IV: Data Matrices): 1. IBTCI staff coded the data into five topic categories. The topic categories were environment, fidelity, functionality, activities and impacts. 2. Staff grouped responses that cohered into “chunks” when they dealt with the same or a similar fieldwork question. 3. Staff conceptually organized the data chunks into recognizable clusters according to identifiable themes, patterns, leitmotivs, or causal links. 4. Staff brainstormed to review and transform the data systematically so that it answered the research questions in the SOW and articulate lessons learned. 5. Staff synthesized the answers and lessons learned with a review of the relevant literature on youth violence to draw the report’s recommendations. The process progressively condensed the data along coherent lines of inquiry until we were able to understand whether participants’ attitudes changed, how they changed, and the reasons for the changes (See Annex V: Data Reduction). STUDY FINDINGS Question 1: How effective was the CSP Youth Engagement Program in reducing incentives for participation in violent conflict? As shown in Box A, perceptions of the adult stakeholders were that the primary incentives for youth to participate in violent conflict were mainly money and religious reasons. Respondents recognized the difficulty for the young to remain in one place for a long time without anything to do. When this inability is coupled with being poor and uneducated, many are said to have joined to have money and authority. However, some youth may have joined for other reasons, such as to avenge the death of someone or to have a gun. 6 A desk review of youth related conflict mitigation and violence reduction literature; a desk review of documents and reports related to the CSP; and an analysis of CSP program project tracking sheets. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 10 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box A: Adult Perceptions on Incentives for Youth to Participate in Violence Quotes from KIIs and Focus Groups “[The methods and incentives were] exacerbated by the lack of entertainment, poverty, frustration, and the absence of proper outlets for youth to expend their energy, abilities, and talents (Salah ad Din).” Most young people were unemployed and joined armed groups because of external pressures, so they could earn some money and help secure neighborhoods (Al-Sadr City) "Many young people here in Mosul joined the militia either for money or for power. Those young people were unemployed, poor, and uneducated. They were given cars, weapons, and money.” "They [militias and armed groups] attracted young people who were uneducated, unemployed and had nothing to do by giving them money and power (Mosul).” The majority of people said there was no comparison between the level of violence now and the level of violence 18 months ago. The current level is significantly lower than the previous level. One government official said that the CSP program restored confidence and enabled people to socialize and make friendships. Another person said that youngsters compared the actions of the terrorists and the activities of CSP and realized the positive difference. As a result of this realization, violence retreated and was reduced (See Box B). Box B: Adult Perceptions on Changes in the Level of Violence in their Community during the CSP Youth Engagement Program Quotes from KIIs and Focus Groups "Today our life is safe and the whole situation in Sadr City is normal. There is not a lot of violence or attacks occurring, although we do sometimes have violent troubles (Al-Sadr City)." "The situation now is very good compared with two years ago (Mosul).” “[The militia] can go everywhere now. Last week they went to an area that was very bad. [In the past,] they used it to dump bodies. There is now a soccer field there (Al-Sadr City).” When asked if CSP could be attributed to the decrease in violence, everyone said that it was not possible to directly attribute the decrease to the CSP program. However, they said they knew that it was at least one of several contributing factors. They believed that CSP contributed to the decrease in violence and to other results. In fact, one member of a Youth and Sports Committee Council said that CSP had a greater role in decreasing the violence than the government. All groups said that the program taught tolerance and conflict mitigation. As a result of the program, youth improved their skills, made new friends, and obtained a sense of normalcy, hope, and pride. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 11 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box C: Some Observations on the Impact of Youth Engagement Activities Regarding Changes in Attitude and Behavior Quotes from KIIs and Focus Groups ““Youth wave more as they [coalition forces] pass through. They are more receptive and [there are fewer] rock throwing incidents. The violence against coalition forces has decreased. Violence against Iraqi Security Forces went up (Mosul).” There was a young man from Al-Adhmiya’s team who shook hands with a young man from the Al-Shaab team whose cousin had killed his brother and he knew him well (Al-Sadr City).” “The terrorists had created an emptiness of time for youngsters; but the CSP programs filled their time again with something useful to do. For example, one of the football team captains was about to join the terrorists because he had nothing to do but now he is all right (Salah ad Din)." “Soccer and sports participation makes players focus on their games and sport and they have no time for violence. The caring that they receive from their teammates and coaches is most important to reduce their violent activities. Knowing that someone cares about them is very important (Al-Sadr City).” “Effective” under the COIN strategy means the ability of the Youth Engagement program to produce more engagement days dollar for dollar than alternative CSP programs. In this respect the CSP programs (see the analysis under Question 2 below) were evenly matched. What the study could not measure was any multiplier effect that sports events may have had in attracting crowds of spectators to these events and whether this resulted in any added normalizing/stabilizing benefit. The study generally accepts that there was positive effect beyond what the number of program participants alone will measure. The KIIs and the focus group interviews found a large number of interviewees who discussed how the youth activities had a positive effect on the community and were able to provide anecdotal evidence to support their statements. However, the CSP program did not attempt to assess the effect the program had on non-participants. Question 2: How cost-effective has the Youth Engagement Program been in achieving results? Cost was defined as the verified project cost as it appears in the IRD tracking sheets dated May 9, 2009 for completed or closed out projects. The amounts used in the analysis and shown in Table 2 below were the grant amounts, and do not include any contributions made by the community or GOI. The amounts also exclude overhead and staff costs that may be associated with each program. The summary table shows cumulative amounts since 2006 (column [c]) and these may vary slightly from the CSP Activities Report for the week of May 3-9 because on-going projects were excluded. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 12 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Effectiveness under the COIN concept was measured by the number of youth engaged in CSP program activities. Youth were assumed to be “at risk” youth, although they were assessed on the basis of the community they lived in rather than any individual characteristics that might identify them as more likely to join a militia. Engaging youth as participants was done under the premise that it deters them from joining other violence prone activities (from the literature search we were cautioned not to treat youth as a violence prone monolith). Youth engagement was counted in the CSP weekly reports as the cumulative number of participants in each of the CSP programs since 2006.7 Short￾term employment, long-term employment, vocational education training, apprenticeships and participation in a Youth program were all counted equally as engagement. One method for considering cost effectiveness was to divide cumulative program cost by the number of persons that have been engaged since the program began in 2006. Column [d] and [e] in Table 2 below show these estimates. Using this measure the Youth Engagement Program was clearly the more economical option with a cost per person engaged estimated at $70 compared with $470 (Apprenticeships), $709 (Vocational Training), $1037 (CIES) and $1888 (BDP) estimated for the other CSP programs (See Table 2). This method assumes that all CSP program engagements were equal but the study readily acknowledges that some programs engage participants for longer periods than others. An alternative method was to measure the duration of the engagement in man-days across the CSP programs. This method has the effect of leveling the measure for comparing across programs. The Youth program tends to sponsor short term events perhaps engaging youth for no more than a few days (this was measured weekly). The CIES, BDP, vocational education and apprenticeship programs engage participants for longer periods. For the CIES program the number of person-months employment was recorded, and this was converted to man-days of employment by multiplying person￾months by 24 working days each month. The BDP program provides long term employment to grant awardees, which means that their program engagement was a minimum of 90 days, but typically longer.8 The study used a conservative estimate of 120 days engagement per BDP grantee. Vocational education retains participants for the duration of the course, usually 60 to 90 days. The study used 60 days as an estimate. Apprentices remain under a paid stipend for an average of 82 days based on an analysis of survey data.9 Youth were assumed to have been engaged for just a few days each time they were counted as a participant (weekly or bi-weekly).10 Table 2 below summarizes how the duration of engagement for each CSP program component was calculated. Results show that there was little to 7 This conforms to how IRD reports engagement in the weekly CSP Activities Report. 8 Studies of the BDP program show that most grantees were still in business one year since the grant was awarded. 9 H. Herr, R. Mason, & M Zubaidy (2009). MEPPII Community Stablization Program (CSP) Vocational Educational and Apprenticeship Special Study. Vienna, VA: International Business and Technical Consultants, Inc. 10 The youth program did not maintain a database of participants so the study was unable to determine how long an individual remains in the youth program. Also, the same individual can be counted each time he/she participates in a sponsored event. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 13 Report on Community Stabilization Program: An Examination of the Youth Engagement Program distinguish between the programs based on cost effectiveness. These results were sensitive to the assumptions made about the “average duration in days” and caution is advised. The duration that youth were actually engaged was not clear from the data and how the data were collected. A comparative advantage that the Youth Engagement program enjoys was the ability that it has to attract audiences and spectators through some of the activities that it sponsors. Consequently, parts of the Youth Engagement program, like sports and theatre, have a multiplier effect that expands its influence on the community. This expanded influence remains unmeasured, and the study unable to understand its consequences. Table 2: Cost Effectiveness of CSP Program Elements short term engagement (persons)      [a] long term engagement (persons)      [b] program budget/grants [c] cost/person forshort term engagement [d] =[c]/[a] cost/Person forlong term engagement [e] =[c] /[b] estimated man/days duration of engagement [f] = [a] or[b] * [h] cost per day of engagement [g] = [c]/[f] average duration in days[h] BDP 36482 68,892,214 $     $            1,888 4377840 $                  16 120 CIES 212613 $  220,438,913 $              1,037 12053376 $                  18 see footnote Vocational education 40785 $    28,932,787 $                    709 2447100 $                  12 60 Apprenticeships 9269 $       4,357,423 $                    470 760058 $                    6 82 Youth Engagement 288157 $    20,087,305 $                    70 1008549.5 20 $                   3.5 Total 550824 36482 273,816,428 $   $                 497 $            2,801 20646924 $                  13 Cumulative totalsfromtracking sheets as of May 9, 2009 Relative cost per number engaged CSP ProgramElement 7. Youth Engagement: Based on enrolmentrecords, number of participants completing non formal courses, collected weekly. 6. CIES short termengagement =man‐months  multiplied by 24daysfor engagement duration = 112053376man days engagement 5. CIES short termengagementis given inman‐monthsin the tracking sheet and is converted to personsin column [a] 1. Data are taken fromthe tracking sheets as of May 9, 2009for completed and closed out projects only. 2. Relative cost per number engaged = the number of persons employed or participating in a CSP project, or as a result of a CSP project divided by the program/grant cost. 3. Short termengagement = engagementfrom1day to < 3months. 4. Long termengagement =engaged 3months orlonger. Each of the other CSP programs has objectives beyond engagement, but these objectives spill over into areas beyond the COIN objectives that were of interest to this study. In terms of keeping youth occupied, the Youth Engagement program may not deliver the greatest “bang for the buck.” However, the Youth Engagement program brings with it an ability to widely engage the community beyond the number of participants counted. Apprenticeships provide the most economical way to engage youth: a result of the private sector matching the CSP apprentice stipend (apprenticeships follow-on from vocational education). The SOW suggests changes in attitude should be a key measure of program effectiveness. Measuring changes in attitude should start with a baseline assessment followed by continuous tracking of participants over time. The study established that such a measure is not easily obtained. Some practitioners suggest using the “participation retention” model that tracks participants to establish whether a program is International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 14 Report on Community Stabilization Program: An Examination of the Youth Engagement Program responsive to the needs and interests of participants so that they are motivated to remain actively engaged in the program for long periods of time. The “participant retention” indicator can also be used to monitor changes in participants’ attitudes over time and it can be complemented by the Knowledge, Attitudes and Practice (KAP) surveys to compare new program entrants with long-term participants. Question 3: Did the Youth Engagement Program of CSP make use of prevailing methodologies and practices in the areas of youth and conflict mitigation? There are several common methodologies employed in the conflict mitigation and violence reduction field such as: peace education programs that broadly aim to build capacity by developing knowledge, skills, attitudes and values that prevent conflict and/or contribute to a culture of peace and generally fall into four types including dialogue, social events or social games, creative, and life skills; encounter programs based on “contact theory” that brings youth together for dialogue or exchanges; and violence reduction programs that can fall include hard (e.g., more police arrests, juvenile detention) and soft approaches (e.g., conflict resolution, peer mediation, after-school activities, sports and recreation, and life skills).11 The Youth Engagement Program employed all of these methodologies though the overwhelming number of activities fell into the soft violence reduction and encounter program categories with the primary purpose of changing attitudes of youth toward other people from different ethnic groups, religions, or tribal affiliations. Recently, IRD has made efforts to include a larger number of peace education activities.These peace education activities that broadly aim to build the capacity of youths by developing their knowledge, skills, attitudes, and those values that prevent conflict and/or contribute to a culture of peace (See Annex II: Desk Review, pages 5-6). The activities generally fell into four types of activity: dialogue; social events or social games; creative activities and life skills oriented activities. The Youth Engagement Program primarily provided opportunities for youths to engage in sports and recreation activities. However, several communities also implemented seminars that brought youth together for dialogue and art and theatre activities. Salah ad Din CSP staff received training that included positive conflict mitigation messages. The majority of the interviewees said that conflict mitigation was indirectly addressed in the sport events. However, several Youth Engagement Program components had structured activities related to conflict mitigation. For example, the Basrah Youth Engagement Program sports events included conflict mitigation messages during halftime or between games through mobile theaters and poetry readings. The Baqubah CSP program had staff conduct seminars where youth talked about their futures without violence. Additionally, the Mosul Youth Engagement Program conducted lectures and reconciliation camps and the Salah ad Din program included conflict mitigation messages in boy scouting activities. 11 See Desk Review: Annex II, pages 5 – 7. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 15 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box D: Provision of Conflict Mitigation Quotes from KIIs and Focus Groups “Opening ceremonies include leaders that should give speeches about the larger goals. This would not be at every game (Al-Sadr City).” “Youth programs are not really appropriate in affecting violence levels. CSP doesn’t have a large impact, as the individual’s ideology is the predominant factor (Al-Sadr City).” [We have] a good history of scouting here, which is a good conflict mitigation strategy and they (Scouts) received lots of support for this from the community (Salah ad Din), also in Samarra.” “We held many lectures on cultural and religious varieties and about the necessity to socialize and interact with those who [are] different from you in ideas and beliefs in Tel Afar (Mosul).” Although CSP did employ prevailing methodologies and practices in the areas of youth and conflict mitigation, findings from the desk review indicated that it is questionable whether programs based on changing attitudes can achieve their conflict mitigation or prevention objectives in a conflict setting (See Box E). Therefore, the implementation of prevailing practices does not guarantee that attitudes among youth will change. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 16 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box E: Lessons Learned: Implementing “Attitudinal Change” Activities in Violent Conflict Settings Desk Review Findings • Research shows only a tenuous link between changes in attitude and resulting changes in behavior or deep-seated beliefs, particularly in intractable conflict settings. (Desk Review, Annex II, page 8, referencing the Seeds of Peace Annual Report, Seeds of Peace, 2006, p.18). • Interventions in conflict settings that focus on encounters for young people based on contact theory may be of questionable value when adults in a community still display behaviors based on conflict. (Desk Review, Annex II, page 8, referencing Hart, Jason, “Children’s Participation in Humanitarian Action: Learning from Zones of Conflict”. • Conflicts [are] usually, if not always characterized by development and/or social inequalities and encounter programs may be counter-productive in that they actually increase frustration for some participants. (Desk Review, Annex II, page 8) • Young people in conflict settings face a number of issues such as personal security, or other hardships (lack of work, food security, and basic services) that may cause frustrations or apathy that affect the young people attending such programs, their experience, and their participation. (Desk Review, Annex II, page 8, referencing: Women’s Commission for Refugee Women and Children (2008), Living in Limbo: Burma’s Youth in Thailand. 12 Question 4: While acknowledging that CSP was not a sustainable development program, how successful has the Youth Engagement Program been in transitioning youth activities to the GoI, the communities, CSO, NGOs, and/or others? Respondents from four of the five communities reported that the municipality, Ministry of Youth and Sports, other local organizations, or the youth themselves continued some of the CSP activities (See Box F). Mosul and the other target communities in Ninawa was the only governorate where respondents stated unanimously they did not know of other organizations continuing CSP activities. What is clear is that in most cities CSP’s youth activities showed that organized youth activities and events were again possible. The activities and events appear to have provided motivation that sparked and emboldened others to emulate and also organize activities. MoYS and other local government officials repeatedly stated that they would like to maintain the level of youth activities initiated by CSP but lacked the funding to do so. One member of a Youth and Sport Committee Council said he heard there were attempts made by other organizations to offer similar programs but the organizations needed support they could not provide. 12 http://wwww.reliefweb.int/rw/RWFiles2008.nsf/FilesByRWDocUnidFilename/MYAI-7LFAJG￾full report.pdf/$File/full report.pdf International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 17 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box F: Sustainability of the Youth Engagement Program Quotes from KIIs and Focus Groups “There are several projects we are aware of as we continue to support them by assisting them with the local municipality as well as the Sports & Youths Office. These include health awareness (breast cancer awareness), literacy courses, and mural painting by the Fine Arts Institute (Basrah).” “The campaigns have influenced lots of other institutions. They supported CSOs, such as the Fine Arts Institute, youth (with CSP support) went back to schools that CSP had renovated and decorated the classroom walls with appropriate art and painting and decorations (Basrah).” “We do know of soccer tournaments and training courses that have been independently organized by former CSP youth activities participants (Baqubah).” “Every day I see youth playing sports like football and interacting in many activities in the community…CSP had a hand in creating these activities in Al￾Sadr City (Sadr City).” "We found now that the youth have organized themselves into many football teams and without any external kind of support…just their own efforts to play and keep in contact by playing and feeling happy (Sadr City).” “Participants in the soccer tournaments continued to organize tournaments on their own and recycled the trophies won at the CSP tournaments to use as prizes … while CSP did not support billiard or ping-pong tournaments, the fact that CSP had shown such tournaments were possible encouraged…to independently organize billiard and ping-pong tournaments (Salah ad Din).” Question 5: Where CSP was active, was there any apparent correlation between site-by-site historical reported incidents of violence with CSP youth activities? IRD analyzed average daily insurgent attacks in CSP cities in a December, 2008 report.13 Their report used Brookings Institution Iraq Index data that was a daily average of insurgent attacks measured quarterly. These data originate with Iraq Significant Activities (SIGACTS) reporting from maneuver units in the field. In the IRD analysis average daily incidents were again averaged across the CSP provinces. This ‘average of the averages’ was the dependent variable in a regression analysis where the independent variables represent the three phases of the CSP program. The regression parameters and tests show that reductions in insurgent activities correlate to a decrease in average insurgent attacks. 13 Sidibe, Mamadou (2008). Analysis of the Number of Daily Insurgent Attacks in CSP Cities in Iraq. Baghdad: International Relief & Development. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 18 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Using the same data this study validated the analysis, but used the sum of the provincial daily averages as the dependent variable as it gives a better sense of the magnitude of the insurgent attacks (the bottom row in Table 3 below). Table 3 shows that the average incidents per day decreased from a peak of 156.7 in the quarter February to April 2007, down to 11.8 for the latest quarter. Our regression analysis results confirmed the correlation between CSP program phases and reductions in insurgent attacks. Correlation does not prove a causal link, and as stated in IRD’s report this “result was the combined effect of CSP’s effort, the surge policy, and all other interventions in the country affecting the level of insurgency.” Table 3: Average Daily Insurgent Attacks by Province and by Quarter Province Feb‐ June 05 Aug 05‐  Jan 06 Feb‐ May 06 May‐ Aug 06 Aug‐ Nov 06 Nov 06‐ Feb 07 Feb‐ Apr 07 May‐ July 07 July‐ Nov 07 Dec 07‐ Feb 08 Feb‐ May 08 Sep‐ Nov 08 Dec 08‐ Feb 09 Baghdad 20.3 21 28.9 30.3 39.5 44.8 50.7 58 27.5 15.7 24 5.9 3.8 Anbar 12.3 23.3 22 31.1 41.2 35.3 25.8 11.1 5.2 2.4 2 1.6 0.8 Salah ad Din 8 13.8 13.7 15.5 20.3 22.8 26.2 28.4 17 8.8 6.2 2.8 1.9 Diyala 3.1 5.4 8.3 14.5 15.7 16.8 21.8 25.2 13.4 5.2 3.8 1.8 1.5 Ninawa 10.4 8.5 7.6 10.3 9.8 11.5 15 14.2 14.1 16.3 13.7 7.4 2.3 Tamim 3.1 4.7 4.3 4.8 4.2 5 5.7 7 5.8 2.7 1.9 1.5 1 Basra 1.2 1.1 2 2.4 4.9 7.8 8 8.8 3.9 1.6 1.5 0.2 0.2 Babil 1.5 1.8 1.2 1.9 2.8 2 3.5 3.7 2.1 0.7 0.8 0.7 0.3 Total All CSP 59.9 79.6 88 110.8 138.4 146 156.7 156.4 89 53.4 53.9 21.9 11.8 CSP GOI Phase 1: pre CSP Phase 2: CSP Implemented Phase 3: Surge/CSP ramp up GOI "Enforcing the Law" Average Daily Insurgent Attacks Measured by Quarter (Source: Brookings Institute Iraq Index) Interestingly, Ninawa has not seen the same level of reduction in violence observed in other areas and remains an insecure area even though CSP has invested heavily in youth and other programs in that area. Question 6: What were the key lessons learned of CSP’s Youth Engagement Program as a COIN activity/strategy? The lessons learned reported here are a summation of responses received during the interviews and focus groups conducted by this study. As such they do not necessarily reflect the opinions of MEPP II: 1. As reported by Baqubah CSP staff, youth activities can be the initial step toward building trust and, as a result, gaining entry into a community. 2. Members from all of the communities indicated that an indirect outcome of youth activities was the engagement of adults. 3. The ability to respond to and adjust the program to current issues in the community is important. For example, Baqubah experienced a sudden increase of female suicide bombers. CSP staff immediately created activities that targeted female youth. The number of suicide bombs by female youth decreased from 19 in 2007 to 3 in 2008. 4. Several community leaders indicated that implementing programs in local schools not only improves the schools but also provides additional International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 19 Report on Community Stabilization Program: An Examination of the Youth Engagement Program opportunities to children who are currently attending school, provides a means for dropouts to re-enter the mainstream, and increases the number of re-enrollment among dropouts. 5. It is important to engage even younger children. 6. The quality of equipment and materials was dependent on the quality of the quality assurance system that was in place. 7. Media is a positive and efficient means to get messages to the general public. RECOMMENDATIONS Implementer 1) Develop indicators that show a clear link between program interventions and resulting changes in youth behavior and attitudes. 2) Conduct research that examines the relationship between the change in behavior and attitudes among youth with those of adults in their communities. 3) Employ a more comprehensive and systematic M&E of activities that continuously tracks participants over time and includes observation and interviews with spectators, audiences and parents. 4) Continue to take advantage of national events to enhance the impact of programs, e.g., when Iraq won the Asia Cup, CSP rolled-out soccer tournaments that capitalized on nationwide enthusiasm. 5) Pay attention to ensuring the selection of beneficiaries matches the program rationale. USAID 6) Conduct youth activity needs assessments in each target community to inform program planning prior to program rollout in target communities. 7) Encourage youth engagement activities that involve the wider community as spectators. 8) Establish youth-activity CAGs (e.g., as is done under CAP III) as advocates/lobbyists to MOYS and local governments, and to source greater community contribution and perform a secondary M&E and quality control of events and the resources used including: CSOs/clubs; venues; equipment; and materials. 9) Require the encouragement of public-private partnerships to sponsor teams and support other youth activities. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 20 Report on Community Stabilization Program: An Examination of the Youth Engagement Program 10) Require conflict mitigation programs take place regularly, and are attended by symmetrical (i.e. both sides) or close to symmetrical numbers of participants. 11) Use a “participant retention” model that ensures a program is responsive to the needs and interests of the participants so that they continue to take part, and are motivated to actively engage in it for its duration.14 12) Develop a transition strategy to ensure that local government agencies develop the organizational capacity and sources of funding to maintain and build upon the activities initiated by the program as it phases out. CONCLUSIONS CSP was a multi-faceted program designed to reduce the incentives for participation in violent conflict by developing and implementing activities that supported the social and economic stabilization of Iraqi communities. CSP was a non-traditional program for USAID in that it focused on short-term results in support of the broader USG COIN Strategy in Iraq, rather than focusing on long-term developmental impacts. CSP targeted unemployed youths who were the most vulnerable to overtures from violent elements of Iraqi society. CSP identified youth as an easy target for those seeking to mobilize violence and the challenge was to employ a program that could undermine the urge/need to join violent elements of the society and in the process, offer the youth hope for a viable future. The desk review revealed that it is still unclear whether programs like CSP do in fact change attitudes and reduce violence. However, this study suggests (based on eye witness accounts) that CSP had a positive impact in these areas. Due to the limitations mentioned earlier in the report, the study could not quantify the level of impact that the Youth Engagement program had on youth attitudes and the levels of violence. Youth activities were overwhelmingly popular. When asked why the youth wanted to participate, the majority of interviewees said it was because they wanted to make new friends, develop or use their skills, and to forget the past. 14 This recommendation comes from the Literature Review (Annex II, pages 12 and 16) and refers to the Basic Functioning Approach and “good enough” Peace Education Model described by Ifat Mooz in “Conceptual Mapping and Evaluation of Peace Programs: The Case of Education for Co-existence through Inter-group Encounters between Jews and Arabs in Israel” in Gavriel Salomon and Baruch Nevo, Peace education: the concept, principles, and practices around the world, Lawrence Erlbaum Associates, 2002, p. 264 - 2 International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 21 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Box G: Adult Perceptions on Level of Youth and Community Participation Quotes from KIIs and Focus Groups “Participants welcomed the projects enthusiastically and participated. There was a desire [among participants] to have follow-up [projects] and to continue activities (Mosul).” “All of their youth activities are always overflowing with participants (Basrah).” “We faced many young people and teams who were calling us and demanding to perticipate without getting anything in return. They wanted to participate in an event. During the championship, many teams informed us that they paid to rent buses or buy some things for the championship (Al-Sadr City)." “We had many volunteers who participated in the art gallery and Arabic calligraphy. In addition, people volunteered to work as guards and drivers or cleaners at the football fields. We had 60 male volunteers and 30 females (Mosul).” Initially, the level of participation was low. For security reasons many families did not agree to let their children participate in the project. There was also some suspicion and warnings from religious figures in the neighborhoods that such programs would bring forth more troubles. Families eventually realized that their children were not in danger. Parents played a large role in encouraging their children to participate in CSP. For example, the community of Kalak had parents encourage their girls to participate in youth activities. As one community member said, “Our community is conservative and we believe women should stay indoors. Yet, their fathers brought them to the activities and took them home.” Local councils found the local community members more than willing to volunteer their time to the project. Many young people came to participate or assist without demanding benefits. Baghdad Council received many complaints from young people who did not have a chance to participate. Those young people accused their local councils of being partial in their choices of the participants, or of making specific choices that did not match the real number of young people in a certain area. For example, there were four teams from Al-Hurria. However, some local government officials felt there should have been six teams. By May 9, 2009, the Youth Engagement Program had engaged 332,707 Iraq youth in over 640 different CSP-sponsored sports, arts and life skills programs, including soccer competitions, poetry festivals, t-wall painting, and dramatic presentations.15 The intended goal of these activities has been to create social fora where Arab, Kurdish, Turkmen, Sunni, Shia, Christian, Yezidi and Sabia communities were brought together in vibrant, non-violent interface. 15 International Relief Development (2009). CSP Activities Report (Week of May 3-9, 2009) International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 22 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Often, these activities have impact beyond their direct beneficiaries. For example, the Youth Engagement Program sporting events have attracted large crowds of spectators and been publicized widely on local TV and in the local newspapers as – for the target communities -- they were the first such sporting events in many years and helped to peacefully bring together youth teams from regions that in the past two years were actively engaged in insurgent acts against one another. Such activity and publicity brings a sense of stability and normalcy to communities that may have been divided for years. For example, given the reality of diverse predominantly Sunni and minority Shiite communities living throughout the region of North Babil, such neutral events have helped to facilitate indirect conflict mitigation through the modalities of sports and cultural events. At the opening of one CSP funded soccer tournament the Major of Mussayib publicly announced that this event was helping to “wipe the dust from their faces” after years of war - a metaphor for hope re-emerging in communities that were once strongholds of Al Qaeda in North Babil. The majority of program staff and community members interviewed believed that the activities decreased recruitment to militia or insurgent groups among youth and there was a beginning of diverse groups coming together. However, there is no data from the program that clearly links youth activities to a decrease in violence, recruitment to insurgent groups, or violence among diverse groups. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 23 Report on Community Stabilization Program: An Examination of the Youth Engagement Program ANNEX I: KEY INFORMANT INTERVIEW FORMS Key Informant Interview Guide: Program Officer Director Arrived in (date): ________________________ Program Fidelity: Testing for Correct Knowledge about the Program. 1. What is the purpose of CSP’s youth activities overall? Specifically in your city/area? 2. What do CSP youth activities intend to achieve in terms of specific results? 3. Probe: How? 4. Have the objectives and/or desired results changed over time? If so, why and how? Program Design and Implementation 1. How did this office develop its youth program and activities? 2. What was the involvement of local Iraqi staff in the selection of activities? 3. Please describe the assessment methodology used, if any, to inform the design of this program. 4. What role does the IZ program directors play in the design and implementation of your youth activities? 5. Please describe how well your office’s youth program matches the vision and strategy presented by IRD’s Youth Program administration and senior program management. 6. Please describe your office’s youth portfolio in detail, including the different types of activities, their relative participation and funding levels, and their relative accomplishments. 7. How does your youth programming differ from that in other cities/areas, if at all? 8. How do CSP EGY staff share lessons learned? 9. Can you provide an example of a best practice you shared or learned that was implemented? Functionality 1. Please describe the role of the PRT in the design, selection and implementation of your office’s youth activities? 2. Please describe the ways in which CSP works with local councils to implement youth activities. Can you give an example? 3. Please describe the ways in which CSP works with the Ministry of Youth and Sports. Can you give an example? 4. Please describe the ways in which CSP works with local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities. Can you give examples? 5. Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 24 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Impacts 1. How do you know if your office’s youth programs are working or achieving their intended objectives? Probe: anything from the PRT or military? 2. Please describe the different data sources and metrics that your office uses to track the effects of your youth activities and program. 3. What records does your office keep in regards to projects and participants? 4. In your view, are there particular strengths or weaknesses of your office’s different youth interventions that makes them more or less effective at reducing incentives for youth to engage in violence? 5. What is the most effective program your office has implemented to accomplish this objective? 6. Have CSP’s youth activities had an impact on the programs run by other community organizations? Can you provide an example? Why do you think that was? Prompt: Are you aware of any activities that were started by CSP that have been taken up by the municipality, Ministry of Youth and Sports, or other local organization? 7. Going forward, what changes would you make to the youth program based on the lessons you have learned? General 1. Is there anything we haven’t covered that you would like to tell me about the way CSP’s youth activities may have helped to reduce incentives for violence in your community? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 25 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Key Informant Interview: Program Office Officer Arrived in (date): ________________________ Program Fidelity: Testing for Correct Knowledge about the Program. 1) What is the purpose of CSP’s youth activities overall? Specifically in your city/area? 2) What do CSP youth activities intend to achieve in terms of specific results? Probe: How? 3) Have the objectives and/or desired results changed over time? If so, why and how? Program Design and Implementation 1) How did this office develop its youth program and activities? 2) What was the involvement of local Iraqi staff in the selection of activities? 3) Please describe the assessment methodology used, if any, to inform the design of this program. 4) What role does the IZ program directors play in the design and implementation of your youth activities? 5) Please describe how well your office’s youth program matches the vision and strategy presented by IRD’s Youth Program administration and senior program management. 6) Please describe your office’s youth portfolio in detail, including the different types of activities, their relative participation and funding levels, and their relative accomplishments. 7) How does your youth programming differ from that in other cities/areas, if at all? 8) How do CSP EGY staff share lessons learned? 9) Can you provide an example of a best practice you shared or learned that was implemented? Functionality 1) Please describe the role of the PRT in the design, selection and implementation of your office’s youth activities? 2) Please describe the ways in which CSP works with local councils to implement youth activities. Can you give an example? 3) Please describe the ways in which CSP works with the Ministry of Youth and Sports. Can you give an example? 4) Please describe the ways in which CSP works with local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities. Can you give examples? 5) Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 26 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Impacts 1) How do you know if your office’s youth programs are working or achieving their intended objectives? Probe: anything from the PRT or military? 2) Please describe the different data sources and metrics that your office uses to track the effects of your youth activities and program. 3) What records does your office keep in regards to projects and participants? 4) In your view, are there particular strengths or weaknesses of your office’s different youth interventions that makes them more or less effective at reducing incentives for youth to engage in violence? 5) What is the most effective program your office has implemented to accomplish this objective? 6) Have CSP’s youth activities had an impact on the programs run by other community organizations? Can you provide an example? Why do you think that was? Prompt: Are you aware of any activities that were started by CSP that have been taken up by the municipality, Ministry of Youth and Sports, or other local organization? 7) Going forward, what changes would you make to the youth program based on the lessons you have learned? General 1) Is there anything we haven’t covered that you would like to tell me about the way CSP’s youth activities may have helped to reduce incentives for violence in your community? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 27 Report on Community Stabilization Program: An Examination of the Youth Engagement Program PRT Interview Guide 1) PRT Interviewee: a) Assignment in the PRT USAID Representative b) How long at current assignment/ in the PRT c) Parent agency (DOS, 3161, USAID, Mil, …) d) Invite a general rant about how the PRT works : i) Support from HQ ii) Strengths and weaknesses of the PRT organization iii) Ability to engage with GOI counterparts (operations/logistics support) iv) What programs are operating that support the PRT (i.e., CSP is one among several, and may not be the dominant one) (if you wanted to do a program for youth, where would you look for support? CERP, IRAP, CSP, ICCM, CAP, LGP…) 2) CSP Program Overview (Fidelity) a) Describe how the CSP program fits under USAID Mission strategy? Prompt: How does it dovetail with the PRT mission? b) Is the CSP best described as a development program or a COIN program? c) How familiar are you with CSP’s activities? d) What is CSP doing in your Area of Responsibility? e) Do you know how CSP activities were designed and selected in your AoR? f) Please describe the assessment methodology, if any, is used to inform the design of this program. [How did you come up with stuff for the CSP Youth Program to do?] g) Are you involved with the design or selection or oversight of CSP activities here? How? h) What do you think is the appropriate mix of programs to compliment an effective COIN strategy in your area? Please explain your answer. i) What mix of these programs has been implemented in your area? j) How does your youth programming differ from that in other cities/areas, if at all? If there are youth activities: k) What are CSP youth activities in your AoR intended to achieve in terms of specific results? l) How involved have you been in the selection, design or oversight of these activities? m) Have the objectives and/or desired results changed over time? If so, why and how? n) Are there different components of the youth program that are designed/tailored to achieve specific results within the overall strategy? (Such as Life skills for certain youth and soccer/sports for other groups based on their profile or some other factor)? 3) Functionality a) Do you know if CSP supports local councils to implement youth activities? Can you describe this and give examples? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 28 Report on Community Stabilization Program: An Examination of the Youth Engagement Program b) Do you know if CSP collaborates with the Ministry of Youth and Sports? Can you describe this and give examples? c) Do you know if CSP supports local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities? Can you describe this and give examples? d) Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? e) Do you know if CSP’s youth activities had an impact on the programs run by other community organizations? Can you provide an example? Why do you think that was? Prompt: Are you aware of any activities that were started by CSP that have been taken up by the municipality, Ministry of Youth and Sports, or other local organization? 4) Impacts a) How do you know if CSP’s youth programs are working or achieving their intended objectives in your AoR? b) Have you seen changes in the youth in your AoR in the past 18 months? c) Can you attribute these to CSP youth activities in any way? If so, how? d) Please describe the different data sources and metrics that the PRT uses to track both the scope and the effects of CSP youth activities and programs in your AoR. e) In your view, are there particular strengths or weaknesses of CSP’s different youth interventions that makes them more or less effective at reducing incentives for youth to engage in violence in your AoR? f) What is the most effective youth activity CSP has implemented in your AoR to accomplish this objective? g) Going forward, what changes would you make to the CSP youth program based on the lessons learned in your AoR? 5) General a) Is there anything we haven’t covered that you would like to tell me about the way CSP’s youth activities may have helped to reduce incentives for violence in your community? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 29 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Civil Military Affairs (CMA) Interview Guide 1) Civil Military Affairs: a) How long at current assignment b) Parent agency (DOS, 3161, USAID, Mil, …) c) Invite a general rant about how the CMA works : i) Strengths and weaknesses of the CMA ii) Ability to engage with GOI counterparts (operations/logistics support) iii) What programs are operating that support the CMA (i.e., CSP is one among several, and may not be the dominant one) (if you wanted to do a program for youth, where would you look for support? CERP, IRAP, CSP, ICCM, CAP, LGP…) 2) CSP Program Overview (Fidelity) a) Describe how the CSP program fits in the overall strategy out here? Prompt: How does it dovetail with the CMA mission? b) Is the CSP best described as a development program or a COIN program? c) How familiar are you with CSP’s activities? d) What is CSP doing in your Area of Responsibility? e) Do you know how CSP activities are designed and selected in your AoR? f) Please describe the assessment methodology, if any, is used to inform the design of this program. [How did you come up with stuff for the CSP Youth Program to do?] g) Are you involved with the design or selection or oversight of CSP activities here? How? h) What do you think is the appropriate mix of programs to compliment an effective COIN strategy in your area? Please explain your answer. i) What mix of these programs has been implemented in your area? j) How does youth programming in your AoR differ from that in other cities/areas, if at all? If there are youth activities: k) What are CSP youth activities in your AoR intended to achieve in terms of specific results? l) How involved have you been in the selection, design or oversight of these activities? m) Have the objectives and/or desired results changed over time? If so, why and how? n) Are there different components of the youth program that are designed/tailored to achieve specific results within the overall strategy? (such as Life skills for certain youth and soccer/sports for other groups based on their profile or some other factor) 3) Functionality a) Do you know if CSP supports local councils to implement youth activities? Can you describe this and give examples? b) Do you know if CSP collaborates with the Ministry of Youth and Sports? Can you describe this and give examples? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 30 Report on Community Stabilization Program: An Examination of the Youth Engagement Program c) Do you know if CSP supports local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities? Can you describe this and give examples? d) Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? e) Do you know if CSP’s youth activities had an impact on the programs run by other community organizations? Can you provide an example? Why do you think that was? Prompt: Are you aware of any activities that were started by CSP that have been taken up by the municipality, Ministry of Youth and Sports, or other local organization? 4) Impacts a) How do you know if CSP’s youth programs are working or achieving their intended objectives in your AoR? b) Have there been changes in the youth in your AoR in the past 18 months? Explain. c) Can you attribute these to CSP youth activities in any way? If so, how? d) Please describe the different data sources and metrics that the CMA uses to track both the scope and the effects of CSP youth activities and programs in your AoR. e) In your view, are there particular strengths or weaknesses of CSP’s different youth interventions that makes them more or less effective at reducing incentives for youth to engage in violence in your AoR? f) What is the most effective youth activity CSP has implemented in your AoR to accomplish this objective? g) Going forward, what changes would you make to the CSP youth program based on the lessons learned in your AoR? 5) General a) Is there anything we haven’t covered that you would like to tell me about the way CSP’s youth activities may have helped to reduce incentives for violence in your community? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 31 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Focus Group Discussion Guide: EGY Staff The Program Environment 1) Can you describe what a typical neighborhood in your community looked like 18- months to 2 years ago? PROMPTS: • Can you describe how families socialize in the community? Were they able to socialize like 18 months ago? • Many neighborhoods are said to have had militia or security groups; can you describe the situation in your community? • Can you describe what the militia and/or local security groups did and how their presence affected young people’s lives? • Did young people in neighborhoods join these groups? • Do you have any idea how and why young people join these groups? • Compared to now, was there more violence 18 months ago than there is now? If yes: Can you describe how things have changed? If yes: Can you give an example of how violence has changed; what can the community do now that they could not do before? 2) Can you describe what might be the incentives for young people to become involved in violence in your community? 3) In your own words, please describe what life was like for young people in your community 18-months to two years ago? By young people I am including high school aged boys and girls as well as young men and women in their late teens, 20s and early 30s. PROMPTS: • How wide were the circles in which they socialized? Were they restricted to family members only or did they include other people? Who? • What prevented young people from socializing? • Describe how young people ventured beyond their neighborhoods. Can you give an example? • Did they feel safe in their neighborhoods? Were they remaining in their home and not going outside? Can you give an example? • What sporting, cultural and/or other group activities did young people do to socialize and enjoy life? How has this changed over the last two years? Program Fidelity: Testing for Correct Knowledge about the Program. 1) What is the purpose of CSP’s youth activities overall? 2) Specifically in your city/area? 3) What do CSP youth activities intend to achieve in terms of specific results? 4) Have the objectives and/or desired results changed over time? If so, why and how? 5) Are there different components of the youth program that are designed/tailored to achieve specific results within the overall strategy? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 32 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Program Design and Implementation 1) How did this office develop its youth program and activities? 2) What was the involvement of local Iraqi staff in the selection of activities? 3) Please describe the assessment methodology used, if any, to inform the design of this program. 4) What role does the IZ program directors play in the design and implementation of your youth activities? 5) Please describe how well your office’s youth program matches the vision and strategy presented by IRD’s Youth Program administration and senior program management. 6) Please describe your office’s youth portfolio in detail, including the different types of activities, their relative participation and funding levels, and their relative accomplishments. 7) How does your youth programming differ from that in other cities/areas, if at all? 8) How do CSP EGY staff share lessons learned? 9) Can you provide an example of a best practice you shared or learned that was implemented? Functionality 1) Please describe the ways in which CSP collaborates with the local council, and community groups and leaders. Can you give an example? 2) Did CSP support local councils to implement youth activities? Can you describe this and give examples? 3) Please describe the ways in which CSP collaborates with the Ministry of Youth and Sports. Can you give an example? 4) Did CSP support local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities? Can you give examples? 5) Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? Impacts 1) How do you know if your office’s youth programs are working or achieving their intended objectives? 2) Please describe the different data sources and metrics that your office uses to track the effects of your youth activities and program. 3) What records does your office keep in regards to projects and participants? 4) In your view, are there particular strengths or weaknesses of your office’s different youth interventions that makes them more or less effective at reducing incentives for youth to engage in violence? 5) What is the most effective program activity your office has implemented to accomplish this objective? 6) Have CSP’s youth activities had an impact on the programs run by other community organizations? Can you provide an example? Why do you think that was? Prompt: Are you aware of any activities that were started by CSP that have been taken up by the municipality, Ministry of Youth and Sports, or other local organization? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 33 Report on Community Stabilization Program: An Examination of the Youth Engagement Program 7) Going forward, what changes would you make to the youth program based on the lessons you have learned? General 1) Is there anything we haven’t covered that you would like to tell me about the way CSP’s youth activities may have helped to reduce incentives for violence in your community? Focus Group Discussion Guide: Iraqi Stakeholders Introduction Welcome and thank you for agreeing to participate in today’s interview/focus group. Over the next 3 weeks, we will be performing interviews and focus groups like this in several locations across Iraq. The purpose of these interviews/focus groups is to provide data for a study of CSP’s youth activities. Specifically, we are interested in understanding how the program functioned and the impacts it had, if any, on the attitudes of its young participants and the Iraqi stakeholders who helped implement the events and activities. Why the program worked on may not have worked are of equal importance. By “youth activities” we are referring to all activities that CSP performed with young persons aged 12-35 years old -- with the exception of vocational training courses, apprenticeships, the provision of business grants and business training. The interview should last approximately one to one-and-a-half hours. It is designed to look at certain subject areas. Each subject area will begin with me asking a general question and then some more specific questions. I will be asking the questions and guiding the discussion while my colleague takes notes. We will also be taping the conversation so that my colleague and I can listen to it again to refresh our memories when writing our summary. It is your thoughts, opinions and experiences we are interested in, so please attempt to give full answers and use examples of things that happened to help explain your answers. Let’s get started: The Program Environment 1) Can you describe what a typical neighborhood in your community looked like 18- months to 2 years ago? PROMPTS • Can you describe how families socialize in the community? Were they able to socialize 18 months ago? • Many neighborhoods are said to have had militia or security groups; can you describe the situation in your community? • Can you describe what the militia and/or local security groups did and how their presence affected young people’s lives? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 34 Report on Community Stabilization Program: An Examination of the Youth Engagement Program • Did young people in neighborhoods join these groups? • Do you have any idea how and why young people join these groups? • Compared to now, was there more violence 18 months ago than there is now? If yes: Can you describe how things have changed? If yes: Can you give an example of how violence has changed; what can the community do now that they could not do before? 2) Can you describe what might be the incentives for young people to become involved in violent in your community? 3) In your own words, please describe what life was like for young people in your community 18-months to two years ago? By young people I am including high school aged boys and girls as well as young men and women in their late teens, 20s and early 30s. PROMPTS: • How wide were the circles in which they socialized? Were they restricted to family members only or did they include other people? Who? • What prevented young people from socializing? • Describe how young people ventured beyond their neighborhoods. Can you give an example? • Did they feel safe in their neighborhoods? Were they remaining in their home and not going outside? Can you give an example? • What sporting, cultural and/or other group activities did young people do to socialize and enjoy life? How has this changed over the last two years? Program Fidelity: Testing for Correct Knowledge about the Program. 1) Please tell me what you know about the CSP Youth Activities program. PROMPTS: • What is the purpose of CSP’s youth activities? • What do CSP youth activities intend to achieve? • How are they organized? • How are they financed? Functionality 1) Please describe the ways in which CSP collaborates with the local council, and community groups and leaders. Can you give an example? 2) Cid CSP support local councils to implement youth activities? Can you describe this and give examples? 3) Please describe the ways in which CSP collaborates with the Ministry of Youth and Sports. Can you give an example? 4) Did CSP support local NGOs, CSOs (Civil Society Organizations) and other community-based groups to implement youth activities? Can you give examples? 5) Which community organizations, i.e., the local council, Ministry of Youth and Sports, local CSOs, etc., worked the most with CSP on youth activities and which worked less with CSP? Can you apportion a percentage to each? 6) Can you please describe the things about the way CSP organized its youth activities that were good and the things that were not so good. 7) Please describe CSP’s relationship with you. Is it based on friendship, family or tribal connection with someone at CSP, purely professional, or something else? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 35 Report on Community Stabilization Program: An Examination of the Youth Engagement Program 8) Are you aware of attempts by CSP to keep records of participant attendance and retention, and track the development of participants over time in certain activities? Can you provide examples? Activities 1) Can you please describe the types of youth activities that CSP has held in your community? 2) How did CSP select activity participants? 3) Approximately how many people participated in the program at any one time? What events led to higher or lower levels of participation (e.g., Iraq winning the Asia Cup; local security incidents)? 4) Was participation voluntary? 5) If participation was voluntary: Why do you think youth participated? 6) Did parents encourage their children to participate? 7) Was participation primarily from one group in the community? Which group? Why? 8) What do you think the participants got out of the activity/event? 9) To the best of your knowledge, did any of these events/activities include efforts to dissuade or discourage young people from becoming involved in violence – either directly or indirectly? If yes, do you think they were they successful? Why? Did you observe or hear of anything that makes you believe this? 10) To the best of your knowledge, did any of these events/activities include efforts to bring together young people from different groups in your community? If yes, how did the young people benefit? Did you observe positive changes in participants’ behavior? Can you provide an example? Impacts 1) What impacts, if any, did CSP’s youth activities have on your neighborhoods and community and, in particular, the lives of the young people who participated in events and activities? Example? PROMPT: Describe improvements to young people’s quality of life? How? Example? 2) Describe how the youth activities directly or indirectly affected the levels of violence among youth in your community? How? Example? 3) If violence levels lowered: Was CSP the only program to lower violence in your community? If not, how can you be sure it affected the levels of violence and this was not due to the other programs and activities in your community? 4) Describe how youth activities helped to improve or worsen relations between youth from different groups in your community? How? Example? 5) If responses indicate CSP was effective ask: Why do you think CSP’s youth activities achieved these positive impacts? 6) If violence levels were not reduced: Describe why the programs did not affect levels of youth participation in violence? 7) Do you know of any instances where CSP’s youth activities have led to young participants initiating similar activities in their community? 8) Describe any instances where youth may have formed their own organizations in the community? If instances are offered: Is this in any way attributable to CSP? If yes: How? Internatio Monitoring and Evaluation Per nal Business & Technical Consultants, Inc. formance Project, Phase II (MEPP II) 36 Report on Community Stabilization Program: An Examination of the Youth Engagement Program 9) Have CSP’s youth activities had an impact on the programs you or other community organizations have subsequently run? Can you provide an example? Why do you think that was? International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 37 Report on Community Stabilization Program: An Examination of the Youth Engagement Program ANNEX II. IRD AND USAID RESPONSES TO REPORT RECOMMENDATIONS Recommendation Implementer (IRD) Response Action to be taken In your response, indicate whether you Accept, Partially Accept or Reject the recommendation and provide a brief explanation Action Timeline Follow￾Up/Status USAID Comments 1 Develop indicators that show a clear link between program interventions and resulting changes in youth behavior and attitudes. Currently, there is no indicator required in the approved M&E plan to show this link. Indicator IR 7.2.2, titled: Percentage of Youth Participants Who Indicate a Positive Change in Their Attitude Towards Conflict was included in the August 2008 approved PMP. However, USAID directed IRD to delete this indicator in the February 2009 PMP revision and update, and later approved the CSP PMP, with this specific indicator having been deleted as requested. Indicator 7.2.2 was defined as the “percentage of youth who, when interviewed, indicate a more positive attitude towards other ethnic or religious groups, and a more negative view towards conflict and violence disaggregated by target area.” CSP accepts this recommendation for Year 4 PMP indicators if USAID agrees. Assume this would require a Modification to the CoAg. CSP requests immediate feedback from USAID whether to include again indicator IR 7.2.2 in the Year 4 PMP. Pending action by USAID. USAID will consider this for future initiatives. The early termination notice to IRD for CSP delivered was on July 24, 2009. The youth component as a "directly implemented" activity was terminated immediately with that notice. 2 Conduct research that examines the relationship between the change in behavior and attitudes among youth with those of adults in their communities. Such research would involve a broad spectrum of Iraq’s communities. CSP does not have the capacity to conduct such research. Execution would need to be by an independent research firm. CSP rejects this recommendation. No action by CSP. Not applicable Not applicable . While this suggestion was rejected by CSP, this is good information for future USAID initiatives. The need for good baseline assessment with follow￾up is documented in other IBTCI MEPPII CSP studies. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 38 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Recommendation Implementer (IRD) Response Action to be taken In your response, indicate whether you Accept, Partially Accept or Reject the recommendation and provide a brief explanation Action Timeline Follow￾Up/Status USAID Comments 3 Employ a more comprehensive and systematic M&E of activities that continuously tracks participants over time and includes observation and interviews with spectators, audiences and parents. CSP references response outlined in Recommendation No. 1: CSP further adds that a more comprehensive and systematic M&E of activities that continuously tracks participants over time and includes observation and interviews with spectators, audiences and parents can be a part of the indicator identified in Recommendation No. 1. CSP partially accepts this recommendation. Action required by USAID. We advise immediate Pending action by USAID. USAID will consider this for future initiatives. The early termination notice to IRD for CSP was delivered on July 24, 2009. The youth component as a "directly implemented" activity was terminated immediately with that notice. 4 Continue to take advantage of national events to enhance the impact of programs, e.g., when Iraq won the Asia Cup, CSP rolled-out soccer tournaments that capitalized on nationwide enthusiasm. CSP accepts this recommendation although notes that USAID has requested a reduced focus on large sports events for EGY youth activities in Year 3 and 4 with the goal of enhancing and increasing the number of conflict mitigation seminars and peace-building training events for youth. CSP Chief of Party and EGY National Director will advise CSP cities of appropriate events in order to enhance the impact of programs. July 26, 2009 DCOP will follow-up. USAID will consider this for future initiatives. The early termination notice to IRD for CSP was delivered on July 24, 2009. The youth component as a "directly implemented" activity was terminated immediately with that notice. 5 Pay attention to ensuring the selection of beneficiaries matches the program rationale The selection of youth program beneficiaries is based on program design that targets youth in conflict and post￾conflict areas. In the CSP cities, at-risk youth register and enroll in CSP EGY youth programs based on their preferences. Additionally CSP does not directly select EGY youth beneficiaries. That is normally done by the GOI Directorate of Youth and Sports representatives in each location or by Not applicable. Not applicable. Not applicable . While this suggestion was rejected by CSP, this is good information for future USAID projects. International Business & Technical Consultants, Inc. Monitoring and Evaluation Performance Project, Phase II (MEPP II) 39 Report on Community Stabilization Program: An Examination of the Youth Engagement Program Recommendation Implementer (IRD) Response Action to be taken In your response, indicate whether you Accept, Partially Accept or Reject the recommendation and provide a brief explanation Action Timeline Follow￾Up/Status USAID Comments the Iraqi NGO implementing partner. CSP can advise GOI and implementing partner on program rationale, but cannot directly select the youth beneficiaries. CSP rejects this recommendation.