February 2016 This publication was produced at the request of the United States Agency for International Development. It was pre￾pared independently by Checchi and Company Consulting, Inc. MID-TERM EVALUATION Favorable Opportunities to Reinforce Self-Advancement for Today’s Youth (FORSATY) MID-TERM EVALUATION: Favorable Opportunities to Reinforce Self￾Advancement for Today’s Youth (FORSATY) This report was contracted under USAID Task Number: AID-608-M-15-00002. Assignment Title: Mid-term Evaluation: Favorable Opportunities to Reinforce Self￾Advancement for Today’s Youth (FORSATY) Team Leader: Philip Boyle Team Members: Ettibari Bouasla Mhammed Abderebbi Activity Start Date: September 8, 2015 Activity Completion Date: February 29, 2016 Checchi and Company Consulting, Inc. Rabat, Morocco DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i TABLE OF CONTENTS Executive Summary..........................................................................................................................................iii Introduction .................................................................................................................................................... 1 Methodology ................................................................................................................................................... 2 Findings........................................................................................................................................................... 5 CVE APPROACH AND THEORY OF CHANGE ...................................................................................... 5 PROGRESS TOWARD ACHIEVING INTENDED RESULTS....................................................................... 6 BENEFICIARY AND STAKEHOLDER PERCEPTIONS OF THE MOST SIGNIFICANT CHANGES ACHIEVED ................................................................................................................................................................13 MOST AND LEAST EFFECTIVE PROJECT COMPONENTS ........................................................................19 Most Effective Components ...................................................................................................................19 Effectiveness To Be Determined .............................................................................................................21 Least Effective Components ...................................................................................................................22 PROGRESS TOWARD CSO CAPACITY BUILDING ..................................................................................23 CSO Capacity Building Results................................................................................................................24 Capacity Strengthening Aspects for Sustainability and Replication..............................................................25 OSSs and Institutional Action Plans.........................................................................................................27 A Sustainable System of Mentoring and Cooperation between OSSs.........................................................29 EFFECTIVENESS OF PROJECT MOBILIZATION AND FOSTERING OF RELATIONSHIPS BETWEEN RELEVANT ACTORS ..........................................................................................................29 Overall Conclusions.......................................................................................................................................32 Recommendations .........................................................................................................................................33 Annexes........................................................................................................................................................36 ANNEX A: EVALUATION STATEMENT OF WORK..................................................................................37 ANNEX B: PERSONS INTERVIEWED...................................................................................................49 ANNEX C: KEY DOCUMENTS REVIEWED ..........................................................................................52 ANNEX D: BENEFICIARY SURVEY QUESTIONNAIRE .......................................................................54 ANNEX E: FOCUS GROUP AND INTERVIEW GUIDES.......................................................................69 ANNEX F: FORSATY MID-TERM EVALUATION MATRIX....................................................................73 ANNEX G: FORSATY PHASE I and PHASE II RESULTS, BY SEX .....................................................75 ANNEX H: RESULTS OF FOCUS GROUP DISCUSSIONS..................................................................81 ANNEX I: KEY SURVEY RESULTS AND CROSS-TABULATIONS BY SEX AND LOCATION............87 ii Acronyms AFAK A Moroccan Non-formal Education and Vocational Training Project ANAPEC Agence Nationale de Promotion de l’Emploi et des Compétences CSO Civil Society Organization CV Curriculum Vitae CVE Countering Violent Extremism FGD Focus Group Discussion FORSATY Favorable Opportunities to Reinforce Self-advancement for Today’s Youth FY Fiscal Year GOM Government of Morocco INAS Institut National de l’Action Sociale IO Intermediate Objective IOM International Organization for Migration IP IR ISIS Insertion Professionnelle (Employment Placement) Intermediate Result Islamic State of Iraq and the Levant IYF International Youth Foundation LS Life Skills M&E Monitoring and Evaluation MELP Monitoring, Evaluation, and Learning Plan MNE Ministry of National Education (MEN in French) NFE Non-Formal Education NGO Non-Governmental Organization OCA Organizational Capacity Assessment (tool) OFPPT OD Office de la Formation Professionnelle et de la Promotion du Travail Organizational Development OS Organizational Strengthening OSS One-Stop-Shop (association) PTA Association des Parents d’Elèves PYA Participatory Youth Assessment SFCG SO Search for Common Ground Strategic Objective SOP Standard Operating Procedures TOT Training of Trainers UNFM Union des Femmes Marocaines USAID United States Agency for International Development WORDE World Organization for Resource Development and Education iii EXECUTIVE SUMMARY PROJECT BACKGROUND The FORSATY project was designed in 2012 to reach disaffected youth aged 12 to 25 at risk of socioec￾onomic marginalization in deprived neighborhoods of Tangier and Tétouan in northern Morocco. By im￾proving opportunities for youth in education, employment, and community involvement, FORSATY’s goal is to help this population avoid delinquency and become productive citizens. An allied objective is to integrate disenfranchised and marginalized youth into their communities as productive citizens who no longer offer a recruitment target for violent extremists. The International Organization for Migration (IOM) was selected to implement the project. It began with a two-year Phase I (October 2012 – September 2014), and in the first year carried out an extensive Participatory Youth Assessment upon which the drivers of violent extremism in each of the selected neighborhoods were identified and Phase I pilot activities were designed. Phase II, which began in October 2014 and will last three years, maintains many of the academic support and employment preparedness objectives of Phase I but adds a new youth mobilization and advocacy component implemented in part by Search for Common Ground (SFCG). In Phase I FORSATY implemented project activities through One￾Stop-Shop (OSS) neighborhood associations in four underserved neighborhoods (two in each city) and early in Phase II expanded into three new neighborhoods, each with its own OSS. This mid-term evaluation covers the performance of FORSATY from the start of the project (October 2012) through the latest period of data availability (September 2015). At project mid-term, the evaluation objectives are to: (1) measure progress against plans and expectations, (2) describe factors of relative success and failure, (3) provide guidance on sustainability and replication in other areas, and (4) provide recommendations that USAID and IOM can use to improve project processes and outcomes. FORSATY clearly targets youth at-risk of radicalization from economic and social marginalization. The project theory of change involves the mitigation of the following drivers of radicalization and potential extremist recruitment: (1) social isolation, reduced by improving academic success and school retention leading to higher levels of education, maturity and improved employment options; (2) poverty, economic frustration, and social marginalization, mitigated by improving access to employment; (3) youth coping strategies, maturity, and judgment, improved by building internal assets through Life Skills training and extracurricular activities; and (4) neighborhood quality-of-life and security, improved by youth mobilization and involvement in youth-oriented organization, advocacy, and recreational activities. METHODOLOGY Nearly 700 persons were consulted through various research tools during the mid-term evaluation of FORSATY, as shown by the table below. The evaluation was carried out by a team consisting Team Leader Dr. Philip Boyle, At-risk Youth Expert Dr. Ettibari Bouasla, and Survey Specialist Mhammed Abderebbi, assisted by nine surveyors. Following USAID’s Evaluation Policy, the team gathered evidence from various sources, so that conclusions and associated recommendations would be supported by strong evidence. Data collection methods consisted of a document review, a beneficiary survey of 512 youth in three of the four original neighborhoods, 15 focus group discussions with youth and parents (five in each neighborhood), and over 60 key informant interviews with project implementers, partners, and other stakeholders. iv Table 1: Individuals Consulted by FORSATY Mid-term Evaluation Team Data Collection Tool Males Females Total Quantitative Survey 277 235 512 Focus Groups 38 84 122 Interviews 39 26 65 All Methods 354 (51%) 345 (49%) 699 (100%) The focus group discussions and beneficiary survey relied on random selection of respondents from lists of participants in the major activities carried out by the OSSs. These included beneficiaries of academic tutorials, extracurricular activities, Life Skills training, summer camps, vocational training, internships, work readiness skills and job placement assistance, and vocational orientation and counseling. At least 150 par￾ticipants were administered the quantitative survey on the premises of each of the three OSSs selected: Raouabit Assadaka, Bir Chifae, and Al Amal. Focus groups were conducted at the same time with youth and parents, usually separated by sex. The survey, focus group discussions, and key informant interviews were conducted in Tangier and Tétouan over a period of three weeks in October 2015. KEY FINDINGS Progress toward Intended Results FORSATY target objectives and results were achieved in Phase I and are on track to achievement by end of Phase II (2017). These include the three major objectives: (1) Marginalized and disaffected out-of-school youth are successfully integrated into society; (2) In-school youth receive improved support to increase retention and success and decrease dropout; and (3) Marginalized and disaffected youth enjoy a safer community environment. FORSATY performance indicators report 11,825 youth beneficiaries (7,623 males, 4,202 females) in Phase I and 12,860 youth beneficiaries (8,022 males, 4,838 females) in Phase II. The Phase II figure includes 2,907 in-school youth benefitting from extracurricular activities which is no longer an indicator. Sex-disaggrega￾tion of this last group of beneficiaries is an estimate. Major academic and vocational results since the project began include 1,682 out-of-school youth (999 males, 683 females) placed in project-supported vocational training, 1,482 out-of-school youth (1,017 males, 465 females) employed or placed in internships, 4,453 in-school youth (2,415 males, 2038 females) received academic tutoring, and 5,310 in-school youth (3,821 males, 1,489 females) engaged in extracur￾ricular activities in schools or local OSSs. What is crucially important is to keep students in formal educa￾tion long enough to qualify for higher levels of vocational training, such as those offered by the Office de la Formation Professionnelle et de la Promotion du Travail (OFPPT). Other accomplishments to date include: 3,726 youth (2,425 males, 1,301 females) participated in innova￾tive awareness campaigns around relevant youth issues in Phase I, 1,333 youth (816 males, 517 females) engaged in work readiness and Life Skills activities (Phase II), and 2,299 youth (1,487 males, 812 females) participated in training center vocational orientation or job placement services (Phase II). The Life Skills component was included in school extracurricular activities and as part of preparing youth for internships and employment. It received general acclaim by teachers, educators, trainers, and private v sector employers. Teachers felt that Life Skills training affected their students positively. Ninety percent of teachers in a separate Life Skills evaluation declared that there had been a significant change in their relationship with students, and that students had become notably more confident in interactions. Teachers noted behavioral changes consistent with improved self-confidence, ability to listen, and interaction with others. With respect to performance in school, beneficiaries claimed to be more disciplined in listening, better at time management, more skilled in setting objectives, more attentive and receptive in class, and generally more capable in school. In terms of rendering local communities safer, 704 youth (689 males, 15 females) participated in Commu￾nity Dynamics (CD), FORSATY’s new initiative in collaboration with smaller neighborhood associations. Among the participants were eight youth who were convinced not to join ISIL in Syria, as reported by the FORSATY CD manager, although the evaluation team could not corroborate this fact independently. The CD component only began in April 2015, but accomplishments reported to date are promising. Since this component had just gotten under way, these project participants were not included in the evaluation. Thus far, IOM has implemented these activities directly with second-tier associations in project neighbor￾hoods, rather than through the OSS partners, given the complex relationship-building needed for the CD component to be established correctly. FORSATY has affected the lives of thousands of youth in terms of improvements in professional skills, schooling, knowledge, and civility, and this appears to have led to an improving security environment. With more time spent in learning and participating in the classroom and on the playground, such youth interac￾tion has resulted in an increased feeling of confidence, solidarity, and cohesion among youth of the same neighborhood. Evidence from the focus group discussions points to a decrease in criminality among youth resulting from awareness raising activities by OSSs. Beneficiary Perceptions of Most Significant Changes Focus group discussions and the youth beneficiary survey showed strong youth appreciation of project activities. This is very important to confirm at the midterm when there is still time to modify the content of activities if beneficiaries indicate dissatisfaction or inappropriateness. There was no significant difference in appreciation by gender reflected in the survey or the focus group discussions. Among academically tutored youth, 88% said their grades improved, and 91% felt personally improved by the tutorial experi￾ence. Some 96% of youth participating in extracurricular activities and 95% of summer camp participants felt personally improved by the experience. Focus group discussions revealed that personal improvement meant greater self-confidence, greater academic confidence, and a sense of greater social connection. This sense of improvement also extended to their neighborhoods: nearly three-quarters (72%) of youth sur￾veyed indicated that the quality of life in their neighborhood had improved under the FORSATY project. With respect to participation in internships, 72% reported being paid for their internship and 57% said they were offered employment in the firm following the internship. Virtually all those in internships (98%) felt their internship experience was what they had been seeking. In respect to vocational training, 97% of vocational trainees surveyed claimed to be personally improved and 60% felt this assistance package had helped them find employment. Regarding job counseling provided by the OSS associations, 86% felt better organized in their job search, 76% of youth got interviews as a result, and 59% of these youth landed a job. Most and Least Effective Project Components In the opinion of the evaluators, vocational training and job counseling is the most effective component of the project, as it mobilizes marginalized and disaffected youth for quick integration into the labor market. vi Criteria for this assessment include the number of beneficiaries involved and the linkage to factors related to youth joblessness, frustration, and social marginalization. Academic support has also been highly effec￾tive. Vocational training and academic support were also the two original strategic objectives, along with OSS capacity building. The success of academic support has resulted in a reportedly sharp decrease in school dropout, with parents strongly appreciating the improved academic results of their children that keep them in school. The overall rate of academic improvement following academic tutorials for 2015 is 76% in primary schools and 65% in middle schools. Focus group discussions with parents indicate that mothers in particular have become more involved in the follow-up of their children with the OSS association and the school admin￾istration. There has been a significant drop in student failure at school and increased learning capacity, according to education officials. According to both education officials and OSS staff, extracurricular activities supported through FORSATY are also contributing to sustained school retention, performance in the classroom, and personality devel￾opment. Life Skills training implemented through the International Youth Foundation (IYF) to youth in and out of school has also been particularly popular and successful. Youth, teachers, and employers all attested to its positive impact on disadvantaged youth. In particular, local education authorities and school princi￾pals consider Life Skills as very useful in strengthening young people’s personality, making them more expressive, letting them feel freer, opening them up to be more participatory in class and more considerate of others in outlook and practice with peers. With regard to workplace impact, on the whole young employees or interns furnished through FORSATY obey the rules in the work place, accept intermediation, and keep their jobs. Company officials directly ascribe this positive workplace attitude to the Life Skills and work readiness training received in the OSS associations. The human resource personnel interviewed in LARINOR were quite emphatic in comparing the superior personal qualities of interns arriving from OSS Chifae compared to those from the govern￾mental program OFPPT. One beneficiary from a Chifae focus group stated: “I changed a lot. Now I am working in LARINOR, I have a good salary (3,000 dirhams per month), I feel happy, I now help my family, I am ambitious, and I want to keep my job, to marry and have a family.” Evidence from focus group discussions with out-of-school youth shows that employed and vocationally qualified youth enjoy enhanced esteem and respect in their neighborhoods. This is socially important for youth whose very marginalization stems from low education attainment and resulting unemployment. Because of its recent start-up, the Community Dynamics component cannot yet be evaluated with respect to other components. As a new element, its potential for reaching large numbers of at-risk youth in poor neighborhoods that are not being reached by the vocational and academic components is promising. The most recent FORSATY interim report (April-June 2015) indicated poor results with bringing school dropouts aged 15 and older to the desired level of academic certification. Indicator data for FY 2015 indicate that only 35 of this group successfully completed the NFE program. Non-formal education (NFE) in general, for either age group, appears to be least promising in terms of impact for the project going forward, although FORSATY has plans to strengthen this component in partnership with AFAK, a non￾formal education and vocational training project of the government. Progress toward the CSO Capacity Building Intended Result A final project objective was to strengthen neighborhood OSS associations to provide better quality ser￾vices and render them sustainable by the end of FORSATY. Strengthening of the four original OSS partners vii has occurred since 2013, but it only became systematized in November 2014 with the use of an Organi￾zational Capacity Assessment tool (OCA). A set of institutional action plans was devised for these four OSSs in February 2015. About 50% of the organizational development targets with these associations were reached by June 2015 when the process was discontinued for the summer. It will be reactivated in November 2015 after an assessment of sustained progress. Weaknesses in OSSs exist in governance,1 general administration, human resource management, financial management, organizational management, and program management. The three OSSs added to the project under Phase II (Aide et Secours, Hay Benkirane, and UNFM) are expected to be administered the organ￾izational self-assessment tool in November 2015. This overall Organizational Development Action Plan is ambitious but achievable. These actions are clearly well chosen and comprehensive, but quite numerous for some OSSs. Even in the absence of a repeated OCA or use of other assessment tool, it is clear that since the imple￾mentation of FORSATY, OSS service delivery capacity has strengthened in quantity and quality. In spite of some delay in the training of OSS educators, service provision in and through OSSs to at-risk youth has clearly increased beyond anything these organizations had done in this regard prior to FORSATY. Focusing project services for youth through neighborhood OSSs and placing them as intermediaries be￾tween the outside donor and local public and private institutions is a major innovation, the sustainability of which requires that OSSs continue to be proactive in developing institutional relationships well beyond their own neighborhoods. Capacity Strengthening Aspects for Sustainability and Replication Stronger vocational training and orientation and capacity is the key to prevent delinquency among at-risk￾youth and induce change in personality and attitudes. The strong dependency of OSSs on external funding is a threat to their sustainability. All FORSATY OSS implementing partners are heavily dependent on financial resources from various donors, without any sustained action to diversify their sources of funding for core project activities. OSS Perception of Institutional Action Plans The four OSSs engaged in institutional strengthening measures based on self-assessment and IOM exper￾tise deem these measures to be necessary and useful to their future strength and sustainability. While there is time remaining in the project to implement the remaining measures, inevitable changes in key personnel are likely to weaken their sustainability. The question of key personnel change is likely to be exacerbated by reduced financial support following the end of FORSATY. Mentoring and Cooperation between OSS Associations A sustainable process of mentoring and cooperation between OSS partners to share and transfer knowledge, skills, and expertise between them has not yet been devised and will need to be promoted by the IOM institutional development team. The focus until now has been on addressing the various institu￾tional weaknesses exhibited by each OSS. Depending on the OSS, some of these are numerous and sub￾stantial. 1 Governance refers to the overall supervision of policies and policy implementation carried out by OSS administrators. viii Effectiveness of Relationship Mobilization Efforts FORSATY has been successful in terms of mobilization of public, private, and civil society institutions and actors. The project has brought together OSS partners and a comprehensive set of public institutions. It has mobilized educators, teachers, school directors, and regional education officials. FORSATY has fos￾tered links between private sector enterprises and OSSs to receive vocational trainees as interns. All the components of the project have generated strong attendance by at-risk youth and follow-up by parents. According to the head of NFE at the Education Delegation in Tangier: “FORSATY is a success on peda￾gogical and cultural levels. It has helped to curb dropout and school failure, freed pupils from the routine of classical learning and developed their potential of creativity and mutual understanding.” Focusing project services through neighborhood OSSs and placing them as intermediaries between the outside donor and local public and private institutions is a major innovation, the sustainability of which requires that OSSs continue to be proactive in developing institutional relationships well beyond their own neighborhoods. FORSATY outreach and involvement of relevant entities and actors has been impressive, but it has kept its focus on the OSS model, with one association in each neighborhood that can establish local relation￾ships with schools, governmental vocational training facilities, private sector employers, and job placement entities. In regard to the Community Dynamics component, OSSs are currently actively being strength￾ened in establishing linkages with “second-tier” associations. Each OSS has a full-time staff member dedi￾cated to the community component and full management is being transferred to the OSSs this fiscal year. Linkages have been forged between these two levels of community organizations, even if the process was constructed with direct control by IOM. This community dynamics experience, if it is to be sustainable, must be incorporated within the overall implementation model. Reduced competition and a new comple￾mentarity between the OSSs and the second-tier associations is now a reality, according to FORSATY. OVERALL CONCLUSIONS • FORSATY is on schedule to reach its 2017 youth support objectives and has generally surpassed its annual targets. • Focusing project services for youth through neighborhood OSSs and placing them as intermediaries between the outside donor and local public and private institutions is a major innovation under FOR￾SATY. • The FORSATY project has produced an institutional dynamic that has brought together OSSs and public institutions, such as the Ministry of Education and Entraide Nationale, in joint action to address the issue of at-risk youth rehabilitation. The programming of educational support, Life Skills clubs, extracurricular activities, and non-formal education has mobilized educators, teachers, school princi￾pals, and regional governmental officials. • FORSATY has fostered linkages between private enterprises and OSSs throughout the whole process of rehabilitation of at-risk youth. • The project has so far mobilized beneficiaries and their families and created a strong link with the OSSs. • While FORSATY appears to be on track to reach its OSS capacity-strengthening objectives by 2017, about 50% of strengthening outcomes remain to be accomplished. A sustainable system of current service delivery beyond FORSATY is not yet in place. ix • More successful project components include vocational training and academic support, as well as Life Skills activities with youth in and out of school. These key activities have produced large numbers of beneficiaries and represent the core FORSATY activities from project beginning. • Follow-up information on vocational trainees indicates a strong commitment to work and social sta￾bility among these beneficiaries in large part due to Life Skills and work readiness training alongside technical training. This is reintegration into society of disenfranchised and marginalized youth in the fullest sense. • The overall rate of academic improvement in primary schools following academic tutorials for 2015 is 76%, compared to 65% in middle schools. • The success of academic support has resulted in an improvement in student grades and examination success with a concomitant drop in school abandonment, and strong parental appreciation of the improved academic results of their children. Focus group discussions with parents indicate that mothers in particular have become more involved in the follow-up of their children with the OSS association and the school administration. • Extracurricular activities supported through FORSATY are also contributing to sustained school retention, performance in the classroom, and personality development. This is attested to by OSS staff and school principals. • Life Skills training for out-of-school beneficiaries has been extremely beneficial to their success in internships and future employment. On the whole, young employees or interns obey the rules in the work place, accept intermediation, and keep their jobs. Company officials directly ascribe this superior attitude compared to other interns to the Life Skills and work readiness training received in the OSS associations. • Non-formal education, which has produced relatively few successful cases of reintegration, has been less successful to date, although it is now being strengthened through a new partnership with the government. • The community dynamics component is only now getting under way. It shows promise, particularly from a youth governance and CVE perspective, but neighborhood activities undertaken by IOM and city-level council activity by SFCG had not yet been well coordinated by time of evaluation. Coordi￾nation with neighborhood youth committees is expected to be established as the city-level councils become active. For this reason, this component must be evaluated at a later date. • Personnel costs are overly high with about 100 staff now supported in the seven OSSs, in addition to the 22 in the IOM project office in Tangier. There are also 72 youth leaders supported by stipends when actively engaged in neighborhood activities. • Organizational sustainability is possible by 2017, but FORSATY is unlikely to achieve financial sus￾tainability by project end under foreseeable circumstances. KEY RECOMMENDATIONS • Current activities are appropriate and should continue, but cost containment should be stressed. The project has less than two years to render OSS activities financially sustainable. • Gender issues in FORSATY neighborhoods are being addressed, but should be given increased importance, since gender roles still restrain women’s access to project activities and employment opportunities. Fathers should be actively mobilized to support project activities. Stronger project staff outreach to fathers will be necessary to involve more men in their children’s welfare. x • A follow-up survey of youth satisfaction has not been conducted since a baseline was derived from the PYA and a target proposed for end of Year 2. • The Livelihoods Assessment conducted by Mercy Corps should be reviewed for its pertinence to employment and self-employment opportunities in project neighborhoods, especially the new ones under Phase II. • The three new OSS associations need to begin the process of organizational assessment and strength￾ening. • A new Results Framework that includes the grassroots push of the community dynamics component needs to be adopted with indicators that represent feasible and measurable achievement outcomes. • A new project monitoring tool (MELP) should include clearly defined outcomes with indicators for the community dynamics component. To the extent possible, measures of CVE-drivers should be included as performance indicators. • The new Results Framework and MELP currently proposed represent the third versions in three years, a lot of shifting of objectives and related performance indicators, but FORSATY feels that the new framework simply realigns a large number of inputs and outputs to eliminate overlap and improve legibility. These modifications reflect experience in implementation over the last few years and can thus be considered an example of experience-based adaptive management. • Since two-thirds of schools do not have an adequate space to conduct Life Skills sessions and teachers are not paid for these extra hours, FORSATY should explore means to promote Life Skills club activities within OSS premises. • Inclusion of Life Skills into teachers’ normal workloads would help sustain that effort within schools. The project staff should explore how such educational policy change could be encouraged. • Current potential for vocational training in OSSs needs to be upgraded with a focus on those job profiles much in demand in the regional labor market. In the case of the Tangier-Tétouan region this means more workshops for sewing, car wiring, tourism, and catering. FORSATY indicates this has been an orientation under FY 2015, but it should be intensified. • Partnerships with private sector employers should continue to be pursued aggressively with consid￾erable potential still available in Tangier. • Job placement of trainees through private employment agencies should be explored. Many of the larger private sector employers do not wish to deal with neighborhood associations. Although FOR￾SATY indicates it has held exploratory meetings with some private sector placement agencies, this thrust should be pursued more aggressively moving forward. • The potential for public-private partnerships to sustain OSS activities after project end should be seriously explored since FORSATY ends in less than two more years. • In view of strong stakeholder appreciation, if costs can be contained and sustainable sources of funding secured, FORSATY may consider: • Increasing academic tutorials in beneficiary institutions, while including new school districts where school dropout is also high. • Supporting more tutors and more teaching hours, especially in math, French and physics. • Increasing Life Skills clubs in middle schools and providing adequate training to tutors. • Increasing the number of vocational training workshops, in order to double the capacity of OSSs in response to rising demand for training from both out-of-school youth and local employers. xi • Provide more support to non-formal education (NFE) in OSSs, in order to improve the rate of school reintegration of youth under fifteen, which is currently estimated at 35%. • The project can be replicated in other cities with much the same components, but these locations should show economic growth and increasing demand for skilled labor or employment results will not match those of Tangier. • In view of the likelihood of key staff turnover in OSS partners, FORSATY institutional strengthening staff should concentrate on measures documenting internal procedures and processes to truly institutionalize them. Those key procedures and processes most related to sustaining FORSATY activities would be the best investment in time and resources. • To achieve viable and sustainable OSSs after the withdrawal of FORSATY or for replication elsewhere in Morocco, organizational strengthening measures should focus on governance, human resource administration, and financial capacity. • Expanding the Organizational Development team may be a possible solution, with the goal of passing responsibilities for organizational assessment and future strengthening activities to the OSS partners by end of project, if not before. Currently this is projected to occur in 2016, but this may be a bit optimistic. • The prospect of self-financing for the OSSs lies far ahead. Aide et Secours charges fees for services, Al Amal manages a gymnasium, and Hay Benkirane charges for placement and raises money from small donors. Increasing cost coverage will require: • Administration of a general fee for services provided by the OSS with an exception for very poor beneficiaries. • Provision of a sustained grant from the State through project submissions or other special agreements. • Formation of property assets through endowments and acquisitions which will guarantee fixed revenues. • Financial sustainability of various levels of future OSS activities patterned on current FORSATY activities should be carefully examined within these organizations. This is a key deliverable of the FORSATY financial capacity expert and should be carried out as soon as possible, in order to launch the financial capacity and resources mobilization plan for the OSSs. • It is conceivable that a public policy designed to support OSS associations, such as the adoption of a mechanism of tax rebates for enterprises training and employing at-risk youth received directly from OSSs, could encourage private sector enterprises to provide more technical and material support to associations and help to professionalize their training activities. • Representatives from the LARINOR garment company expressed an interest in supporting the OSSs financially and technically given their reliance on a steady provision of partially trained labor to their internship programs. • FORSATY should promote a formal process of OSS mentoring and cooperation that should include the following: • Increase the stress placed on training and capacity building activities to enhance OSS upgrading and certification to international standards, which FORSATY feels is part of its current strategy. This will increase their ability to receive grants and to collaborate with each other on project development and implementation, enabling them to design projects that respond to key issues at neighborhood and city levels and mobilize stakeholders, including public institutions, to support them. xii • Increase the stress placed on development of OSS information sharing and communication capabilities, which FORSATY feels is part of its current strategy. This will create information sharing and strengthens the capacity to communicate with others on common issues relevant to their target population – at-risk youth. • Technical capacity building activities, such as technical workshops and seminars on thematic issues, all directed toward further replication in smaller neighborhood associations. 1 INTRODUCTION The FORSATY project was designed in 2012 to reach disaffected youth aged 12 to 25 at-risk of marginal￾ization in selected deprived neighborhoods of Tangier and Tétouan in Morocco. By improving opportuni￾ties for youth in education, employment, and involvement in the larger community, FORSATY’s ultimate goal is to help this target population avoid delinquency and contribute positively to the community. An allied objective is to integrate disenfranchised and marginalized youth into their communities as productive citizens who are no longer a recruitment target for violent extremists. The International Organization for Migration (IOM) was selected to implement the project and began work with a two-year Phase I (October 2012 – September 2014). Phase II, which began in October 2014, differs from Phase I in that it contains a new civic participation component implemented in part by Search for Common Ground (SFCG). IOM implements FORSATY in partnership with Moroccan public institutions and key neighborhood civil society organizations (CSOs) in an effort to promote sustainability of project accomplishments. FORSATY assessed each neighborhood and partner CSOs thoroughly, so that it can assist the CSO either by provid￾ing complementary services or referring it to the appropriate service provider. In Phase I, FORSATY operated its “one-stop-shop” approach through focal CSOs in four underserved neighborhoods (two in Tangier and two in Tétouan); early in Phase II, FORSATY expanded into three new neighborhoods, each with its One-Stop-Shop (OSS) focal association. The OSS CSOs each offered a similar package of in-school and out-of-school project activities. The eval￾uation focused on three CSOs and the three neighborhoods in which they are active: Raouabit Assadaka in Dchar Bendibane (Tangier), Association Chifae in Bir Chifae (Tangier), and Association Al Amal in Boujarrah (Tétouan). These three associations have been involved in the FORSATY project since 2012. They began implementing activities with beneficiaries in 2013, following the first project year in which an extensive Participatory Youth Assessment led to the selection and planning of activities. ATIL, a fourth CSO working across several neighborhoods in Tétouan, decided in 2015 to concentrate on policy advo￾cacy and opt out of direct service provision. ATIL continues to work with FORSATY and the Ministry of National Education (MNE) in the AFAK project with school dropouts 15 and over. Early in 2015, three new CSOs and their focal neighborhoods were added to the program: Aide et Secours in El Mers (Tangier), Association Benkirane in Hay Benkirane (Tangier), and UNFM in Jamaa Mzouak (Tétouan). While their staffs were interviewed during the evaluation, their responses are not as closely examined in this evaluation since their activities have only recently gotten under way. Although the evaluation team relied on indicator project data available through September 2015 in compiling the overall evaluation, the annual report for the entire 2014 - 2015 project year had not been completed at the time of evaluation drafting. EVALUATION PURPOSE AND SCOPE This mid-term evaluation covers the performance of FORSATY under Phases I and II from the start of the project (October 2012) through the latest period of data availability (June 2015). Following USAID’s Evaluation Policy, the evaluation team gathered evidence from various sources so that conclusions and associated recommendations are well-supported. The evaluation focuses on the following evaluation questions posed by USAID in the original task order: 1a. To what extent is the FORSATY project on target to achieving its intended results? i. Marginalized and disaffected out of school youth are successfully integrated into society. 2 ii. In school youth receive improved support to increase retention and success and decrease drop out. iii. Marginalized and disaffected youth enjoy a safer community environment (cross-cutting objective). 1b. What do targeted beneficiaries (including male and female at-risk youth) and other actors see as the most significant changes (positive or negative) to which the project has contributed, and what else is required? 1c. Which project components are the most and least effective in contributing to the project objectives and may have the most or least promise for the project going forward? 2a. To what extent is the FORSATY project on target to achieving the CSO capacity building intended result: youth service organizations are reinforced to provide better services for at risk youth/young of￾fenders? 2b. In order to achieve CSO capacities that are likely to be sustained beyond the life of the program, what aspects of CSO capacity strengthening should be adopted, adapted or replicated within existing regions and to new regions? 2c. How useful do participating CSOs perceive the institutional action plans and how could they be sup￾ported to improve them? 2d. What is required to establish a sustainable system of mentoring and cooperation between the partic￾ipating CSOs? 3. How effectively did the project mobilize and foster relationships between the different actors serving at-risk youth (including CSOs, government services and departments, the private sector, families and the youth themselves), over the course of the project? METHODOLOGY DATA COLLECTION METHODS This mid-term evaluation was carried out by a three person team consisting of a team leader, an at-risk youth expert, and a survey expert. The latter carried out the beneficiary survey with the assistance of one female research assistant and gender expert and eight more surveyors (six women and two men). Including the research assistant, nine surveyors worked in each of the three sites (OSS premises). The evaluation relied on a mixed-methods approach, combining a beneficiary survey with focus group discussions (FGDs) and semi-structured interviews with key individuals. The beneficiary survey was con￾ducted with youth in their neighborhoods and focused on their perceptions and experiences. Collection of information through FGDs and interviews was intended to complement survey data to provide trian￾gulation of findings from multiple data sources. The evaluation matrix, found in Annex F, shows which methods were used to answer each evaluation question. Results were compared, contrasted, and dis￾cussed among evaluation team members prior to report writing. 3 Gender of Evaluation Respondents Data Collection Tool Males Females Total Quantitative Survey 277 235 512 Focus Groups 38 84 122 Interviews 39 26 65 All Methods 354 (51%) 345 (49%) 699 (100%) Survey: The beneficiary survey relied on random selection of respondents from lists of beneficiaries of the most common types of project activities. The lists were used to select every “nth” person, with the interval determined by the size of the beneficiary pool for each activity. With the help of the OSS, the team attempted to contact selected beneficiaries to come for a face-to-face interview. In total, 750 bene￾ficiary names were selected, resulting in 512 interviews. The vast majority of selected but not interviewed beneficiaries were not interviewed because they could not be found by the address or phone number. Approximately 2% of those reached refused to participate. Survey participants were categorized in two groups: those still in school and those that had dropped out, since program activities were generally different for each. Beneficiaries were asked specific questions based on their participation in activities such as academic support, vocational training, life skills training, work readiness activities, and summer camps. Some beneficiaries benefitted from more than one project-sup￾ported service and were thus asked questions about each activity. The English version of the survey is found in Annex D and the focus group discussion guide is found in Annex E. The beneficiary survey obtained responses from 196 beneficiaries in Bendibane, 166 in Bir Chifae, and 150 in Boujarrah. The goal had been to obtain at least 150 valid responses from each of these neighborhoods. The yield got increasingly close to the target as the teams learned how many beneficiaries were necessary to invite to achieve the target. Focus Group Discussions (FGDs): The OSS associations were instructed to randomly select from the various participant lists a small number of discussants who were not included in the survey. The FGDs provided a means to interpret the survey results, since many of the questions were similar. FGDs also provided a means to gather information on a variety of activities that participants engaged in under FOR￾SATY but which had fewer participants than the eight activities covered by the survey. The focus groups allowed greater in-depth exploration of beneficiary perceptions of their situation before and following the services received from FORSATY. In addition to their perception of personal impact, respondents were asked about the effects on the wider community. Discussants also offered their ap￾praisal of the quality of project services and their relevance to their own personal needs and expectations.. FGDs were organized with in-school and out-of-school youth separately since they received distinct ser￾vices, with the exception of life skills training. FGDs with in-school youth combined boys and girls in mixed groups (as they are mixed in school), but discussions with out-of-school male and female youth were conducted separately. In addition, mothers and fathers of both in-school and out-of-school beneficiaries were asked to attend, but in all cases only mothers attended. This was due to local cultural reasons, since men in these neighborhoods do not generally take responsibility for child care, including their children’s education. These four basic FGDs were conducted in each of the three neighborhoods for a total of 12 FGDs. 4 Key Informant Interviews: The team interviewed a total of 65 key informants in single or small group interviews (39 males and 26 females). The interviewer asked semi-structured questions that focused on themes of participation, organization, and coordination relating to the key evaluation questions (see the evaluation matrix, Annex F). For those organizations and institutions receiving organizational strengthen￾ing assistance through FORSATY, an assessment of those activities and their impact was also explored. The Interview Guide is found in Annex E. The partner and stakeholder organizations interviewed consisted of: • IOM FORSATY management and various component teams in Tangier and Tétouan, • Seven One-Stop-Shop CSO organization managers and key personnel, • School directors, teachers, and project educators, • International Youth Foundation (IYF), • Search for Common Ground (SFCG), • Regional education officials, • Entraide Nationale (EN) officials, • ANAPEC, • OFPPT, and • Various private sector partners. Document Review: The key documents reviewed were the original project document, the Participatory Youth Assessment (2013), quarterly reports from FY 2013 - 2015, the Annual Report for FY 2014, and the project extension document (2014). Thus, this evaluation has been informed by the data contained in the FORSATY quarterly reports covering the period from October 2012 through June 2015. The team also reviewed an excellent Livelihoods Assessment conducted by Mercy Corps. The original Performance Monitoring Plan for Phase 1 activities (2013-2014) was replaced by a Monitoring, Evaluation, and Learning Plan (MELP) for the Phase II program extension. Although a new results framework is proposed for the next two years (2015-17), this evaluation has focused on program results described by the Phase II MELP, dated April 2015. DATA ANALYSIS Survey: The survey was processed in SPSS and response frequencies were calculated for each question. Since there was no baseline data for beneficiaries, the results reflect the range of current perceptions of respondents, as well as their reports of change over time. Cross-tabulations have been carried out to explore relationships among different respondent types and particular responses by sex, neighborhood, type of service received, and CSO implementing organization (see Annex I for key results including se￾lected cross-tabulations). The few open responses were reviewed, including the answers to the final open response question “Is there anything else you wish to share”, which was asked of all respondents. Re￾sponses to that question can be found in Annex I. FGDs and Interviews: In addition to notes taken by the analyst or an assistant, FGDs were sometimes recorded in order for the analyst to refresh his or her memory of the sessions. The recording was made known to all participants and none objected. Quotes for key themes noted in the FGD guide were sum￾marized by the analyst for inclusion in the evaluation report. Interviews were not recorded, but extensive notes were taken for each key question of the interview guide. A software program such as NVIVO or Atlas.ti was not deemed necessary to analyze themes and key issues, given the manageable number of discussions. 5 LIMITATIONS The beneficiary survey was conducted with randomly selected participants from lists of participants in major FORSATY activities. The ability to find these participants and bring them in for interviews defined who responded to the survey. Beneficiary lists from FY 2014 were only available for one of the three associations. In addition, it was easier to locate participants from FY 2015, due to change in address or phone number. Therefore, the survey results are more representative of 2015 participants than partici￾pants from both years. Since there was no baseline survey against which to compare current survey data, all that could be done was to examine beneficiaries’ perceptions of their previous and current participation in the major project activities. This is a limitation that could have been avoided with a proper baseline survey. The selection of focus group discussants was done as randomly as possible from the same OSS participant lists. FGDs involved between four and 15 discussants. Compared to a survey, the results of FGDs are more influenced by individual participant views. However, the semi-structured nature of FGDs can lead to information not previously suspected, constituting an important complement to survey data. Time constraints in the data collection period meant that the survey, FGDs, and interviews were carried out at the same time in the field. Consequently, survey data could not be used as a reference point for further probing through qualitative methods. FINDINGS This section begins with findings related to the FORSATY theory of change, after which findings are pre￾sented in order of the key evaluation questions, with other emergent themes also highlighted. CVE APPROACH AND THEORY OF CHANGE A number of risk factors can lead to radicalization of youth: ideological factors (beliefs and values), psy￾chological factors, economic factors, sociological motivators, and political grievances (WORDE, pp.1-4). These can act independently or in concert. While only a small minority of disaffected and frustrated youth can be expected to embrace violent extremism, the northern region of Tangier-Tétouan has already been linked to this phenomenon. For historical reasons, this area has remained very conservative socially and is reported to have supplied around one-third of Moroccan recruits to the war in Syria and Iraq (Cem￾brero). The FORSATY project seeks to address several push and pull factors potentially leading to ideological or political radicalization of youth. Primary among these are push factors of poverty and rootlessness from residence in peri-urban, immigrant neighborhoods and social marginalization resulting from school drop￾out with insufficient formal education to qualify for decent or even any employment (WORDE, pp. 1-4). Pull factors of local extremist leaders and ideologues, violent extremist websites, the appeal of a sense of importance, belonging, or mission all help to explain how marginalized youth, particularly young men, can be mobilized into radical activities. Project Theory of Change: Among other objectives, the FORSATY project seeks to mitigate youth grievances and hopelessness by linking them to educational and economic opportunities, as well as helping them to connect positively to their communities and local institutions. Youth so benefitting should be 6 much less likely to turn to extremist messaging and local radicals leaders. FORSATY features the following strategies: (1) Social isolation is mitigated by improving academic success and retaining youth in school leading to higher levels of education, maturity, and judgment, as well as improved employment options. (2) Poverty, economic frustration, and social marginalization are mitigated by improving access to em￾ployment. (3) Youth coping strategies, maturity, and judgment are improved by building internal assets through Life Skills training and extracurricular activities, including summer camps. (4) Neighborhood quality-of-life and security are improved by youth mobilization and involvement in youth-oriented activities (sports, arts, other recreational outlets) as part of the community dynamics component. PROGRESS TOWARD ACHIEVING INTENDED RESULTS Evaluation Question: To what extent is the FORSATY project on target to achieving its intended results? Under Phase I (October 2012 - 2014), FORSATY identified three Intermediate Results plus a Cross￾cutting Result under an overall Program Objective of preventing delinquency and reducing recidivism. These results were: IR 1: Marginalized and disaffected out-of-school youth are successfully integrated into society. IR 2: In-school youth receive improved support to increase retention, success, and decrease drop-out. IR 3: Youth-serving organizations provide better services for at-risk youth. CCR: Marginalized and disaffected youth enjoy a safer community environment. Under Phase II (October 2014 – September 2017), FORSATY developed a Monitoring, Evaluation and Learning Plan (MELP) that included three Intermediate Objectives plus two Cross-cutting Supporting Ob￾jectives under a broad Program Objective that sought to improve the social and economic integration of marginalized youth. The overarching goal was to ensure that marginalized youth reject violent extremist messaging and actors. These objectives were: IO 1: Augmented youth assets. IO 2: Improved Implementation of policies that benefit marginalized youth by elected officials. IO 3: Improved youth access to jobs. SO 4.1: Youth-serving organization capacities to provide adequate services to youth at risk improved. SO 4.2: Marginalized youth’s safety and security at the community level improved. In spite of differences in terminology, the basic Phase II activities remained the same, except that IO 2 brought in SFCG as a subcontractor to establish youth councils and train youth leaders and local officials to effect increased communication and understanding between youth and local authorities. Under IO 2, IOM is working more with second-tier neighborhood associations that will eventually lead to neighbor￾hood youth committees that can be linked to city councils formed by SFCG. Examining the reported indicator achievements and comparing them to previously set targets is essential to answering whether FORSATY at the mid-term is on target to achieving its intended results. The per￾formance tracking data for Phases I and II are presented in Tables 1 and 2 in Annex G. In Table 2 the FORSATY project has provided its most recent indicator values through September 2015. 7 FORSATY Phase I Phase I included an indicator for the Overall Objective of decreased delinquency among the at-risk youth population, but results are not presented in the relevant annual report. Instead, the Program Objective has been raised to the level of a Goal and is not tracked by any indicator. The FORSATY baseline data presented in Annex G refer to results obtained in the first year of project activity (October 2012 – September 2013), when the major activity was the Participatory Youth Assessment. Data gathered in this evaluation related more to 2014-2015 activities, although some beneficiaries had taken part in similar activities the previous year as well. IR 1: Out-of-School Youth are Successfully Integrated into Society:2 Achievements surpassed targets for the first two years of project activity, except in the case of youth returning to school after dropout. Throughout the project this result has been more difficult to achieve than expected. Youth 15 and over cannot return to public school by law, but they can be given a certification of achievement and then integrated into the job market. FORSATY is currently collaborating with the MNE in the AFAK project designed to address the 15+ issue. Youth under 15 can rejoin school, but the number successfully reintegrated in Phase I was small (27 of a target of 60). With the exception of the non-formal education component, the project reached most of its target indicators for out-of-school youth over the course of Phase I. IR 2: In-School Youth Receive Support to Increase Retention and Success and Decrease Drop-out: Targets for in-school youth were achieved, according to the Phase I report, in all cases. In most cases, they were largely surpassed. Notably, the project does not report on retention and dropout rates - only participation and support activities. Therefore, neither the project nor this evaluation team can quantitatively confirm the link between dropout and project activities. IR 3: Youth-Serving Organizations Provide Better Services for At-Risk Youth: It appears that a value could not be assigned to the baseline or succeeding indicators for the quality of youth services. This is not surprising since the services are varied and disparate and no easy tool exists for their meas￾urement. Apparently no follow-up survey of youth satisfaction was conducted, although a baseline value was derived from the PYA and a target proposed for end of Year 2. On the other hand, the number of youth-serving professionals trained surpassed its target value. While difficult to say based on these indica￾tors, it is likely the project was on target to achieving a higher quality of youth services. Cross-Cutting IR: Marginalized and Disaffected Youth Enjoy a Safer Community Environ￾ment: Apparently the first indicator on the percentage of youth demonstrating a better understanding of relevant youth issues identified during the PYA could not be given a value, because no follow-up survey was conducted as planned. On the other hand, indicator values largely surpassed target expectations. The project was largely on target to achieving its cross-cutting neighborhood quality/security objective, at least as measured by its five indicators. Unfortunately, none of these indicators directly addresses the issue of neighborhood safety. The FORSATY Phase I project results were summarized and published in the FY 2014 annual report, but not disaggregated by gender. Disaggregation of these results by gender, as presented in Annex G, is a rough estimate based on information collected from the field and from project reports. During Phase I, FORSATY was not yet able to establish a solid M&E system with partner OSS associations. Capacity 2 FORSATY Interim Report to USAID October 2013 to September 2014. Cumulative achievements reflect the total Phase I period from October 2012 through September 2014. 8 strengthening under the project was focused on collecting basic field performance data through improved basic reporting skills. FORSATY Phase II FORSATY did not track the Program Goal of enabling youth to reject violent extremist messaging and actors with an indicator. The Program Objective of improving youth’s social and economic integration is tracked by one indicator: “percentage of marginalized youth that are productively engaged after exposure to the project.” Its value will be reported on only once at the end of the project. In the Phase II results framework, indicators are attached to the three supporting outcomes under IO 1, the two outcomes under IO 2, and the two outcomes for IO 3. These indicators all track the outcomes and objectives that constitute FORSATY’s intended results. The tables for FORSATY Phase II results, presented in Annex G, have been provided by the project and reflect achievements through FY 2015 (September 30, 2015). They also contain helpful explanatory com￾ments on each of the indicators. The Case of Objective 2: Youth Participation in Local Governance Increased Phase II’s new Intermediate Objective 2, “Elected Officials Implement Policies that Benefit Marginalized Youth,” was similar to the old Cross-cutting Support Objective 4.2, except that the latter appears to focus on safety and security only, while IO 2 encompasses youth advocacy from below and elected official actions from above. Moreover, IO 2 was originally seen to be largely the responsibility of SFCG, an International NGO with considerable experience in mobilizing youth in deprived neighborhoods in Morocco and form￾ing youth councils, among other activities, in urban neighborhoods. Under Phase II, the Cross-cutting Support Objective 4.2 includes a package of activities that are designed to increase youth participation in public affairs, improve security services, and build community resistance to drugs and clandestine migration. The CVE objective of reducing susceptibility to ISIS recruitment mes￾sages is also part of these “community improvement” actions. At the same time, FORSATY chose to revive its earlier Community Dynamics component (2013) under IO 2, convinced that most at-risk youth continue to “fall under the radar and are not sufficiently reached by One-Stop-Shops.” As the FY 2015 Quarter 3 report indicates, FORSATY “recruited additional resources to reinforce its actions at the grass￾roots and developed with OSS partners a comprehensive approach to mobilizing youth and readying them for inclusion.”3 These Community Dynamics activities all fit under Objective 4.2 and represent a desire to broaden the impact of OSS activities. In particular, this will allow FORSATY to reach a large number of youth most at-risk of falling prey to social marginalization, frustration, and possible extremist recruitment. Until now these youth have been out of reach of OSS activities. OSS associations are not yet able to produce the volume of qualified youth to fill the 7,000 good jobs that local authorities estimate will be created in the project area in the coming years. Consequently, OSS associations will need to build linkages with smaller neighborhood associations to reach these youth sustainably. Over the last six months, the new CD component, according to the component manager for FORSATY, has reintegrated 241 youth into new life paths, including 26 in school, 109 in vocational training programs, 51 into professional sports, and 15 into music (traditional bands). More importantly, 20 youth were pulled away from illegal migration, 20 stopped using drugs heavily, and eight were convinced not to join ISIS in Syria. At least 52 of the 109 youth oriented toward vocational training programs benefitted also from OSS 3 FORSATY. 2015. “Interim Report to USAID: April 1 – June 30, 2015.” Submitted August 14, 2015. 9 professional counseling activities. These data were supplied late to evaluators and could not be corrobo￾rated with other sources. Currently IOM implements Community Dynamics activities directly with associations. However, IOM re￾ports that in the next year (FY 2016) they will transfer activities over to the OSSs. According to their plans, by the last project year (FY 2017), the OSSs will come to manage their community programs inde￾pendently of FORSATY. The new vision reported in the FY 2015 Quarter 3 report for IO 2 (Component 2) is a set of comple￾mentary phases in communities to promote youth personal growth and safety: • Mobilizing and organizing large groups of youth through various community-based activities. • Engaging neighborhood associations in attracting and organizing youth through a continuous program of community-based activities. • Establishing neighborhood-based youth committees to advocate and influence youth policies from local decision makers. • Connecting local authorities, elected officials and other decision makers with established youth committees at the neighborhood level and youth councils at the city level. To reflect this expansion of Component 2 activities, a new results framework with four strategic objectives has recently been submitted to USAID. In addition to former objectives focused on OSS performance and sustainability, academic retention and extracurricular development, and skills development and employ￾ment, Objective 1 now focuses on youth personal growth and sense of neighborhood belonging. This directly reflects the activities under the FORSATY Community Dynamics component now getting under￾way. According to FORSATY, this new results framework was not devised to reflect an expansion of Com￾ponent 2. It realigns a great number of inputs and outputs to eliminate overlap and improve clarity. Since there is overlap between results sought under Intermediate Objective 2 and Support Objective 4.2 in the current MELP, the new results framework seeks to merge what is sought under each objective and reword them. FORSATY feels strongly that while the approach has evolved in implementing SO 4.2, activities all fit into the original scope of work. The FORSATY project has used OSS associations to play a key role in mobilizing youth, their parents, and various civic and institutional actors. A sustained program of services has been provided to two distinct youth groups, those in school and those out of school, with the following outcomes. Marginalized and Disaffected Out-of-School Youth are Successfully Integrated into Society Examples from interviews conducted during fieldwork suggests that the number of marginalized youth in the region are growing due to early school dropout, lack of professional skills, and unemployment. This was pointed out by representatives of the two MNE delegations in Tangier and Tétouan. The FORSATY project has contributed to improving the chances of economic integration of out-of-school youth mobi￾lized within OSS associations for vocational training. Through FORSATY during FY 2015, 718 youth (271 males, 447 females) had benefited from vocational training in Tangier and in Tétouan, exceeding the pro￾jected target of 704. The length of vocational training varied between three and seven months and was followed by internships in local enterprises for advanced qualification. Garment manufacturing (cutting and sewing) seems to be the predominant vocational choice within these associations, given a strongly rising demand from clothing manufacturing companies. Traditionally practiced by women, as pointed out to the evaluation team by 10 various OSS personnel in Assadaka and Chifae and by human resource managers in LARINOR (a Tangier Free Zone company), clothing assembly and sewing is now preferred by young men due to the paid in￾ternships available (800 - 1,500 dirhams per month) and job opportunities, often in the Tangier Free Zone, that pay an average of 3,000 dirhams per month with social security coverage. Other training activities, such as electricity and car mechanics, painting and cookery, are chosen by other youth, who have the possibility subsequently of joining the enterprise or opting for self-employment. Through FORSATY, 1,006 youth (627 males, 379 females) are reported to have engaged in internships or been employed through September 2015 against a target of 690. Again, gender social roles influence the choice of internships. Male youth tend to pick electricity, house painting, vehicle mechanics, and – as explained above – increasingly garment manufacturing, while females opt generally for cooking, pastry, baking, and sewing. Vocational counseling has also contributed to linking out-of-school youth to the local labor market, with 1,333 (816 males, 517 females) of them exposed to Life Skills, work readiness, and job counseling against a target of 425. The provision of FORSATY Life Skills training to this age group has contributed to building individuals’ social values, leading to an improved sense of responsibility, participation, self-control, and ambition. This is attested to by a number of interested observers (teachers, project educators, school principals, benefi￾ciaries, and parents) in interviews and FGDs, as well as in evaluation survey responses and the Life Skills evaluation carried out by IYF. It is particularly reflected in rising self-esteem and declining violent behavior in the neighborhood and at work. In short, the young employed or intern groups obey the rules in the work place, accept intermediation, and keep their jobs. Company officials directly ascribe this superior attitude compared to other interns to the Life Skills and work readiness training received in the OSS associations. Evidence from one company (LARINOR) in the Tangier Free Zone revealed only two minor cases of conflict over two years. Youssef, 24 years old Instruction level: 2nd year in high school Vocational training: Sewing + Life Skills (Bir Chifae OSS) Current job: Works at LARINOR in the Tangier Free Zone Question: Please tell me, what direct results did you experience from these activities in the association? Answer: Since I joined the sewing workshop in the association, I first learned a new skill which I did not have be￾fore, and second I learned how to be patient and not to be a trouble maker. I feel now a sense of responsibility and respect for myself and others. Question: Do you feel that you permanently changed from these activities and in what ways? Answer: I changed a lot. Now I am working in LARINOR, I have a good salary (3,000 dirhams per month), I feel happy, I now help my family, I am ambitious and I want to keep my job, marry and have a family. Source: Focus group at OSS Bir Chifae, October 2015 In-school Youth Receive Improved Support for Retention and Success The local education authorities in Tangier and Tétouan are participating in FORSATY through the involve￾ment of two school districts in each city that include a major secondary school in each marginal district and four to five neighboring primary schools. Educational support, extracurricular activities and Life Skills sessions are provided to limited numbers of pupils aged between 10 and 15 years by tutors hired by the OSS associations and paid by the FORSATY project. In a few cases academic support activities are pro￾vided by teachers, in order to complete their weekly teaching load of 24 hours. A total of 52 (against a target of 75) educators in target schools have been trained by FORSATY partner IYF in the “Passport to 11 Success” Life Skills methodology. This series of modules creates a supportive learning environment that enhances youths’ results at school. Some 30 clubs are estimated to have used at least six of the Life Skills modules by the end of September 2015. With about 24 students per club, this means that up to 720 youth were exposed to these messages during the 2014-2015 school year. Young pupils in the last (6th) year of primary school and those in middle school with poor grades benefit from extensive school tutorial support, particularly in mathematics, French, Arabic, and physics. Through September 2015, a total of 1,779 in-school youth (916 males, 863 females) received academic support, which led to a sustained improvement in terms of higher grades and overall success. The overall rate of academic improvement in primary schools following academic tutorials for 2015 is 75% compared to 52% in middle schools, or an overall success rate of 64%. These figures are obtained from the OSSs that in turn obtain the data from the schools involved. No information was available to the evaluation team on non-participants and school principals in interviews were not able to give precise figures on the results of tutoring. According to informants, the success of academic support has resulted in a sharp decrease in school drop￾out. Parents strongly appreciate the improved academic results of their children that keep them in school. FGDs indicate that mothers in particular have become more involved in the follow-up of their children with the OSS association and the school administration. For local cultural reasons, mothers rather than fathers tend to engage in overseeing their children’s education. However, boys and girls appear to face similar challenges in the classroom. Extracurricular activities supported through FORSATY are also contributing to sustained school reten￾tion, performance in the classroom, and personality development. This has been attested to by youth in FGDs in each project neighborhood. In 2013-2014, 2,403 in-school youth were given the chance to play sports, paint, and learn theater, cinema and photography that materialized in many inter-school competi￾tions and exhibitions. In 2015, a total 2,907 youth4 participated in these activities, with further extracur￾ricular activities dealing with school violence provided to 1,239 youth (a program supervised by ATIL in Tétouan). Other security, health, and environment issues have been discussed in the monthly Youth Wednesday gatherings hosted by the OSS associations in their neighborhoods that result in increased youth interest in their school.5 Marginalized and Disaffected Youth Enjoy a Safer Community Environment In the districts of Bendibane, El Mers, Hay Benkirane in Tangier and Ancienne Medina, Boujarrah and Jamaa Mezouak in Tétouan, a community building process is underway that is aimed at improving safety. To date, FORSATY has affected the lives of 11,825 youth in Phase I and 12,860 in Phase II by endowing them with professional skills, academic performance assistance, extracurricular activities, career knowledge, Life Skills, work readiness, and employment orientation. According to surveyed beneficiaries and focus group discussants, the growing influence of “second-tier“ neighborhood associations among marginalized youth and the positive effect of FORSATY community dynamics activities are already leading to an improved security environment. With more time spent interacting in the classroom and on the playground, youth reported an increased feeling of confidence, solidarity, and cohesion among young people of the same neighborhood. This is further enhanced by activities undertaken by small neighborhood associations to mobilize other youth against violence, crime and extremism. The initiation by OSS associations of a debate 4 These beneficiaries are not reported under Phase II performance indicators. The figure was supplied directly to the evaluation team by the M&E manager in FORSATY. 5 Intermediate Result 2 in FORSATY Interim Report to USAID (October 2013 to September 2014) and Achievement Reports (bilans) of the OSS associations for 2014-2015. 12 on security with the participation of local police representatives has begun to ease tension between the communities and the police. Evidence from the FGDs points to a decrease in criminality among youth resulting from awareness raising activities by small neighborhood and OSS associations. Some responsive￾ness by the police to community concerns regarding street aggression and disorder is acknowledged by local youth in FGDs, although it is still considered insufficient to sustain a strong feeling of security within the whole community. Gender Approach While not one of its core objectives, FORSATY has striven to be equitable in inclusion of male and female staff members and beneficiary youth. This has not always been possible given the cultural constraints of the neighborhoods where FORSATY works. The number of staff supported by FORSATY in the project field offices and in the seven OSS partners is 87, plus 72 local association leaders (coaches) whose job it is to mobilize youth under the Community Dynamics component in the various neighborhoods. The coaches earn a small stipend but only when involved in activities with youth, such as sports tournaments. The table below summarizes the gender composition of these paid staff members. Gender of Paid Staff Organization Male Staff Female Staff Total IOM Tangier Office 6 8 14 IOM Tétouan Office 6 3 9 Raouabit Assadaka 8 4 12 Hay Benkirane 8 6 14 Chifae 6 2 8 Aide et Secours 6 4 10 Al Amal 5 10 15 UNFM 1 4 5 ATIL 0 0 0 Totals 46 (53%) 41 (47%) 87 (100%) Total Beneficiaries by Gender 6 Summing the number of beneficiaries of both phases suggests a total of 24,685 beneficiaries since the beginning of FORSATY; however, the total number of unique beneficiaries for Phases I and II is not avail￾able. Not all data have been disaggregated by sex and estimates have been made. The available data point to 15,645 male beneficiaries (62.4%) and 9,040 female beneficiaries (37.6%) reported since project initia￾tion. This gender breakout has not changed much between the two phases. Males were 64.5% of benefi￾ciaries under Phase I and 62.4% under Phase II. Specific figures are presented in the indicator tables for the two project phases (Annex G). According to final FY 2015 indicator data, the actual number of youth participating in community improve￾ment actions at the end of FY 2015 was 704: 689 males and 15 females. The coaches chosen by the small neighborhood associations probably account for this gender disparity as 69 are men and only three are women. The focus for now is on the key target population: young, unemployed, socially marginalized men, who are considered to be vulnerable to extremism. At the beginning, FORSATY staff felt that locals should 6 Phase I numbers are estimates based on partner reports. An indicator on in-school extracurricular participants is no longer included in performance indicators for Phase II but has been supplied by FORSATY to the evaluation team. 13 be free to choose their own leaders and coaches and the bias has definitely been in favor of male youth. However, FORSATY indicates that a target of 30% females for this group of local leaders has been agreed with the second-tier associations. In terms of indicators of youth-serving professionals trained, 362 (331 males, 31 females) were trained under Phase I and 52 (34 males, 18 females) Life Skills trainers and 113 OSS educators (82 males, 31 females) have been trained to date under Phase II. There is clearly a bias in favor of males among benefi￾ciaries, trainers, and local leaders/coaches that was more apparent in Phase I. Under Phase II the gap is narrowing. BENEFICIARY AND STAKEHOLDER PERCEPTIONS OF THE MOST SIGNIFI￾CANT CHANGES ACHIEVED Evaluation Question 1b: What do targeted beneficiaries (including male and female at-risk youth) and other actors see as the most significant changes (positive or negative) to which the project has con￾tributed, and what else is required? This section draws from the key results of the youth beneficiary survey, as well as relevant findings from the focus group discussions and key informant interviews with OSS personnel and stakeholders, including the MNE delegations, primary and secondary schools, Entraide Nationale, ANAPEC, and private sector firms hosting interns in Tangier and Tétouan. In general, these findings reveal a positive perception of FORSATY accomplishments. The data revealed few significant differences between male and female beneficiaries in terms of perceptions of the FORSATY activities in which they had participated (see Annex I containing key cross-tabulations by sex). No discernible difference existed among other male and female stakeholders either. Although inter￾views unearthed occasional complaints, such as wanting to spread the project to far more needy neigh￾borhoods and occasional lack of communication between entities (OSSs and schools) within FORSATY, these complaints were few compared to the general praise placed on FORSATY project activities of the last two years. Evidence from FGDs with youth and parents corroborates information collected through interviews with key stakeholders. Survey Sample Characteristics The beneficiary survey involved 512 youth participants in FORSATY activities over the previous two years in two neighborhoods of Tangier and one in Tétouan. Of the total, 196 youth were surveyed in Dchar Bendibane (Tangier), 166 in Bir Chifae (Tangier), and 150 in Boujarrah (Tétouan). Males made 54% of the sample, while females made 46%. In terms of age, 55% were aged 12 to 15, 25% aged 16 to 19, 14% aged 20 and 23, and 6% aged 24 or older. Some 69% of these 512 youth were born in their current neighbor￾hood, while another 16% had been born in another part of the same city (Tangier or Tétouan). Respondents were of two types: those in school (331) that could have benefitted at some point over the previous two years from academic support, extracurricular activities, and Life Skills training; and those out-of-school (181) that had benefitted at some point over the same period from vocational training, internships, work readiness training, job counseling, and Life Skills. Overall, 54% of respondents (56% of males and 51% of females) had not gone beyond primary school. There were slight differences by neighborhood surveyed: 57% in Dchar Bendibane, 55% in Bir Chifae, but only 47% of respondents in Boujarrah had not gone beyond the primary level. Not surprisingly, 93% of 14 youth (88% of males and 98% of females) still live with their family, with whom 75% (74% of males and 77% of females) claimed to have a very good relationship. While 17% (25% of males and 8% of females) claim to earn some money on their own, 83% (75% of males and 92% of females) state that they generate no income through their own activities. There were some differences by neighborhood: 25% of youth surveyed earned some income on their own in Bir Chifae, 15% in Boujarrah, and only 13% in Dchar Bendibane. In-school Youth Activities Academic Tutorials In-school youth receiving educational support generally expressed satisfaction with how the tutorials were carried out and reported measurable improvement in their academic performance. Similarly, the vast majority of mothers of youth reported being happy with their children’s achievements and recognized the benefit beyond the classroom. Mothers of youth attending Assadaka tutorials, for example, reported that their sons were more likely to be found at the OSS rather than playing in the street. In an Al Amal FGD, youth explained that their parents seemed to have more respect for them as they have become more serious about their studies, which improved family life as well as their grades. Representatives of the local education authorities and principals stressed the contribution of FORSATY’s academic support activities to a significant drop in student failure and increased learning capacity. Some 88% of in-school youth surveyed had been involved in academic support activities, with most of these (92%) receiving tutoring in mathematics and French, and a smaller percentage (69%) receiving tutoring in Arabic. About two-thirds (64%) of tutored youth received special tutoring focused on succeeding on the examinations at the end of primary or secondary school. A very large majority (94%) of tutored youth felt the tutoring had been successful and a slightly smaller number (88%) reported that their grades had improved as a direct result. There was virtually no difference between girls and boys in feeling their grades had improved. A large majority of beneficiaries agreed that they had personally changed for the better as a result of the academic assistance offered by the neighborhood OSSs through FORSATY: 60% fully agreed and 31% partially agreed with this appraisal. Only 9% disagreed to any degree. The most common complaints were about insufficient teachers (seven survey respondents volunteered this opinion) or that some teachers were not skilled or sometimes absent (six respondents). Some mothers participating in the Assadaka and Chifae FGDs also mentioned insufficient teachers as a problem. Both students and parents expressed a desire to see the OSSs offer more subjects (such as English, Spanish and science), and increase the amount of teachers and teaching time. There was almost no difference between male and female youth’s reports about personal change. Those who reported positive personal change most often reported that they felt more focused on education (78%), more confident (58%), or that they were on a better path to a career or higher education (19%). “The big achievement of FORSATY is that it has contributed to shape a more expressive, coura￾geous, and creative pupil at school.” Source: Primary school principal in Tangier 15 Extracurricular Activities In the survey and FGDs, both male and female in-school youth described positive experiences with the sports (football, basketball) and recreational activities (theater, cinema, photography, and painting) availa￾ble in school or at the OSS association, with few qualifications or complaints. Respondents talked about the importance of extracurricular activities as being rich in learning outcomes, enabling social interaction, and providing ways to render school a more attractive experience. Male youth and their parents were more likely to talk about extracurricular activities as an alternative to being “on the streets” while female youth were more likely to discuss the confidence-building effects. According to Imane, a female participant in a Dchar Bendibane FGD: “Through extracurricular activities in Assadaka association, I learned about cinema and film making, and together with my friends we have produced a short film. I also play football and would like for the association to set up a women’s football team, like the men’s team, so we can play in local competitions.” School officials also think highly of the extracurricular activities promoted by FORSATY in their schools. According to one middle school principal in Tangier: “FORSATY has improved extracurricular activities in Hassan Ibn Tabit College, we have a theater venue where youngsters learn and perform theater, we have sustained sports activities, and we or￾ganize cultural activities as well. The academic tutorial is successful with 82.5% grade improvement, and non-formal education has allowed 16 pupils to reintegrate the school track. FORSATY has also created a synergy of efforts by teachers, educators and administrators. Volunteering is developing in our institution as many teachers are involved in life skills, and extracurricular activities. They have even formed after-school clubs which are a success to which the association Bir Chifae is also con￾tributing. FORSATY is a good project that should continue to sustain such improvements in the fu￾ture.” Of the in-school beneficiaries surveyed, 61% had participated in extracurricular activities that are normally engaged in by youth receiving academic assistance. There was little difference in responses between girls and boys. Most (88%) reported engaging in educational and cultural games, followed by theater (86%), sports (84%), and Life Skills (81%). On the other hand, youth found the most valuable activity for them personally to be sports (37%), followed by theater (32%), with male youth showing more interest in sports and female youth more likely to prefer theater. Life Skills, which was popular according to FGD partici￾pants, was favored by only 13% of survey respondents. It is unclear why, and the survey result was not available to use in probing FGD participants in this regard. Virtually all of those engaging in extracurricular activities (96%) felt they had improved personally as a result of their participation. Of those reporting such improvement, 58% felt more organized in life, 50% were more optimistic, and 40% felt they had a better social life. Life Skills Clubs Life Skills clubs in schools are considered by beneficiaries as useful in terms of enhancing individual re￾sponsibility, self-confidence, self-respect and respect for others, goal setting and achievement, and problem solving. Representatives of local education authorities and school principals mentioned that Life Skills helped strengthen young people’s personalities, making them more expressive and opening them up to be more participatory in class and more democratic in outlook and practice with peers. 16 Summer Camps Only about one-third of in-school beneficiaries (37%) indicated that they had attended a summer camp during the previous two years. Of those attending, 76% engaged in sports and 75% in theater activities. More than half engaged in games (59%), academic strengthening (57%), and art and drawing (56%). Getting a head start academically for the next school year is one of the important objectives of the summer camps. Some 95% of those attending summer camp reported that they felt personally changed for the better by the experience. Of those claiming personal improvement, about two-thirds (67%) felt more organized in life, nearly half (47%) felt more optimistic, and 37% felt more outgoing as a result of their camp experience. Out-of-school Youth Internships To the extent that employment expectations were discussed in FGDs with out-of-school youth, females identified cultural barriers to employment opportunities, particularly parental control over their freedom of movement and visibility outside the home. This was also true of their participation in project extracur￾ricular activities offered in the OSSs or in schools. Females in FORSATY neighborhoods tend not to see themselves as having to be employed to establish a socially-acceptable identity among peers and neighbors. If women do earn income, they usually prefer to work from home, whether that of their parents or that of their husbands. This is why Al Amal in Tétouan prefers to train women in activities that they can carry out from home, such as sewing, cooking, and baking goods for sale. Exactly half of the out-of-school beneficiary respondents had participated in an internship over the pre￾ceding two years (56% of males and 40% of females). Of those in internships, 72% stated that the internship had been paid, with boys (74%) paid slightly more often than girls (69%). Overall, 78% claimed that they had been able to choose the internship they felt they wanted. Virtually all those in internships (98%) felt their internship experience was what they had been seeking. Somewhat over half (57%) had been offered employment following the internship with very little difference between males and females. The survey results correspond with interview data from private sector firms (e.g., LARINOR in Tangier) and FGDs with youth. Of the 90 youth respondents that had participated in internships, 90% felt changed for the better by the experience, with a small difference between boys (92%) and girls (86%). Of those that felt improved, 79% felt more skilled, 57% felt more confident, 49% were more optimistic, and 36% felt more organized in life. Life Skills and Work Readiness Activities Of the 181 out-of-school beneficiaries that responded to questions on Life Skills and work readiness training, only 85 (47%) indicated that they had received at some point such training in Life Skills and/or work readiness skills (job searching, CV preparation, interviewing). Of those receiving this assistance, most (79%) agreed that these activities had been useful, while the remainder felt that it had been somewhat useful. The IYF’s own evaluation of its Life Skills activities provides a good deal more information on beneficiary appreciation of this activity. 7 7 International Youth Foundation. « Evaluation du Programme Aptitudes de Vie: Programme Passport to Success. » FORSATY Project Education Component. Tangier and Tétouan. 2015. 17 Vocational Training Most of those receiving vocational training also received Life Skills (85%) or work readiness training (86%). Of those who did, 60% felt that receiving Life Skills and work readiness training during their vocational training helped them find employment. However, there was substantial difference by sex in responses, with 65% of boys compared to 49% of girls who felt that the training helped them to find employment. This may suggest some gender bias in employment following vocational training, although OSS personnel and representatives from the garment industry indicated that girls were still preferred. The Human Re￾sources officer at LARINOR in the Tangier Free Zone pointed directly to the Life Skills and other work readiness training provided to vocational trainees by Chifae Association that made them clearly superior to others from OFPPT. Respondents tended to mention the quality of their instructors and the availability of good equipment as important factors in their perception of vocational training usefulness. In Chifae, male FGD participants ranked both as excellent. In Al Amal, while many were still very satisfied with the training, some male youth voiced some dissatisfaction with the quality of instruction, as well as the insufficient work space and tools and difficulty travelling to the center. In the FGDs, about half of the female youth had reservations about the quality of instruction for the same reasons, although they were overall very happy with the opportunity that vocational training offered. According to mothers in Boujarrah, more parents were now sending their out-of-school girls to the Association and considered the idea of letting them work outside, instead of keeping them at home. Of those participating in vocational training, virtually all (97%) felt personally changed for the better by the experience. Of those feeling changed, 76% felt more skilled, 63% had more confidence, 59% felt more optimistic, and 30% felt more organized in life. The one survey respondent who explained why he did not feel improved by the training said that he only needed a job, not training. The most common suggestions offered by survey respondents in relation to vacationing training were to address shortages in equipment and increase the amount of training. Out-of-school youth tended to express the belief that vocational training was a great life achievement after abandoning school. Learning a professional skill is a source of pride and hope, as they no longer felt useless in their family and have good work opportunities. Evidence from FGDs shows that employed and vocationally qualified youth enjoy esteem and respect in their neighborhoods, and many of their friends are subsequently willing to join a vocational training program at the nearby OSS. Representatives of OSSs and Entraide Nationale stress the positive contributions of the FORSATY project to economic integration of out-of-school youth, since a majority of their trainees have been employed by local enterprises or are productively self-employed. Follow-up information on vocational trainees indicates a strong commitment to work and social stability among these beneficiaries in large part due to Life Skills and work readiness training alongside technical training. This is reintegration into society of disenfran￾chised and marginalized youth in the fullest sense. “There is simply no comparison between the intern trainees sent to us by Chifae Association and those coming from OFPPT. Those coming in from Chifae are clearly more disciplined, even tempered, rule following, and serious than those arriving from OFPPT. We very much prefer the Chifae interns.” Source: Head of Intern Training in LARINOR garment factory (Tangier Free Zone) 18 Vocational Counseling Of the out-of-school youth surveyed, 62% had received job counseling through OSS partners under FOR￾SATY, although this differed strongly by sex with 70% of boys and only 49% of girls reporting participation. About three-quarters of these (76%) had received interviews as a result of this counseling, 86% became more organized in their job search, and 59% actually felt this counseling had led them to employment. However, boys reported more success in securing employment as compared to girls (62% of boys; 53% of girls), revealing gender bias most likely related to cultural reasons. The FGDs did not reveal specific problems encountered by women compared to men (survey results were not available at the time). Dis￾cussants reported satisfaction with the opportunities opened up by vocational training. As in the case of those with vocational training, virtually all (96%) of those receiving vocational counseling felt changed for the better as a result of the counseling. Of those who reported feeling improved, 76% felt more skilled, 60% were more confident, 60% were more optimistic, and 37% felt more organized in life. Two respondents reported negative change following the vocational counseling experience. While clearly in the minority, they explained that they felt the training was either “not beneficial” or “gave them false promises.” Non Formal Education Of the 181 out-of-school beneficiaries, only 26 (14%) answered that they had participated in non-formal education. Twenty of these were aged 15 years or older, thus unable to return to school. The remaining six were under 15 and could be reintegrated into formal education activities. For those 15 or older, most (65%) continued in non-formal education activities and only two youth (10%) chose to discontinue the non-formal education work and were assisted to access vocational training. Five youth (25%) discontinued non-formal classes but were not assisted to enter into vocational training activities. Of the 26 youth engaged in non-formal education activities at some point in the preceding two years, 22 (85%) claimed to have successfully passed the examination they were preparing for, 23 (88%) were satisfied with the academic assistance they received, and 21 (81%) also received Life Skills training during their non￾formal education classes. Neighborhood Improvement Nearly three-quarters (72%) of youth surveyed indicated that the quality of life in their neighborhood had improved under the FORSATY project, while 27% indicated that there had been no change. While there was no difference by sex of respondent, there were some differences by neighborhood: 79% felt the OSS had raised the quality of life in Dchar Bendibane, 72% in Bir Chifae, but only 63% in Boujarrah. Over half (53%) of those perceiving improvement felt that youth were now more successful in their neighborhoods. Two problems were most commonly selected by respondents as those more affecting their quality of life: violence/crime and unemployment. Overall, 42% of respondents picked violence and crime as the most common problem affecting their quality of life, with no difference between male and female respondents. By neighborhood, differences were significant: 50% of respondents in Dchar Bendibane and 46% in Bir Chifae picked violence and crime, while only 27% did so in Boujarrah. On the other hand, unemployment was considered the most important problem for quality of life overall by 25% of males and 14% of females, reflecting the greater social pressure placed on male youth to be gainfully employed. By neighborhood, 26% in Boujarrah felt this to be the primary problem, while only 16% did so in both Bir Chifae and Dchar Bendibane. Other problems seen as most serious in their neighborhood were general poverty (9%) and 19 family problems (8%). Only 6% of respondents felt that lack of educational opportunities was their most serious problem. While only 7% of the 512 respondents felt unsafe or somewhat unsafe at home, 52% felt unsafe or some￾what unsafe in the street (60% of females and 45% of males), 38% felt some degree of insecurity in leisure places (cafes, restaurants), and 45% in public transportation. Only 10% felt some degree of insecurity existed in schools. While 44% felt unsafe in the street in Boujarrah, 56% felt unsafe in both Bir Chifae and Dchar Bendibane. Interestingly, 42% of surveyed youth felt that neighborhood security had improved over the previous two years (40% of youth in Bir Chifae, 45% in Dchar Bendibane, and 39% in Boujarrah). Overall, 48% felt security had not changed and only 10% of respondents felt that neighborhood security had worsened. MOST AND LEAST EFFECTIVE PROJECT COMPONENTS Evaluation Question 1c: Which project components are the most and least effective in contributing to the project objectives and may have the most or least promise for the project going forward? The selection of most and least effective components in FORSATY depended on the evaluation team’s appraisal of the number of beneficiaries reached, their susceptibility to drivers of extremism, and the effect of the intervention on those driving factors. The team feels that unemployment and resulting social mar￾ginalization (especially lack of income, peer respect, and hope) are major drivers of radicalization and are successfully addressed by FORSATY. Academic support is important for remaining in school long enough to qualify for advanced vocational training and to achieve greater social maturity. Here, too, large numbers of beneficiaries are being reached with the intended results. Non-formal education produces relatively few successful outcomes (see Annex G), and there is no reason to believe these youth are more at risk than other would-be dropouts or unemployed neighborhood youth, making the rationale for investing in this activity weaker. The prospects for the Community Dynamics component is too early to judge and has been classified as “To be determined.” It is certainly an intervention in which large numbers of at-risk youth can be reached through small neighborhood associations to give them outlets and lessen the frustration in their lives. Yet, this can really only be a temporary palliative. Ultimately, they will need vocational training and assistance to qualify for jobs. Most Effective Components Vocational Training and Job Counseling In the evaluators’ opinion, vocational training and job counseling is the most effective component of the project, as it targets marginalized and disaffected youth and mobilizes them for quick integration into the labor market. Vocational skills that require shorter training periods, such as clothing manufacture and car wiring, are most coveted by young men and women since they offer strong job opportunities. Other skills that offer both employment and self-employment opportunities are in mechanics, pastry, electricity, and hairstyling. Vocational training offers the most promise for project impact going forward. Decent employment or self-employment is key to integrating young men and women into their society. Those out of school without a sufficient level of education are generally condemned to marginality and lack of respect in their neighborhoods. Many turn to using drugs or delinquency of various types, including drug dealing. They are susceptible to recruitment by religious or political extremists. These consequences 20 are reported by focus group discussants, as well as opinions of FORSATY educators, OSS staff, school principals, and MNE delegation officers. While the survey solicited open responses from participants, very few respondents indicated no improvement from various project activities and no conclusions could be drawn based on a very limited number of brief open responses. Academic Tutorials Educational support is judged to be the second most effective FORSATY component. Results thus far are generally positive, with average performance improvement seen in 75% of primary school and 52% of middle school students, as reported to the OSSs by the schools involved. In addition, drop-out and repe￾tition rates are decreasing as a result. Unfortunately, dropout rates in FORSATY schools are not part of Phase II indicators, because of the difficulty found in Phase I in obtaining useful data and tracking benefi￾ciaries from one year to the next. Keeping youth in school at least through middle school is extremely important to their inclusion into society, as well as developing their overall judgment and maturity. The work with youth to prepare them for the 6th Grade and the 9th Grade completion examinations is a key focus of the academic tutorials under FORSATY. Beneficiaries assisted under this project component are even larger than for the em￾ployment-oriented activities. While the numbers of beneficiaries are larger in the in-school component, in the opinion of the evaluators most youth having serious academic difficulties are likely to end up in vocational training (see comment in Annex G). What is crucially important is to keep them in formal education long enough to qualify for higher levels of vocational training, such as those offered by OFPPT. Life Skills Activities Life Skills clubs in schools and training in employment-oriented activities is also an effective project com￾ponent, targeting at-risk youth to promote enhanced potential for mutual understanding, responsibility and general social inclusion. Through September 2015, FORSATY reports that 30 school clubs have im￾plemented at least six Life Skills modules. Each club normally contains 24 students. According to project reports, 1,333 youth in internships and vocational training also received Life Skills (and work readiness) training during 2014-2015. The result has been stronger commitment to work, workplace solidarity, and worker problem solving as strongly attested to by employers (e.g., LARINOR in Tangier). The Life Skills component included in school extracurricular activities and as part of preparing youth for internships and employment received general acclaim from teachers, educators, trainers, and private sec￾tor employers (e.g., LARINOR in the Tangier Duty Free Zone). A recent evaluation of this activity was carried out by IYF in 16 project schools between February and June 2015. Primary and secondary-level teachers had originally been trained for four days and then proceeded to use the modules in clubs con￾sisting of about 24 students. Of these teachers, 65% created Life Skills clubs and held at least one interac￾tive session each week. Responses from interviews with all teachers involved and from a sample of stu￾dents in four schools were evaluated for perceptions of appreciation, value, and impact. According to the IYF study, teachers almost uniformly felt the modules to be very useful and considered their greatest value to be the innovative, interactive, school-focused, yet recreational manner in which they are used with students. Teachers felt there had been not only definite impact on students, but also on their own behavior and relationship to students. Some 90% of teachers declared that there had been 21 a complete change in relationship with their students and that students had become notably more confi￾dent in dealing with them.8 Evaluation of responses from a sample of students revealed that students felt that impact was greatest in three skills: self-respect and respect for others; cooperation and teamwork; and decision making. Over three-quarters of students sampled also reported having changed in the other life skills: emotion management, critical judgment and problem solving, communication and interpersonal skills, personal responsibility, conflict resolution, and self-confidence. According to the IYF study, behavioral changes were attested to by discussants in self-confidence, ability to listen, and interaction with others. With respect to performance in school, discussants claimed to be more disciplined in their listening, better at time management, more skilled in setting objectives, more attentive and receptive in class, and generally more capable in school. The discussants also agreed that because of changes in themselves, their parents had more confidence in them. Both teachers and students wanted these Life Skills modules to be taught as part of the regular school curriculum, but for the moment these activities are only part of FORSATY-supported extracurricular activities. According to the IYF re￾port, 65% of schools do not have an adequate space to conduct these interactive learning sessions. More￾over, teachers are not paid for these extra hours, thus, as pleased as they may be, there may be a tendency for enthusiasm to flag over time. Extracurricular Activities According to interviews with school principals, extracurricular activities with students are introducing a socio-cultural dynamic within the school system leading to the rise of creativity, individual talents, and improvement of language capacity (Arabic, French) through artistic activities (theater, songs, poetry). Effectiveness To Be Determined Community Dynamics This component of the project has not yet reached its objectives in terms of youth mobilization at the neighborhood level, since it has only gotten under way in the last few months. It does have promise to address large numbers of neighborhood youth not involved in existing in-school or out-of-school activities. One problem to resolve is the weak coordination between the FORSATY OSSs and smaller local neigh￾borhood associations, which reduces the grassroots impact of activities of this kind. It is also partly due to gender bias that some OSSs associations are unwittingly being identified locally as women’s associations. This is the case of UNFM and Al Amal in Tétouan, where community dynamics mobilization activities are receiving little support from the male youth population in the very conservative Jamaa Mezouak and Bou￾jarrah neighborhoods. On the other hand, for the 1,077 youth participating in community improvement actions (updated through October 2015), some real achievements have recently been reported by FORSATY: • 26 were reintegrated into their schools. • 109 were reoriented to vocational training and job placement. • 51 were inserted into professional sports clubs (Clubs MAT, Ajax, Kac). • 68 were reintegrated into their neighborhoods, including some 20 that had attempted to emigrate to Europe by spending months waiting next to Ceuta. This meant in most cases that these youth returned to their homes. 8 International Youth Foundation. «Evaluation du Programme Aptitudes de Vie: Programme Passport to Success. » FORSATY Project Education Component. Tangier and Tétouan. 2015. 22 • 15 youth have created their own musical groups (3 groups); 2 of these groups are already earning their living playing in festive events. • About 20 report that they have stopped taking drugs. • 8 report that they were candidates for ISIS enrollment but that they abandoned the idea of trav￾elling to Syria, in spite of having already made such plans. The preceding data were provided directly to the evaluation team by the Community Dynamics compo￾nent manager in FORSATY. The mid-term evaluation did not validate project data as part of its scope. In its Community Dynamics component, IOM has temporarily abandoned its previous implementation model of providing services to at-risk youth through OSSs. It did so because it found that these OSSs were not able to form effective linkages with “second-tier” neighborhood associations where effective mobilization of large numbers of marginalized youth could occur with minimal cost. Materials, such a sports equipment supplied, were distributed through these very small associations and over 1,000 youth involved in various activities, including those indicated above. Nevertheless, it is not clear how community dynamics activities relate systematically to other project components, particularly in-school academic support and out-of-school vocational training and counseling. Since this component has not yet proven its worth compared to the original objectives focused on keeping youth in school and employing them once having abandoned it, the evaluation team cannot judge it as well as the other FORSATY components. Least Effective Components Non-formal Education Activities The non-formal education component of the project has quite limited impact, as it targets only a small group of out-of-school youth, those under age 15 with a low prospect of school reintegration, and offers virtually no educational support for those beyond age 15. The most recent FORSATY interim report (April-June 2015) indicated poor results with the 15 and over age group: Chifae reported drop-out of all 17 students, with eight referred to vocational training; Raouabit Assadaka suffered drop-out of 18 NFE students of a total of 30, with 10 referred to vocational training. Al Amal, on the other hand, reported working with 25 youth (20 females and 5 males) of which none had dropped out. Compared to the large numbers of beneficiaries in other components, non-formal education (NFE) would appear to be least promising in terms of impact for the project going forward and cost-effectiveness. Not only is it proving to be difficult to redirect many youth below 15 back into school or provide educational certification to those 15 and over through special classes, but it would appear that they will all very likely need to be integrated into vocational training at some point. It is the joblessness and the social marginalization that accompanies it that are most likely to be drivers of religious radicalization. 9 9 FORSATY agrees with this assessment of somewhat meager results, but wishes to point out that FORSATY will be part of a major new program called AFAK, in turn the regional pilot for MNE’s E2CNG project. AFAK is designed to work with 15+ youth with low educational achievement to bring them up to a level of qualification and place them into vocational training. The program is designed to last two years. FORSATY will be partnered with ATIL and Spanish NGO CODESPA to implement AFAK in the region of Tangier and Tétouan. Current FORSATY partner OSSs will work in non-formal education for 15 + henceforth within the framework of AFAK. For non￾FORSATY CSOs, ATIL will implement the program and FORSATY will provide parallel support in OD and Life Skills. 23 PROGRESS TOWARD CSO CAPACITY BUILDING Evaluation Question 2a: To what extent is the FORSATY project on target to achieving the CSO capac￾ity building intended result: youth service organizations are reinforced to provide better services for at-risk youth/young offenders? As indicated in the FORSATY results frameworks for Phase I and Phase II, capacity building of the OSS CSOs was intended to result in the provision of more efficient and better services to at-risk youth. Or￾ganizational strengthening was also assumed to result in the sustainability of these focal organizations and their services to youth beyond the life of project (2017). Organizational strengthening of the original four OSS associations occurred during Phase 1, but it became systemized and reinforced in Phase II under the current specialist in financial and administration manage￾ment. Beginning in October/November 2014 the four OSSs engaged in a capacity self-evaluation using an Organizational Capacity Assessment (OCA) tool consisting of eight sets of organizational skills divided into 49 questions with four levels of capacity from which to choose. An action plan to address perceived weaknesses was developed in each OSS, but actual implementation of agreed changes proved challenging. By mid-January 2015, no actions had been taken by these key implementing partners. Workshops were held to address these difficulties and priority actions were revised. In a revised February 2015 Action Plan, the four OSSs had a number of organizational development actions to take between March and September 2015. Although getting off to a good start, these organizational development (OD) actions were suspended with elections in June, Ramadan, summer camps, and summer vacation for youth in school and had not yet been resumed at the time of the mid-term evaluation, according to the OD specialist. They are ex￾pected to resume shortly and continue to the end of the project. The various actions to be taken by OSS partners were in the areas of governance, administration, human resource management, financial management, organizational management, program management, and lead￾ership and team dynamics. These seven sets of actions were further divided into specific themes to be strengthened: Chifae had 22 themes, Al Amal had 15, Assadaka had 15, and ATIL had 7. IOM methods consisted of monitoring (used in 14 cases), coaching (entirely or partially used in 33 cases), participatory workshops (entirely or partially in 15 cases), and training (in one case). Actions within the various themes were quite specific, such as revise and improve the manual of procedures, develop an evaluation schema for staff, update the organizational chart, develop job descriptions, draw up a plan of action, and develop a self-financing plan. Time periods for completing these actions ran from one to three months, but not all were to be initiated concurrently. The types of individuals involved were noted in the OD plan: deciders, directors, or coordinators, and sometimes all of them, were to be involved in carrying out the desired action. Actual achievement of these 59 specific actions by the four OSSs, and in particular their institutionalization, will not be measured again until October/November 2016, when a follow-up OCA will be administered to compare scores with the 2014 exercise. If all went well until OD actions were suspended in July 2015, 30 of the 59 actions have been completed. The OD specialist with FORSATY stated to the evaluation team that more than 50% of actions had thus far been completed, but this varies by organization. Al Amal should have completed 7 of 15 actions, Assadaka 7 of 15, ATIL 5 of 7, and Chifae 11 of 22. This was confirmed by the OD specialist. A recalibration exercise is planned for October/November 2015, at which time progress can be reassessed and actions relaunched appropriately. This should leave sufficient time for the remaining actions to be completed by November 2016 when the next OCA should reveal substantial institutional changes in these OSS associations. The ultimate issue is whether these key implementing partner organizations can reach 24 a sustainable level of organizational self-assessment by the end of FORSATY. Not surprisingly, changes in key personnel have been deleterious thus far to their progress. The objective is to transfer all responsi￾bilities for self-assessment and self-development to the OSS associations in 2016. This overall OD action plan is ambitious but achievable. The actions are clearly well chosen and compre￾hensive, but quite numerous for some OSSs. Their sustainability will depend on limiting key staff turnover and documenting procedures and internal processes to truly institutionalize them. Of course, if these organizations no longer have the financial resources to operate at current levels, key staff will depart and internal documentation may no longer correspond to organizational activities. CSO Capacity Building Results Evaluation Question 2b: In order to achieve CSO capacities that are likely to be sustained beyond the life of the program, what aspects of CSO capacity strengthening should be adopted, adapted, or repli￾cated within existing regions and to new regions? Service Capacity Even in the absence of a repeated OCA or use of another assessment tool, it is clear that since the implementation of FORSATY, OSS service delivery capacity has strengthened in quantity and quality. All the components of the project have been implemented by OSSs, with most activities hosted in their compounds, which has eased follow up and control. During 2014-2015, approximately 200 OSS personnel, training center trainers, and other personnel participated in project-supported training, forums or round table meetings. A total of 79 teachers or educators hired to provide Life Skills activities in schools were trained in the methodology. However, the planned training of 61 OSS educators for tutorials and 35 educators for Life Skills did not occur on schedule in 2015. The OSS educators will be trained in the next project year, but the funds are not available for IYF to train the 35 OSS personnel in Life Skills. All in all, in spite of some delay in the training of OSS educators, service provision in and through OSSs to at-risk youth has clearly increased beyond anything these organizations had done in this regard prior to FOR￾SATY. Orientation and Insertion Capacity OSS orientation and job placement capacity has improved first and foremost in workplace preparation, which benefitted 305 youth in Tangier and Tétouan over the last year, and orientation and information, which was provided to 1,633 youth. At the neighborhood level, it is estimated that 25% of target out-of￾school youth participating in the project have received vocational counseling. This has helped youth to learn about labor market requirements and job seeking techniques and procedures. This orientation ca￾pacity can be boosted to reach the project target of 35% by 2017. The expected sustained support by the government placement agency ANAPEC to OSSs will help to achieve this result. Based on the interview with the head of ANAPEC in Tangier, an agreement is under way with FORSATY outlining the targets for each OSS that the agency will back up with an action plan for achieving them. A target-based contract will be signed, but it will not include a capacity building component beyond job orientation for candidates. A future employment facility will be set up within OSSs to inform youth of job offers and register candidates in the ANAPEC database. Individual coaching will be provided to help youth prepare their CV for an anticipated job position or set up a project of self-employment. An agreement has been signed with OSSs Al Amal and UNFM in Tétouan, integrating them into the project “Min Ajliki” that supports self-employment and income-generating activities launched by women. 25 Synergy with Neighborhood Associations The OSS associations have begun to develop ties with local neighborhood associations through community dynamics activities targeting at-risk youth. Within the last few months, some 19 community improvement actions at the neighborhood level have mobilized 704 youth and required sustained follow-up and coor￾dination between representatives of FORSATY and neighborhood associations. This has contributed to the selection of 72 community youth leaders that receive a modest stipend (70 dirhams for transportation, food, and telephone calls when actually working with youth). For the time being, 69 of these 72 youth leaders are male and are focused on mobilizing young, unemployed men in project neighborhoods. These leaders have participated in project supported training activities and are very active in devising sports, music and other cultural activities much demanded by youth in the neighborhood. The prospect of creating neighborhood youth councils linked to OSSs (and to city youth councils) will enhance the organizational capacity of OSSs as entities for channeling information, actions, and support that improve the conditions of youth in neighborhoods. The prospect of increasing community advocacy actions for youth (15 targeted for 2017) should take place within a youth mobilization process around common interests that will enhance participation in community development. However, most of these community mobilization and advocacy actions and targets were originally to be implemented by Search for Common Ground, which has only recently completed its Conflict Assessment and has not yet joined in IOM community-based actions. Capacity Strengthening Aspects for Sustainability and Replication The seven core areas of organizational efficiency and performance within which 59 discrete actions have been developed for the four OSS partners represent an undeniably ambitious set of actions for these associations to acquire and institutionalize over two years in the midst of efforts to provide a range of services to in-school and out-of-school youth. Beyond this, the OSSs will be increasingly involved in build￾ing linkages with “second-tier” neighborhood associations under the Community Dynamics component currently underway in FORSATY after an initial pause under Phase II. The question is which among these important organizational attributes might be prioritized, given the relatively short span of time remaining before OD responsibility handover, the limited number of core staff, likely staff turnover, and increased organizational demands as the OSSs demonstrate project success and accomplishments. This should be examined carefully in the next annual OD recalibration exercise (projected for November 2015). To achieve viable and sustainable OSSs after the withdrawal of FORSATY, or for replication of project activities in other parts of the northern region or elsewhere in Morocco, organizational strengthening measures should focus on the aspects discussed below. Vocational Training and Orientation Capacity To ensure sustained socioeconomic impact of the OSS in each target neighborhood, its vocational training and orientation capacity should be strengthened. The rehabilitation of at-risk youth in marginalized urban districts requires first and foremost their empowerment through the acquisition of professional skills and job orientation knowledge. Evidence from FGDs shows that youth attendance in vocational training is strongly linked to their expectation of job preparation and training, while parallel activities are appreciated only secondarily. In the case of Bir Chifae, male youth joined the OSS to get vocational training in sewing – usually reserved for young women – for the job opportunities that this professional skill now provides in Tangier. In spite of strong capacity to improve in-school academic performance and retention, stronger vocational 26 training and orientation capacity is the key to retrieve at-risk youth from delinquency and induce change in personality and attitudes. The OSSs are currently the appropriate entities dedicated to youth with a low educational level and no chance of access to professional training elsewhere (e.g., in OFPPT). Current potential for vocational training in OSSs needs to be upgraded with a focus on those job profiles much in demand in the regional labor market. In the case of the Tangier-Tétouan region, this means more work￾shops for sewing, car wiring, tourism, and catering. Financial Capacity The strong dependency of OSSs on external funding is a threat to their sustainability. All FORSATY OSS implementing partners are heavily dependent on financial resources from various donors, without any sustained action to diversify their sources of funding for core project activities. Only two OSSs are in￾creasingly relying on self-generated funds from service activities or partnerships: Aide et Secours and Al Amal. In the first case, a project has been adopted by the management board to set up service projects capable of generating sufficient revenues to cover some of the OSS operating costs. This includes the construction of a children’s nursery and a primary school attended by well-off families who pay substantial fees, although these fees are lower than those in the private school system since the OSS is a non-profit entity. To run these service units, Aide et Secours has created for each unit a new association acting as a managerial body. Their current operational costs are estimated at 25% of total budget, and the OSS aims at cutting them gradually to 15%. In addition, Aide et Secours charges fees for services: 100 dirhams per month for vocational training (over three months), 20 dirhams per month for its alphabetization program (over ten months), 50 dirhams per month for school support (over ten months), 100 dirhams per month for hairdressing (over two years). It estimates its potential for financial self-sufficiency at 75%. In the second case, Al Amal has been managing a sports gymnasium under its facility that is securing steady income. The amount of income generated from this facility is unclear, but in each of these OSSs there are now several FORSATY-paid positions that are unlikely to be taken into account in these calculations. Other self-funding initiatives of lesser importance are being undertaken by OSS Hay Benkirane. A fixed fee of 100 dirhams per month is paid by an average youth cohort attending a sewing workshop over a year, and all of them are placed in paid internships followed by recruitment. The OSS has created a support club called “Club des Amis de l’Association Benkirane” composed of local enterprises, each of which pays an annual financial contribution of 5,000 dirhams and 250 dirhams per trained youth employed. This is the kind of initiative that needs to be expanded among the FORSATY OSS partners. Association Benkirane estimates its potential for self-funding and sustainability at 50%. The prospect of self-financing for these OSSs lies far ahead and needs to be planned around the following: • Administration of a general fee for services provided by the OSS with an exception for very poor beneficiaries. • Provision of a sustained grant from the State through project submissions or other special agree￾ments. • Formation of property assets through endowments and acquisitions which will guarantee fixed revenues. Replication The jury is still out at this point in project implementation regarding which capacity building activities will bear most fruit in OSSs and be more or less sustainable following the departure of IOM. The follow-up OCA exercise is not due for another year. Beyond the 59 actions scheduled to be adopted by the four 27 original OSSs, most of which will surely be required as well for UNFM and Hay Benkirane, the very expe￾rience of implementing the various activities promoted by FORSATY is likely to be most formative for these CSOs. These organizations have functioned in the past with little organizational capacity and have managed to secure outside donor funding to implement a range of activities in their neighborhoods. The most important attributes that they will need going forward will be good governance, effective program management, and financial management. In terms of maintaining current activities at reasonable levels after the end of FORSATY and the 87 staff members paid by IOM in the seven OSSs, financial management will be crucial. Financial sustainability of various levels of future OSS activities patterned on current FORSATY activities should be carefully examined within these organizations. OSSs and Institutional Action Plans Evaluation Question 2c: How useful do participating CSOs perceive the institutional action plans and how could they be supported to improve them? Key personnel in all FORSATY OSS associations expressed satisfaction with their action plans in the face of evaluation team probing during interviews. The FORSATY project coordinators all know the various actions to be taken, according to the OD specialist. Of course, current staff cannot be responsible for future staff actions if turnover occurs, and the associational culture from which these organizations derive does not stress rigor in administrative, financial, and personnel management. Nevertheless, it appears that those in management positions in OSSs want this OD process to continue, respect the FORSATY OD team’s abilities, and are willing to internalize new procedures and practices. Should they leave their current employers, they may well replicate much of it in new organizations. It will be important to involve the Boards of Directors to the extent possible in awareness of the key institutional practices that FORSATY seeks to internalize in its OSS organizations. Support to all seven OSSs should continue, although that would seem to be a tall order for the three￾person team charged with overseeing implementation of numerous actions in each association. The three new OSS partners will certainly need dozens of OD actions as have the original four, although Aide et Secours appears to be relatively sophisticated compared to Hay Benkirane and UNFM. Maintaining ad￾vances and institutionalized changes in all seven OSSs may well be a Sisyphean task for FORSATY. Expand￾ing the OD team may be a possible solution, with the goal of passing responsibilities for organizational assessment and future strengthening activities to the OSS partners by end of project, if not before. Cur￾rently this is projected to occur in 2016, but this may be a bit optimistic. Institutional action plans have been devised by IOM that are tailored to the capacity-building needs of these OSSs, since their managerial experience is recent and human resources are limited. With the ex￾ception of Aide et Secours, the organizational performance of most OSSs shows shortcomings concerning the following components. Governance The organizational structure of most OSSs is weak and staff qualifications are not well defined. There is lack of internal institutional communication between the executive board and the technical team, and even the absence of the position of managing director (the case of Al Amal) and succession planning for key positions. The enhancement of these functions and positions within the organizational structure will im￾prove their effectiveness. Institutional improvement is carried out through coaching, participatory work￾shops, and monitoring. The OSSs have generally adhered to this process of capacity building, although with relatively low commitment in an organizational structure involving three partners in the case of Assadaka (Raouabit+ Entraide Nationale+ PAIDEIA) or in an organizationally weak structure as in the case 28 of Bir Chifae. In their OCA scores, the OSSs recognize their organizational weaknesses and are aware of the need to improve them, but they are very short of staff capable of undertaking the tasks of restructur￾ing, since they are still in the stage of prioritizing project management and performance. Administration The administrative performance of OSSs is affected by the lack of an upgraded manual of administrative and financial procedures, which affects the procurement system and control of fixed assets, as well as by the lack of reports on workshops to share experience, which prevents the dissemination of information throughout the organizations. Institutional strengthening actions target leaders through coaching and fol￾low-up to facilitate adoption of professional administrative practices leading to efficiency and transparency. Human Resources Management Most OSSs have limited experience in managing their human resources, due to their recent growth history and shortage of personnel. Institutional strengthening actions will help to develop a general policy of staff management, which includes management defining staff positions, qualifications, wage levels and social security, time and work management and control, and procedures for performance evaluation for perma￾nent staff, trainees and volunteers. Support measures should focus on staff qualified in human resource management to overcome the above mentioned shortcomings, securing staff retention at all levels, and adopting a regulatory framework for permanent and temporary work. Financial Management Capacity building regarding financial management within most OSSs concerns improvements to compo￾nents of their financial system: accounting practices and adequate internal audit controls, certification of accounts, and sharing of the manual of financial procedures with the staff. Evidence from fieldwork shows that OSSs are preparing for an adequate financial system based on transparency and efficiency, as in the case of Bir Chifae where, according to the president, a process of restructuring is taking place with a new organization chart, an accounting section, and certification. Organizational Management Institutional strengthening action aiming at enhancing organizational management is to be achieved through the development of annual and strategic planning and the prospect of new opportunities for partnership and funding that will allow the OSSs to manage their external environment and internal resources, in order to create competitive advantages that enhance sustainability. Through coaching and participatory workshops this capacity building will result in the adoption of short and long term planning, institutional communication with an annual forum, regular reports, an updated and interactive website, and identifica￾tion of funding opportunities. Program Management OSSs have good program management capacity in terms of donor compliance, sub-grant management, and community involvement. Yet their technical reporting should be enhanced though systematic data collec￾tion and monitoring, which will feed a culture of learning. Institutional development actions will lead to systematic data collection among these organizations. A typical response of OSSs to institutional action plans is stated by the president of Bir Chifae: “The value added of FORSATY is that it is working with us on governance. Since the implementation 29 of OCA in December 2014, we have adopted a new action plan proposed by IOM which targets the restructuring of our association, we have engaged an accountant – paid by FORSATY – to prepare for the certification of accounts, and financial visibility. We are reviewing the association organization chart with clearly defined responsibilities, coaching of staff, strategic plan, and action plan, and institu￾tional information, in order to develop a brand image and new funding opportunities. Bir Chifae will become a social enterprise.” A Sustainable System of Mentoring and Cooperation between OSSs Evaluation Question 2d: What is required to establish a sustainable system of mentoring and coopera￾tion between the participating CSOs? At the heart of the concept of OSS mentoring among themselves (or of smaller neighborhood associa￾tions) is the transfer of knowledge, skills, and expertise from a more experienced CSO to one less expe￾rienced for improvement, growth, and positive change. In the case at hand, organizational strengthening and learning by one OSS can be shared with other neighborhood OSSs to improve and sustain the services currently provided to FORSATY beneficiaries. At present, there is little sense that this is happening be￾tween these FORSATY partners. However, building such a structure should be promoted going forward. It should be noted that such mentoring and cooperation actions between OSSs in the FORSATY project have not been common and run counter to their organizational culture that stresses competition for resources from donors and benefactors. It will require special efforts (workshops, coaching) from the FORSATY organizational strengthening staff to instill such values and practice among its OSS partners. EFFECTIVENESS OF PROJECT MOBILIZATION AND FOSTERING OF RELATION￾SHIPS BETWEEN RELEVANT ACTORS Evaluation Question 3: How effectively did the project mobilize and foster relationships between the different actors serving at-risk youth (including CSOs, government services and departments, the pri￾vate sector, families, and the youth themselves) over the course of the project? One way to gauge the relative effectiveness of FORSATY mobilization and development of relationships between institutions and parties serving at-risk youth is to ask whether some entity has been left out or whether a relationship has been underexploited. FORSATY outreach and involvement of relevant entities and actors has been impressive, but it has kept its focus on the OSS model, with one association in each neighborhood that can establish local relationships with schools, governmental vocational training facilities, private sector employers, and job placement entities. In respect to placement of trainees, a Renault exec￾utive who was interviewed stated that the only effective way for OSSs to integrate youth at his plant was to go through a private recruitment agency, which could not only channel demand for labor from employ￾ers but could also assist OSSs to improve their supply of job candidates to match demand. The company (and many other private sector entities) is not interested in direct coordination with OSSs. Finally, the Comité Régional pour l’Amélioration de l’Emploi et de l’Employabilité is a regional body in Tangier that identifies sectors with high employment potential and could guide OSSs to provide vocational skills that will be in increased demand in the garment manufacturing, automotive, and tourism industries, given the forecast by this body of an aggregate 30,000 new jobs in the region in upcoming years. This is particularly relevant for the large automotive companies now establishing themselves in Tangier. One budding relationship that was lost to FORSATY recently is the linkage forged earlier with the Institut National d’Action Sociale (lNAS) seeking to institutionalize a permanent retraining curriculum for social workers focused on youth problems. The INAS mandate apparently does not allow this refresher training 30 activity. Relationships with youth are certainly solid through OSSs but relationships with parents’ associations in schools appear weak in spite of project efforts to involve parents. The school-oriented PTAs (Associations des Parents d’Eleves) are generally inactive or ineffective, according to school directors. Moreover, most parents in target neighborhoods do not show much interest in their children’s education, especially men. Those parents that showed up for evaluation FGDs were almost always mothers. Some stronger outreach to fathers will be necessary to bring in at least a few men. The relationships forged between FORSATY and current OSSs, such as Assadaka, Aide et Secours, and Hay Benkirane on the one hand, and with governmental agencies, such as Entraide Nationale and OFPPT on the other, predated FORSATY and have been part of the earlier Youth Rehabilitation and Reinsertion Project. That project did not rely on an OSS approach and consequently mobilized a wide variety of youth￾serving organizations without establishing coherent relationships among them. Focusing project services for youth through neighborhood OSSs and placing them as intermediaries between the outside donor and local public and private institutions is a major innovation, the sustainability of which requires that OSSs continue to be proactive in developing institutional relationships well beyond their own neighborhoods. Mobilization of Public Institutions and OSSs The FORSATY project has produced an institutional dynamic that has brought together OSSs and public institutions, such as MNE and Entraide Nationale, in joint action to address the issue of at-risk youth rehabilitation. The programming of educational support, Life Skills clubs, extracurricular activities and non￾formal education has mobilized educators, teachers, school principals, and regional governmental officials. Various means have been employed to reach in-school youth outside of the classroom, including sports facilities, art and theater activities, and Life Skills training. Beyond building youth internal assets, these activities are designed to make being in school more attractive. Volunteer work has developed among OSSs and schools to help advance the project and make it succeed. Mobilization of the MNE is further enhanced by the implementation of the AFAK project intended for out-of-school youth between 15 and 20 years of age. Over three years, a total of 2,100 youth will be trained in the Tangier-Tétouan region. In the first year, FORSATY OSSs in their neighborhoods will assist in providing academic tutorials to this marginalized group, who will enter in the second year into vocational training in OFPPT and Entraide Nationale centers. The MNE will fund the AFAK project at the rate of 1,200 dirhams per beneficiary and another 1,500 dirhams will be provided by other donors. The prospect of further mobilization of the MNE in support of youth activities already engaged in by FORSATY will be through the projected education reform nationwide, which will integrate extracurricular activities as part of the educational program and enroll trained staff to implement this. A pilot experience will take place in Tangier (and other cities) during the next academic year 2015-2016. According to the head of NFE at the Education Delegation in Tangier, “FORSATY is a success on pedagogical and cultural levels, it has helped to curb dropout and school failure, freed pupils from the routine of classical learning and developed their potential of creativity and mutual understanding.” Youth Wednesdays Another type of mobilization promoted by FORSATY OSS partners has involved other public institutions, such as the police within the Youth Wednesday events, where security is often debated at the neighbor￾hood level, and at which representatives of the Ministry of Health have raised the issues of drug addiction and prevention. Evidence from focus group discussions shows a strong appreciation of these meetings, as 31 in-school youth find them very instructive, consciousness raising, and responsible for promoting attitudinal change regarding institutions like the police, whom they now consider as more involved in improving security and crime control than repression. They also consider these meetings as changing the routine of school life and allowing encounters with adults other than teachers. Mobilization of the Private Sector FORSATY has fostered linkages between private enterprises and OSSs throughout the whole process of rehabilitation of at-risk youth. In Tangier, partnerships have been built over the last three years and set the framework for professional internships and employment. This is particularly the case of the clothing enterprises, which take virtually all the cohorts of trained youth in the OSSs, keep open a communication link, ask for intermediation in case of conflict, and help maintain the sewing workshops in the OSSs through small contributions. Representatives from the LARINOR garment company expressed an interest in sup￾porting the OSSs financially and technically, given their reliance on a steady provision of partially trained labor to their internship programs. In Tétouan, the private sector is relatively well-mobilized in favor of OSSs, although most youth on in￾ternships are not paid and the prospect of employment of trained youth is limited compared to Tangier. The car wiring enterprises are in contact with OSSs for the provision of trained youth and give some technical support. In tourism and catering, hotels are offering training for a limited number of waiters and chef’s assistants, paying 1,500 dirhams over a two-month internship in the high season. Large restaurants are also involved in seasonal internships of trainees from OSS vocational training programs. Nevertheless, the mobilization of the private sector remains limited thus far for many reasons: • The professional profiles required by enterprises are rarely matched by youth trained by FORSATY OSSs. Youth completing training from these OSSs are not skilled workers and technicians and usually lack the formal educational level that allows further training by enterprises. Nevertheless, they have begun to prove themselves in some sectors, such as the garment industry and vehicle wiring. • Strong competition with neighborhood OSSs by OFPPT, which provides more needed professional profiles, usually youth with a secondary school education (9th Grade) who qualify themselves in OFPPT as skilled workers in mechanics and electricity. The OFPPT is far better prepared to respond to rising demand by enterprises for higher skilled workers than OSSs. This advantage will increase as the State continues to allocate more financial resources to OFPPT than to OSSs. • The lack of State incentives to private enterprises to mobilize them in favor of OSSs. It is conceivable that a public policy designed to support these civil society entities, such as the adoption of a mechanism of tax rebates for enterprises training and employing at-risk youth received directly from OSSs, will encourage private sector enterprises to provide more technical and material support to associations and help to professionalize their training activities. This needs to be examined by FORSATY staff and specific recommendations made to local officials through the activities of both IOM and SFCG under Component 2. Mobilization of Beneficiaries and Parents The project has so far mobilized beneficiaries and their families and created a strong link between them and the OSSs. All the components of the project have benefitted from strong attendance by at-risk youth, and follow up by parents is generally sustained. FORSATY supported extracurricular activities are mobi￾lizing increased numbers of in-school youth, who find in the local OSS a space for self-fulfillment. Evidence from focus group discussions with in-school youth shows that more young girls than previously are at￾tending sports and other cultural workshops organized by the OSS with the agreement of their families. 32 Families are not traditionally keen on letting girls attend associations or any public venue other than school. The safe experiences of girls in OSS and school extracurricular activities are directly linked to this easing of restrictions on girls in such activities. As far as evaluators know, no parents called for “safe spaces” in these activities for their girls, since boys and girls are mixed in school without problems. Mobilization of beneficiaries also includes the participation of primarily out-of-school youth in community dynamics through cultural animation and coordination with “second-tier” neighborhood associations. For￾merly trained youth come back and participate in the training of beginners in various workshops. OVERALL CONCLUSIONS • FORSATY is on schedule to reach its 2017 youth support objectives and has generally surpassed annual targets. • Focusing project services for youth through neighborhood One-Stop-Shop organizations and placing them as intermediaries between the outside donor and local public and private institutions is a major innovation under FORSATY with respect to its precursor project. • The FORSATY project has produced an institutional dynamic that has brought together OSSs and public institutions, such as the MNE and Entraide Nationale, in joint action to address the issue of at￾risk youth rehabilitation. The programming of educational support, Life Skills clubs, extracurricular activities, and non-formal education has mobilized educators, teachers, school principals, and regional governmental officials. • FORSATY has fostered linkages between private enterprises and OSSs throughout the whole process of rehabilitation of at-risk youth. • The project has so far mobilized beneficiaries and their families and created a strong link with the OSSs. • While FORSATY appears to be on track to reach its OSS capacity-strengthening objectives by 2017, the next formal stocktaking will not occur until November 2016. About 50% of strengthening out￾comes remain to be accomplished. A sustainable system of current service delivery beyond FORSATY is not yet in place. • More successful project components include vocational training and academic support, as well as Life Skills activities with youth in school and out of school. These key activities have produced large num￾bers of beneficiaries and represent the core FORSATY activities from project inception. • Follow-up information on vocational trainees indicates a strong commitment to work and social sta￾bility among these beneficiaries, in large part due to Life Skills and work readiness training alongside technical training. This is reintegration into society of disenfranchised and marginalized youth in the fullest sense. • The overall rate of academic improvement in primary schools following academic tutorials for 2015 is 76%, compared to 65% in middle schools. • The success of academic support has resulted in an improvement in student grades and examination success with a concomitant drop in school abandonment. Parents, especially mothers who are most involved, strongly appreciate the improved academic results of their children that keep them in school. • Extracurricular activities supported through FORSATY are also contributing to sustained school retention, performance in the classroom, and personality development. This is attested to by OSS staff and school principals. • Life Skills training for out-of-school beneficiaries has been extremely beneficial to their success in internships and future employment. The young employed or intern groups obey the rules in the work place, accept intermediation, and keep their jobs. Company officials directly ascribe this superior attitude compared to other interns to the Life Skills and work readiness training received in the OSS associations. 33 • The least successful component has been non-formal education, which has produced relatively few successful cases of reintegration, although it is currently being strengthened through a new partnership with a government program. • The Community Dynamics component is only now getting under way. It shows promise, particularly from a youth governance and CVE perspective, but neighborhood activities undertaken by IOM and city-level council activity by SFCG had not yet been well coordinated at the time of the evaluation. Coordination with neighborhood youth committees is expected to be established as the city-level councils become active. For this reason, this component has not yet proven its worth compared to the original objectives focused of keeping youth in school and employing them once having abandoned it. • Personnel costs are overly high with about 100 staff now supported in the seven OSSs, in addition to the 22 in the IOM project office in Tangier. There are also 72 youth leaders supported by stipends when actively engaged in neighborhood activities. • Organizational sustainability is possible by 2017, but FORSATY is unlikely to achieve financial sustain￾ability by project end under foreseeable circumstances. RECOMMENDATIONS • Current activities are appropriate and should continue, but cost containment should be stressed. The project has less than two years to render OSS activities financially sustainable. • Gender issues in FORSATY neighborhoods are being addressed, but should be given increased importance, since female roles still restrain their access to project activities and employment opportunities. Men should be actively mobilized to support such activities for women and girls. • The Livelihoods Assessment conducted by Mercy Corps should be reviewed for its pertinence to employment and self-employment opportunities in project neighborhoods, especially the new ones under Phase II. • The three new OSS associations need to begin the process of organizational assessment and strength￾ening. • The new Results Framework should include the grassroots push of the community dynamics compo￾nent. • A new project monitoring tool (MELP) should include clearly defined outcomes with indicators for all objectives, including the community dynamics component. To the extent possible, increased measures of drivers of violent extremism should be included as performance indicators. • The new Results Framework and MELP currently proposed represent the third versions in three years, a lot of shifting of objectives and related performance indicators, but FORSATY feels that the new framework simply realigns a large number of inputs and outputs to eliminate overlap and improve legibility. These modifications reflect experience in implementation over the last few years and can thus be considered an example of experience-based adaptive management. • FORSATY should take care to ensure that indicators are tracked throughout the life of the project, despite shifting objectives and MELPs, so that results can be properly documented. • All project data must be sex-disaggregated wherever feasible. • Since two-thirds of schools do not have an adequate space to conduct Life Skills sessions and teachers are not paid for these extra hours, FORSATY should explore means to promote Life Skills club activities within OSS premises. • Inclusion of Life Skills in teachers’ normal workloads would help sustain that effort within schools. The project staff should explore how such educational policy change could be encouraged. • Current potential for vocational training in OSSs needs to be upgraded with a focus on those job profiles much in demand in the regional labor market. In the case of the Tangier-Tétouan region this means more workshops for sewing, car wiring, tourism, and catering. The rehabilitation of at-risk 34 youth in marginalized urban districts requires first and foremost their empowerment through the acquisition of professional skills and job orientation knowledge. • Partnerships with private sector employers should continue to be pursued aggressively with consid￾erable potential still available in Tangier. • Job placement of trainees through private employment agencies should be explored. Many of the larger private sector employers do not wish to deal with neighborhood associations. • The potential for public-private partnerships to sustain OSS activities after project end should be seriously explored since FORSATY ends in less than two more years. • In view of strong stakeholder appreciation, if costs can be contained and sustainable sources of funding secured, FORSATY may consider the following as high priority areas: • Increasing academic tutorials in beneficiary institutions, while including new school districts where school dropout is also high. • Supporting more tutors and more teaching hours in math, French and physics. • Increasing Life Skills clubs in middle schools and providing adequate training to tutors. • Increasing the number of vocational training workshops, in order to double the capacity of OSSs in response to rising demand for training from both out-of-school youth and local employers. • Providing more support to non-formal education in OSSs, in order to improve the rate of school reintegration of youth under 15 years of age, which is currently estimated at 35%. Performance in this activity has been poor thus far. • The project can be replicated in other cities with much the same components, but these locations should show strong potential for future economic growth and resulting job demand or employment results will not match those of Tangier. • In view of the likelihood of key staff turnover in OSS partners, FORSATY institutional strengthening staff should concentrate on measures documenting internal procedures and processes to truly institutionalize them. Those key procedures and processes most related to sustaining FORSATY activities would be the best investment in time and resources. • To achieve viable and sustainable OSSs after the withdrawal of FORSATY or for replication of project activities in other parts of the northern region or elsewhere in Morocco, organizational strengthening measures should focus on governance, human resource administration, and financial capacity. • Expanding the OD team may be a possible solution, with the goal of passing responsibilities for organizational assessment and future strengthening activities to the OSS partners by end of project, if not before. Currently this is projected to occur in 2016, but this may be a bit optimistic. • The prospect of self-financing for the OSSs lies far ahead. Aide et Secours charges fees for services, Al Amal manages a gymnasium, and Hay Benkirane charges for placement and raises money from small donors. Increasing cost coverage will require: • Administration of a general fee for services provided by the OSS with an exception for very poor beneficiaries. • Provision of a sustained grant from the State through project submissions or other special agreements. • Formation of property assets through endowments and acquisitions that will guarantee fixed revenues. • Financial sustainability of various levels of future OSS activities patterned on current FORSATY activities should be carefully examined within these organizations. This is a key deliverable of the FORSATY financial capacity expert and should be carried out as soon as possible, in order to launch the financial capacity and resources mobilization plan for the OSSs. • OSSs could improve their administrative performance with by recruiting administrative and maintenance staff, holding an annual information forum, and distributing regular newsletters. • It is conceivable that a public policy designed to support FORSATY OSS associations, such as the adoption of a mechanism of tax rebates for enterprises training and employing at-risk youth received 35 directly from OSSs, will encourage private sector enterprises to provide more technical and material support to associations and help to professionalize their training activities. • Representatives from the LARINOR garment company expressed an interest in supporting the OSSs financially and technically given their reliance on a steady provision of partially trained labor to their internship programs. • FORSATY should promote a formal process of OSS mentoring and cooperation that should include the following: • Increase the stress placed on training and capacity building activities to enhance OSS upgrading and certification to international standards, which FORSATY feels is part of its current strategy. This will increase their ability to receive grants and to collaborate with each other on project development and implementation, enabling them to design projects that respond to key issues at neighborhood and city levels and mobilize stakeholders, including public institutions, to support them. • Increase the stress placed on development of OSS information sharing and communication capabilities, which FORSATY feels is part of its current strategy. This will create information sharing and strengthens the capacity to communicate with others on common issues relevant to their target population – at-risk youth. • Technical capacity building activities, such as technical workshops and seminars on thematic issues, all directed toward further replication in smaller neighborhood associations. 36 ANNEXES 37 ANNEX A: EVALUATION STATEMENT OF WORK Mid-term Performance Evaluation of USAID/Morocco Favorable Opportunities to Rein￾force Self-Advancement for Today’s Youth (FORSATY) Project # AID-608-A-13-00001 Implementer: International Organization for Migration (IOM) USAID/Morocco Development Objective 2: Increased Civic Participation in Governance BACKGROUND INFORMATION Project Information Project Title: Favorable Opportunities to Reinforce Self-Advancement for Today’s Youth (FORSATY) Award Number: AID-608-A-13-00001 Award Dates: October 1, 2012- September 2017 Funding: $8.77 million Implementing Partner: International Organization for Migration (IOM) Agreement Officer Representative: Idriss Touijer Development Context The political and social situation in Morocco has been influenced by recent developments in countries in the Middle East and North-Africa (the so-called ‘Arab Spring’). While these devel￾opments led to revolutionary change at the political, social and economic level in Egypt, Libya and Tunisia, average Moroccans embraced demands for change much more cautiously. In re￾sponse to these demands for democracy and good governance, King Mohammed VI initiated a reform of the constitution, which was adopted in a public referendum in June 2011 and an￾nounced a comprehensive judicial reform in spring 2012. Earlier the Government had already launched an ambitious education reform, aiming to deliver quality education for all. The implementation of these ambitious reform packages is still pending and risks outlasting the patience of disadvantaged Moroccans, expecting measurable change in their individual lives. Lack of opportunities for political participation, wider social and economic divides and persistent poverty has been recognized as a central and cross-cutting issue facing Moroccan society today. Multiple factors contribute to a disproportional effect on youth, including lack of access to qual￾ity education, high rates of unemployment (population between 15-24 years of age make up 25.1% of the labor force while constituting 35.7% of the unemployed). Despite compulsory schooling up to the age of 16, Morocco’s illiteracy rate stands at 43% and some 300,000 Moroc￾can school-aged children and youth do not attend school each year, thus increasing their risk of social and economic marginalization, which may contribute to or drive individuals towards crim￾inal behavior, tendencies to engage in irregular migration, violent extremism and insurgency. In addition, in many neighborhoods, young people lack opportunities to participate in activities that are critical in the development of confidence, leadership abilities, and other necessary life skills. 38 Exclusion from the education system, high rates of economic inequality, and political marginali￾zation lead many youth to regard mainstream institutions with a mixture of skepticism, apathy or hostility that can lead to extremism. Indeed, the high rates of school drop-outs and the poor em￾ployment situation have been identified as significant contributors to the risks of extremism and irregular migration in Northern Morocco. Not only does marginalization as a phenomenon matter in itself, but also the perception of exclu￾sion and marginality that the individual may have may differ among marginalized groups, with urban youth representing a particularly vulnerable group. Target Area and Target Population The cities of Tangier and Tétouan are the target geographic areas for this program. IOM works in up to four neighborhoods, such as the Tangier neighborhoods of Ben Dibane, El Mers and the Tétouan neighborhoods of Boujarah and Medina. Other neighborhoods will be added moving forward with the second phase of the program’s implementation. All selected Program neighborhoods in Tangier and Tétouan are marginalized. They have been chosen based on a field assessment that indicated a very high demographic density due to rural migration; they suffer a severe lack of public services (public schools, youth centers, health facil￾ities, recreational green spaces, etc), high criminality rates and drug abuse. The target population of this Program are at-risk youth (12 to 25 years old) who live in marginal￾ized neighborhoods of Tangier and Tetouan that lack public services and support systems, which increases the risk factor of dropping-out of school and being vulnerable to illegal migration, drugs, and other extremist ideologies. Intended Results The Project Objective is ‘to prevent delinquency and reduce recidivism among target youth in selected areas of Tangier and Tétouan’. Underpinning this objective is the Specific Objective ‘to improve the educational and socio-economic integration of marginalized youth in selected areas of Tangier and Tétouan’. To reach these objectives the Program works towards three intended results: Result 1: Marginalized and disaffected out-of-school youth are successfully integrated into soci￾ety; Result 2: In-School youth receive improved support to increase retention, success and decrease drop- out Result 3: Youth serving organizations provide better services for at risk youth; Cross Cutting Result: Marginalized and disaffected youth enjoy a safer community environ￾ment The Program complements both the efforts of the Moroccan authorities to address youth margin￾alization and the focus of the USAID Country Assistance Strategy (CAS) (2008-2013) and the Country Development Cooperation Strategy (CDCS) 2013- 2018). Furthermore, it falls squarely within the priorities of the United Nations Development Assistance Framework 2012-2016, un￾der which IOM operates. 39 See Attachment 1: Results Framework. Project Approach The Program aims at enhancing prevention efforts to tackle the broader issues of social marginal￾ization, with an emphasis on the role of the community in reducing marginalization. As such, Program activities encourage communities to improve social services for marginalized youth. Best practices in positive youth development show that when youth have positive outlets to ex￾plore their interests, test their abilities, and take on leadership roles, they are more likely to stay in school, be active in their communities, avoid negative influences, and grow up to become pro￾ductive and involved citizens. Furthermore, the Program aims to forge links between partnering local associations and governmental institutions thus reinforcing the idea of strong governance. Following a needs assessment of the selected target neighborhoods, the Program aims to increase the offer of activities at community level. Social marginality and unemployment also mean that large numbers of young people have too much time on their hands. The Program supports initia￾tives that promote the labor-force participation of at-risk youth as a means of preventing margin￾alization and producing positive alternatives to delinquent behavior. In addition and in line with best practices in positive youth development, the Program aims at strengthening the capacity of Government and non-government institutions to improve the qual￾ity and delivery of services provided to the targeted youth; this approach will ensure the sustaina￾bility and long-term value of the Program. For each selected target neighborhood, the Program has identified a local NGO to serve as a sin￾gle point of contact for beneficiaries to deliver a standard package of holistic youth-friendly ser￾vices. At the beginning of the Program, a neighborhood needs assessment and an association self-assessment is conducted. Following these assessments, it is determined which services the association can deliver itself, where it has training needs to implement them and where it will need to refer to other partners. This single point of contact (a “one-stop-shop”) for the beneficiaries can facilitate contact and enhance the trust of beneficiaries. The selected associations are specialized in working with mar￾ginalized and at risk-youth in their respective mandates. The Program relies on their existing knowledge and expertise to identify beneficiaries, which (besides fitting the geographical focus and age criteria) include for example levels of school attendance and exam results, school drop outs without job qualifications, drug abuse, etc. The Program is based on a hybrid system of service delivery through the selected “one-stop￾shop” associations’ own approaches, as well as referrals to other service providers. Existing in￾formal referral networks may be strengthened and formalized to ensure that associations do com￾plement each other based on their particular expertise and that gaps identified during the needs assessment and self-assessments are programmed into the work and training plans during the im￾plementation of the Program. 40 Via the provision of sub-grants, the Program aims at strengthening the participant associations’ existing projects reaching at-risk youth and young offenders, and allows the local associations to improve the quality and expand the delivery of their service. See Project reports and work plans for detailed descriptions of activities and attached Perfor￾mance Management Plan (Appendix 1 in separate file of appendices) for results, sub-results and indicators. RATIONALE Evaluation Purpose, Scope and Audience USAID/Morocco intends to conduct a mid-term performance evaluation of the FORSATY Pro￾ject. This evaluation will serve the following purposes: 1. To provide the Mission with concrete evidence regarding the extent to which FORSATY achieved its objectives through December 2014; 2. Determine any recommended mid-course adjustments for the program to improve results moving forward, particularly focusing on the capacity strengthening of CSOs; 3. To inform how this project would be best replicated in other regions of Morocco. The primary task of the evaluation team will be to analyze the performance of FORSATY mid￾way through program implementation and make recommendations for improvements in ongoing and future programming. This evaluation will cover the period from the award date through De￾cember 2014. The USAID/Morocco Mission, specifically the Office of Democracy and Govern￾ance, will use the analysis of this evaluation to guide the direction of FORSATY in the remain￾ing project life. The implementing partner, IOM, will learn about the strengths and weaknesses of project activities to date and adjust the upcoming work-plan based on the evaluation recom￾mendations. Evaluation Questions The following questions shall guide the evaluation in order of importance: 1. To what extent is the FORSATY project on target to achieving its intended results? i. Marginalized and disaffected out of school youth are successfully integrated into so￾ciety. ii. In school youth receive improved support to increase retention and success and de￾crease drop out. iii. Marginalized and disaffected youth enjoy a safer community environment (cross-cut￾ting objective). a) What do targeted beneficiaries (including male and female at risk youth) and other ac￾tors see as the most significant changes (positive or negative) to which the project has contributed, and what else is required? b) Which project components are the most and least effective in contributing to the project objectives, and may have the most or least promise for the project going forward? 41 2. To what extent is the FORSATY project on target to achieving the CSO capacity build￾ing intended result; ‘Youth service organizations are reinforced to provide better ser￾vices for at risk youth / young offenders’? a) In order to achieve CSO capacities that are likely to be sustained beyond the life of the program, what aspects of CSO capacity strengthening should be adopted, adapted or rep￾licated within existing regions and to new regions? b) How useful do participating CSOs perceive the institutional action plans and how could they be supported to improve them? c) What is required to establish a sustainable system of mentoring and cooperation between the participating CSOs? 3. How effectively did the project mobilize and foster relationships between the different actors serving at-risk youth (including CSOs, government services and departments, the private sector, families and the youth themselves), over the course of the project? In the draft and final evaluation report, the evaluation team should: • Identify critical learning to inform the second phase of FORSATY and the scaling up to new regions in Morocco. Recommendations are required to improve the attainment of the development objective and the intended results. • Include an examination of overall program coherence and make specific recommenda￾tions for each activity component; • Document lessons learned and identify enabling factors related to: - Targeting, mobilizing and communicating with different actors; - Coordinating the participation of different actors into the program; - CSO capacity strengthening moving forward. Proposed Background Materials: • FORSATY snapshot evaluation report. • 2013 Annual Report GENERAL METHODOLOGICAL GUIDELINES USAID/Morocco seeks a mixed methodological approach to effectively address this performance evaluation and effectively answer the evaluation questions. This scope of work requires that the team develop and submit for approval a work plan with proposed methodology within the first 7 days of the evaluation schedule. The evaluation should utilize an evaluation design matrix. The matrix should include research questions, methods, data sources and sampling, and a data analysis plan for each question. An illustrative and abbreviated design matrix (see Attachment 1 to this Scope of Work) has been de￾veloped. Evaluation teams are encouraged to refine and expand on this using the design matrix template provided (see Appendix 2 in the separate file of Appendices). 42 The following approaches should be included in the team’s methodology: • Completion of desk review (see proposed information sources). The desk study is in￾tended to prepare the evaluation team for fieldwork. In this period, the team will analyze available program documentation (including performance and management documenta￾tion, baseline youth-led assessments, as well as organizational capacity assessments), as well as identify, obtain, and analyze relevant non-program data sources. • The evaluation must include the use of three methods of data collection, suggested data collection tools include: - A survey of a random stratified sample of beneficiary individuals (male and female at risk youth who have taken part in project activities). - Focus group discussions with a random stratified sample of: o Beneficiary individuals (male and female at risk youth); o Project beneficiaries’ families; o Participating CSOs. - Semi structured interviews with a sample of stakeholders, including: o IOM o 4 CSOs o School Directors and Teachers o IYF o NME and Regional Delegation of the Ministry of National Education o OFPPT o EN o INAS o Other Key Informants identified by the evaluation team. • Evaluators shall work with USAID to determine an appropriate number of beneficiaries and partners to include in the representative sample. The evaluation design matrix should include a data analysis plan for each evaluation question. The data analysis plan should clearly describe the evaluation team’s approach for analyzing quantitative and qualitative data. It is expected that quantitative data will be analyzed using sta￾tistical analysis software (e.g. SPSS, SAS, and STATA). The evaluation team may use qualita￾tive data analysis software (e.g. NVivo) to analyze qualitative data. While the use of qualitative data analysis software is optional, qualitative data should be coded as part of the analysis ap￾proach, and the coding used should be included in the appendix of the final report. Gender, geo￾graphic, and role (beneficiary, implementer, government official, NGO) disaggregation must be included in the data analysis where applicable. **All conclusions made by the evaluation team must be supported by clear, verified evi￾dence. ** The evaluation team should avoid generalities that are not supported by specific evidence. The evidence cited should include, for instance, interview data (for example, “x number of x said ….”), secondary evidence, respondent’s quotes for key informants and focus group interviews, tables, graphs, and charts where needed, etc. The report must let data speak for itself in order to 43 minimize, as much as possible, subjective judgments or at least the impression of judgments be￾ing subjective. Data sources must also be cited. The more data sources are used to provide evi￾dence on a particular finding, the better. Limitations in Methodology Internal reliability and external reliability limitations must be considered during the evaluation process. Due to internal reliability issues, it is not certain that specific project interventions are the cause of changes which may be observed over the course of the evaluation. Not unrelated, researchers and readers must be mindful of external reliability limitations and the fact that find￾ings of this study may not be entirely generalizable. While these limitations exist, the mixed￾methods approach suggested in this section will help boost confidence in internal reliability by establishing and comparing separate findings. It is important to note, therefore, that the data must be analyzed separately but conclusions should be based on the results of all the findings in total. Specifically, the researchers and the readers must not rely too heavily on any one method but consider how they fit together to inform our understanding of the project. The final evalua￾tion report should clearly document limitations and how the evaluation team addressed these lim￾itations in its methodological and/or analysis approach. Any limitations that have potentially sig￾nificant bearing on the validity of the evaluation findings should be clearly identified. TEAM COMPOSITION USAID/Morocco anticipates the evaluation team to be comprised of an external team leader, one subject-matter experts, and a one research/logistics assistant. USAID/Morocco highly recom￾mends that at least one evaluation team member be an experienced evaluation expert. The Team Leader should be an evaluation expert combining academic credentials, demonstrable CVE experience, knowledge of the Moroccan socio-economic and political environment scene, and USG foreign policy and foreign assistance programming. S/he should have considerable ex￾perience working as a part of, as well as leading, democracy and governance evaluation teams and writing analytic but actionable reports in a clear, coherent and compelling way. The ability to speak and write in English is required. The evaluation team together should have expertise in youth development, gender, and CSO ca￾pacity strengthening. One of the two experts shall have a solid knowledge of English and either French, Moroccan Darija, or Arabic. These languages are preferred as a skill for the other team members. At least one member should have extensive experience in facilitating interviews and/or group discussions in the targeted languages. Gender analysis skills are desirable. The youth de￾velopment expert should preferably be Moroccan. It is recommended that the research assistant be young Moroccan researcher or university student with a mixed set of skills and backgrounds in civil society, youth development, evaluation, gen￾der equity and related areas. Other personnel and skill mixes may be proposed. 44 Written disclosures of conflicts of interest are required from all evaluation team members. The Program Management Office will work with the technical team to ensure the quality and objec￾tivity of the evaluation process and report. DELIVERABLES, REPORTING AND DISSEMINATION REQUIREMENTS The following reports and deliverables are required: 1. Detailed Evaluation Design and Work Plan: A research design and evaluation work plan shall be submitted for approval to the USAID/Morocco Office of Democracy and Gov￾ernance on day 7 (from start of evaluation). This plan will include: 1.1 A detailed evaluation design matrix (including the key questions, methods and data sources to address each question and the data analysis plan for each question). 1.2 Draft data collection instruments; 1.3 Known limitations to the evaluation design; and 1.4 A detailed workplan: The workplan must include the anticipated schedule and logistical arrangements and delineate roles and responsibilities of members of the evaluation team. This plan will be discussed with and approved by USAID prior to implementation. 2. Interim Briefing: An interim briefing on the assessment team’s overall progress and pre￾liminary findings shall be presented to USAID/Morocco after 3 full weeks of fieldwork in country. A short briefer with preliminary observations shall be presented to USAID in ad￾vance of this briefing. 3. Debriefing with USAID: Prior to departure, the contractor must present an outline (in bullets, possibly in power point or as a handout) of the draft evaluation report with prelimi￾nary findings, conclusions, and anticipated recommendations. During this briefing, the team leader, as well as key members of the evaluation team shall be present. During this briefing the evaluation team shall submit a suggested table of contents for the final evaluation report to USAID for approval. 4. Presentation of Results to Implementing Partners and Stakeholders: The evaluation team shall make an oral presentation of preliminary evaluation findings, conclusions and rec￾ommendations to project partners and stakeholders before departure of expat team member(s) from Morocco. 5. Draft Report: A draft evaluation report, with executive summary and major findings and recommendations, shall be presented in writing to USAID during week 4. This document should explicitly respond to the requirements of the SOW, should answer the evaluation questions, be logically structured and adhere to the standards of USAID Evaluation Policy of January 2011. USAID will provide comments on the draft evaluation report during the 5 business days following submission of the draft. A second, revised draft shall be submitted for USAID comments before the final evaluation report is completed. 45 6. Final Report in English: A final report shall be submitted that incorporates and ade￾quately addresses all Mission comments received. The final report should not exceed 30 pages, excluding executive summary and annexes. 7. Thorough Executive Summary in Arabic: An executive summary (10-15 pages) in Ar￾abic will be provided as part of the final report. 8. One Page Report Summary in English, French and Arabic: A one page overview of evaluation findings in French, Arabic and English shall be provided as part of the final re￾port. 9. The final report shall be sent to the Development Experience Clearinghouse (DEC) within 30 days after completion of the evaluation. The final report is subject to approval by USAID and should clearly address each of the research questions and their supporting issues. The evaluation team shall submit a suggested table of con￾tents for the final report to USAID for approval. The table of contents should take USAID Eval￾uation Policy into consideration (see attached “USAID How To” note). The evaluation report will include the following appendices: • A copy of the scope of work; • Team composition and study methods (1 page maximum); • Full description of evaluation methods, including evaluation design matrix; • Any survey, interview, and other tools developed by the evaluation team; • A list of documents consulted, and of individuals, communes, associations, and partners contacted; • A list of sources of information (key informant, documents reviewed, other data sources). The Team Leader shall be responsible for providing the final report to USAID in electronic for￾mat (in Microsoft Word, Excel, maps in PDF and hard copies). Maps, graphs, charts, etc. should also be used appropriately throughout the report. The Executive Summary, and main report with￾out appendices, should also be presented as separate electronic documents for easier internal USG dissemination. Language of the final report is English. The Team Leader is expected to keep close communication with USAID/Morocco activity man￾ager throughout the evaluation (at minimum, 3x per week – via email or phone). All data and records in their raw form (i.e. questionnaires, surveys, observation notes etc.) shall be provided to the USAID/Morocco activity manager in an organized, electronic format. To help ensure the quality of the evaluation report, the evaluation team is asked to adhere to the following criteria: USAID Evaluation Policy: Criteria to Ensure the Quality of the Evaluation Report • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. 46 • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, un￾observable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical, and specific, with defined respon￾sibility for the action. • Recommendations should be detailed separate from findings and conclusions. For further guidance on USAID quality evaluation policies, please visit http://www.usaid.gov/evaluation. Timeline for Deliverables and Logistics USAID/Morocco expects that the overall length of this evaluation is about 7 weeks, including 3 weeks of fieldwork. The expected start date for this evaluation is February 2015. Illustrative schedule Week 1 and 2 • Day one: Post Award Conference/Meeting: A post-award conference will be held to re￾view the Statement of Work, clarify any questions that may arise, and address any con￾cerns related to methodology, and implementation timetable. • Logistical preparations, including hiring of research assistants. • Desk review of literature. • Develop and submit evaluation design and work plan with detailed methodology and de￾sign matrix to for USAID approval. Week 3: • Meet with USAID/Morocco to discuss proposed schedule and meetings list. • Submit evaluation instruments to USAID for approval. To the extent possible, test evalu￾ation instruments. • Team consultations/meetings, including early consultation with stakeholders (including country-level) on evaluation approach and instruments. • Work plan and instruments approved. Data collection process starts. 47 Week 4-5: • Interviews with stakeholders and staff in Morocco. • Meetings, interviews and field visits. • Debrief with USAID. • Presentation of preliminary results to implementing partners and stakeholders. Weeks 6 through 7: • Tabulate and Analyze data. • Submit draft report 1. • Incorporate USAID feedback into second draft. • Submission of draft report 2. • Incorporate any further USAID feedback before submission of final report. • Prepare PPT presentations and one-pagers for dissemination of results. Submit to USAID for approval. • Presentations of results to stakeholders: USAID, GOM counterparts, CSOs, other donors • Insert feedback from Mission in Report. Submit final report in English as well as a thor￾ough summary in Arabic and a one pager in Arabic, English and French before expat team member(s) depart Morocco. Logistics Travel: In general, in-country travel will be the responsibility of the evaluation team. Most in￾terviews and other meetings are anticipated to occur in the region of Tangier-Tetouan, in addi￾tion to Rabat. Taxis are readily available in Morocco and there is good train to the project’s tar￾get areas. Support: The evaluation team will be responsible for providing the workspace, office supplies, computers, communications (cell phone rental), and administrative services it may require. USAID will provide contact information for key informants and, in agreed cases, provide sup￾port in securing appointments, but in general the evaluation team will also be responsible for set￾ting up and managing most of its meetings. 48 49 ANNEX B: PERSONS INTERVIEWED USAID / Morocco Idriss Touijer FORSATY Project Manager (AOR), Office of Democracy/Governance Riad Berdayi Deputy FORSATY Project Manager, Office of Democracy/Governance Alice Rowley FORSATY Evaluation Manager, Program Office Alaa Eddin Serrar Office of Democracy and Governance Chuck Sweigman Head, Office of Democracy and Governance IOM / Morocco Ana Fonseca Chief of Mission to Morocco IOM / FORSATY Vincent Carbonneau Program Director Karim Bribri Specialist in Community Mobilization/Dynamics Ahmed El Haddioui Specialist in Capacity Development/Organizational Strengthening Abdenour Boukamhi Youth Specialist (head of professional placement) Khadija Ramran Education Specialist Loubna Lahssaini Support Agent in Education (Tangier) Mona El Hamdani M&E Advisor Hajar Benfaida Communications Specialist Hisham Bilali Support Agent in Education (Tetuan) Ali Dahdah Support Agent in Professional Placement (Tetuan) Jihane El Mechrah Support Agent in Community Mobilization/Dynamics (Tetuan) International Youth Foundation (FORSATY Partner) Imane Mourabiti Project Manager (Life Skills) Search for Common Ground (FORSATY Partner) Noufal Abboud Morocco Country Director RENAULT (FORSATY Partner) Salaheddine Sadik Director, Professional Relations (Morocco) Entraide Nationale Zineb Oulhajene Regional Coordinator (Tangier-Tetuan) Tangier Education Delegation Aziz Filali Sadouk Head of Non-formal Education Tetuan Education Delegation Abdelkader Mehdi Head, Anti-Illiteracy and Non-formal Education Saif Ennasr El Khayati Head, Bureau of Scholastic Support 50 Assadaka Social Complex (Association Raouabit Assadaka) Belkar El Mokhtar Complex Director (Entraide Nationale) Brahim Jerdouj FORSATY Project Coordinator Asiya Al Wahdi Middle School (Assadaka partner) Abdellatif Cheghraoui Director Imam Boukhari Elementary School (Assadaka partner) Rachid El Bakkali Director Association Chifae Tahar El Quor Association President Houda Zlali FORSATY Project Coordinator Amar Mlahi Association General Coordinator Anwar Zemmouri Administrative/Finance Officer Hassan Ibn Tabet Middle School (Chifae partner) Hassan El Ajy School Director and Chifae Vice President LARINOR (Chifae Internships) Sabah Azougagh Head of Intern Training Rokia Ben Mbarek Administrative and Finance Head OFPPT Assabil Center Samir Kadmiri Pedagogical Director Jamal Oulboub Director, Training Center Association Al Amal Aicha El Haddad FORSATY Project Coordinator Imane Lezaar Specialist in Employment Placement Sidi Saidi Elementary School (with Al Amal) Ahmed Lamchoudi Director Muhammed Hassan El Laiti Life Skills Instructor (volunteer) Ibn Sina Middle School Youssef El Morabit Director Entraide Nationale Center For Apprenticeship Training Hassan I (Al Amal interns) Nassarine El Bardouni Director Ahmed Fitian Automotive Mechanics Trainer 51 Auto Hall (hosting Al Amal interns) Bahaa Lebbar Agency Director Zoubir Amarir Head, Post-Sales Services Rachid Marjani Mechanics Technician (intern supervisor) Hotel Chams (hosting Al Amal interns) Es Soussi Ikbal Head of Personnel Restaurant Widji (hosting Al Amal interns) Brahim Chahboun Master Chef and Intern Trainer ANAPEC (Tangier) Latifa Rabbaj Regional Director Association Aide Et Secours Noureddine Temsamani Allouka Director Hajja Qtib FORSATY Project Coordinator Association Hay Benkirane Mohamed Taibi President Said Dakir Director Mohamed Akalaii Head of Orientation and Professional Placement Farid Zahraoui Head of Education Latifa Talha Educator UNFM (Union des Femmes Marocaines) Hafida Charif President Assia El Achouch FORSATY Project Coordinator Basma Boukir Administrative and Finance Head Najwa Rayane Head of Education Component Monaim El Achouch Head of Community Mobilization/Dynamics Nouria Aouinti Social Worker Association Atil Mohamed Fouad Amrani Director General Halima Haddi FORSATY Project Coordinator 52 ANNEX C: KEY DOCUMENTS REVIEWED Checchi. 2015. “Mid-term Performance Evaluation of the USAID/Morocco FORSATY Program.” Second Revised Technical Proposal. Washington, D.C. May 4, 2015. IOM. 2015. “FORSATY Interim Report to USAID: April 1 to June 30, 2015.” Tangier, Morocco. August 14, 2015. IOM. 2015. “FORSATY Interim Report to USAID: January 1 to March 31, 2015.” Tangier, Morocco. May 1, 2015. IOM. 2015. “FORSATY Monitoring, Evaluation, and Learning Plan: October 2014 to September 2017.” Tangier, Morocco. January 31, 2015. Revised on April 10, 2015. USAID. 2015. “Mid-term Performance Evaluation of FORSATY Program: Scope of Work.” USAID/Morocco. Cembrero, Ignacio. “Un tercio de los yihadistas marroquies procede del entorno de Ceuta. “ El Mundo, January 12, 2014. IOM. 2014. “FORSATY Interim Report to USAID: October to December 2014.” Tangier, Morocco. January 31, 2015. Resubmitted March 1, 2015. IOM. 2014. “FORSATY Annual Report: October 1, 2013 to September 30, 2014.” Tangier, Morocco. October 27, 2014. USAID. 2014. Program Description: FORSATY Extension. Rabat, Morocco. IOM. 2014. “FORSATY Baseline Report.” Tangier, Morocco. May 5, 2014. IOM. 2014. “FORSATY Performance Monitoring Plan.” Tangier, Morocco. January 2014. IOM. 2014. “FORSATY Interim Report to USAID: January to March 2014.” Tangier, Morocco. April 30, 2014. IOM. 2014. “FORSATY Interim Report to USAID: October to December 2013.” Tangier, Morocco. January 31, 2014. Mercy Corps. 2014. “Livelihoods Assessment.” Tangier, Morocco. January 2014. IOM. 2013. “Participatory Youth Assessment Report.” Tangier, Morocco. July 2013. IOM. 2013. “FORSATY Interim Report to USAID: April to June 2013.” Tangier, Morocco. July 25, 2013. IOM. 2013. “FORSATY Interim Report to USAID: January to March 2013.” Tangier, Morocco. April 22, 2013. USAID. 2013. “USAID/Morocco Country Development Cooperation Strategy: 2013 – 2017.” Rabat, Morocco. 53 USAID. 2012. “Cooperative Agreement with IOM. Attachment B: Program Description.” Rabat, Morocco. November 5, 2012. WORDE. Nd. “CVE Program Evaluation Worksheet.” No date. Washington, D.C. 54 ANNEX D: BENEFICIARY SURVEY QUESTIONNAIRE Final English Version October 8, 2015 Good morning/afternoon/evening. My name is [NAME]. I am an interviewer for Checchi Consulting. I am conducting research about your participation in programs for youth (FORSATY). You have been selected for an interview by random sampling. It is really important to us to hear about your experi￾ences and opinions during this interview. We are using these tablets to make it easier to record your responses clearly. All your responses and all your other information will be treated completely anonymously and confidentially – no names or phone numbers will be written down. The interview will last for approximately 30 minutes. Do you agree to participate? [Interviewer: Proceed only if participant gives full consent.] ------------------------------------------------------------------------------------------------------------------------------- Questionnaire A. Demographics and Background: (for all) 1. Interviewer, record the location of Interview: 1. BirChifa 2. DcharBendibane 3. Ancienne Medina 4. Boujarrah 2. Interviewer, record the gender of the participant: 1. Male 2. Female 3. How old are you? ___ 4. With whom do you live? 1. With family 2. With friends 3. Alone 4. Other 5. Where were you born? (Interviewer -- ask questions to categorize their place of origin by region and urban/rural status) 1. This neighborhood: If “this neighborhood” skip to 7. 2. Another area of this city 3. Another city in this region 4. Another city in a different region 5. A rural area in this region 55 6. A rural area in a different region 6. When did you move to this neighborhood? 1. 5 years+ ago 2. 3 – 4 years ago 3. 1 – 2 years ago 4. Less than a year ago 7. Are you earning money on your own now? Read answer choices and mark one. 1. Yes, in a full-time job 2. Yes, in a part-time job 3. Yes, I earn money in other ways 4. No, I do not early money on my own 8. How would you describe your relationship with your family? 4 – Very good, 3 – Somewhat good, 2 – Somewhat poor, 1 – Very poor, 0 – I have no family In the following school years, were you in school? [ (1) In school (2) Out of school ] 9. 2013/2014: [ (1) In school (2) Out of school] 10. 2014/2015: [ (1) In school (2) Out of school] 11. This year: [ (1) In school (2) Out of school] 12. What is the highest grade of formal education you completed? 1. None 2. Some Primary 3. Primary 4. Some Secondary 5. Secondary 6. Post-Secondary If In school 2013/2014 & 2014/2015 they will be asked only “In-School” questions If out of school 2013/2014 & 2014/2015 they will be asked only “Out of school” questions If some combination, they will be asked both B. Neighborhood Quality of Life and Safety: (FOR ALL RESPONDENTS) Considering this list of common problems, which do you think affects you and the quality of your life the most? Which is second? 13. Biggest problem: 56 1. Violence and crime 2. Lack of employment 3. Lack of educational opportunity 4. Family problems 5. Problems with friends and peers 6. General poverty 7. Political problems 8. Lack of opportunities for recreation 9. Lack of services like water, electricity and garbage collection 14. Second biggest problem: 1. Violence and crime 2. Lack of employment 3. Lack of educational opportunity 4. Family problems 5. Problems with friends and peers 6. General poverty 7. Political problems 8. Lack of opportunities for recreation 9. Lack of services like water, electricity and garbage collection In the following places/situations, how safe do you feel? [Response Format: 4: Safe, 3: somewhat safe, 2: somewhat unsafe, 1: unsafe, 0 does not apply] 15. In your own home : 4 - 3 - 2 - 1 - 0 16. In the street : 4 - 3 - 2 - 1 - 0 17. At school : 4 - 3 - 2 - 1 - 0 18. Leisure places (cafes / cyber cafes etc) : 4 - 3 - 2 - 1 - 0 19. On public transport : 4 - 3 - 2 - 1 - 0 20. Thinking about the last two years, how has security in your neighborhood changed? 1. Worse 2. Same ( If “same” skip to 22). 3. Better 21. Why do you think things have changed? Mark all that the respondent mentions. 1. Change in police 2. Change in community 3. Change in politicians 4. Change in NGOs 5. Change in youth 6. Change in criminals 7. Change in economic situation 8. Change in social situation 9. Other (Specify: ________) 57 People use violence for various reasons. In the following cases, do you think violence is of￾ten justified, sometimes justified, or never justified? [Response Format: 1. often justified, 2.sometimes justified, 3. never justified] 22. To defend yourself against physical attack 23. To defend yourself against insults or verbal threat 24. To defend your family or friends against insults or verbal threat 25. To defend your religion or culture against insults or verbal threat 26. Have the activities of the CSO with youth here changed the quality of life in the neighborhood? How? 1. Worse ( If “same” skip to 28). 2. Same ( If “same” skip to 29). 3. Better 27. What kind of improvement have you seen? [Mark all that apply] 1. Youth are more successful 2. Youth are happier 3. Parents are happier 4. Community is improved 5. Better communication in community 6. Other (Specify: ___________) 28. Why do you think that things have gotten worse? [Open] ------------------------------------------------------------------------ C. In-School Youth (ONLY ASK FOR THOSE IN SCHOOL DURING 2013/2014 OR 2014/2015) Academic Tutorials 29. Have you received academic tutoring over the last 2 years? 1. Yes 2. No (If No, skip to 42) 30. When did you receive tutoring? Mark all that apply. 1. 2013/2014, 2. 2014/2015 31. In total, how many months of tutoring did you receive? ____ 32. What subjects did you receive tutoring in? Mark all that apply 1. Mathematics, 58 2. Arabic, 3. French, 4. Sciences, 5. Other:…………………………………………… 33. Did you receive special tutoring for the 6th Grade examination or the 9th Grade examination? 1. Yes 2. No 34. Did you take a placement (positioning) test before tutoring? [Y/N] 1. Yes 2. No If No, skip to 36 35. Do you think that the placement test put you in the right tutorial class? 3. Yes 4. No 36. Do you feel the teaching methods were right for you and your particular academic needs? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree. 37. Was the tutoring successful in your opinion? 1. Yes 2. No 38. Did your grades change as a direct result of the tutorial? 1. Yes 2. No 39. If yes, how? 1. Got worse, 2. Same, 3. Improved 40. Do you believe you have changed as a person as a result of the tutorial experience? 4. Disagree, If 4 skip to 45. 3. Somewhat Disagree, If 3 skip to 45. 2. Somewhat Agree, 1. Agree. 41. If Agree/ Somewhat Agree, how? Mark all that apply. 1. More confidence 2. More focused on education 59 3. Better path to career/higher education 4. Other (Specify: ______) Extracurricular and Life Skills Activities 42. Did you participate in extracurricular and life skills activities over the last 2 years? 1. Yes 2. No If No, skip to 59: For each of the following activities, tell me if you participated. If so, which year? For how many months? Table: 2013/2014 (check if yes), 2014/2015 (check if yes), Number of total months ____ (enter if either year is checked) 43. Sports: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 44. Life Skills: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 45. Theater: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 46. Excursions: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 47. Educational/cultural games: 1) Yes 2) Now 3) 2013/2014 4) 2014/2015 / Number of total months: ……….. 48. Singing: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 49. Other: 1) Yes 2) No 3) 2013/2014 4) 2014/2015 / Number of total months:………. 50. Were you able to choose which activities to engage in? 1) Yes 2) No 51. Did you find these activities well carried out? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree. 52. Which of these activities were most valuable to you personally? 1. Sports 2. Life Skills 3. Theater 4. Excursions 5. Educational/cultural games 6. Singing 7. Other 60 53. Do you believe you have changed as a person as a result of the activities? 1. Yes 2. No If No, skip to 59 54. If yes, how? 1. Worse , If worse, skip to 56 2. Same, If same, skip to 57 3. Improved 55. How did you improve as a result? Mark all that apply? - More optimistic - More organized - More outgoing - Healthier - Better social life - Other (Specify: _______) 56. Why do you say “worse”? [Open] 57. Are there any activities that were not included that you would like to be in next time? 1. Yes 2. No If No, skip to 59 58. What activities would you like to see next time? [Open] ……………………………………… ………………………………….. Summer Camp Activities 59. Did you participate in a summer camp in the last 2 years? 1. Yes 2. No If No, skip to 124 60. What year (s) did you participate? Mark all that apply. 1. Summer of 2013 2. 2014, 2015 61. What was the organization that ran the camp? [Choices?] 1. Assadaka 2. Amal Féminine 3. Atil 4. UNIFEM 61 5. Aide et secoure 6. Chifae 7. Hay Benkirane 62. What camp activities did you engage in personally? Mark all that apply 1. Academic strengthening 2. Sports 3. Hiking 4. Art and drawing 5. Theater 6. Games 7. Field Trips 63. What was the one activity most appreciated by you personally in the camp experience? 1. Academic strengthening 2. Sports 3. Hiking 4. Art and drawing 5. Theater 6. Games 7. Field Trips 64. Do you feel personally changed by this camp experience? 1. Yes 2. No If No, skip to 124 65. If yes, how? 1. worse, If worse, skip to 67 2. same, If same, skip to 124 3. improved 66. How did you improve as a result? Mark all that apply? 1. More optimistic 2. More organized 3. More outgoing 4. Healthier 5. Better social life 6. Other (Specify: _______) Skip to 68 (IF the student should be asked Out of School questions) 67. Why do you say worse? [Open] ----------------------------------------------------------------------------------------------------------------------------------- 62 D. Out of School Youth (ONLY ASK OF STUDENTS WHO WERE OUT OF SCHOOL 2013/2014 OR 2014/2015) Summer Camp Activities Reads as summer camp module above Non-formal Education (NFE) 68. Did you participate in non-formal education classes over the last 2 years? (Y / N) 1. Yes 2. No If No, skip to 76 69. Which type of NFE class did you participate in: under 15 or 15 and over? 1. Under 15, (If under 15, skip to 72) 2. 15 and over 70. Were you assisted in accessing vocational training, if you did not continue in the NFE? Mark one choice. 1. I did continue in NFE 2. I did not continue, but I did get help accessing vocational training 3. I did not continue, and I did not get help accessing vocational training 71. What subjects did you receive tutoring in? Mark all that apply. 1. Mathematics, 2. French, 3. Arabic, 4. Sciences, 5. Other 72. Do you feel you were placed in the right activities for your academic needs? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree 73. Were you able to pass the examination you were preparing? 1. Yes 2. No 74. Were you satisfied with the academic assistance you received? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree 75. Did you participate also in life skills training? 1. Yes 63 2. No Youth with Internships 76. Did you participate in an internship activity during the last 2 years? 1. Yes 2. No If No, skip to 89 77. In what years? Mark all that apply. 1. 2013, 2. 2014, 3. 2015 78. For how many total months? ______ 79. What were you doing before the internship? 1. School 2. Looking for a job 3. Part-time work 4. Sitting idle 5. Other: _____________ 80. Was the internship paid? 1. Yes 2. No 81. Were you able to choose the type of internship? 8. Yes 9. No 82. Do you feel the experience was what you were seeking? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree 83. Were you offered employment in the firm after the internship? 1. Yes 2. No If yes, skip to 85 84. Do you feel you have the skills necessary to be employed by that firm? 1. Yes 2. No 85. Do you feel personally changed by the internship experience? 1. Yes 64 2. No If No, skip to 89 86. If yes, how? 1. Worse , If worse, skip to 88 2. Same, If same, skip to 89 3. Better 87. How did you improve as a result? Mark all that apply? 4. More confident 5. More optimistic 6. More skilled 7. More organized 8. Other (Specify: _______) 88. Why do you say worse? [Open] ……………………………………. Youth in Life Skills (Work Readiness) Training 89. Have you participated over the last 2 years in life skills or work readiness training? 1. Yes 2. No If No, skip to 99 90. When did you participate? Mark all that apply. 1. 2013, 2. 2014, 3. 2015 91. For how many total months? ____ 92. What type of sessions did you participate in? Mark all that apply. 1. Self-esteem 2. Leadership 3. Decision-making 4. Problem resolution 5. Other 93. Do you feel this training was personally useful to you? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree. 94. Do you feel personally changed by the experience? 1. Yes 65 2. No If No, skip to 98 95. If yes, how? 1. Worse, If worse, skip to 97 2. Same, If same, skip to 98 3. Better 96. How did you improve as a result? Mark all that apply? 1. More confident 2. More optimistic 3. More skilled 4. More organized 5. Other (Specify: _______) 97. Why do you say worse? [Open] ………………………………………… 98. What other training could be added to these life skills to help you with employment? [Open] …………………………………………………… Youth in Project-Supported Vocational Programs 99. Have you participated in a vocational training program over the last 2 years? 1. Yes 2. No (If No, skip to 112) 100. If yes, when did you participate? Mark all that apply. 1. 2013, 2. 2014, 3. 2015 101. For how many total months did you participate?____ 102. What organization directly provided this vocational training to you? 1. Assadaka 2. Amal Féminine 3. Atil 4. UNIFEM 5. Aide et secoure 6. Chifae 7. Hay Benkirane 66 103. Were you able to choose the type of training you wanted? 1. Yes 2. No 104. Do you feel you learned the skills that you need to find work of the same type? 4) Disagree, 3) Somewhat Disagree, 2) Somewhat Agree, 1) Agree. 105. Were you given life skills training as well during the vocational training? 1. Yes 2. No 106. Were you given work readiness training during the vocational training? 1. Yes 2. No 107. Did this help you find employment? 1. Yes 2. No 108. Do you feel personally changed by the experience? 1. Yes 2. No If No, skip to 112 109. If yes, how? 1. Worse, If worse, skip to 111 2. Same, If same, skip to 112 3. Better 110. How did you improve as a result? Mark all that apply? 1. More confident 2. More optimistic 3. More skilled 4. More organized 5. Other (Specify: _______) Skip to 112 111. Why do you say worse? [Open] Youth in Project-Supported Career Information and Counseling Activities 112. Did you participate in a vocational counseling activity in the last 2 years? 1. Yes 67 2. No (If No, go to 124) 113. In what years? Mark all that apply. 1. 2013, 2. 2014, 3. 2015 114. For how many total months? ____ 115. Which organization provided this to you? 1. Assadaka 2. Amal Féminine 3. Atil 4. UNIFEM 5. Aide et secoure 6. Chifae 7. Hay Benkirane 116. Was this a neighborhood career caravan activity? 1. Yes 2. No What was the result of this counseling activity? Did you. . . 117. Get interviews? : 1) Yes 2) No 118. Get a job? : 1) Yes 2) No 119. Get more organized in your search? : 1) Yes 2) No 120. Do you feel personally changed by the experience? 1. Yes 2. No If No, skip to 124 121. If yes, how? 1. Worse, If worse, skip to 123 2. Same, If same, skip to 124 3. Better 122. How did you improve as a result? Mark all that apply? 1. More confident 2. More optimistic 3. More skilled 4. More organized 5. Other (Specify: _______) Skip to 124 68 123. Why do you say worse? [Open] Now I have just two final questions. 124. Overall, how would you describe your experience with the youth activities we just discussed? (Read choices): 4 – Very beneficial, 3 – Somewhat beneficial, 2 – Not beneficial, 1 – No answer/ don’t know 125. Is there anything else you want to share about your experience? [Open] ………………………………………………………..; …………………………………………………………. THANK YOU VERY MUCH FOR YOUR COOPERATION! 69 ANNEX E: FOCUS GROUP AND INTERVIEW GUIDES A. Beneficiary Focus Groups Focus Group size: 8-10 persons Time: 1 hour 1. Focus Group of In-School Girls and Boys (mixed) 2. Focus Group of Out-of-School Girls 3. Focus Group of Out-of-School Boys Basic themes: • Activities or services provided • Who provided these? • Personal participation in activity selection • Satisfaction with the services/activities • What worked well and what did not? • What direct personal results/outcomes resulted? • What permanent personal changes resulted? • What changes occurred in others or in the community as a whole? • Recommendations for improvements in the future. Major Activities/Services for In-School Youth (1) Youth in academic tutorials (2) Youth in extracurricular activities and life skills clubs Major Activities/Services for Out-of-School Youth (1) Youth in non-formal education classes (2) Youth in internships or given employment (3) Youth in work readiness, life skills development, and entrepreneurial training (4) Youth in project supported vocational programs (5) Youth in project-supported information, vocational counseling, or job placement activities Discussion Points: • Please introduce yourselves and give us your age and neighborhood. • Please tell me about which activities you participated in and when. • Who provided these activities and were they of good quality in your opinion? • Why do you feel you were selected for participation in these activities? 70 • How much were you able to freely choose your activities? • Please tell me how satisfied you were with the way these activities were conducted? • What worked well and what did not in these activities in your opinion? • Please tell me what direct results you experienced personally from these activities? • Do you feel that you permanently changed from these activities and in what ways? • What changes occurred to others or to your neighborhood as a result of these activities? • Do you have any recommendations for improving these activities in the future? B. Beneficiary Parents Focus Groups Focus Group size: 8-10 persons Time: 1 hour Parents Focus Groups 1. Mixed Focus Group of In-School Beneficiary Parents 2. Mixed Focus Group of Out-of-School Beneficiary Parents Basic themes: • Activities or services provided • Who provided these? • Parental participation in activity selection or implementation • Satisfaction with the services/activities • What worked well and what did not? • What direct children’s results/outcomes resulted? • What permanent children’s personal changes resulted? • What changes occurred in other youth or in the community as a whole? • Recommendations for improvements in the future. Major Activities/Services for In-School Youth (1) Youth in academic tutorials; (2) Youth in extracurricular activities and life skills clubs Major Activities/Services for Out-of-School Youth (1) Youth in non-formal education classes (2) Youth in internships or given employment (3) Youth in work readiness, life skills development, and entrepreneurial training (4) Youth in project supported vocational programs (5) Youth in project-supported information, vocational counseling, or job placement activities. 71 Discussion Points: • Please introduce yourselves and tell us your neighborhood. • Please tell me about which activities your children participated in and when. • Who provided these activities and were they of good quality in your opinion? • Why do you feel your children were selected for participation in these activities? • How much were they or you able to choose your children’s activities? • Please tell me how satisfied you were with the way these activities were conducted? • What worked well and what did not in these activities in your opinion? • Please tell me what direct results your children experienced personally from these activities? • Do you feel that your children permanently changed from these activities and in what ways? • What changes occurred to other children or to your neighborhood as a result of these activities? • Do you have any recommendations for improving these activities in the future? C. Semi-Structured Interview Guides Group or individual semi-structured interviews: 1. Implementers (IOM and One-Stop-Shop CSOs, SFCG) 2. Implementing partners and other stakeholders (donors, small CSOs, schools, government entities) Following lead-in questions by interviewers in each of these themes (not provided here), follow-up and probing will occur to explore fully the responses given by interviewees. Interviews may last up to an hour, depending on the richness and complexity of the responses. Implementer Perceptions Themes: • Progress toward FORSATY objectives (Phase I) • Progress toward FORSATY objectives (Phase II) • Melding of Phase I and Phase II objectives and orientations • Progress toward the CSO capacity strengthening objective • Progress toward the cross-cutting community environment objective • Effectiveness of various component contributions to project objectives • Beneficiary impacts on in-school and out-of-school beneficiaries • Neighborhood impacts and future Community-Oriented Policing component • Organizational coordination • Mobilization and involvement of various youth-serving actors and organizations • Issues in organizational capacity between implementers (IOM and CSOs) • Effectiveness of organizational assessments and action plans • Replicability and sustainability of project accomplishments • Recommendations for future emphases and actions 72 Partner and Stakeholder Perceptions (below and around IOM/CSOs) Themes: • Nature, relevance, and coherence of FORSATY objectives • Progress made toward these objectives • Involvement in capacity strengthening activities • Impact on youth of project-supported activities and services • Impact on neighborhoods of project-supported activities • Organizational coordination between implementers and partners • Mobilization and involvement of various youth-serving actors and organizations • Issues in organizational capacity • Replicability and sustainability of project activities and outcomes • Recommendations for future emphases and actions 73 ANNEX F: FORSATY MID-TERM EVALUATION MATRIX Evaluation Questions Data Collection Methods and Sources 1a. To what extent is the FOR￾SATY project on target to achieving its intended results? iv. Marginalized and disaf￾fected out of school youth are successfully in￾tegrated into society. v. In school youth receive improved support to in￾crease retention and suc￾cess and decrease drop out. vi. Marginalized and disaf￾fected youth enjoy a safer community envi￾ronment (cross-cutting objective). 1b. What do targeted benefi￾ciaries (including male and fe￾male at-risk youth) and other actors see as the most signifi￾cant changes (positive or nega￾tive) to which the project has contributed, and what else is re￾quired? 1c. Which project components are the most and least effective in contributing to the project objectives and may have the most or least promise for the project going forward? • Survey of a random stratified sample of project benefi￾ciaries (male and female at risk youth) • Focus Group Discussions with a random stratified sam￾ple of project beneficiaries • Focus Group Discussions with a random sample of pro￾ject beneficiaries’ families • Focus Group Discussions with participating CSOs (group interviews) • Semi-structured interviews with relevant implementers and stakeholders: - IOM - 4 original CSOs - 3 new CSOs - School directors and teachers - IYF - National and regional education officials - OFPPT - ANAPEC - EN - INAS - SFCG - Other key informants • Desk review, including - Participatory Youth Assessment (PYA) - Livelihoods Assessment - FORSATY Annual Report (2013-2014) - Project Description: FORSATY Extension - FORSATY Interim Reports FY 2015 (1-3) - Existing performance monitoring and evaluation data - Existing CSO capacity assessments (OCAs) - Existing capacity development action plans - 2a. To what extent is the FOR￾SATY project on target to achieving the CSO capacity building intended result: youth • Desk Review: Existing CSO capacity assessments (OCAs) and Action Plans • Focus Group Discussions with participating CSOs 74 service organizations are rein￾forced to provide better ser￾vices for at risk youth/young of￾fenders? 2b. In order to achieve CSO ca￾pacities that are likely to be sus￾tained beyond the life of the pro￾gram, what aspects of CSO capac￾ity strengthening should be adopted, adapted or replicated within existing regions and to new regions? 2c. How useful do participating CSOs perceive the institutional action plans and how could they be supported to improve them? 2d. What is required to establish a sustainable system of mentoring and cooperation between the par￾ticipating CSOs? • Survey of a random stratified sample of project benefi￾ciaries (male and female at-risk youth) • Focus Group Discussions with a random stratified sam￾ple of project beneficiaries • Semi-structured interviews with implementers and stakeholders: o IOM o 4 original CSOs o 3 new CSOs o School directors and teachers o SFCG o IYF o National and regional education officials o OFPPT o EN o INAS o Others 3. How effectively did the pro￾ject mobilize and foster rela￾tionships between the different actors serving at-risk youth (in￾cluding CSOs, government ser￾vices and departments, the pri￾vate sector, families and the youth themselves), over the course of the project? • Semi-structured interviews with implementers and rele￾vant stakeholders: - IOM - 4 old and 3 new CSOs - School directors, teachers, tutors - SFCG - IYF - National and regional education officials - OFPPT - ANAPEC - EN - INAS - Others • Focus Group Discussions with a random stratified sam￾ple of project beneficiaries • Focus Group Discussions with a random sample of pro￾ject beneficiaries’ families • Focus Group Discussions with participating CSOs. 75 ANNEX G: FORSATY PHASE I and PHASE II RESULTS, BY SEX Table 2: FORSATY Results, Phase I Indicators Baseline / Achieved Y1 Target YR1 & 2 Achieved YR2 Cumulative Intermediate Result 1: Out-of-School Youth are Successfully Integrated into Society 1. # of target youth that have returned to school after dropping out 0 60 27 (21M, 6F) 27 (21M, 6F) 2. # of youth that have integrated professional training programs 202 500 964 (728 M, 236F) 964 (728 M, 236F) 3. # of youth employed and/or self-employed and/or undertak￾ing internships 48 400 476 (390M, 86F) 476 (390M, 86F) 4. # of youth who have completed training classes 48 320 374 (290M, 84F) 374 (290M, 84F) Intermediate Result 2: In-School Youth Receive Support to Increase Retention and Success and Decrease Drop-out 1. # of target in-school youth who receive school support 184 1,600 2,674 (1,499M, 1,175F) 2,674 (1,499M, 1,175F) 2. # of institutional school support systems created through partnerships between CSOs and education public institutions 0 1 1 1 3. # of tutorial after-school support classes offered 39 1,800 2,277 2,277 4. # of target in-school youth who participate in extracurricular activities 116 1,800 2,403 (1,728M, 675F) 2,403 (1,728M, 675F) Intermediate Result 3: Youth-Serving Organizations Provide Better Services for At-Risk Youth 1. Quality of services provided to at-risk youth NA NA 0 0 2. % of youth satisfied with services provided by target youth￾serving organizations 44% 60% 0 0 3. # of youth-serving professionals trained 0 320 362 (331M, 31F) 362 (331M, 31F) Cross-Cutting Intermediate Result: Marginalized and Disaffected Youth Enjoy a Safer Community Environment 1. % of youth demonstrating better understanding of relevant youth issues identified during the PYA 28% 50% 0 0 2. # of target youth who benefit from adapted psychological support 0 500 762 (281M, 481F) 762 (281M, 481F) 3. # of innovative awareness campaigns held around relevant youth issues identified during the PYA 0 25 39 39 4. # of youth participating in innovative awareness campaigns held around relevant youth issues identified during the PYA 0 1,800 3,726 (2,425M, 1,301F) 3,726 (2,425M, 1,301F) 5. # of youth participating in neighborhood-based volunta￾rism/civic activities 0 400 419 (261M, 158F) 419 (261M, 158F) 76 Table 2: FORSATY Results, Phase II10 Intermediate Objective 1: Youth Assets Built Support Outcome 1.1: In-school Youth Academic Achievement Improved Indicators Target FY 15 Achieved FY15 Comments 1. % of at-risk school youth participating in project supported after school tutoring pro￾gram whose grades im￾prove 75% 64% (F 49%; M 51%) (Primary: 75%) (Secondary: 52%) Major difficulties were faced with students coming from middle school be￾cause they have very low academic standings. The tutoring program was adapted to work with these youth to strengthen their basic learning skills (that should have been treated in Primary) before attacking academic per￾formance, which did not immediately reflect on their grades. Compared to the previous year OSSs were for the first time able to report on students’ progress using simplified monitoring tools. FORSATY has worked with OSSs to establish that effective tutoring is not about enrolling students and delivering tutoring, it is more about measuring progress against deployed efforts. 2. # of at-risk, in-school youth who attend FOR￾SATY IP after-school tutoring programs Target 1,260 1,779 (863 F; 916 M) During the previous year 2,674 students benefited from school tutoring program compared to 1779 students for the current year. FORSATY has worked with its partner OSSs to limit the number of enrolled students in function of available resources. Controlled numbers give an opportunity to the associations and to the project to put in place more efficient monitor￾ing systems and test improved education approaches and solutions. 3. # of teachers in tar￾get schools trained by project on “Passport to Success” life skills methodology Target 75 52 (18 F; 34 M) The Life Skills TOT was planned to be conducted in partnership with the delegation of Education in Tétouan for a large number of NFE educators. However, because of IYF’s busy schedule and limited human resources this TOT was postponed to the upcoming year (pending additional funding). Compared to the previous year the Education Delegations in Tangier and Tétouan showed greater interest and deeper commitment to the imple￾mentation of more Life Skills programs in public schools. The results have been outstanding among teachers and students as Life Skills helped improve the general learning environment in classrooms. 4. # of after school clubs that implement at least 6 life skill modules Target 35 30 When the Life Skills TOT was conducted last year with teachers, only 19 were able to establish Life Skills clubs in their respective school and many did not restart in September 2014. Thanks to regular follow-up and tech￾nical support from FORSATY, this year 21 more clubs were created and they are holding regular Life Skills sessions with more than 800 students. 5. # of school initiatives organized by target school PTAs or other parent-involved school governance committees in partnership with CSO IPs Target 29 42 The anti-violence program conducted by ATIL and Youth Wednesdays or￾ganized by Assadaka are the main programs that engaged parents. Other OSSs organized events with parents, but the approach was weak and is in need of improvement. Parental involvement remains one of the main chal￾lenges faced by partner CSOs because parents are difficult to approach. Fa￾thers continue to be absent and disengaged; mothers are present but they take no action with CSOs or with schools. PTAs are also very difficult to approach, and they are either inactive or totally disengaged in the majority of target schools. Support Outcome 1.2: Youth Participation in Activities that Build Internal Assets Increases 10 Baseline was “N/A” or “0” for each indicator. 77 Indicators Target FY15 Achieved FY15 Comments 1. % of target youth who improve their score on the project Internal asset self-assessment Target 70% N/A The test will be conducted retrospectively over the upcoming year. It was postponed to give enough time to beneficiaries to grasp the concept of in￾ternal assets. Their answers will be more grounded if they can understand the concepts they will be asked to evaluate. 2. # of CSO educators who receive training from the project Target 61 113 (31F, 82M) These educators include animators/educators from CSOs working with youth aged 18+. They had a leading role in organizing youth and supporting them to work on different productions. The number of trainees increased significantly because of the growing number of CSOs interested in taking part in FORSATY community-based activities. Educators also demonstrated a high interest in training because of the very limited opportunities they have to sharpen their capacities. 3. # of CSO educators/ other staff who partici￾pate in IYF Life Skill training programs Target 35 0 Because of IYF’s busy schedule and limited human resources this TOT was postponed to the upcoming year (pending additional funding). Support Outcome 1.3: Out-of-School Youth Access to Educational Opportunities Increased Indicators Target FY15 Achieved FY15 Comments 1. % of FORSATY sup￾ported NFE program participants who rein￾tegrate into formal ed￾ucation or vocational training or obtain a pri￾mary, brevet or bac certificate (diploma) N/A 130 (90%) An exceptionally high reintegration rate was recorded mainly because of the result of the NFE program for youth aged between 9 and 15. The socio-psy￾chological support provided, in addition to academic support by educators, was very effective in helping more youth reintegrate the formal school sys￾tem. Youth enrolled under NFE programs are generally youth with major social, economic and family problems, which make them a very challenging category to work with. They require very close socio-psychological follow-up in paral￾lel with academic learning. They are prone to dropping out, even after inte￾gration, compared to any other category of youth. 2. # of target out-of￾school youth aged 15+ who complete IP NFE program Target 0 35 This program yielded very mixed results as OSS reported very contrasted results. This situation triggered an immediate reaction from FORSATY to reevaluate its NFE 15+ strategy and adjust it to guarantee a more stable and competitive program that can allow youth to access real integration oppor￾tunities in education and professional integration. 3. # of target out-of￾school youth aged <15 who complete IP NFE program Target 25% 110 (42F, 68M)* Refer to comment on indicator 1. 4. % of target out-of￾school youth partici￾pating in an IP NFE program who receive vocational counseling services Target 25% 73% (60 out of 93) en￾rolled This performance concerns youth enrolled under the NFE 15+ program. Af￾ter detecting unsatisfactory results among a significant number of youth in the NFE 15+ FORSATY with its partner OSSs started conducting exit plans and referring youth to career counseling services, especially those with no real chance of completing the program. # of at-risk students who benefitted from Life Skills activities A part of the at-risk youth taking part in these activities also benefited from the school tutoring pro￾grams offered by partner OSSs (roughly 1779). These activities are very popular especially among in￾school students, because they provide a rare opportunity to explore their talents and stimulate their creativity. They also have been linked, according to educators, faculty members and schools princi- # of at ples, to improved academic performance and better behavior among students. -risk students benefitted from parallel activities offered by partner IPs Intermediate Objective 2: Youth Participation in Local Governance Increased 78 Support Objective 2.1: Youth Advocacy for Marginalized Youth Needs Increased/Improved Indicators Target FY15 Achieved FY15 Comments 1. # of advocacy actions in favor of marginalized youth needs car￾ried out by project supported community youth leaders Target 5 0 This result is implemented through FORSATY sub-grantee Search for Common Ground (SFCG). The activities programmed under this results were delayed (upon SFCG request and FORSATY’s ap￾proval) until SFCG performs a ground conflict assessment to better plan its strategy. SFCG will use the results of this assessment to im￾prove its intervention and its plans to create youth councils and en￾sure the implication and engagement of local actors. 2. Proportion of target popula￾tion reporting increase agree￾ment with the concept that males and females should have equal ac￾cess to social, economic and po￾litical opportunities Target N/A 0 3. # of community youth leaders in target neighborhoods who par￾ticipate in project supported training activities Target N/A 0 The activities programmed under this result were scheduled to begin in January. Discussions and implementation issues on a modi￾fied scope of work delayed activities by 3 months. The original scope was reintroduced and SFCG launched in early April. SFCG staffing issues further delayed the work plan. Support Objective 2.2: Elected Officials and Local Authorities’ Use of Youth Inputs to Inform Youth Policies Increased Indicators Target FY15 Achieved FY15 Comments 1. # of youth policies created, modi￾fied or implemented taking into con￾sideration community youth input Target 0 0 The activities programmed under this result were scheduled to begin in January. Discussions and implementation issues on a modified scope of work delayed activities by 3 months. The original scope was reintroduced and SFCG launched in early April. SFCG staffing issues further delayed the work plan. 2. # of elected officials and other local authorities who participate in project supported capacity building programs Target 15 0 Intermediate Objective 3: Improved Youth Access to Jobs Support Outcome 3.1: Youth Job Qualifications Enhanced Indicators Target FY15 Achieved FY15 Comments 1. # of target youth who are employed or under￾taking an internship Target 690 1006 (379F;627 M)* The partnership and cooperation links between partner OSS and ANAPEC helped a higher number of youth access the job market. The work readiness program (job search, CV, interviews, administration paper-work) has tremen￾dously increased youth’s chances in passing interviews and getting hired with a number of businesses. 2. # of target youth par￾ticipate in work readiness activities, life skills devel￾opment programs, or/and entrepreneurship training programs Target 425 1333 (517 F;816 M)* In addition to the work readiness program implemented by partner OSSs and ANAPEC, Life Skills sessions have been provided to a large number of youth. These sessions tremendously help youth work on their interpersonal skills in preparation to the job market. Time management, communication and conflict resolutions are one of the big values enterprises look for when hiring youth. The number of participating youth increased significantly because three public training centers and seven Centre de Formation par l’Apprentissage -IE held reg￾ular life skills session with youth in their training workshops. 3. # of target youth who complete a project sup￾ported vocational training program Target 704 718 (447 F; 271 M)* The target has been surpassed due to increased commitment and investment by the project and its partners, in response to high demand from potential benefi￾ciaries. Support Outcome 3.2: Youth Job Qualifications Enhanced 79 Indicators Target FY15 Achieved FY15 Comments 1. # of project supported training programs that form partnerships with employers Target 3 12 As associations recognized the added value of establishing partnerships with the private sector from last year, many increased their efforts to sign part￾nership this year. These partnerships allow OSSs to offer better integration chances for youth; while, they allow private sector companies to access qualified labor with adequate training and, most looked for, professional conduct. 2. # of CSO personnel and other training center trainers and other person￾nel who participate in project supported train￾ing, forums or round table meetings Target 70 200 FORSATY has created a number of exchange opportunities between OSSs and the private sector and youth’s employability. The event held with Confédération Générale des Entreprises Marocaines has been very success￾ful and has attracted a significant number of local and regional actors. 3. # of target youth who participate in FORSATY supported CSO and train￾ing center vocational ori￾entation or job placement services Target 2100 2299 (812 F; 1 487 M) Information campaigns have been very well received among youth in mar￾ginalized neighborhoods, because of the lack of information on available training and placement opportunities. Public training institutions have also been very keen to participate in such events, as they provided them with an outreach opportunity in neighborhoods they usually do not cover. Carrier counseling services established within partner association and cer￾tain public centers attracted a very important number of youth seeking counseling at the neighborhood level. The reputation of these centers is growing by the day, attracting larger numbers of youth. Cross-cutting Support Objective 4.1: Youth-Serving Organizations’ Capacities to Provide Adequate Services to Youth At Risk Improved Indicators Target FY15 Achieved FY15 Comments 1. # of FORSATY CSO implementing partners who diversify their sources of fund￾ing for core project activities Target 2 3 in pro￾gress Chifae, AL Amal, and Assadaka OSSs have successfully created financial re￾source development committees, in charge of diversification of funding sources. So far they have responded to several requests for projects and have been trained and supported in this process by FORSATY. These OSSs were not able to raise actual funding during 2015, but their pro￾ject proposals have been accepted and are being examined. They have very good chances to engage new funds during the upcoming year. 2. # of FORSATY CSO implementing partners that reach 50% or more of their OCA action plan ob￾jectives Target 6 3 Chifae, AL Amal , and Assadaka were able to achieve more than 50% of their OCA action plans for FY2015. Hay Benkirane, Aide et Secours, and UNFM have not gone through their organizational capacity assessment workshop (OCA) during FY2015 in order to develop an action plan. OCA could not be conducted with FORSATY’s new partners because priority was given to the following: • Strengthening of technical and financial planning, coordination and reporting capacities • Sufficient follow-up time for implemented activities • Adopt and master adequate management methods for program funds • Enough learning and integration time for the staff members newly hired to work on project activities 80 Cross-cutting Support Objective 4.2: Marginalized Youth’s Safety and Security at the Community Level Improved Indicators Target FY15 Achieved FY15 Comments 1. # of community improvement ac￾tions implemented by FORSATY imple￾menting partners in collaboration with second sphere CSOs in 7 target neigh￾borhoods Target 4 50 Youth who are not affiliated with any formal sys￾tem represent the majority in targeted marginal￾ized neighborhoods. FORSATY is working with these youth to bring them around activities pro￾moting positive values and favoring personal growth. The absence of activities at the com￾munity level explains the high demand from youth on FORSATY activities organized with lo￾cal CSOs. 2. # of youth who participate in com￾munity improvement actions imple￾mented by FORSATY implementing partners in collaboration with second sphere CSOs in 7 target neighborhoods Target 150 704 (15F; 689 M) NB: the gender disaggregation in marked indicators (*) was calculated based on the gender ratio extracted from the gender disaggregation reported by partner OSSs concerning enrolled youth. This method was adopted because certain associations are still struggling with reporting exact gender disaggregation for marked indicators. 81 ANNEX H: RESULTS OF FOCUS GROUP DISCUSSIONS Twelve focus group discussions (FGDs) were held, four in each of the three OSS association chosen for examination in the mid-term evaluation (Raouabit Assadaka, Bir Chifae, and Al Amal). FGDs included in each place: (1) a mixed group of in-school youth, (2) a group of male out-of-school youth, (3) a group of female out-of-school youth, and (4) a mixed group of parents of both in-school and out-of-school youth, although only mothers actually showed up. According to discussion leaders, there is little difference between male and female responses in focus group discussions, but the following generalizations can be made. • In extracurricular activities, females are more interested in creative activities than males. Theater, painting, and cinema are preferred by women, while sports (particularly football) are preferred by young men. • Females insist more than males on the necessity to increase teaching hours for academic tutorials. • In their satisfaction with the FORSATY project, out-of-school females appreciate equally much the vocational component and the cultural and extracurricular activities, using the terms of skills qualification, self-expression, freedom, and creativity. On the other hand, males refer more to the vocational component and use expressions of skills qualification, employment, and hope for success. The key group discussion points focused on degree of satisfaction with FORSATY activities, sense of personal impact from participation, and changes observed at the community level ascribable to these activities. In-School Focus Groups Discussants were beneficiaries of academic support (tutorial) activities, plus the extracurricular activities that ran in parallel to them. Their responses did not vary significantly between the three neighborhoods (Dchar Bendibane, Bir Chifae, and Boujarrah). Satisfaction with Activities All focus group discussants expressed satisfaction with academic support activities as is the case for the vast majority (94%) of survey respondents. They claimed these tutorials helped them to learn better and improve their test results, as well as improving their linguistic skills (French and classical Arabic). Improved verbal and writing levels in Arabic and French were emphasized in Assadaka by girls more than boys. Those that had had math tutoring also felt that they had improved their skills. 88% of survey re￾spondents indicated that their grades had improved. 82 “School support classes are very good for us, and they help us understand and improve our results, but we think that there should be more classes in French and math for us to understand them well.” Source: Female in-school participant in Assadaka Association “What is required to strengthen academic support is more educators and more teaching hours through the three years of middle school.” Source: Female in-school participant from Assadaka Association Extracurricular activities were greatly appreciated by both males and females in the FGDs, although the survey tells us that only 62% of females and 59% of males surveyed actually participated in extracurricular activities compared to 88% overall that had been involved in academic support. Sports, particularly bas￾ketball and football, were considered very satisfying by both sexes, and in Assadaka the women expressed a desire to organize a team for themselves that could compete in tournaments as did the men’s team. Women particularly liked theater activities in Chifae. “Extracurricular activities are very good for us girls, because usually we are stranded at home after school. Now we participate in sports and theater, we attend the association and Hassan Ibn Tabit mid￾dle school for that purpose, and this make us feel free and satisfied as a result.” Source: Female in-school participant Personal Change from Participation Members of the three FGDs claimed substantial personal impact from the academic tutorials, cultural activities, and Life Skills. In Chifae discussants felt that their level of instruction had advanced and that their sense of responsibility, participation, and determination had improved. Girls felt their cultural horizon had expanded and they were now more creative and expressive. In Assadaka, discussants indicated improved verbal and writing levels in both Arabic and French, with girls stressing this more. They also indicated that determination, goal setting, and problem solving are personal qualities they acquired from the Life Skills sessions. Theater and cinema were perceived as improving self-esteem and making youth more creative and open minded. Discussants in Al Amal (all girls) also felt that participation in activities had changed them greatly. They felt more self-confident and have learned to get along better with others. They have become calmer, more optimistic, and more active. They now like challenges, take decisions, and have become more competitive. These results seem to be linked particularly to the Life Skills sessions, but also to the other extracurricular activities (sports, cultural activities). Survey respondents also overwhelmingly (91%) agreed that they had changed personally as a result of academic support activities and 96% felt changed as a result of extracurricular activities. Community-wide Changes In-school discussants in Assadaka affirmed that an increasing number of male and female youth have be￾come involved in academic tutorials (over 900 youth in Assadaka in 2014-2015) and that parents are extremely pleased to see this, now believing their children can indeed succeed in school. The same group 83 claimed that male youth no longer played in the streets but rather frequented the OSS premises to engage in sports. They also stressed the fact that girls are now playing football and hope to form a women’s team to represent their district in city sports competitions. Discussants in Bir Chifae echoed the fact that there are more female youth involved in extracurricular activities and that they engage in competitive play. They claimed that female youth are no longer looked down upon by males, since male youth now perceive them as participating in OSS activities and achieving more in school. Finally, parents are now more interested in maintaining their children in school, since they can benefit from free services offered by the OSS. The focus group in Al Amal also attested to the fact that their families have become more tolerant, since they see them succeeding in school. They felt that their parents have become more understanding of them and that both parents and their friends now have more respect for them as more studious and serious persons. Out-of-School Males Focus Groups Satisfaction with Activities Most members of these groups generally expressed satisfaction with the training they had received. This corresponds to the 96% overall in the survey that felt they had receiving the type of training they wanted, but only 81% who felt they had learned the skills needed to find work. In Assadaka, discussants felt very satisfied with the skills they had acquired that had pulled them out of marginality. They now felt personally useful and also useful to their families, feeling more ambition and determination to be active and earning an income. In Chifae the youth praised the quality of the garment cutting and sewing workshop and felt the instructor was competent and engaged in a good deal of personal coaching as well. They also felt content with the Life Skills sessions which helped them to be more patient and responsible. On the other hand, in Al Amal the youth, while basically satisfied, complained of the lack of skill of both the non-formal education and vocational training teachers, lack of transportation to the center, insufficient work space and tools, and the amount of time for training. They recommended adding more hours of training, some transportation allowance or assistance, a stipend or scholarship, more tools, more non-formal education teachers and teachers of French for all activities, and a psychologist, since many felt they have psychological problems. Personal Change from Participation Focus group discussants felt they had changed personally from their participation in FORSATY activities. This corresponds to the 97% of survey respondents who claimed such change after vocational training. Regarding these personal changes induced by training, the youth in Assadaka were proud of their tech￾nical qualification and felt it gave them an identity. They also attested to feeling a sense of responsibility and self-reliance and that they were now on the right track in life and rescued from delinquency. They felt strong enough now to cope with real life. In Chifae the discussants were happy with the training re￾ceived and felt that the transformation from unskilled and jobless to the opposite situation was crucial for their self-esteem. They now felt determined and ambitious to become useful for themselves and their families. They felt the Life Skills training had helped them retain their jobs and avoid workplace conflict, an opinion echoed by employers (e.g., LARINOR). In regard to working in the Tangier Free Zone, one young man said, “My life has changed positively after I finished my vocational training (sewing) 84 in the Bir Chifae association and started working in LARINOR in the Tangier Free Zone. I now have a stable job, earn 3,000 dirhams a month, and help my family. I plan to marry and have children.” Beyond workplace behavioral improvement, these youth felt more respectful of others in general and law abiding in society, capable of expressing solidarity and engaging in mutual support activities with oth￾ers. The discussants in Al Amal also attested to considerable personal change from their participation in project activities. The beneficiaries of the non-formal education activities felt pulled out of ignorance and illiteracy, while vocational trainees felt optimistic that they now had a new opportunity in life and could pull themselves out of a precarious existence through their new knowledge and skills. One youth in As￾sadaka expressed his personal change in this way, “Since I joined vocational training in house painting, I feel that I have regained hope in the future. I was on the edge of delinquency and despair, but now I have a profession, I earn some money painting houses, and my parents are happy with that, and our neighbors now respect me as well as my friends who now want to join the training program in Assadaka.” Community-wide Changes The Assadaka discussants felt that project activities had generated hope among families in their neighbor￾hood (Dchar Bendibane) that their uneducated children could be integrated into the job market and saved from delinquency. The discussants also felt that their participation in the project had fostered a more positive perception of these youth among neighbors, peers, and friends, who are now interesting in imi￾tating this experience. The same focus group also indicated that those now in paid internships or employed were contributing to family expenses. The discussants in Chifae Association stressed that a sense of en￾thusiasm had been generated in the community as a whole regarding Chifae and FORSATY. Parents are now encouraging their unemployed children to join the vocational training, and more youth are actually asking to join Chifae activities based on conversations with their friends in the program. Working youth are no longer loitering aimlessly in the street or hanging out on street corners. Those having participated were said by discussants to be feeling more responsible and had personal projects of marrying and having a family. As in Assadaka, the discussants felt that FORSATY participants enjoyed increased respect in their neighborhood (Bir Chifae), particularly when they landed jobs in the Tangier Free Zone (e.g., LARINOR). Al Amal discussants echoed the comments of the other two groups regarding enhanced respect from family and neighbors because of project participation. Out-of-School Female Focus Groups Satisfaction with Activities As in the case of male participants, satisfaction with vocational training was high for female participants. In Assadaka, they felt that this training made them perform at something for the first time in their lives. Vocational training was a very good and easy way to make them learn and practice a skill. They also expressed great satisfaction for the cultural activities and Life Skills which they felt had opened their minds. They had been included as well in theater and cinema activities along with the in-school youth. The focus group from Chifae had all been involved in garment manufacture (cutting and sewing) training and Life Skills sessions and recognized that FORSATY was giving them a new opportunity to succeed in life. They felt that thanks to their training they now had decent employment. Prior to the training, they claimed not 85 to have been doing anything and felt excluded and marginalized from society. In Al Amal both beneficiaries of non-formal education and vocational training activities were very satisfied with their activities and felt that FORSATY through Al Amal was providing them with a new opportunity to succeed in life, just as the group from Chifae had recognized. As in the case of Chifae, the participants felt that prior to their in￾volvement with Chifae they felt excluded and marginalized in society, particularly those that enrolled in non-formal education. However, half of the beneficiaries had some reservations about the quality of the training, whether non-formal education or vocational training. There were complaints about the scarcity of teachers and trainers, lack of tools in vocational training, lack of skilled non-formal education teachers, insufficient hours for vocational training, and lack of sufficient textbooks in non-formal education. Personal Change from Participation The direct results of non-formal education assistance and vocational training were considered by discus￾sants as the primary personal impacts from their participation in FORSATY activities for out-of-school youth. Thus, the discussants in Assadaka Association felt that vocational training has given them a qualifi￾cation to prepare them for finding a job. Life Skills were considered as effective in building their self-con￾fidence, responsibility, problem solving, and social solidarity. Activities such as theater and cinema, which they also engaged in, enabled them to express themselves, lose their timidity, feel freer, and be able to share ideas and visions with others. One woman said, “It’s been a year since I joined the FORSATY pro￾gram. I left school in the third year of middle school, got married, had children, then I came back to reg￾ister in non-formal education which gave me a strong impetus to study. In two years, I will pass the bac￾calaureate exam as a free candidate.” Chifae discussants insisted that they had benefitted personally as well from participation in FORSATY activities. They claimed to be more responsible, have many female friends, have more confidence in them￾selves, and feel more courageous. Al Amal discussants also attested to feeling more self-confident and responsible in life. Community-wide Changes In Assadaka the women stressed the fact that change had occurred in their neighborhood (Boujarrah). More parents were now sending their out-of-school girls to the Association and not keeping them at home as previously the case. They were now willing to give them a chance at training and eventually work. In Chifae, the discussants also pointed to the easing of parental and social controls for girls in the program and the greater respect shown them by their families and friends. Discussants in Al Amal also pointed out the greater freedom and respect shown them by their entourage (family, neighbors, and friends). Parental Focus Groups Satisfaction with Activities Parents (actually all mothers) claim to be satisfied with the services received by their children in Assadaka Association. Regarding academic support the mothers appear to have pushed for the inclusion of their 86 children, given their poor performance in school. On the other hand, the choice of extracurricular activ￾ities was made by the children. The discussants agreed that their children’s grades had indeed improved. Although satisfied in principle, the mothers in Assadaka complained of the lack of sufficient teachers in academic support activities and that their children were still very weak in French. The focus group of mothers in Chifae also expressed satisfaction with the services received by their children, but complained also of a lack of sufficient teachers, since one instructor only appeared to teach all subjects. They did admit to a small improvement in their children’s grades. In terms of extracurricular activities, the mothers were aware of sports, theater, summer camp, and Life Skills. They asked for sports for their children and the addition of other sports, such as Karate. Discussants also requested more teachers in academic support with more additional hours. Finally, they asked for a psychologist to help them and their children. In Al Amal also, mothers knew of academic support, sports, theater, music, and Life Skills activities. They claimed to be satisfied with the activities and academic support. They were sure that their children now had greater success with their examinations. In Al Amal there were no complaints concerning the pro￾gram, but the discussants did ask for more academic support in mathematics, French, and also English. Personal Change from Participation Mothers in all groups felt their children had changed considerably for the better following program activ￾ities. In Assadaka, they felt that their children now liked to go to school and participated more in class. They had stopped playing in the street and had become more responsible. In Chifae the discussants felt their children had become more courageous and self-confident. Discussants in Al Amal felt their children had changed a great deal following their participation in the program. They also felt their children were less perturbed and more courageous and self-confident, with an open spirit. They were more optimistic, sociable, and more active. Community-wide Changes The focus group discussants pointed to few larger impacts of their children’s benefits from the various FORSATY activities, except in the case of Al Amal where as a result of their increased friendliness with neighbors, relations with neighbors had improved. 87 ANNEX I: KEY SURVEY RESULTS AND CROSS-TABULATIONS BY SEX AND LO￾CATION Below are key survey results, followed by open response answers to question 125, and cross-tabulations by sex and location. In-school Youth Activities Academic Tutorials Have you received academic tutoring in the last two years? What subjects did you receive tutoring in? N = 331 No. % N = 291 No. % Yes 291 88% Mathematics 269 92% No 40 12% Arabic 229 69% Did you receive special tutoring for the 6th or 9th grade examination? French 269 92% N = 291 No. % Sciences 57 17% Yes 187 64% Other 57 17% No 104 36% Was the tutoring successful in your opinion? Do you believe you changed as a person as a result of the tutoring? N = 291 No. % N = 291 No. % Yes 274 94% Agree 175 60% No 17 6% Somewhat Agree 90 31% Did your grades improve as a result? Somewhat Disagree 17 6% N = 291 No, % Disagree 9 3% Yes 257 88% No 34 12% Extracurricular Activities Did you participate in extracurricular and Life Skills activi￾ties in the last two years? Do you believe you changed as a person as a result? N = 331 No. % N = 201 No. % Yes 201 61% Yes 193 96% No 130 39% No 8 4% For each of the following, tell me if you participated. If yes, how? N = 201 No. % N = 193 No. % Sports 168 84% Improved 193 100% Life Skills 163 81% Got worse 0 - Theater 269 86% Excursions 140 70% Educational/cultural games 176 88% Singing 121 60% 88 What was the most valuable to you personally? (Of those who reported they changed for the better) How did you improve as a result? N = 200 N = 193 N = 193 N = 193 No. % Sports More op￾timistic More op￾timistic More optimistic 96 50% Life Skills 25 13% More organized 111 58% Theater 63 32% More outgoing 60 31% Excursions 16 8% Healthier 44 23% Educational/cultural games 19 10% Better social life 78 40% Singing 4 2% Other 17 9% Summer Camps Did you participate in summer camp in the last two years? Do you believe you changed as a person as a result? N = 331 No. % N = 124 No. % Yes 124 37% Yes 118 95% No 207 63% No 6 5% For each of the following, tell me if you participated. If yes, how? N = 124 No. % N = 118 No. % Academic strengthening 71 57% Improved 118 100% Sports 94 76% Got Worse 0 - Hiking 59 46% Art and drawing 70 56% Theater 93 75% (Of those who reported they changed for the better) How did you improve as a result? Games 73 59% N = 118 No. % Field Trips 56 45% More optimistic 55 47% More organized 79 67% More outgoing 44 37% Healthier 28 24% Better social life 43 36% Out-of-school Youth Internships Did you participate in an internship activity in the last two years? Do you feel the experience was what you were seeking? N = 181 No. % N = 90 No. % Yes 90 50% Yes 88 98% No 91 50% No 2 2% Was the internship paid? Do you believe you changed as a person as a result? N=90 No. % N = 90 No. % Yes 65 72% Yes 83 92% No 25 28% No 7 8% 89 Were you able to choose the type of internship? If yes, how? N = 90 No. % N = 81 No. % Yes 70 78% Improved 81 98% No 20 22% Got worse 2 2% Were you offered employment as a result? (Of those who reported they changed for the better) How did you improve as a result? N = 90 No. % N = 81 No. % Yes 51 57% More confident 46 57% No 39 43% More optimistic 40 49% More skilled 64 79% More organized 28 36% Life Skills and Work Readiness Activities Did you participate in life skills or work readiness training in the last two years? Do you believe you changed as a person as a result? N = 181 No. % N = 85 No. % Yes 85 47% Yes 82 96% No 96 53% No 3 4% Was this personally useful for you? If yes, how? N=85 No. % N = 82 No. % Agree 67 79% Improved 82 100% Somewhat Agree 18 21% Got worse 0 - Disagree 0 0% (Of those who reported they changed for the better) How did you improve as a result? N = 82 No. % More confidence 61 74% More optimistic 46 56% More skilled 46 56% More organized 32 39% Vocational Training Did you participate in a vocational training program in the last two years? Were you given work readiness training as well? N = 181 No. % N = 90 No. % Yes 118 65% Yes 102 86% No 63 35% No 16 14% Were you able to choose the type of training you wanted? Do you believe you changed as a person as a result? N = 118 No. % N = 118 No. % Yes 113 96% Yes 114 97% No 5 4% No 4 3% 90 Do you feel you learned the skills you need to find work of the same type? If yes, how? N = 118 No. % N = 114 No. % Yes 95 81% Improved 114 100% No 23 19% Got worse 0 - Did it help you find employment? (Of those who reported they changed for the better) How did you improve as a result? N = 118 No. % N = 114 No. % Yes 71 60% More confidence 72 63% No 47 40% More optimistic 67 59% More skilled 87 76% More organized 34 30% Vocational Counseling Did you participate in a vocational counseling program in the last two years? Do you believe you changed as a person as a result? N = 181 No. % N = 112 No. % Yes 112 62% Yes 107 96% No 69 38% No 5 4% What was the result? If yes, how? N = 112 No. % N = 107 No. % Got an interview 85 76% Improved 107 100% Got more organized in search 96 86% Got worse 0 - Got a job 66 59% (Of those who reported they changed for the better) How did you improve as a result? N = 107 No. % More confident 64 60% More optimistic 64 60% More skilled 81 76% More organized 40 37% Non Formal Education Did you participate in non-formal education in the last two years? Were you satisfied with the academic assistance you re￾ceived? N = 181 No. % N = 26 No. % Yes 26 14% Yes 23 88% No 155 86% No 3 12% Which age group did you participate in? Did you also participate in Life Skills? N = 26 No. % N = 26 No. % under 15 6 23% Yes 21 81% 15 and over 20 77% No 5 19% 91 Were you assisted in accessing vocational training if you did not continue to 15+ NFE? N = 20 No. % I did continue in NFE 13 65% I did not continue/did get help accessing training 2 10% I did not continue/did not get help accessing training 5 25% Were you able to pass the examination you were preparing? N = 26 No. % Yes 22 85% No 4 15% Neighborhood Improvement How have the activities of the FORSATY OSS with youth changed the quality of life in the neighborhood? What kind of improvement have you seen? (more than one re￾sponse allowed) N = 512 No. % N = 370 No. % Improved 370 72% Youth are more successful 196 53% Same 139 27% Community is improved 167 45% Worse 3 1% Parents are happier 141 38% Youth are happier 131 35% Better communication in community 107 29% Open response answers to question 125 125. Is there anything else you want to share about your experience? No. % Need help in my study 8 1,6 I learned alot/ it was a good experience for me 215 42 Need more/ additional course 27 5,3 Need more activities or entertainment 38 7,4 Camp for girls only 4 0,8 More hours in sports 5 1 Academic tutorial in Language (Spanish and English mentioned) 18 3,5 Help young people find work 14 2,7 Free courses in the association and not paid 6 1,2 Need more content (mentioned either science or Quoran) 3 0,6 A shortage of some equipment 15 2,9 Need more help in training 6 1,2 Number of teacher is insuffiant 7 1,4 Need more hours in computer science 4 0,8 Need more hours in study end training 9 1,8 92 I just took advantage/ did nothing 5 1 More organization and the cleanliness of the center 7 1,4 We want library 1 0,2 Need psychological assistance for students 1 0,2 Poor service by some staff 9 1,8 Awareness for the parents 1 0,2 Need to study mechanics 2 0,4 Lack in Methods of teaching/many teachers are absent 6 1,2 Need to assist students from high school in the association center 1 0,2 Cross-tabulations, by sex (select questions) 4. With whom do you live? With Family With friends Alone Assadaka Center Total Male 244 12 7 14 277 Female 231 0 2 2 235 Total 475 12 9 16 512 5. Where were you born? This neigh￾borhood Another area of this city Another city in this re￾gion Another city in a different region A rural area in this re￾gion A rural area in a different region Total Male 188 38 8 29 8 6 277 Female 167 42 2 17 4 3 235 Total 355 80 10 46 12 9 512 7. Are you earning money on your own now? Yes, in a full￾time job Yes, in a part￾time job Yes, I earn money in other ways No, I do not early money on my own Total Male 33 24 12 208 277 Female 11 5 3 216 235 Total 44 29 15 424 512 [9. 2013/2014] Questions 9 to 11: In the following school years, were you in school? In Out Total Male 185 92 277 Female 175 60 235 Total 360 152 512 [10. 2014/2015] Questions 9 to 11: In the following school years, were you in school? In Out Total Male 270 70 277 Female 184 51 235 Total 391 121 512 93 11. This year] Questions 9 to 11: In the following school years, were you in school? In Out Total Male 186 91 277 Female 166 69 235 Total 352 160 512 12. What is your highest grade of formal education? 1. None 2. Some Primary 3. Primary 4. Some Second￾ary 5. Sec￾ondary 6. Post￾Second￾ary Total Male 8 68 78 94 25 4 277 Female 4 54 62 88 22 5 235 Total 12 122 140 182 47 9 512 13- Considering this list of common problems, which do you think affects you and the quality of your life the most? Biggest problem: Vio￾lence and crime Lack of em￾ployment Lack of educa￾tional op￾portunity Family prob￾lems Prob￾lems with friends and peers Gen￾eral pov￾erty Political prob￾lems Lack of op￾portunities for recrea￾tion Lack of services To￾tal Male 116 69 15 16 10 23 3 13 12 277 Female 100 32 18 23 12 21 3 11 15 235 Total 216 101 33 39 22 44 6 24 27 512 [25. To defend your religion or culture against insults or verbal threat ] 22 to 25 - People use violence for various reasons. In the following cases, do you think violence is often justified, sometimes justified, or never justified? often justified sometimes justi￾fied never justified Total Male 92 71 114 277 Female 83 58 94 235 Total 175 129 208 512 26. Have the activities of the CSO with youth here changed the quality of life in the neighborhood? How? Worse Same improved Total Male 1 75 201 277 Female 2 64 169 235 Total 3 139 370 512 Academic Tutorials 29. Have you received academic tutoring over the last 2 years Yes No Not concerned Total Male 146 23 108 277 Female 145 17 73 235 Total 291 40 181 512 36. Do you feel the teaching methods were right for you and your particular academic needs? Agree Somewhat Agree Somewhat Disagree Disagree Not Con￾cerned Total Male 94 46 4 2 131 277 Female 114 25 4 2 90 235 94 Total 208 71 8 4 221 512 37. Was the tutoring successful in your opinion? Yes No Not concerned Total Male 139 7 131 277 Female 135 10 90 235 Total 274 17 221 512 38. Did your grades change as a direct result of the tutorial? Yes No Not concerned Total Male 130 16 131 277 Female 131 14 90 235 Total 261 30 221 512 40. Do you believe you have changed as a person as a result of the tutorial experience? Agree Somewhat Agree Somewhat Disagree Disagree Not Con￾cerned Total Male 81 54 8 3 131 277 Female 94 36 9 6 90 235 Total 175 90 17 9 221 512 [More confidence] 41. If Agree/ Somewhat Agree, how? Mark all that apply Not selected Yes Not concerned Total Male 58 76 143 277 Female 53 77 105 235 Total 111 153 248 512 [More focused on education] 41. If Agree/ Somewhat Agree, how? Mark all that apply Not selected Yes Not concerned Total Male 36 98 143 277 Female 22 108 105 235 Total 58 206 248 512 [Better path to career/higher education] 41. If Agree/ Somewhat Agree, how? Mark all that apply Not selected Yes Not concerned Total Male 107 27 143 277 Female 108 22 105 235 Total 215 49 248 512 Extracurricular and Life Skills Activities 42. Did you participate in extracurricular and life skills activities over the last 2 years? Not selected Yes Not concerned Total Male 101 68 108 277 Female 100 62 73 235 Total 201 130 181 512 51. Did you find these activities well carried out? Agree Somewhat Agree Somewhat disagree Disagree Not con￾cerned Total Male 75 22 3 1 176 277 Female 82 16 2 0 135 235 Total 157 38 5 1 311 512 95 52. Which of these activities were most valuable to you personally? Sports Life Skills Theater Excur￾sions Educational/cultural games Sing￾ing Not con￾cerned Total Male 51 9 19 9 9 3 177 277 Female 22 16 44 7 10 1 135 235 Total 73 25 63 16 19 4 312 512 53. Do you believe you have changed as a person as a result of the activities? Yes No Not concerned Total Male 97 4 176 277 Female 96 4 135 235 Total 193 8 311 512 54. If yes, how? Improved Not concerned Total Male 97 277 277 Female 96 235 235 Total 193 512 512 [More optimistic ] 55. How did you improve as a result? Not selected Yes Not concerned Total Male 56 41 180 277 Female 41 55 139 235 Total 97 96 319 512 [More organized ] 55. How did you improve as a result? Not selected Yes Not concerned Total Male 32 65 180 277 Female 50 46 139 235 Total 82 111 319 512 [More outgoing] 55. How did you improve as a result? Not selected Yes Not concerned Total Male 69 28 180 277 Female 64 32 139 235 Total 133 60 319 512 [Healthier] 55. How did you improve as a result? Not selected Yes Not concerned Total Male 71 26 180 277 Female 78 18 139 235 Total 149 44 319 512 [Better social life] 55. How did you improve as a result? Not selected Yes Not concerned Total Male 60 37 180 277 Female 55 41 139 235 Total 115 78 319 512 59. Did you participate in a summer camp Activities in the last 2 years? Yes No Not concerned Total Male 69 100 108 277 96 Female 55 107 73 235 Total 124 207 181 512 64. Do you feel personally changed by this camp experience? If yes, how? Yes No Not concerned Total Male 66 3 208 277 Female 53 2 180 235 Total 119 5 388 512 65. If yes, how? Same Better Not concerned Total Male 1 65 211 277 Female 0 53 182 235 Total 1 118 3888 512 [More optimistic ] 66. How did you improve as a result? Not selected Yes Not concerned Total Male 36 29 212 277 Female 27 26 182 235 Total 63 55 394 512 [More organized] 66. How did you improve as a result? Not selected Yes Not concerned Total Male 19 46 212 277 Female 20 33 182 235 Total 39 79 394 512 [More outgoing] 66. How did you improve as a result? Not selected Yes Not concerned Total Male 44 21 212 277 Female 30 23 182 235 Total 74 44 394 512 [Healthier] 66. How did you improve as a result? Not selected Yes Not concerned Total Male 49 16 212 277 Female 41 12 182 235 Total 90 28 394 512 68. Did you participate in non-formal education classes over the last 2 years? Yes no No concerned Total Male 11 97 169 277 Female 15 58 162 235 Total 26 155 331 512 [69. Were you able to pass the examination you were preparing? Yes No Total Male 266 10 1 277 Female 220 12 3 235 Total 486 22 4 512 97 [71. Were you satisfied with the academic assistance you received Yes No total Male 266 9 2 277 Female 220 14 1 235 Total 486 23 3 512 [72. Did you participate also in life skills training? Yes No total Male 266 9 2 277 Female 220 12 3 235 Total 486 21 5 512 Youth with Internships 76. Did you participate in an internship activity during the last 2 years? Yes No 99 Total Male 61 47 169 277 Female 29 44 162 235 Total 90 91 331 512 [80. Was the internship paid? ] Yes No Notconcerned Total Male 45 16 216 277 Female 20 9 206 235 Total 65 25 422 512 [81. Were you able to choose the type of internship? Yes No Notconcerned Total Male 45 16 216 277 Female 20 4 206 235 Total 70 20 422 512 [82. Do you feel the experience was what you were seeking] Yes No Notconcerned Total Male 59 2 216 277 Female 29 0 206 235 Total 88 2 422 512 [83. Were you offered employment in the firm after the internship? ] Yes No Notconcerned Total Male 35 26 216 277 Female 16 13 206 235 Total 51 39 422 512 [84. Do you feel you have the skills necessary to be employed by that firm] Yes No Not concerned Total Male 11 5 261 277 Female 6 3 226 235 Total 17 8 487 512 98 85. Do you feel personally changed by the internship experience? Yes No Notconcerned Total Male 58 3 216 277 Female 25 4 206 235 Total 83 7 422 512 86. If yes, how? [worse, same, better] Worse Better Not concerned Total Male 2 56 49 277 Female 0 25 210 235 Total 2 81 429 512 [More confident ] 87. How did you improve as a result? Not selected Yes Total Male 250 27 277 Female 216 19 235 Total 466 46 512 [More optimistic ] 87. How did you improve as a result? Not selected Yes Total Male 246 31 277 Female 226 9 235 Total 472 40 512 [More skilled] 87. How did you improve as a result? Not selected Yes Total Male 231 46 277 Female 217 18 235 Total 488 64 512 [More organized] 87. How did you improve as a result? Not selected Yes Total Male 258 19 277 Female 226 9 235 Total 484 28 512 Youth in Life Skills (Work Readiness) Training 89. Have you participated over the last 2 years in life skills or work readiness training? Yes No Not concerned Total Male 48 66 169 277 Female 37 36 162 235 Total 85 96 331 512 93. Do you feel this training was personally useful to you? 1) Agree 2) Somewhat Agree Not concerned Total Male 35 13 229 277 Female 32 5 198 235 Total 67 18 427 512 94. Do you feel personally changed by the experience? 99 Yes No Not concerned Total Male 47 1 229 277 Female 35 2 198 235 Total 82 3 427 512 95. If yes, how? Better Not concerned Total Male 47 230 277 Female 35 200 235 Total 82 430 512 [More confident ] 96. How did you improve as a result? Not selected Yes Total Male 243 34 277 Female 208 27 235 Total 451 61 512 [More optimistic ] 96. How did you improve as a result? Not selected Yes Total Male 251 26 277 Female 215 20 235 Total 466 46 512 [More skilled] 96. How did you improve as a result? Not selected Yes Total Male 244 33 277 Female 222 13 235 Total 266 46 512 [More organized] 96. How did you improve as a result? Not selected Yes Total Male 260 17 277 Female 220 15 235 Total 480 32 512 Youth in Project-Supported Vocational Programs 99. Have you participated in a vocational training pro￾gram over the last 2 years? Yes No Not concerned Total Male 81 27 169 277 Female 37 36 162 235 Total 118 63 331 512 [104. Do you feel you learned the skills that you need to find work of the same type? ] Yes No 99 Total Male 66 15 196 277 Female 29 8 198 235 Total 95 29 394 512 [105. Were you given life skills training as well during the vocational training? Yes No 99 Total Male 68 13 196 277 Female 32 5 198 235 Total 100 18 394 512 100 [106. Were you given work readiness training during the vocational training? Yes No 99 Total Male 72 9 196 277 Female 30 7 198 235 Total 102 16 394 512 [107. Did this help you find employment? ] Yes No 99 Total Male 53 28 196 277 Female 18 19 198 235 Total 71 47 394 512 108. Do you feel personally changed by the experience? Yes No 99 Total Male 78 3 196 277 Female 37 0 198 235 Total 115 3 394 512 109. 9 If yes, how? Worse Better Notconcerned Total Male 1 77 199 277 Female 0 37 198 235 Total 1 114 397 512 Youth in Project-Supported Career Information and Counseling Activities 112. Did you participate in a vo￾cational counseling activity in the last 2 years? Yes No Not concerned Total Male 76 32 169 277 Female 36 37 162 235 Total 112 69 331 512 [112. Get interviews?] What was the result of this counseling activity? Yes No Not concerned Total Male 63 13 201 277 Female 22 14 199 235 Total 85 27 400 512 [113. Get a job? ] What was the result of this counseling activity? Yes No Not concerned Total Male 47 29 201 277 Female 19 17 199 235 Total 66 46 400 512 [114. Get more organized in your search? ] Yes No Not concerned Total Male 67 9 201 277 Female 29 7 199 235 Total 96 16 400 512 120. Do you feel personally changed by the experience? Yes No Not concerned Total Male 73 3 201 277 101 Female 36 0 199 235 Total 109 3 400 512 121. If yes, how? Worse Better Not concerned Total Male 2 71 204 277 Female 0 36 199 235 Total 2 107 403 512 124. Overall, how would you describe your experience with the youth activities we just discussed? Not beneficial Somewhat benefi￾cial Very beneficial Total Male 7 70 200 277 Female 4 40 191 235 Total 11 110 391 512 Cross-tabulations, by location 4. With whom do you live? With Family With friends Alone Sadakka Center Total Bir Chifa 150 10 6 0 166 Dchar Bendibane 175 2 3 16 196 Boujarrah 150 0 0 0 150 Total 475 12 9 16 512 5. Where were you born? This neigh￾borhood Another area of this city Another city in this re￾gion Another city in a different region A rural area in this re￾gion A rural area in a differ￾ent re￾gion Total Bir Chifa 118 18 7 15 6 2 166 Dchar Bendibane 145 27 3 17 1 3 196 Boujarrah 92 35 0 14 5 4 150 Total 355 80 10 46 12 9 512 7. Are you earning money on your own now? Yes, in a full-time job Yes, in a part-time job Yes, I earn money in other ways No, I do not early money on my own Total Bir Chifa 32 8 1 125 166 Dchar Bendibane 6 9 10 171 196 Boujarrah 6 12 4 128 150 Total 44 29 15 424 512 102 [9. 2013/2014] Questions 9 to 11 : In the following school years, were you in school? In Out Total Bir Chifa 116 50 166 Dchar Bendibane 158 38 196 Boujarrah 86 64 150 Total 360 152 512 [10. 2014/2015] Questions 9 to 11 : In the following school years, were you in school? In Out Total Bir Chifa 132 34 166 Dchar Bendibane 159 37 196 Boujarrah 86 64 150 Total 360 152 512 [11. This year] Questions 9 to 11 : In the following school years, were you in school? In Out Total Bir Chifa 115 51 166 Dchar Bendibane 154 42 196 Boujarrah 83 67 150 Total 352 160 512 13- Considering this list of common problems, which do you think affects you and the quality of your life the most? Biggest problem: Violence and crime Lack of employ￾ment Lack of edu￾cational op￾portunity Family problems Problems with friends and peers Gen￾eral poverty Political problems Lack of opportu￾nities for recrea￾tion Lack of services Violence and crime Total Bir Chifa 77 31 9 11 4 16 3 7 8 77 166 Dchar Bendiba ne 98 31 13 18 12 8 0 9 7 98 196 Boujar￾rah 41 39 11 10 6 20 3 8 12 41 150 Total 216 101 33 39 22 44 6 24 27 216 512 [25. To defend your religion or culture against insults or verbal threat ] 22 to 25 - People use violence for vari￾ous reasons. In the following cases, do you think violence is often justified, sometimes justified, or never justified? often justified sometimes justified never justified Total Bir Chifa 54 42 70 166 Dchar Bendibane 71 42 83 196 Boujarrah 54 42 70 150 Total 71 42 83 512 103 26. Have the activities of the CSO with youth here changed the quality of life in the neighborhood? How? worse same improved Total Bir Chifa 1 45 120 166 Dchar Bendibane 0 41 155 196 Boujarrah 2 53 95 150 Total 3 139 370 512 [Youth are more successful] 27. What kind of improvement have you seen? Not selected Yes Not con￾cerned Total Bir Chifa 49 71 46 166 Dchar Bendibane 83 72 41 196 Boujarrah 42 53 55 150 Total 174 196 142 512 [Youth are happier] 27. What kind of improvement have you seen? Not selected Yes Not con￾cerned Total Bir Chifa 79 41 46 166 Dchar Bendibane 99 56 41 196 Boujarrah 48 47 55 150 Total 226 144 142 512 [Parents are happier] 27. What kind of improvement have you seen? Not selected Yes Not con￾cerned Total Bir Chifa 70 50 46 166 Dchar Bendibane 93 62 41 196 Boujarrah 66 29 55 150 Total 229 141 142 512 [Community is improved] 27. What kind of improvement have you seen? Not selected Yes Not con￾cerned Total Bir Chifa 54 66 46 166 Dchar Bendibane 94 61 41 196 Boujarrah 55 40 55 150 Total 203 167 142 512 [Better communication in community] 27. What kind of improvement have you seen? Not selected Yes Not con￾cerned Total Bir Chifa 87 33 46 166 Dchar Bendibane 105 50 41 196 Boujarrah 71 24 55 150 104 Total 263 107 142 512 [Other] 27. What kind of improvement have you seen? Infrastructurel reform Activity of En￾tertainment and outreach Not con￾cerned Others Total Bir Chifa 116 1 3 46 166 Dchar 150 1 4 41 196 Boujarrah 92 2 0 55 150 Total 358 4 7 142 512 Academic Tutorials 29. Have you received academic tutoring over the last 2 years? Yes No Not Con￾cerned Total Bir Chifa 102 7 57 166 Dchar Bendibane 126 20 50 196 Boujarrah 63 13 74 150 Total 291 40 181 512 36. Do you feel the teaching methods were right for you and your particular academic needs? Agree Somewhat Agree Somewhat Disagree Disagree Not Con￾cerned TOTAL Bir Chifa 75 23 3 1 64 166 Dchar Bendibane 87 32 4 3 70 196 Boujarrah 46 16 1 0 87 150 Total 208 71 8 4 221 512 37. Was the tutoring successful in your opinion? YES NO Not con￾cerned Total Bir Chifa 97 5 64 166 Dchar Bendibane 117 9 70 196 Boujarrah 60 3 87 150 Total 274 17 221 512 38. Did your grades change as a direct result of the tutorial? YES NO Not con￾cerned Total Bir Chifa 94 8 64 166 Dchar Bendibane 110 16 70 196 Boujarrah 57 6 87 150 Total 261 30 221 512 105 39. if yes, how? Got worse Same Improved 9 Not Con￾cerned Total Bir Chifa 0 0 94 8 64 166 Dchar Bendibane 0 2 108 16 70 196 Boujarrah 1 1 55 6 87 150 Total 1 3 257 30 221 512 40. Do you believe you have changed as a person as a result of the tutorial experience? Agree Somewhat Agree Somewhat Disagree Disagree Not Con￾cerned Total Bir Chifa 64 31 4 3 64 166 Dchar Bendibane 68 44 8 6 70 196 Boujarrah 43 15 5 0 87 150 Total 175 90 17 9 221 512 Extracurricular and Life Skills Activities 42. Did you participate in extracurricular and life skills activities over the last 2 years? Yes No Not con￾cerned TOTAL Bir Chifa 57 52 57 166 Dchar Bendibane 96 50 50 196 Boujarrah 48 28 74 150 Total 201 130 181 512 51. Did you find these activities well carried out? Agree Some￾what Agree Some￾what Disagree Disagree Non con￾cerné TOTAL Bir Chifa 43 13 1 0 109 166 Dchar Bendibane 70 22 4 0 100 196 Boujarrah 44 3 0 1 102 150 Total 157 38 5 1 311 512 52. Which of these activities were most valuable to you personally? Sport s Life Skills Thea￾ter Excur￾sions Educa￾tional/cul￾tural games Singing Non con￾cerné TOTAL Bir Chifa 17 9 11 7 10 2 110 166 Dchar Bendibane 49 2 32 6 6 1 100 196 Boujarrah 7 14 20 3 3 1 102 150 Total 73 25 63 16 19 4 312 512 106 53. Do you believe you have changed as a person as a result of the activities? Yes No Not con￾cerned TOTAL Bir Chifa 42 15 109 166 Dchar Bendibane 73 23 100 196 Boujarrah 36 12 102 150 Total 151 50 311 512 54. If yes, how? improved Non con￾cenré TOTAL Bir Chifa 56 110 166 Dchar Bendibane 91 105 196 Boujarrah 46 104 150 Total 193 319 512 59. Did you participate in a summer camp Activities in the last 2 years? Yes No Not con￾cerned TOTAL Bir Chifa 25 84 57 166 Dchar Bendibane 69 77 50 196 Boujarrah 30 46 74 150 Total 124 207 181 512 64. Do you feel personally changed by this camp experience? If yes, how? Yes No No Concerné Total Bir Chifa 23 2 141 166 Dchar Bendibane 66 3 127 196 Boujarrah 30 0 12 150 Total 119 5 338 512 65. If yes, how? Same Batter No Concerné Total Bir Chifa 0 23 143 166 Dchar Bendibane 1 65 130 196 Boujarrah 0 30 120 150 Total 1 118 393 512 107 64. Do you feel personally changed by this camp experience? If yes, how? Yes No No Concerné Total Bir Chifa 3 1 162 166 Dchar Bendibane 8 2 186 196 Boujarrah 2 0 148 150 Total 13 3 496 512 65. If yes, how? Better No Concerné Total Bir Chifa 3 163 166 Dchar Bendibane 8 188 196 Boujarrah 2 148 150 Total 13 499 512 68. Did you participate in non-formal education classes over the last 2 years? YES NO No Concerné Total Bir Chifa 5 52 109 166 Dchar Bendibane 8 42 146 196 Boujarrah 13 61 76 150 Total 26 155 331 512 [68. Do you feel you were placed in the right activities for your academic needs? YES NO TOTAL Bir Chifa 161 5 0 166 Dchar 188 8 0 196 Boujarrah 137 10 3 150 Total 486 23 3 512 [69. Were you able to pass the examination you were preparing? ] YES NO TOTAL Bir Chifa 161 5 0 166 Dchar Bendibane 188 8 0 196 Boujarrah 137 9 4 150 Total 486 22 4 512 [71. Were you satisfied with the academic assistance you received] YES NO TOTAL 108 Bir Chifa 161 5 0 166 Dchar Bendibane 188 6 2 196 Boujarrah 137 12 1 150 Total 486 23 3 512 [72. Did you participate also in life skills training? YES NO TOTAL Bir Chifa 161 4 1 166 Dchar 188 6 2 196 Boujarrah 137 11 2 150 Total 486 21 5 512 Youth with Internships 76. Did you participate in an internship activity during the last 2 years? YES NO No Concerné TOTAL Bir Chifa 30 27 109 166 Dchar Bendibane 17 33 146 196 Boujarrah 43 31 76 150 Total 90 91 331 512 [80. Was the internship paid? ] YES NO No Concerné Total Bir Chifa 26 4 136 166 Dchar Bendibane 12 5 179 196 Boujarrah 27 16 107 150 Total 65 25 422 512 81. Were you able to choose the type of internship? YES NO No Concerné Total Bir Chifa 22 8 136 166 Dchar 14 3 179 196 Boujarrah 34 9 107 150 Total 70 20 422 512 [82. Do you feel the experience was what you were seeking] YES NO No Concerné Total Bir Chifa 29 1 136 166 Dchar Bendibane 16 1 179 196 109 Boujarrah 43 0 107 150 Total 88 2 422 512 [83. Were you offered employment in the firm after the internship? ] YES NO No Concerné Total Bir Chifa 18 12 136 166 Dchar Bendibane 8 9 179 196 Boujarrah 24 19 107 150 Total 51 39 422 512 [84. Do you feel you have the skills necessary to be employed by that firm] YES NO No Concerné Total Bir Chifa 2 2 162 166 Dchar Bendibane 3 2 191 196 Boujarrah 12 4 134 150 Total 17 8 487 512 85. Do you feel personally changed by the internship experience? YES NO No Concerné Total Bir Chifa 30 0 136 166 Dchar Bendibane 15 2 179 196 Boujarrah 38 5 107 150 Total 83 7 422 512 86. If yes, how? [worse, same, better] Worse Better No Concerné Total Bir Chifa 1 29 136 166 Dchar Bendibane 0 15 181 196 Boujarrah 1 37 112 150 Total 2 81 429 512 93. Do you feel this training was personally useful to you? Agree Somewhat agree No Concerné Total Bir Chifa 22 4 144 166 Dchar Bendibane 12 5 179 196 Boujarrah 33 9 108 150 110 Total 67 18 427 512 94. Do you feel personally changed by the experience? Yes No No Concerné Total Bir Chifa 25 1 140 166 Dchar Bendibane 15 2 179 196 Boujarrah 42 0 108 150 Total 82 3 427 512 95.If yes, how? Better No Concerné Total Bir Chifa 25 141 166 Dchar Bendibane 15 181 196 Boujarrah 42 108 150 Total 82 430 512 104. Do you feel you learned the skills that you need to find work of the same type YES NO NC TOTAL Bir Chifa 34 4 128 166 Dchar Bendibane 16 8 172 196 Boujarrah 45 11 94 150 Total 95 23 394 512 [105. Were you given life skills training as well during the vocational training? YES NO NC TOTAL Bir Chifa 34 4 128 166 Dchar Bendibane 20 4 172 196 Boujarrah 46 10 94 150 Total 100 18 394 512 [106. Were you given work readiness training during the vocational training? YES NO NC TOTAL Bir Chifa 37 1 128 166 Dchar Bendibane 18 6 172 196 Boujarrah 47 9 94 150 Total 102 16 394 512 [107. Did this help you find employment? YES NO NC TOTAL Bir Chifa 30 8 128 166 Dchar Bendibane 10 14 172 196 Boujarrah 31 25 94 150 111 Total 71 47 394 512 108. Do you feel personally changed by the experience? YES NO NC TOTAL Bir Chifa 37 1 128 166 Dchar Bendibane 22 1 172 196 Boujarrah 56 25 94 150 Total 115 3 394 512 109. If yes, how? Worse Better NC Total Bir Chifa 0 37 129 166 Dchar Bendibane 1 21 174 196 Boujarrah 0 56 74 150 Total 1 114 397 512 Youth in Project-Supported Career Information and Counseling Activities 112. Did you participate in a vocational counseling activity in the last 2 years? Yes No No Concerné Total Bir Chifa 38 19 109 166 Dchar Bendibane 23 17 146 196 Boujarrah 51 23 76 150 Total 112 69 331 512 [112. Get interviews?] What was the result of this counseling activity? Yes No Not Con￾cerned Total Bir Chifa 35 3 128 166 Dchar Bendibane 14 9 173 196 Boujarrah 36 14 99 150 Total 85 27 400 512 [113. Get a job? ] What was the result of this counseling activity? Yes No Not Con￾cerned Total Bir Chifa 34 4 128 166 Dchar Bendibane 4 19 173 196 Boujarrah 28 23 99 150 Total 66 46 400 512 112 [114. Get more organized in your search? ] What was the result of this counseling activity? Yes No Not Con￾cerned Total Bir Chifa 35 3 128 166 Dchar Bendibane 17 6 173 196 Boujarrah 44 7 99 150 Total 96 16 400 512 120. Do you feel personally changed by the experience? Yes No Not Con￾cerned Total Bir Chifa 38 0 128 166 Dchar Bendibane 21 2 173 196 Boujarrah 50 1 99 150 Total 109 3 400 512 121. If yes, how? Worse Better Not Con￾cerned Total Bir Chifa 1 37 128 166 Dchar Bendibane 1 20 173 196 Boujarrah 0 50 99 150 Total 2 107 400 512 124. Overall, how would you describe your experience with the youth activities we just discussed? 1. Not beneficial 2. Somewhat beneficial 3. Very beneficial Total Bir Chifa 2 35 129 166 Dchar Bendibane 8 50 138 196 Boujarrah 1 25 124 150 Total 11 110 391 512