This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of UBO consulting and do not necessarily reflect the views of USAID or the United States Government. MIDTERM IMPACT EVALUATION UP TO YOUTH ACTIVITY AUTHOR: DRENUSHË TELAKU, UBO CONSULTING DATE: 3 JUNE 2022 DILINA GASHI FOR USAID (DELETE THIS BLANK PAGE AFTER CREATING PDF. IT’S HERE TO MAKE FACING PAGES AND LEFT/RIGHT PAGE NUMBERS SEQUENCE CORRECTLY IN WORD. BE CAREFUL TO NOT DELETE THIS SECTION BREAK EITHER, UNTIL AFTER YOU HAVE GENERATED A FINAL PDF. IT WILL THROW OFF THE LEFT/RIGHT PAGE LAYOUT.) 1 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV LIST OF ACRONYMS CAM Community Asset Mapping CATI Computer Assisted Telephone Interviewing CSO Civil society organization GBV Gender-based violence IADK Initiative for Agricultural Development of Kosovo LYAC Local Youth Action Council MEL Monitoring, evaluation, and learning NGO Non-governmental organization PEN Peer Educators Network TNA Training Needs Analysis UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 2 TABLE OF CONTENTS LIST OF ACRONYMS 1 TABLE OF CONTENTS 2 TABLE OF FIGURES 3 EXECUTIVE SUMMARY 5 EVALUATION PURPOSE 5 EVALUATION QUESTIONS AND KEY FINDINGS 5 BACKGROUND INFORMATION 9 METHODOLOGY 10 DESK RESEARCH 10 IN DEPTH INTERVIEWS 11 SURVEYS 11 FOCUS GROUP DISCUSSIONS 13 RESULTS, CONCLUSIONS AND RECOMMENDATIONS 14 RESEARCH GOAL 1: WHAT ARE THE BARRIERS AND ENABLERS THAT AFFECT THE SUCCESSFUL ENGAGEMENT OF YOUTH, INCLUDING MARGINALIZED AND VULNERABLE YOUTH, IN ACTIVITIES DESIGNED TO CREATE POSITIVE CHANGE? 14 RESEARCH GOAL 2: HOW HAS THE PROGRAM IMPACTED SOCIAL INCLUSION OF MARGINALIZED AND VULNERABLE YOUTH? 17 RESEARCH GOAL 3: WHAT PROGRAM STRATEGIES AND PRACTICES INCREASE EFFECTIVENESS OF TEACHING, LEARNING, AND APPLYING NEW SKILLS FOR YOUTH PROGRAM PARTICIPANTS? 21 RESEARCH GOAL 4: IDENTIFY AND QUANTIFY BEHAVIOR CHANGES WITHIN YOUTH IDENTIFIED BY COMMUNITIES AND UP TO YOUTH STAFF. 26 RESEARCH GOAL 5: IDENTIFY AND QUANTIFY CHANGES AND/OR IMPROVEMENTS IDENTIFIED AND/OR IMPLEMENTED BY COMMUNITIES AS A RESULT OF UP TO YOUTH INITIATIVES. 37 CONCLUSION 39 ANNEXES 40 ANNEX 1: LIST OF SEMI-STRUCTURED AND IN-DEPTH INTERVIEWS 40 ANNEX II: LIST OF IN-DEPTH INTERVIEWS WITH BENEFICIARIES 41 ANNEX III: INVITEES FOR FOCUS GROUP DISCUSSIONS 42 ANNEX 1V: YOUTH SURVEY QUESTIONNAIRE 43 ANNEX V: YOUTH IN-DEPTH INTERVIEW QUESTIONNAIRE 52 ANNEX VI: UP TO YOUTH MANAGEMENT TEAM INTERVIEW QUESTIONS 52 ANNEX VII: USAID OFFICIALS INTERVIEW QUESTIONS 53 ANNEX VIII: UP TO YOUTH PARTNERS AND GOVERNMENT INSTITUTIONS INTERVIEW 53 3 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV TABLE OF FIGURES Figure 1. Are there barriers that affected your successful engagement in activities designed to create positive change? Main Group and Control Group .................................................................................................15 Figure 2. Are there enabling effects that affected your successful engagement in activities designed to create positive change? Main Group and Control Group ...................................................................................16 Figure 3. As a self-reported member of a marginalized community, how included do you feel in society? Main Group and Control Group ..................................................................................................................................17 Figure 4. How do you feel about the social inclusion of members of marginalized communities? Main Group and Control group...............................................................................................................................................18 Figure 5. Do you believe that your opinion of youth of other backgrounds has improved since participating in the U2Y program activities (Main Group) and during the last two years (Control Group)....................................................................................................................................................................................18 Figure 6. Do you believe that your opinion of youth of other backgrounds has improved since participating in the U2Y program activities? * Gender.........................................................................................19 Figure 7. To what extent have specific advocacy efforts that promoted minorities and marginalized groups influenced your attention to the issues that affect these groups? Main Group.............................20 Figure 8. Can you rate how the activities implemented in the program have helped you overcome the community challenges you have faced? Main Group..............................................................................................22 Figure 9. Do you consider that your life/soft skills (i.e., critical thinking, time management, problem solving etc.) have improved due to trainings you have attended/ during the last years? Main Group and Control Group ....................................................................................................................................................................22 Figure 10. Have you been offered a training on life/soft skill within the past year? Main Group and Control Group ....................................................................................................................................................................23 Figure 11. What kind of trainings have you participated in? Main Group and Control Group..............23 Figure 12. How satisfied were you with these trainings? Main Group and Control Group....................24 Figure 13. Have you taken any action for positive improvement in your community in the last two years? Main Group and Control Group .....................................................................................................................26 Figure 14. Have you participated in facing/identifying any community challenges, within the last 2 years? Main Group and Control Group ..................................................................................................................................26 Figure 15. Can you rate how the participation in community improvement initiatives has helped you? Main Group...........................................................................................................................................................................27 Figure 16. Have you completed any participatory analysis workshops to provide solutions to your community challenges, within the last 2 years? Main Group and Control Group ......................................27 Figure 17. Which of the following workshops have you participated in? Main Group and Control Group ...................................................................................................................................................................................................28 Figure 18. Have you participated in meetings related to youth priorities with local and or national officials? Main Group and Control Group .................................................................................................................28 Figure 19. Have you participated in any leadership roles, within the last 2 years?.....................................29 Figure 20. Can you rate how effective were leadership roles in your daily life within your community? ...................................................................................................................................................................................................29 Figure 21. What life/soft skills trainings/clubs have you participated in during the last 2 years? Main Group and Control Group .............................................................................................................................................30 Figure 22. Can you rate how the participation in "Advocacy Trainings" has affected your participation as a member of the youth?..............................................................................................................................................30 Figure 23. Do you think your friends are exposed to any of the following? Main Group and Control Group .....................................................................................................................................................................................31 Figure 24. How much you agree with all of the following statements concerning your perceived “self￾efficacy”? Main Group and Control Group ...............................................................................................................33 USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 4 Figure 25. Have you been involved in a paid internship within the first year? Main Group and Control Group .....................................................................................................................................................................................33 Figure 26. For older youth only: Have you completed any trainings focused on entrepreneurship? ..34 Figure 27. Has being involved in project activities made you feel more connected to the youth of your community members? Main Group and Control Group......................................................................................34 Figure 28. Have you participated in any processes established by networks to support youth, within last year? ................................................................................................................................................................................35 5 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE The Up to Youth activity's midterm evaluation explored the program's medium and long-term effects on youth and stakeholders. The evaluation was conducted externally by UBO Consulting. The evaluation research team employed mixed methods for its methodological approach and ensured data triangulation. The methods utilized included desk research, a survey with youth from the program and youth outside of the program, key informant interviews, and focus group discussions. These are discussed in depth in the methodology section. The study's five research goals, and the subsequent critical findings for each, are highlighted below: EVALUATION QUESTIONS AND KEY FINDINGS 1. WHAT ARE THE BARRIERS AND ENABLERS THAT AFFECT THE SUCCESSFUL ENGAGEMENT OF YOUTH, INCLUDING MARGINALIZED AND VULNERABLE YOUTH, IN ACTIVITIES DESIGNED TO CREATE POSITIVE CHANGE? In general, no barriers affected youth's successful engagement in activities. Two elements served as enablers to youth's successful engagement: (1) Up to Youth staff's high level of involvement in project activities and (2) the opportunities for youth to strengthen their soft and professional skills. - Most youth respondents (including marginalized and vulnerable youth), with no significant difference between youth participants from the program (83%) and youth outside the program (76%), stated there are no barriers that affect their successful engagement in activities designed to create positive change. The youth that did list barriers, mentioned the COVID-19 pandemic, issues with transportation, and difficulties juggling between school and work engagements. When focusing on gender differences, more than 17% of female respondents declared that there are barriers that affect their successful engagement in activities designed to create positive change, while only 8% of men declared the same. - Most youth participants from the program (69%) and youth outside the program (76%) listed no enablers that allow them to engage with activities successfully. From the youth that did list enablers, these included the trainings and capacity building on how to communicate better; the trainings and capacity building on decision making; the trainings and capacity building on getting more business knowledge, and the opportunities to network and collaborate with other young people. All these enabling factors are activities that Up to Youth heavily invests in. Additional enablers included the opportunities for youth to strengthen their soft and professional skills, their high level of willingness to participate in program activities, and the high level of involvement made by Up to Youth staff in project activities. 2. HOW HAS THE PROGRAM IMPACTED SOCIAL INCLUSION OF MARGINALIZED AND VULNERABLE YOUTH? Youth participating in the program feel more included in society, are more accepting of the social inclusion of members of marginalized communities and have changed opinions of youth of other backgrounds since participating in the program’s activities. The program helped them establish social relations, thus influencing the creation of positive personal thoughts about them and knowledge about the prospects of their lives. - Specific advocacy efforts that promoted minorities and marginalized groups have significantly influenced youth’s attention to issues that affect these groups. As a result, youth had the USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 6 opportunity to become friends with youth from other backgrounds and increased their awareness of how different communities live and where they come from. - To assess the impact of Up to Youth on the social inclusion of marginalized and vulnerable youth, the evaluation relied heavily on data from the evaluation’s survey with youth from the program and youth outside the program and on in-depth interviews with program youth from marginalized communities, including ethnic minorities and youth with disabilities. Youth participating in the program tend to feel more included in society when compared to youth outside the program. However, overall, young women reported a lower scale of inclusion when compared to young men. - All interviewed youth from the program affirmed that Up to Youth had a beneficial effect on the inclusion of marginalized and vulnerable youth from other backgrounds. Interviewed youth stated that, before Up to Youth, they did not have contact with youth from other backgrounds, and the program helped them establish friendships, influenced youth to have positive thoughts about others with whom they had no contact before, and to understand more about other youths’ way of life. - When surveyed, youth from marginalized backgrounds that were also participants of the program self-reported higher feelings of being included in society than youth from marginalized backgrounds outside of the program: 64% of Up to Youth participants stated that they felt somewhat to completely included in society, as opposed to 57% of non-participants. - The evaluation explored how specific advocacy efforts helped raise awareness on issues that affect youth from marginalized and vulnerable communities. Two-thirds of youth participants from the program (62%) stated that they were affected or significantly affected by these advocacy efforts, and another 20% stated that they were somewhat affected. These advocacy efforts helped youth better understand youth perspectives from other backgrounds. Some of the most common responses from youth from the program on this matter included: youth became friends with youth from other backgrounds (23%), youth increased their awareness of equal rights for all (18%), youth increased their awareness of how youth from other backgrounds live and what their beliefs on life are (15%), and youth learned how to fight prejudices against youth from other backgrounds (10%). Up to Youth created and supported youth groups with youth from different backgrounds. By providing a setting for youth to regularly work together towards common goals, the Up to Youth program can likely continue to increase the positive influence that youth have in each other’s development. 3. WHAT PROGRAM STRATEGIES AND PRACTICES INCREASE EFFECTIVENESS OF TEACHING, LEARNING, AND APPLYING NEW SKILLS FOR YOUTH PROGRAM PARTICIPANTS? Most youth consider that their life/soft skills have improved due to trainings they attended as part of the program. They believe that the activities implemented in the program have helped them overcome the community challenges they have faced, and they are completely satisfied with these trainings. The project activities have created/will create capacities that will remain with the beneficiaries, even after the completion of the project. Active learning methodology, group work, games and energizers have increased the effectiveness of teaching, learning, and applying new skills for youth program participants. - According to almost all survey responses from youth participants (91%), Up to Youth helped enhance their life / soft skills. Youth participants from the program were asked how much they needed and how satisfied they were with each type of training provided with Up to Youth. Based on the results, 91% of respondents from the main group were satisfied/very satisfied with the trainings. 7 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV - When asked to rate how the program’s activities helped youth overcome any challenges that they faced, 60% of youth from the main group the stated that the activities helped or helped a lot. When asked to elaborate, respondents stated that the program’s activities helped them increase their awareness on the social inclusion of youth from other backgrounds and on the eradication of bullying. Program activities also helped them with improving their problem-solving skills (12%), socializing with others (11%), increasing their self￾confidence, and improving their communication skills (7%). - The overwhelming majority of youth from the program consider that their life/soft skills have improved due to trainings they have attended (91%), and only nine percent stated the opposite. Comparing this to the responses of youth outside the program, only 78% stated that their life / soft skills had improved during the last year. In addition, 22% stated the opposite – a percentage that is greater compared to the responses from youth from the program. - Similarly, 91% of youth from the program were satisfied or very satisfied with the trainings offered, compared to 80% of youth outside of the program in trainings of similar topics. Whereas 10% of the youth outside the program were somewhat satisfied to not at all satisfied with the trainings offered to them, only 2% of the youth from the program were not satisfied with trainings offered from the program. 4. IDENTIFY AND QUANTIFY BEHAVIOR CHANGES WITHIN YOUTH IDENTIFIED BY COMMUNITIES AND UP TO YOUTH STAFF. In general, youth participating in program activities have a higher sense of social responsibility towards their communities, as they reported to have taken action for positive improvement and participated in facing/identifying community challenges. Project activities made respondents feel more connected their community members. Youth reported to have taken action for positive improvement and participated in identifying community challenges, volunteering, dialogues with a second party other than their group, leading community development activities, and extracurricular activities. - More than half of youth respondents from the program stated that they have taken actions for positive improvement in their community in the last two years (58%). The respondents were further asked about what activities they were involved in: 18% had dialogues with youth or others from other backgrounds; 32% participated in individual initiatives for community improvements; 76% participated in group initiatives; 53% did volunteer work; and 33% led community development activities. Conjointly, 25% of the youth from the program said that they were involved in organizing extracurricular activities, and 18% stated that they were involved in notifying government officials on the lack of a service or asset. - Compared to youth outside of the program (24%), almost half of youth from the program (49%) participated in identifying or solving community challenges. Twenty-six percent of the youth from the program reported completing participatory analysis workshops of any kind, to provide solutions to their community challenges, compared to only 10% of youth outside the program. Similarly, 48% of youth respondents from the program stated that they participated in Community Asset Mapping workshops, and 32% participated in Social Network Analysis workshops. - More than half of all youth respondents (62% of youth from the program; and 73% of youth outside the program) stated that they have not participated in meetings related to youth priorities with local and national officials. However, a solid 38% of youth from the program did participate in such meetings. These included meetings with only local officials (89%), meetings with only national officials (3%) and meetings with both local and national officials (8%). USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 8 - In terms of leadership roles, 19% of youth participants from the program stated that they participated in leadership roles, compared to 11% of youth outside of the program. - During the last two years, around the same share of youth from the program (68%) and youth outside of the program (67%) participated in volunteering. Over half (51%) of youth from the program participated in trainings with mentorship or advocacy components, which is significantly higher when compared with respondents from the control group. On the other hand, more youth from outside the program participated in youth-focused clubs (87%) than youth from the program (81%). Project activities have generally made respondents feel more connected to their community members, as stated by 72% of youth respondents from the main group. However, 70% of the respondents from the control group believe that their involvement in these activities did not help in the latter. 5. IDENTIFY AND QUANTIFY CHANGES AND/OR IMPROVEMENTS IDENTIFIED AND/OR IMPLEMENTED BY COMMUNITIES AS A RESULT OF UP TO YOUTH INITIATIVES. In general, the program's initiatives have provided youth with more complex information, which supports and encourages them to participate in public life. The program has impacted youth's awareness of their rights and crucial role within society. As found through data from the evaluation, young people report to: • Participate more in public hearings since their participation in the program. • Raise issues through their learned new skills for effective budget advocacy. • Volunteer more since learning about its importance through the program. • Lead community development activities. • Be involved in bringing to the attention of government official's the lack of a service or asset. Considering that the project is ongoing, Up to Youth management and USAID officials believed that the program's objectives were not completely achieved. Yet, interviewed respondents mentioned that enablers contributed to achieving the objectives so far: these enablers included staff willingness to provide advice, training, and assistance whenever needed; and the vast experience gained by youth from the program. 9 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV BACKGROUND INFORMATION Global Communities, in partnership with LINC LLC, Moonshot CVE, and local organizations, Peer Educators Network (PEN) and NGO LENS, referred to hereafter as Up to Youth, was awarded the USAID/Kosovo, five-year Up to Youth activity on April 15, 2019. Through this contract, the Up to Youth activity addresses youth vulnerable to social exclusion by mobilizing and engaging youth in meaningful ways to effect positive change. Up to Youth works with youth 15 to 24 years old, targeting younger youth (15-18) and older youth (18-24) with age-appropriate activities. The project emphasizes working with marginalized youth, particularly those underserved by current and past youth-focused initiatives, grappling with unaddressed stress and trauma, or vulnerable to developing risky behaviors. Vulnerability characteristics include extreme poverty, exposure to gender-based violence and other violence, lack of employment, and trouble in school. The project also makes concerted efforts to reach youth with disabilities, a group that has historically been excluded from education and community-based programming. Up to Youth applies an evidence-based approach to youth learning and equips youth with the skills they need to demonstrate agency in leading community asset mapping, analyzing data to identify youth￾related challenges or stressors in their community, and work with other youth and local stakeholders to design and implement solutions such as advocacy campaigns, community projects, and social ventures. By empowering youth to use data and analysis, and to leverage and engage their support networks and government, youth will make informed, effective contributions to their communities, and amplify and strengthen their voice and ability to positively affect policies and services that strengthen youth resilience. Up to Youth has the following three objectives: Objective 1: Mobilize Youth to be forces for positive change Objective 2: Develop and demonstrate life/soft skills among youth Objective 3: Develop, build, and support resilience partners in fostering an environment for youth to engage in a positive and meaningful way. The program is implemented in nine municipalities. Table 1. Up to Youth Partner Municipalities Clusters Municipalities Cluster I Han i Elezit/Elez Han, Kaçanik/Kačanik, and Viti/Vitina Cluster II Cluster II Mitrovicë Jugore/Mitrovica South, Vushtrri/Vućitrn, and Skënderaj/Srbica Cluster III Cluster III Istog/k, Klinë/a, and Gjakovë/Đakovica Up to Youth’s Theory of Change states that: “If youth in Kosovo vulnerable to social exclusions are mobilized and empowered to affect positive change, and resilient networks are supported to create meaningful engagement for youth, then youth resilience to social exclusion and extremism will be increased”. USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 10 METHODOLOGY The midterm evaluation utilized a mixed methods approach to ensure adequate data triangulation and to effectively target relevant stakeholder groups through the most appropriate method. The methodology relied on a mixture of both qualitative and quantitative methods: all in all, four main methods were utilized, including (1) desk research, (2) interviews, (3) surveys, and (4) focus group discussions. All research methods and tools were harmonized with the following key research questions and goals: 1. What are the barriers and enablers that affect the successful engagement of youth, including marginalized and vulnerable youth, in activities designed to create positive change? 2. How has the program impacted social inclusion of marginalized and vulnerable youth? 3. What program strategies and practices increase the effectiveness of teaching, learning, and applying new skills for youth program participants? 4. Identify and quantify behavior changes within youth identified by communities and Up to Youth staff. 5. Identify and quantify changes and/or improvements identified and/or implemented by communities as a result of Up to Youth initiatives. Throughout the methodology, the research team paid close attention to including all relevant program stakeholders in the evaluation. This included the Up to Youth management team, relevant officials from USAID, Up to Youth implementing partner representatives, government representatives, youth participants from the program (hereon after referred to as the “main group”) and youth that did not participate in the program (hereon after referred to as the “control group”). Throughout every stage and for every method, the research team ensured that all samples were representative and included participants from all ethnic groups in Kosovo and persons with disabilities. The research team made relevant changes to accommodate youth, stakeholders and other representatives from non-majority communities by translating all data collection tools in English, Albanian and Serbian, and by conducing the data collection process in those languages. Additionally, the research team made efforts to accommodate youth and stakeholders with disabilities by doing remote data collection when possible, and by hosting the in-person focus group discussions in disability accessible venues. Youth under the age of 18 were interviewed or participated in focus group discussions only with the approval of the parent or guardian. DESK RESEARCH The secondary data used for the purpose of this study was collected in the documents provided by the organization. After reviewing the documents, UBO Consulting collected the available data that were most related to the topic, which were further combined and compared. Documents reviewed for the desk research phase included: - Up To Youth Activity in Kosovo - Monitoring, Evaluation, And Learning Plan, June14, 2019 - Up To Youth Activity - Year One Annual Performance Report, April 15, 2019 Through April 14, 2020 - Up To Youth Activity - Year Two Annual Performance Report, April 15, 2020 Through April 14, 2021 - Up To Youth Activity - Quarterly Performance Report (Quarter 4 FY2021), July 1, 2021 through September 30, 2021 - Up To Youth Activity - Quarterly Performance Report (Quarter 1 FY2022) October 1, 2021 through December 31, 2021 - Up to Youth Activity Baseline Assessment for Youth Cohorts - Formative Evaluation of USAID/Kosovo’s Up to Youth Activity, April 2021 11 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV The desk research informed the design process for all data collection tools in the following methods listed below (interview questions, survey questions and focus group questions). IN DEPTH INTERVIEWS The research team conducted 41 interviews with various project stakeholders. These included five interviews with Up to Youth management team representatives, three interviews with USAID representatives, five interviews with Up to Youth implementing partners, four interviews with government institutions and youth officers from Directorates of Culture, Youth and Sports, and 23 interviews with youth participants of the program. The list of all persons who participated in the interviews is available in Annex 1. Among the 23 interviews with youth participants from the program, 15 of them were conducted with Albanian youth, three (3) with youth from Egyptian community, one (1) with youth from Roma community and four (4) with Serbian youth. None of the interviewed young people disclosed whether they had any disabilities or not. The research team conducted the interviews between February 14th, 2022 and March 4th, 2022. Due to the COVID-19 pandemic, the semi-structured and in-depth interviews were conducted through the Zoom platform. Due to the low participation rate of youth from ethnic minority communities in the focus group discussion, additional efforts were made to conduct in-depth interviews with youth from those communities. SURVEYS The research team drafted two surveys for youth participants to capture information relating to the research questions and goals outlined above. One survey was designed for youth that participated in the Up to Youth program (referred to as the “main group”), and the other was designed for youth outside of the program (referred to as the “control group”). Per the research proposal, a total of 400 youth ages 15 to 24 were foreseen to be reached via the surveys. The research team conducted a pre-test (pilot) with a small sample of people from the survey population. A pilot like this is one of the most significant techniques to determine whether the respondents are reading questions as intended and whether the order of the questions may influence responses, thus ensuring the validity and reliability of the questionnaire. To assess the causal effect of the program on youth participants, the evaluation looked at a purposive comparison group with its surveys, to better understand whether participants outside the program had similar results. The Up to Youth master participant list, as of 31-Jan-2022, had a total of 844 entries registered. Working with that master participant list, the survey samples for both survey versions were pre-selected as follows: 1. For the purposive control group with youth outside of the program (n = 138), out of this 844-entry total, 146 entries were pre-selected as potential respondents for the control group sub￾sample. The eligible categories for adding an entry to the control group were youth that during the application process of joining Up to Youth were either: (a) Not selected; (b) Withdrew their application; or (c) Were out of range / Not eligible to participate. The ineligible categories for adding an entry to the control group were: (a) Youth that were selected for the program; and (b) Youth that withdrew from the program after already attending trainings. Entries without valid contact information were removed from the sub-sample. A total of 138 entries remained. The demographic composition for this control group was (n = 138): ⎯ 52.2% girls / women and 47.8% boys / men (72 girls / women; 66 boys / men). USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 12 ⎯ 71.0% older youth and 29.0% younger youth (98 older youth; 40 younger youth). ⎯ 125 are K-Albanian; eight (8) are K-Serb; one (1) is Ashkali; one (1) is Egyptian; three (3) are Roma. ⎯ 13 from Gjakova; seven (7) from Istog; three (3) from Kacanik; 12 from Klina; 60 from South Mitrovica; 17 from Skenderaj; four (4) from Viti; 22 from Vushtrri. Since not everyone from the pre-sampled control group could be reached, the research team extended the criteria for the control group to include youth outside of the program, that were not in the original Up to Youth participant lists (youth from the general population that matched the demographic criteria). For the main group with youth from the program (n = 477), out of this 844-entry total, 477 entries were pre-selected as potential respondents for the main group sub-sample. The eligible category for adding an entry to the main group was youth that during their application process with Up to Youth were: (a) Selected. All 477 entries had valid contact information. The demographic composition for this main group with youth from the program was (n = 477): ⎯ 59.5% girls / women and 40.4% boys / men (284 girls / women; 193 boys / men). ⎯ 74.8% older youth and 25.2% younger youth (357 older youth; 120 younger youth). ⎯ 372 are K-Albanian; 18 are K-Serb; five (5) are K-Bosniak; eight (8) are K-Turk; 12 are Ashkali; 57 are Egyptian; five (5) are Roma. ⎯ 62 from Gjakova; 29 from Hani I Elezit; 59 from Istog; 49 from Kacanik; 65 from Klina; 65 from South Mitrovica; 45 from Skenderaj; 45 from Viti; 58 from Vushtrri. The research team conducted the surveys between February 14th, 2022 and March 4th, 2022. Due to the COVID-19 pandemic, the surveys were conducted through the Computer Assisted Telephone Interviews (CATI) methodology. To encourage the effective participation of youth from marginalized backgrounds, the research team conducted surveys in other languages when needed. In the end, the research managed to collect a total of 355 responses from program and non-program youth. Out of this total number of survey respondents, 70% were beneficiaries of the program, and 30% were part of the control group sample. The latter declared different reasons to not having participated in the program, such as not being accepted (28%), withdrawing their application (5%) and other reasons (11%). This combination of the sample makes it possible to see the real effects of the program on its beneficiaries, by comparing it with the rest of the respondents, who were not beneficiaries of the program. Disaggregation-wise, 60% of the overall respondents were female, while the other 40% were male, followed by 52% who lived in urban areas and 48% live in rural areas. When asked if they considered themselves a member of a marginalized community, one-fourth answered positively (25%) and only 5% preferred to not disclose. Six percent of respondents were youth with disabilities and 13% were youth from non-ethnic majority groups. 13 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV Table 2. Demographic information Demographic information of the surveyed sample Gender Male 40% Female 60% Ethnicity Albanian 87% Serbian 2% Roma 1% Ashkali 2% Egyptian 7% Bosnian 0.3% Turk 1% Settlement Urban 52% Rural 48% Marginalized community Yes 25% No 69% Prefer to not disclose 5% Disability Yes 6% No 93% Prefer to not disclose 1% FOCUS GROUP DISCUSSIONS The Illustrative Research Questions included in Request for Proposals guided the research team in the design of all focus group discussion questions. The backtranslation approach was used for instrument translation in this evaluation, as it places a greater emphasis on conceptual and cultural equivalence in comparison to language. In terms of focus group discussions, the research team organized two groups: one with stakeholders from the program’s Youth Support Network. and one with youth participants from the program. The first discussion was held with Youth Support Network stakeholders, and it targeted 14 persons in total. However, only seven people attended the discussion, three of whom were women and four men. Further, the second discussion targeted 16 youth from the program. However, only five youth (two women and three men) attended the session. One of the participating youths was a person with a disability. The focus group discussions were organized in-person in disability accessible venues to facilitate a smoother exchange between participants. There were three individuals from the research team present in the discussions: two moderators and one note-taker. No difficulties occurred during the process. Due to the low participation rate of youth from ethnic minority communities in the focus group discussion, additional efforts were made to conduct in-depth interviews with youth from those communities to ensure that their inputs are captured. USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 14 RESULTS, CONCLUSIONS AND RECOMMENDATIONS The following paragraphs show a collided approach of the results, conclusions, and recommendations from the evaluation. To address the impact of the Up to Youth program, the evaluation was guided by five research goals: 1. What are the barriers and enablers that affect the successful engagement of youth, including marginalized and vulnerable youth, in activities designed to create positive change? 2. How has the program impacted social inclusion of marginalized and vulnerable youth? 3. What program strategies and practices increase the effectiveness of teaching, learning, and applying new skills for youth program participants? 4. Identify and quantify behavior changes within youth identified by communities and Up to Youth staff. 5. Identify and quantify changes and/or improvements identified and/or implemented by communities as a result of Up to Youth initiatives. These research goals helped assess the medium and long-term effects of the program, including both intended and unintended, and both positive, and negative effects. The results, conclusions and recommendations are presented based on the research goals. RESEARCH GOAL 1: WHAT ARE THE BARRIERS AND ENABLERS THAT AFFECT THE SUCCESSFUL ENGAGEMENT OF YOUTH, INCLUDING MARGINALIZED AND VULNERABLE YOUTH, IN ACTIVITIES DESIGNED TO CREATE POSITIVE CHANGE? RESULTS The first research goal tackled the issue of barriers and enablers that affect the successful engagement of youth, including marginalized and vulnerable youth, in activities designed to create positive change. The results from the survey, in-depth interviews and focus group discussions indicate that, in general, no barriers affected youth’s successful engagement in activities. However, the high level of involvement made by Up to Youth staff in project activities, as well as the numerous opportunities for youth to strengthen their soft and professional skills, were deemed enablers. Most youth respondents (including youth from marginalized communities), with no large difference between youth participants from the program (83%) and youth outside the program (76%), listed no barriers that affect their successful engagement in activities designed to create positive change. From the youth that did list barriers, these included the COVID-19 pandemic, issues with transportation, and difficulties juggling between school and work engagements. When focusing on gender differences, 17% of female respondents declared that there were barriers that affected their successful engagement in activities, while only 8% of men declared the same. 15 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV Figure 1. Are there barriers that affected your successful engagement in activities designed to create positive change? Main Group and Control Group Data collected in focus group discussions showed that activities implemented online did not offer the same experience to youth as when these were held face to face. According to qualitative data gathered from in-depth interviews with program participants, other barriers included internet issues (lack of internet / poor internet), electricity shortages, and the entire process of online activities during the pandemic. As revealed by Up to Youth’s Formative Evaluation, because Up to Youth targets economically and socially marginalized youth living in rural areas, it is not surprising that the challenge of moving to online platforms was greater for these youth than for YAB members, who come from more urban areas, are better off financially, and are more technologically savvy. In addition to the pandemic and implementation of online activities, qualitative data from interviews with Up to Youth management and USAID officials revealed that an additional barrier that could have affected the achievement of program objectives was also the selection process for program youth. According to interviewees, the selection form and analysis during the inception phase ensured that the program included women and men, people with disabilities and other marginalized groups. However, recruiting youth with disabilities and other marginalized groups presented a barrier. When it came to girls from rural areas, recruitment was a barrier in and of itself. The existence of numerous stigmas, such as the community's rejection of girls who choose to prioritize their jobs over raising a family, presented certain challenges in recruiting girls from rural areas. On the other hand, transportation of youth with disabilities was not always possible, which impacted their participation in the program. It is worth mentioning that Handikos helped in this regard. Finally, the program’s work in equipping youth with skills and training in specific areas that assist them in starting their own businesses is insufficient. According to Up to Youth management team, local and central level institutions should create separate financing streams to help youth practice the skills they learn through specific program activities and trainings. The process of establishing relations between institutions and businesses in municipalities presents a barrier. Again, most youth participants from the program (69%) and youth participants outside the program (76%) stated there are no enablers that allowed them to engage with activities successfully. From the youth that did list enablers, these included: trainings and capacity building on how to communicate better; trainings and capacity building on decision making; trainings and capacity building on getting more business knowledge, and opportunities to network and collaborate with other youth. These are all activities that the Up to Youth program heavily invests in. Further, the high level of involvement made by Up to Youth staff in project activities, as well as opportunities for youth to strengthen their soft and professional skills, were seen as enablers. 13% 85% 2% 14% 76% 10% Yes No Don't know Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 16 Figure 2. Are there enabling effects that affected your successful engagement in activities designed to create positive change? Main Group and Control Group According to USAID officials, one of the main enablers that contributed to the achievement of the program objectives so far was the program staff, who were engaged in implementing the program’s activities and adapting those activities during the pandemic. When focusing on other enablers, the opportunities the program offers to youth was considered an enabler as well. Young people of these municipalities have not been active in local organizations previously. As a result, they were unaware of the opportunities and demands they could make, but because of the program and its trainings, they are more motivated to raise their voice and speak for the needs of their local communities. Parents of youth have become more aware of the advantages their children acquire by participating in the program. CONCLUSIONS There are little to no barriers that affect youth’s successful engagement in activities designed to create positive change. The online project implementation during the COVID-19 pandemic, issues with transportation, difficulties juggling between school and work engagements, lack of internet / poor internet and electricity shortages are all mentioned as barriers, with some youth being impacted more than others. Among those impacted more by these barriers are youth from rural areas and youth with disabilities. Further, recruiting young women from rural areas provided a unique challenge, as these youth are already fighting social stigmas regarding women who put education and career before their family life. The transportation of youth with disabilities was not always possible, which impacted their participation in the program. RECOMMENDATIONS 1. The program should adapt its activities to youth participants’ school and work schedules, as youth mentioned this as recurring barrier in survey responses, focus group discussions and interviews. 2. The program should conduct activities near targeted rural areas, due to youth’s inability to access transportation, as youth, and especially youth with disabilities, mentioned this as recurring barrier in survey responses and focus group discussions. 3. The program should avoid implementing activities online: data from focus group discussions show that online activities do not provide the same experience as face-to-face trainings. 30% 69% 1% 23% 76% 1% Yes No Don't know Main Group Control Group 17 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV RESEARCH GOAL 2: HOW HAS THE PROGRAM IMPACTED SOCIAL INCLUSION OF MARGINALIZED AND VULNERABLE YOUTH? RESULTS The second research goal explored how the program impacted the social inclusion of marginalized and vulnerable youth. Youth participating in the program tend to feel more included in society, when compared to youth outside the program. However, overall, young women reported a lower sense of inclusion when compared to young men. One of the main targets of the program has been the social inclusion of marginalized and vulnerable youth. Marginalized and vulnerable youth include young people living in rural areas with few opportunities to participate in community activities, young people with poor economic conditions, youth with disabilities and youth from minority ethnic communities. Therefore, as stated in in-depth interviews with youth, the program has given these youth the opportunity to meet and socialize with people from other communities. Qualitative data from in-depth interviews with USAID officials indicate that the number of youth with disabilities and youth from minority ethnic communities should increase as the program continues. Self-reported marginalized community members from the main group declared they felt somewhat included/completely included in society (64%), rather that 57% of respondents from the control group who declared the same. Further, 13% of respondents from the control group declared that they do not feel included at all in the society, while only 2% of those from the main group declared the same. Figure 3. As a self-reported member of a marginalized community, how included do you feel in society? Main Group and Control Group However, Up to Youth’s management team declared that they saw an increase in tolerance, understanding and acceptance in youth participants from the program. According to them, the program gave these youth the opportunity to learn about and socialize more with youth of other backgrounds, different ethnicities, different sexual orientations etc. Most surveyed respondents from the main group (86%) believe that the social inclusion of members of marginalized communities is necessary / completely necessary, whereas slightly fewer of control group respondents (75%) declared the same. With that said, when asked to elaborate further, 40% of them think that their participation is necessary for the progress of society and 18% stated that they feel good about the inclusion of all communities in their social life. 2% 7% 28% 22% 42% 13% 10% 20% 27% 30% Not at all included 2 3 4 Completely included Control Group Main Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 18 Other respondents believe that members of marginalized communities should be included more (13%) and that all must be equal (11%). Figure 4. How do you feel about the social inclusion of members of marginalized communities? Main Group and Control group Within the main group, 82% of the respondents believe that their opinion of youth of other backgrounds has improved since participating in program activities. Only 5% of the respondents stated the opposite. When asked to elaborate on their assessment, 46% of the respondents declared they now have a more positive attitude toward others. Other respondents stated that it had influenced them to be more self-aware towards different societal backgrounds, as well as the environment in which others live (8%). Additionally, they had the chance to communicate and become friends with members of different groups (6%). Those who declared the contrary gave no specific reasoning behind their assessment, by stating that their opinion simply has not changed since the participation in the program’s activities. The control group were also asked if their opinion of youth from other backgrounds has improved in the past two years. Having said that, 57% of the respondents have affirmed that their opinion has changed. However, 27% stated the contrary – thus declaring that their opinion of other backgrounds did not improve. While 10% of the respondents said that their opinion is the same as two years ago, 17% believe that it has improved and another 9% believe that it has gotten even worse. Figure 5. Do you believe that your opinion of youth of other backgrounds has improved since participating in program activities (Main Group) and during the last two years (Control Group) Gender-wise, more female respondents (85% from the main group and 63% from the control group) than male respondents (77% from main group and 50% from control group) stated that their opinion 3% 1% 10% 14% 72% 5% 6% 14% 23% 52% The social inclusion of members of marginalized communities is not at all necessary 2 3 4 The social inclusion of members of marginalized communities is completely necessary Control Group Main Group 82% 5% 13% 57% 27% 16% Yes No Don't know Main Group Control Group 19 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV of youth of other backgrounds has improved since participating in program activities or generally in the last two years. Figure 6. Do you believe that your opinion of youth of other backgrounds has improved since participating in program activities? * Gender Ethnicity-wise, all Turk and Roma respondents (100%), most Egyptian respondents (95%) and over half of Ashkali respondents (57%) from the main group reported that their opinions of youth from other backgrounds have improved since participating in program activities. The results were also supported by data obtained from in-depth interviews with youth participants. When asked how Up to Youth has affected the social inclusion of marginalized youth, their answers were generally the same. All interviewees stated that the program had a positive effect on the inclusion of youth from other backgrounds. Further, elaborating that many of them had not had frequent contacts with youth from other backgrounds, the program has given them opportunities to establish social relations between them, thus influencing the shaping of positive personal thoughts about them and given them knowledge about their lives. Respondents from the main group were asked to what extent specific advocacy efforts that promoted minorities and marginalized groups influenced their attention to the issues that affect these groups. Having said that, 62% stated that these efforts have affected/greatly affected their attention and 20 percent stated that it has somewhat affected it. The remaining 19% believe that it has not affected/not affected it at all. When elaborating their assessment, 23% of the respondents declared that they became friends with other communities, 18% that it has helped them raise their awareness that all people should have equal rights, 15% that it has helped them get informed on how these groups live and what beliefs they have about life in general and 10% that they now manage to dispel prejudices for minorities. 77% 85% 50% 63% 5% 5% 30% 25% 18% 10% 20% 12% Male Female Male Female Main Group Control Group Yes No Don't know 15% 4% 20% 27% 35% Has not affected it at all 2 3 4 Has greatly affected it USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 20 Figure 7. To what extent have specific advocacy efforts that promoted minorities and marginalized groups influenced your attention to the issues that affect these groups? Main Group CONCLUSIONS Youth participating in the program tend to feel more included in society, are more accepting of the social inclusion of members of marginalized communities and have changed opinions of youth of other backgrounds since participating in program activities. The Up to Youth program allowed participants space to understand the differences and similarities between them, which is a good starting point in learning to address other inequities in society.1 Since many of the program youth had not had frequent contact with youth from other backgrounds before, the program helped them establish social relations, thus influencing the creation of positive personal thoughts about youth from other backgrounds. Specific advocacy efforts that promoted minorities and marginalized groups seem to have influence youth’s attention to the issues that affect these groups greatly. These efforts gave youth the opportunity to become friends with youth of other backgrounds, it increased their awareness towards how different communities live and where they come from, and they now manage to dispel prejudices towards members of minority communities. RECOMMENDATIONS 1. Up to Youth should pay particular attention to the needs of youth with disabilities to ensure that the project’s social inclusion goals are met. Due to youth's inability to obtain transportation, the program should offer activities near selected rural regions, since this was cited as a reoccurring barrier in survey responses and focus group discussions by beneficiaries, and particularly by youth with disabilities. 2. Given the impact the program has had on its beneficiaries when compared to the control group, it is critical for the program to continue to promote social inclusion, social cohesion, and gender equality, thereby encouraging a larger number of minorities and marginalized groups to participate in such activities and trainings. 1 Social Exclusion: Definition, Impact, Examples, Future Opportunities (2021). Retrieved from: https://www.liberties.eu/en/stories/social-exclusion/43579, on 13/04/2022. 21 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV RESEARCH GOAL 3: WHAT PROGRAM STRATEGIES AND PRACTICES INCREASE EFFECTIVENESS OF TEACHING, LEARNING, AND APPLYING NEW SKILLS FOR YOUTH PROGRAM PARTICIPANTS? RESULTS Youth are valuable assets to their communities; however, they are often left out of decision-making and go unheard. The program considered the extent to which activities were in line with participants’ needs, and with the aim of building stronger communities and more resilient youth. Based on the results retrieved from in-depth interviews and desk research, during the design phase of the strategies and practices, the program’s Objective 2 Lead had begun discussions with the Chief of Party concerning potential approaches to the training needs analysis (TNA) tool. This tool was used to determine the specific needs and capacities of youth in each of the nine partner municipalities. This TNA helped program managers develop strategies and practices to increase the effectiveness of teaching, learning, and applying new skills to youth program participants. As found through desk-research, the information gathered from the TNA had two main uses: 1. It helped Up to Youth understand what trainings youth in each community have already received, what competencies they can demonstrate, and what skills they prioritize for further development; and 2. It provided valuable baseline information on project participants that will be used by the Monitoring, Evaluation and Learning team to track youth changes in knowledge and skills over time. The program’s objectives do address the needs of the targeted group; however, given the circumstances of youth in Kosovo, according to Up to Youth management, the project does not address all the needs of young people. Instead, it allows youth to be a positive force for change. According to qualitative data from interviews with Up to Youth management and USAID officials, to the trainings were tested with a smaller group of youth first, to assess the effectiveness of the trainings. Another effective program strategy was the community asset mapping exercise, which allowed youth to identify challenges within the municipalities of the program. The program implements an active learning methodology during its trainings and activities, called “flipping the classroom”. This methodology promotes student-teacher interaction, through different breakouts and fun activities. Lastly, parental cooperation has greatly influenced the involvement of young people in the program. By participating in the program, young people have learned about data, and addressing and communicating various issues of their community. This has mobilized young people to initiate projects with a small budget but with a large impact, in municipalities. Some of the program strategies and practices that contributed to the process of youth gaining and applying new skills as stated by the interviewed youth included: a) “Group work and interactivity between other group members” b) “Trainings regarding cover letters, CV, professionalism, etc.” c) “Trainings for decision making and problem solving” d) “Some of the trainers use energizers, thanks to them and their creativity, we kept the lectures in mind.” e) “During the trainings we had different activities where we were able to put into practice the things we have learned” USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 22 To further identify which strategies and practices have had such an effect, program beneficiaries were asked about the level of need, satisfaction, and type of training provided through the program. To further evaluate the impact of these strategies and practices, the results are derived by comparing the main group and the control group. When asked to rate how the activities implemented in the program have helped them overcome the community challenges that they have faced, 60% of the respondents from the main group declared that these activities have helped/helped a lot. Only 14% stated the opposite, by saying that the activities were not helpful at all, while 22% said that they were somewhat helpful. When asked to elaborate their assessment, 22% of respondents stated that this has helped them raise the awareness of the population to accept people as they are and the young people in the eradication of bullying. Other than that, the respondents stated that it has helped them in problem solving (12%), socializing with others (11%), self-confidence and communication skills (7%). Figure 8. Can you rate how the activities implemented in the program have helped you overcome the community challenges you have faced? Main Group The evaluation provided the opportunity through in-depth interviews with youth participants, to identify program strategies and practices that contributed to creating and applying new skills. Based on the results, some of the most effective strategies and practices were: the interaction between the youth of different groups, the trainings, the space for discussion and expression of opinions, as well as the possibility of applying the things that were discussed and learned in the activities and trainings of the program in their daily lives. The majority respondents from the main group (91%) consider that their life/soft skills have improved due to trainings they have attended, and only 9% stated the opposite. However, when the control group were asked the same question, 78% stated that their life/soft skills have improved during the last year. In addition, 22% of them stated the opposite – a percentage that is greater compared to the respondents that were trained. Figure 9. Do you consider that your life/soft skills (i.e., critical thinking, time management, problem solving etc.) have improved due to trainings you have attended/ during the last years? Main Group and Control Group 14% 3% 22% 26% 34% Not helpful at all 2 3 4 They have helped a lot 91% 9% 78% 22% Yes No Main Group Control Group 23 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV All respondents were asked whether they have been offered a training on life/soft skills within the past year. As depicted in the graph below, 40% of the respondents from the main group claimed that they have been offered a training on life/soft skills, whereas 72% of respondents from the control group said the contrary. Figure 10. Have you been offered a training on life/soft skill within the past year? Main Group and Control Group 42% of the respondents agreed that the trainings were very good and appropriate, where they also had the chance to develop life skills and new skills (24%). Moreover, 11% of the respondents stated that the trainings they attended served them well in their professional development. When focusing on youth’s perception on the School of Entrepreneurship, focus group participants were impressed with the longevity of these trainings, lecturers that included businessmen and university professors. According to them, the two week length of the entrepreneurship school trainings enabled them to acquire professional skills and access the labor market. Figure 11. What kind of trainings have you participated in? Main Group and Control Group However, according to data acquired from interviews with Up to Youth management and USAID officials, to increase the effectiveness of teaching, learning, and applying new skills for program 40% 60% 28% 72% Yes No Main Group Control Group 63% 54% 45% 45% 44% 43% 40% 36% 33% 33% 29% 28% 26% 26% 25% 25% 18% 66% 52% 21% 52% 10% 7% 17% 17% 10% 3% 10% 38% 3% 10% 17% 7% Critical Thinking Active Listening Accessing Mentorship Decision-making Social Media Literacy Leadership Entrepreneurship Issue Advocacy Problem solving Self- motivation Tolerance Empathy Negotiation Self-control Accessing Finance Positive Self-concept Financial Literacy Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 24 participants, trainings should be institutionalized and digitalized, and interaction with the central level should be increased to enable higher durability of program impacts. Furthermore, the declarations gained from the focus group discussion with stakeholders backed up this conclusion. They suggested that a stronger link be built between the commercial sector, particularly local and national SMEs, and these young people, as well as the program itself, in order to expand this partnership. They went on to say that this would allow youth who have been trained in professional skills to put their skills into practice. Figure 12. How satisfied were you with these trainings? Main Group and Control Group Youth participants, on the other hand, had a distinct list of actions that should be implemented in the future to ensure that the outcomes of these programs are long￾lasting. Financial support for the implementation of social entrepreneurship projects, strategic campaigns to raise young people's awareness of their rights and their impact on society and increasing the number of participants in the continuation of the program, including holding trainings / activities on weekends, extending the training time (more than one day), and creating job opportunities were all things that young people said they would change if the project was restarted. It is worth mentioning that other parties interviewed for this assessment listed the same measures. Lastly, the possibilities for applying these skills are not always sufficient. In this context, cooperation with the Ministry of Education, Science, Technology and Innovation, the Ministry of Culture, Youth and Sports and local level institutions should be increased. Based on the data obtained from in-depth interviews with beneficiaries and stakeholders, local-level institutions have always expressed willingness to cooperate during the project, especially in the nine municipalities involved in the project. Still there is a lack of budget, therefore increasing communication between local and central level is important. CONCLUSIONS Most youth consider that their life/soft skills have improved due to training they have attended as part of the program. They believe that the activities implemented in the program have helped them overcome the community challenges they have faced and are completely satisfied with these trainings. The project activities have created/will create capacities that will remain with the beneficiaries, even after the completion of the project. Active learning methodology, group work, games and energizers have increased the effectiveness of teaching, learning and applying new skills for youth program participants. Based on the data retrieved from interviews, the program emphasized interactive teaching methodologies. When it comes to life skills training, merely sharing information is not sufficient for youth to discuss, question, and internalize the competencies taught to use them in their own lives. Participatory training pedagogy was key to inspiring youth and increasing your life skills trainings impact. 2% 7% 29% 62% 7% 3% 10% 21% 59% Not at all satisfied 2 3 4 Completely satisfied Main Group Control Group 25 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV RECOMMENDATIONS 1. According to data gathered from interviews and focus group discussions, the program should continue to use the active learning methodology – such as flipping the classroom, group work and interactivity between group members and energizers – because they were identified as some of the practices that increased the effectiveness of teaching and learning. 2. To increase the effectiveness of the program, trainings should be institutionalized and digitalized, according to interviewed management team and USAID officials. 3. Based on data gathered from in-depth interviews and focus group discussions with beneficiaries and stakeholders, the program should facilitate the cooperation between professional high schools with businesses operating in potential fields of employment, where beneficiaries can apply skills learned from the program. USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 26 RESEARCH GOAL 4: IDENTIFY AND QUANTIFY BEHAVIOR CHANGES WITHIN YOUTH IDENTIFIED BY COMMUNITIES AND UP TO YOUTH STAFF. RESULTS Given the great importance the program placed on behavior changes within youth, the evaluation analyzed actions youth have taken towards positive improvement in their communities, their participation in facing/identifying any community challenges, their participation in analysis workshops to provide solutions to their communities’ challenges and their participation in meetings related to youth priorities with local and/or national officials. In general, youth participating in program activities have a higher sense of social responsibility towards their communities. They reported to have taken more action for positive improvement and participated in facing/identifying community challenges. For reference, 58% of the respondents from the main group and 24% from the control group declared that they have taken actions for positive improvement in their community in the last two years. The respondents were further asked about what activities they were involved in: only 18% had a dialogue with a second party other than their group; and another 32% reported to have undertaken individual initiatives. Figure 13. Have you taken any action for positive improvement in your community in the last two years? Main Group and Control Group Nearly half of the respondents (49%) from the main group have participated in facing/identifying community challenges, with a significant difference between the two groups (49% from the main group and 24% from the control group). However, 50% did not do the same within the last two years. Figure 14. Have you participated in facing/identifying any community challenges, within the last 2 years? Main Group and Control Group 58% 40% 2% 24% 74% 2% Yes No I don't know Main Group Control Group 49% 50% 1% 24% 73% 3% Yes No Don't know Main Group Control Group 27 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV When focusing on the survey results, respondents from the main group were asked to rate how the participation in community improvement initiatives has helped them, where a score of ‘1’ meant “not helpful at all” and a score of ‘5’ meant “they have helped a lot”. With that said, more than half of surveyed youth from the main group (54%) believe that their participation in the aforementioned initiatives have helped/helped a lot, followed by 21% that believe it was somewhat helpful and not helpful at all. Figure 15. Can you rate how the participation in community improvement initiatives has helped you? Main Group Respondents from both groups (main group and comparison group) were asked whether they completed any participatory analysis workshops to provide solutions in their community challenges within the last 2 years. Based on the results, 26% of the respondents from the main group completed participatory analysis workshops of any kind, to provide solutions to their community challenges. A smaller share of respondents from the control group (10%) declared the same. Figure 16. Have you completed any participatory analysis workshops to provide solutions to your community challenges, within the last 2 years? Main Group and Control Group Furthermore, 48% of respondents from the main group participated in Community Asset Mapping workshops, and 32% of respondents of the same group have participated in Social Network Analysis workshop. The remaining 23% stated that they have participated in other participatory analysis workshops. 21% 4% 21% 20% 34% Not helpful at all 2 3 4 They have helped a lot 26% 74% 10% 90% Yes No Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 28 Figure 17. Which of the following workshops have you participated in? Main Group and Control Group More than half of the surveyed respondents from the main group (62%) stated that they have not participated in meetings related to youth priorities with local and national officials. However, 38% of the respondents from the main group stated the contrary, by affirming their participation in such meetings. The meetings that most of the respondents have participated in are the ones with local officials (89%). However, there were also meetings with both local and national officials (8%) and meetings with national officials only (3%). Figure 18. Have you participated in meetings related to youth priorities with local and or national officials? Main Group and Control Group Most of respondents from the main group (78%) have not participated in any leadership roles within the last two years. However, there were other respondents, 19% precisely, that confirmed their participation in the latter. Whereas only 11% of the respondents from control group declared the same. 48% 29% 23% 14% 32% 54% Community Asset Mapping workshop Social Network Analysis workshop Other participatory analysis workshop Main Group Control Group 38% 62% 27% 73% Yes No Main Group Control Group 29 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV Figure 19. Have you participated in any leadership roles, within the last 2 years? A larger number of respondents from the main group (47%) rated the effectiveness of their leadership roles within their community as not effective/not effective at all. When further asked to elaborate on their assessment, these respondents did not give any reasoning behind it. However, 36% of the respondents believe that the leadership roles were effective/very effective. Figure 20. Can you rate how effective your leadership roles were in your daily life within your community? The respondents from both groups were further asked about what life/soft skills trainings/clubs they participated in. During the last two years, around the same share of respondents from the main group (68%) and control group (67%) have participated in in volunteering. More than half of respondents from the main group have participated in trainings that have advocacy components (51%), which is significantly higher when compared with respondents from the control group who declared the same (35%). On the contrary, the share of respondents from the main group (40%) who participated in youth-focused clubs was lower than those from the control group (58%). 19% 78% 3% 11% 87% 3% Yes No Don't know Main Group Control Group 40% 7% 18% 15% 21% Not effective at all 2 3 4 Very effective USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 30 Figure 21. What life/soft skills trainings/clubs have you participated in during the last 2 years? Main Group and Control Group Respondents from the main and control groups were also asked to rate how their participation in trainings affected their active participation in everyday life as a member of the youth community. Most respondents from both groups (84% from the main group; and 89% from the control group) declared that their participation in volunteering affected or completely affected their active participation in everyday life as a member of the youth community. The respondents were further asked to elaborate their assessment, where 41% of them believe that their awareness for volunteering has increased, followed by 29% who stated that they now try to help more people within their community and 12% who stated that their empathy for others has developed. Furthermore, 81% of main group respondents and 87% of control group respondents stated that their participation in youth-focused clubs affected or completely affected their active participation in everyday life as a member of the youth community. Almost half of all respondents (48%) declared that they are now trained on various topics that help them in identifying youth problems and negative phenomena. Further, 30% of respondents affirmed that they learned about youth rights and 23% stated that clubs had a positive impact. Figure 22. Can you rate how the participation in "Advocacy Trainings" has affected your participation as a member of the youth community? 68% 56% 51% 40% 67% 30% 35% 58% Volunteering Mentorship Trainings Advocacy Trainings Youth-focused clubs Main Group Control Group 2% 3% 6% 6% 1% 10% 2% 23% 15% 12% 11% 13% 6% 29% 45% 21% 34% 30% 48% 44% 30% 63% 55% 51% 39% Main Group Control Group Main Group Control Group Main Group Control Group Advocacy Trainings Volunteering Youth-focused clubs Not affected at all 2 3 4 Completely affected 31 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV The evaluation explored the challenges that youth are most exposed to in their daily lives. Overall, the perceived risks reported from the main group respondents is lower than that of control group respondents from outside the program. The share of respondents who believe that their friends are exposed to the following issues, is greater in the case of control group. Based on the results, unemployment is one of the most concerning issues taking place in Kosovo according to respondents from both groups. In addition to this, 89% of the respondents from the main group declared that their friends are exposed to low employability, whereas 93% of the respondents from the control group stated the same. Moreover, a large percentage (72%) of respondents from the main group reported that their friends are exposed to poverty, yet this is still six percentage points lower than those who declared the same from the control group. Half of the respondents from the main group (50%) stated that their friends are also exposed to poor familial support. Further, there is a poor community participation, according to the 74% of the respondents from the control group. Moreover, slightly more than half of the respondents from the main group (51%) claimed that their friends are exposed to psychological violence, a percentage that is smaller when compared with 74% of those from the control group who declared the same. and 46% of them stated the same regards physical violence. One-third of the respondents from the main group (30%) claimed that their friends are exposed to sexual violence, whereas more than half of respondents from control group (52%) declared the same. In this regard, female respondents from both groups tend to report higher perceived physical violence in community (50%), report higher perceived psychological violence in community (61%), and report higher perceived sexual violence in community (43%). Lastly, more than half of the respondents from the main group (53%) claimed that their friends are exposed to discrimination. However, the percentage of those who declared the same from the control group is significantly higher (68%). Figure 23. Do you think your friends are exposed to any of the following? Main Group and Control Group Further, all survey respondents were asked to rate how much they agree with the following statements concerning their perceived “self-efficacy”. With that said, a fairly large percentage 89% 72% 56% 53% 51% 50% 46% 39% 38% 36% 33% 30% 93% 78% 74% 68% 69% 56% 55% 59% 63% 53% 48% 52% Low employability Poverty Poor community participation Being discriminated against Psychological violence Poor familial support Being out of school Being marginalized Physical Violence Being stigmatized Physically located in a community with gangs or violent extremism organizations Sexual Violence Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 32 of the respondents from the main group (85%) somewhat agree or strongly agree that they are strong enough to overcome life’s struggles, whereas 63% of the respondents from the control group declared the same. Most of the survey respondents from the main group (77%) are confident that they can handle the situations that life brings. Only 8% of the respondents from the main group somewhat agree/strongly agree that they usually feel that they are unsuccessful. Most of the respondents from the main group (71%) disagree or strongly disagree that they often feel that there is nothing they can do well. Moreover, a slightly larger percentage of respondents from the main group (76%) are confident that they are competent to deal effectively with the real world, thus agreeing or strongly agreeing with this statement. Lastly, the survey respondents from the main group agree/strongly agree that they usually feel they can handle the typical problems that come up in life (76%). When results were distinguished by gender, female respondents were less agreeable with the following statements: “I am strong enough to overcome life’s struggles.” (84% of male respondents agree/strongly agree compared to 76% of female respondents who declared the same), “I usually feel I can handle the typical problems that come up in life.” (74% of male respondents agree/strongly agree compared to 65% of female respondents who declared the same), and “I often feel like a failure.” (9% of male respondents agree/strongly agree compared to 7% of female respondents who declared the same). 4% 10% 85% 85% 10% 4% 6% 17% 77% 79% 13% 8% 71% 21% 6% 5% 18% 76% 85% 9% 4% 6% 17% 76% 6% 31% 63% 64% 20% 17% 13% 35% 51% 55% 26% 18% 46% 31% 23% 9% 45% 46% 60% 24% 16% 15% 34% 51% Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree Somewhat disagree/Strongly disagree Neutral Somewhat agree/strongly disagree I am strong enough to overcome life’s struggles At root, I am a weak person. I can handle the situations that life brings. I usually feel that I am an unsuccessfu l person. I often feel that there is nothing that I can do well. I feel competent to deal effectively with the real world. I often feel like a failure. I usually feel I can handle the typical problems that come up in life. Main Group Control Group 33 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV Figure 24. How much you agree with all of the following statements concerning your perceived “self-efficacy”? Main Group and Control Group Based on the results obtained from in-depth interviews with youth participants, the program has greatly influenced their perception of "self-efficacy". According to them, participation in the program has influenced them to nurture self-confidence in their abilities, as well as to be aware of their role and rights in society. Data retrieved from focus group discussions shows that before the program, youth were not very aware of the influence they might have as part of their community. According to them, through this program and numerous trainings they were encouraged to participate in local debates and budget hearings. In addition, prior to the launch of this program, young people had the opportunity to engage in their communities only through LYACs, most of which are not functional in certain municipalities. Young participants from the discussion stated that the program gave them the opportunity to meet municipal authorities and cooperate with them, through project proposals, complaint letters or even simple requests. Most of the respondents from the main group (77%) have not been involved in paid internships within the last year. However, there were other respondents (23%) that declared the contrary. When asked to specify, 13% of the respondents stated that they were involved as interns in Up to Youth, 4% were involved in IADK, Nursing and USAID, and 2% were involved in an accounting company, Ani Bar and Art Motion. Figure 25. Have you been involved in a paid internship within the first year? Main Group and Control Group Focus group discussions have highlighted some success stories that are worth mentioning, such as employment with a 6-month employment contract of persons with disabilities (as stated by one participant with disabilities), opportunities for paid internships in local NGOs (as stated by one female participant) and youth employment in some private businesses. Of the young people participating in the discussion, two of them were engaged as interns within the program, (one of which was a female participant), one of them was employed in the private sector and another participant stated that he has become a team leader within Red Cross and other local NGOs. But above all, according to them, the program built future leaders within their communities. Furthermore, 31% of the older youth participants from the main group have completed trainings focused on entrepreneurship, whereas only 13% of respondents from the control group did so. When asked to specify the trainings, 39% reported to have completed the School of Entrepreneurship, 18% attended trainings on how to start a business, business plans (8%), management (5%) and more. According to 94% of the respondents from both groups, their understanding of entrepreneurship has improved since participating in these trainings. However, 6% have denied this statement. Most of the respondents (84%) are satisfied/completely satisfied with the trainings, claiming that they were very informative and the knowledge they have gained 23% 77% 10% 90% Yes No Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 34 about entrepreneurship can be applied in the future. Only 7% reported to be unsatisfied/not satisfied at all. Regarding trainings that explained financing possibilities, most respondents from both groups have not participated in such trainings. When differing the results between the two surveyed groups, 14% of those who confirmed their attendance in the latter belonged to the main group, while 6% belonged to the comparison group. A larger percentage of respondents (44%) declared they were satisfied with these trainings, 39% were completely satisfied, whereas 6% were not at satisfied at all. The respondents further stated that they have gained basic knowledge about finances (95%). Additionally, 86% of the respondents consider that their understanding of finance has increased since participating in these trainings, while the remaining 14% stated the opposite. A total of 11 main group respondents (5%) from the older youth cohorts reported to have registered a business at the Kosovo Business Registration Agency, Ministry of Trade and Industry within the past year. Figure 26. For older youth only: Have you completed any trainings focused on entrepreneurship? Project activities have generally made the respondents feel more connected to the youth of their community members, as stated by 72% of respondents from the main group. Figure 27. Has being involved in project activities made you feel more connected to the youth of your community members? Main Group and Control Group Further, 29% of the respondents from the main group and 18% of respondents from the control group have participated in processes established by networks to support youth. However, a larger percentage (an average of 74% for both groups) did not do so within the last year. 31% 69% 14% 86% 5% 95% 13% 87% 7% 93% 5% 95% Yes No Yes No Yes No Completed any trainings focused on entrepreneurs hip Attended trainings that explained financing possibilities Registered a business at the Kosovo Business Registration Agency, Ministry of Trade and Industry within the past years Main Group Control Group 72% 28% Yes No Main Group 35 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV The networks were established by Up to Youth (36%), LYAC (15%), USAID (6%), Municipality, PEN and VORAE (3%), Art Polis and DIAKONIE YOUTH CENTER (2%) and more. Moreover, the topics discussed in these networks were mostly about preventing bullying (16%), volunteering and awareness of the youth for the protection of environment (9%), inclusion of youth with disabilities in the community and inclusion of all communities in art (3%), as well as other different topics (25%). In addition, 17% of the respondents from the main group reported to have participated in youth related activities where different organizations were involved. They participated in activities from LYAC (16%), LYAC, PEN (6%), Art Polis and CARITAS (3%). More than half of the respondents (70%) reported to be satisfied or completely satisfied with these activities. Only 10% of the respondents were not satisfied at all. When asked to elaborate their assessment, 39% of the respondents believe that these activities were very positive, and that there have been good activities in sports and culture where youth have learned how to treat people with disabilities and accept them in society (7%). A few respondents claimed they managed to create positive change in terms of removing stigma and discrimination (4%). Lastly, 8% of the respondents have designed activities that obtained resources from external stakeholders. The donors/contributors that helped to achieve this were USAID (19%), EU Council (13%), GIZ (13%), UNMIK (13%), U.S Embassy (13%), EU Office (6%), IOM (6%), UNICEF (6%), YNCA (6%), and Up to Youth (6%). The respondents further proclaimed the subjects of the designed activities, which were mostly about COVID-19, the development of various ideas, culture, bullying, as well as awareness raising for young people on employment opportunities (6%). Figure 28. Have you participated in any processes established by networks to support youth, within last year? CONCLUSIONS Youth participating in program activities have a higher sense of social responsibility towards their communities. Project activities made respondents feel more connected to the youth in their communities. As such, they reported to have taken action for positive improvement and participated in facing/identifying community challenges, volunteering, dialogues with a second party other than their group, leading community development activities, and extracurricular activities. They believe that participation in community improvement initiatives was helpful and has increased their awareness towards volunteering. These youth now try to be more helpful to the people within their community and show empathy for others. 29% 17% 8% 18% 15% 9% Participated in any processes established by networks to support youth, within last year Participated in youth related activities where different organizations were involved Designed an activity that obtained resources from external stakeholders Main Group Control Group USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 36 Regarding advocacy trainings, the respondents declared that they have learned about how to be more active in people’s demands for their rights, they have developed critical thinking, and gained necessary general knowledge. Regarding the youth-focused clubs, respondents declared that they helped them identify youth problems and negative phenomena and learned about youth rights. When it comes to self-efficacy, program participants are more likely to agree that they are strong enough to face life's challenges, confident that they can handle the situations that life brings and agree that they feel they can handle the typical problems in life. RECOMMENDATIONS 1. Based on data gathered from interviews and focus group discussions with young beneficiaries and stakeholders, the program should put more effort in organizing workshops that offer solutions to participants’ community challenges. 2. Based on data gathered from interviews and focus group discussions with young beneficiaries and stakeholders, Up to Youth should collaborate more with businesses, companies and NGOs in offering paid or unpaid internship in order to enhance and put into use the skills gained from the program. 37 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV RESEARCH GOAL 5: IDENTIFY AND QUANTIFY CHANGES AND/OR IMPROVEMENTS IDENTIFIED AND/OR IMPLEMENTED BY COMMUNITIES AS A RESULT OF UP TO YOUTH INITIATIVES. RESULTS In general, program initiatives provided youth with more complex information, which supports and encourages them to participate in public life. The program has impacted youth’s awareness of their rights and their crucial role within society. As found through data presented in the above listed evaluation questions and goals, youth from the program report to have: ⎯ Participated more in public hearings, since their participation in the program; ⎯ Raised issues through their learned new skills for effective budget advocacy; ⎯ Volunteered more since learning about its importance through the program; ⎯ Lead community development activities; ⎯ Been involved in bringing to government official’s attention the lack of a service or asset. Before introducing them to their role, the program investigated youth’s interests. Hence, the program’s partners and government institutions included various representatives, including youth officials at the municipal level, representatives of LYACs and representatives of non-governmental organizations in the municipalities where the program operates. These representatives gave feedback by being involved in drafting initiatives to address youth priorities. According to them, this involvement was made through participation in meetings organized by Up to Youth, where they contributed to the design of initiatives based on data from the youth community in those municipalities. Different stakeholders were involved in different ways in youth-designed activities performed as part of the program, such as: a) Youth center Hareja has developed various trainings and meetings, which included topics, such as: compiling CVs and cover letters, English language courses, robotics courses, tailoring, and design courses. b) A three-day activity for children with disabilities, held in restaurant Amazona. c) Lobbying and advocacy as a youth center: how to increase the number of youth activities and capacity building, drafting the budget code for youth in Gjakova and Istog. d) Creating opportunities for meetings with municipal officials, including the mayor, creating space for various meetings and part of the commission in selecting the beneficiaries of the free German language course. When asked whether their institution/organization was engaged in designing and implementing solutions such as advocacy campaigns, community projects, and/or social ventures in collaboration with the program, interviewed stakeholders listed the following: a) Logistical support, including providing the necessary space for training and meetings, for cultural activities. b) Establishing a cooperation with local institutions and businesses. c) Dissemination of information related to leadership, entrepreneurship and businesses to young people. d) Awareness campaigns or more capacity building of young people in soft skills. USAID.GOV UP TO YOUTH MIDTERM EVALUATION | 38 CONCLUSIONS The program's partners and government institutions comprised a diverse group of people, including local youth authorities, LYAC representatives, and non-governmental organization representatives in the municipalities where the program works. These stakeholders provided valuable feedback by participating in the development of initiatives to address youth needs. According to them, they became involved through participating in Up to Youth meetings, where they helped plan programs based on data, they had about the youth community in their communities. While providing youth with more complex information, which supports and encourages them to participate in public life, the program influenced young people's understanding of their rights and the importance of their role in society. Identified changes by youth because of U2Y’s program included: youth’s participation in public hearings, youth raising issues through their learned new skills for effective budget advocacy’ volunteering more since learning about its importance through the program’ leading community development activities and bringing to government official’s attention the lack of a service or asset. RECOMMENDATIONS 1. Based on data gathered from focus group discussions with youth and stakeholders, Up to Youth, in collaboration with municipalities, should continue to encourage program participants to attend public hearings and apply their learned new skills for effective budget advocacy, even after the program is over, as this makes the youth community more aware of their impact on society. 2. Up to Youth, in collaboration with public and private institutions, should offer volunteering activities for participants and other interested individuals even after the program is completed to ensure continuity of skills acquired during the program, as reported via the survey with youth, in￾depth interviews and focus group discussions with youth and stakeholders. Participants should be motivated in leading community development activities applying the skills they gained from the program. 39 | UP TO YOUTH MIDTERM EVALUATION USAID.GOV CONCLUSION Evaluation data shows that the Up to Youth program provides young people with benefits that will last a lifetime, as they include skills that they will use in their everyday life, work, or community engagement. Providing opportunities for young people is essential to support them in creating their own lives, both professionally and personally.2 Therefore, the continuous learning of skills such as empathy, self-awareness, communication, and creativity has become increasingly valuable and important for future success. These "soft skills" or "human skills" can be seen as meta-skills and as necessary prerequisites for the continuous development of expertise and substance skills that change rapidly with changes in industries and work.3 Since engaging in program activities, young people from various backgrounds have felt more included in society, are more accepting of the social inclusion of members of marginalized communities and have changed perceptions about young from other backgrounds. Moreover, most of the youth believe that the training they received as part of the program has enhanced their life/soft skills. They believe that the program's activities have assisted them in overcoming community difficulties, and they are entirely satisfied with these trainings. As further elaborated by them, the project activities have/will generate capacities in the beneficiaries that will last long after the initiative is completed. Youth who take part in program activities have a greater feeling of social responsibility in their communities. They feel more connected to their community's youth because of the project's activities. As a result, individuals reported taking steps toward positive change and participating in activities such as facing/identifying community challenges, volunteering, dialogues with people outside their group, leading community development initiatives, and extracurricular activities. They believe that participating in community improvement activities has benefited them and has raised their knowledge of the importance of volunteering; they now attempt to be more helpful to those in their community and have developed empathy for others. Program participants are more likely to agree that they are strong enough to confront life's obstacles, confident that they can handle the conditions that life throws at them, and that they believe they can deal with the ordinary problems that life throws at them. Lastly, the program improved young people's knowledge of their rights and the relevance of their participation in society by giving them more complex information that supports and encourages them to participate in public life. However, the possibilities for applying these learned skills into their daily lives are not always good. The national government does not invest in positive outlets for youth, as budget lines for youth programming at both the national and municipal levels are woefully inadequate. Strong resilience partners that accept, respect, and incorporate youth contributions, views, skills, and capabilities are directly responsible for meaningful youth participation. Informal and formal resilience actors are crucial in empowering youth to develop self-efficacy and confidence in their potential to affect good change in their communities.4 In this context, cooperation with the Ministry of Education, Science, Technology and Innovation, the Ministry of Culture, Youth and Sports and local level institutions should be increased. While local-level institutions have always expressed willingness to cooperate during the project, especially in the nine municipalities involved in the project, there is still a lack of financial funding towards youth implemented activities, therefore increasing communication between local and central level is important. 2 Global Employment Trends for Youth 2020. Retrieved from: https://www.ilo.org/wcmsp5/groups/public/---dgreports/---dcomm/--- publ/documents/publication/wcms_737648.pdf, on 14/04/2022. 3 Lemmetty, S. & Collin, K. (2020). Throwaway knowledge, useful skills or a source for wellbeing? Outlining sustainability of workplace learning situations. International Journal of Lifelong Education, early online: https://doi.org/10.1080/02601370.2020.1804004 4 Formative Evaluation of USAID/Kosovo’s Up to Youth Activity by Susan Kosinski Fritz, April 2021