DISCLAIMER: This Sri Lanka YouLead Evaluation Report is made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of this report are the sole responsibility of Social Impact and do not necessarily reflect the views of USAID or the United States Government. FINAL EVALUATION REPORT SRI LANKA YOULEAD PERFORMANCE EVALUATION August 31, 2022 PHOTO CREDIT: IESC ABSTRACT This report summarizes the findings of the Midterm Performance Evaluation of the Sri Lanka YouLead contract, implemented by the International Executive Service Corps (IESC), focusing on progress towards youth employability, youth employment and entrepreneurship (particularly among young women), and strategies for strengthening the activity in the current award period. YouLead worked with the Government of Sri Lanka and private sector partners to achieve scale and institutionalization of new and revised technical training curricula whose co-creation YouLead facilitated. However, reliance on other institutions to deliver the training (in the public and private sectors) revealed weaknesses in their quality control processes and opportunities for leakage of benefits to non-youth populations. YouLead contributed to youth employability through the alignment of skills to employer needs, but the impact of the investments to date has been limited by the barriers in the TVET system that have not been addressed. YouLead had no direct impact on women’s participation in the economy, but employment among graduates of YouLead-supported technical training was higher than overall labor market participation rates for young women (through employment or entrepreneurship). YouLead’s entrepreneurship training enhanced the skills of its youth participants (predominantly young women) but contributed little to an increase of self￾employment among youth, as most of the participants were entrepreneurs prior to the training. The difficulties youth face in starting up their businesses, however, largely remained, as YouLead had little influence on the operating environment. ACKNOWLEDGEMENTS The Evaluation Team would like to thank the many people whose support made this evaluation possible. USAID/Sri Lanka provided thoughtful guidance and assistance over the course of the evaluation, and the team would specifically like to thank Chandima Bandara, Darlene Foote, Rajeev Kandanarachchi, Reneera Paul, and Micah Globerson for their contributions. The team would like to thank the team at IESC for their gracious cooperation and support throughout the evaluation process. This evaluation could not have been completed without the team’s support, and especially without the assistance of Charles Conconi, Sudar Theepan, and Chrishan Pereira. Finally, the team also wishes to thank each participant in this evaluation. Without their generous contributions of time and unique perspectives, the evaluation could not have been successfully completed. TABLE OF CONTENTS ABSTRACT..................................................................................................................................I ACKNOWLEDGEMENTS......................................................................................................II ACRONYMS.............................................................................................................................V EXECUTIVE SUMMARY...........................................................................................................I INTRODUCTION....................................................................................................................1 ACTIVITY BACKGROUND ..................................................................................................1 DEVELOPMENT HYPOTHESIS.......................................................................................................................... 1 ACTIVITY OBJECTIVES ....................................................................................................................................... 2 CRITICAL ASSUMPTIONS.................................................................................................................................. 3 EVALUATION QUESTIONS .................................................................................................4 EVALUATION METHODOLOGY.......................................................................................4 EVALUATION DESIGN AND METHODS ..................................................................................................... 4 SAMPLING............................................................................................................................................................... 7 ETHICAL CONSIDERATIONS AND PROTECTIONS ............................................................................... 8 DATA ANALYSIS................................................................................................................................................... 9 LIMITATIONS AND BIASES ............................................................................................................................... 9 FINDINGS AND CONCLUSIONS ................................................................................... 10 EVALUATION QUESTION 1A........................................................................................................................10 EVALUATION QUESTION 1B ........................................................................................................................19 EVALUATION QUESTION 1C........................................................................................................................23 EVALUATION QUESTION 1D........................................................................................................................27 EVALUATION QUESTION 2...........................................................................................................................30 EVALUATION QUESTION 3...........................................................................................................................33 RECOMMENDATIONS ....................................................................................................... 37 ANNEX I: SCOPE OF WORK.......................................................................................... 39 ANNEX II: DOCUMENTS CITED AND REVIEWED.................................................. 48 ANNEX III: DATA COLLECTION PROTOCOLS....................................................... 51 ANNEX IV. EVALUATION TEAM .................................................................................... 71 TABLE OF TABLES Table I: YouLead Core Activities and Partner Organizations..........................................................................2 Table II: Survey Respondents by Age.....................................................................................................................5 Table III: Survey Respondents by Reported Disability.......................................................................................6 Table IV: Distribution of KIIs...................................................................................................................................7 Table V: Distribution of FGDs.................................................................................................................................7 Table VI: Extent to Which Current Business Income Meets Survey Respondents’ Needs by Gender .......................................................................................................................................................................................20 Table VII: Extent to Which Current Business Income Meets Survey Respondents’ Needs by Hours Worked.......................................................................................................................................................................21 Table VIII: Summary of Strengths and Weaknesses of Virtual and In-Person Training ...........................32 TABLE OF FIGURES Figure I: Survey Respondents by Ethnicity............................................................................................................6 Figure II: Survey Respondents by Religious Affiliation .......................................................................................6 Figure III: Survey Respondents’ Leadership Emergence Scores.....................................................................15 Figure IV: Survey Respondents Self-Efficacy Scores..........................................................................................15 Figure V: Survey Respondents Critical Thinking Scores..................................................................................16 Figure VI: Survey Respondents’ Perseverance Scores.....................................................................................16 Figure VII: Survey Respondents’ Assessment of Utility of Job Search Skills Learned...............................17 Figure VIII: Survey Respondents’ Perceptions of Utility of YouLead.Ik Portal and Online Resources 18 Figure IX: Entrepreneurs Whose Businesses Successfully Generated Income by Gender.....................20 Figure X: Number of Unrelated Adults from Whom Survey Respondents Could Seek Career Advice byh Gender................................................................................................................................................................22 Figure XI Number of Unrelated Adults from Whom Survey Respondents Could Seek Career Advice by Ethnicity.................................................................................................................................................................22 Figure XII: Work Experience among Survey Respondents............................................................................24 Figure XIII: Sector of Work among Survey Respondents...............................................................................24 Figure XIV: Days Worked per Week by Survey Respondents .....................................................................25 Figure XV: Impediments to Work Reported by Survey Respondents........................................................26 Figure XVI: Survey Respondent’s Views of Whether Men and Women Should Have Equality ............29 Figure XVII: Survey Respondents Assessment of In-Person versus Virtual Instructor............................32 ACRONYMS ADB Asia Development Bank ALME Asia Learning, Monitoring, and Evaluation CG Career Guidance COR Contracting Officer Representative COVID Coronavirus Disease DCGC District Career Guidance Committee DTET Department of Technical Education and Training EDB Export Development Board EQ Evaluation Questions ET Evaluation Team FGD Focus Group Discussions ICT Information and Communications Technology IDB Industrial Development Board IESC International Executive Service Corps KII Key Informant Interview MOSD The State Ministry of Skills Development, Vocational Education, Research & Innovation NAITA National Apprentice and Industrial Training Authority NVQ National Vocational Qualification Framework of Sri Lanka NYC National Youth Corps NYSC National Youth Service Council SI Social Impact Inc SLASSCOM Sri Lanka Association for Software Services Companies SME Small and Medium Enterprises SOW Scope of Work SPSS Statistical Package for Social Sciences TO Task Order TVEC Tertiary and Vocational Education Commission TVET Technical and Vocational Education and Training UNIVOTEC University of Vocational Technology USAID United States Agency for International Development YouLead The Youth Employment and Business Start-up Activity i | YOULEAD EVALUATION FINAL REPORT USAID.GOV EXECUTIVE SUMMARY INTRODUCTION AND EVALUATION PURPOSE The Youth Employment and Business Start-up Activity (YouLead) is a seven-year activity (June 2, 2017- May 31, 2024) funded by United States Agency for International Development (USAID)/Sri Lanka and implemented by the International Executive Service Corps (IESC) under a Leader with Associate award (Cooperative Agreement EEM-A-00-04-00002-00 Associate Cooperative Agreement No. AID-383-LA-17-00001). YouLead aims to create better employment – including self￾employment – outcomes for Sri Lankan youth. While primarily USAID-funded with a budget of US $17,999,861.00 the activity expanded in 2021 to include cooperation with the Swiss Development Corporation and European Union (EU). Along with IESC’s cost share the total budget for YouLead is US $23,312,118.84. USAID/Sri Lanka contracted Social Impact, Inc. (SI) to conduct a utilization-focused evaluation of YouLead as a service request under the Asia Learning Monitoring and Evaluation (ALME) contract (No. AID-486-I-14-00001) Task Order (TO) 46 between February and August 2022. The purpose of this evaluation is to assess the performance of the YouLead activity on the achievement of its higher￾level results to date and to identify potential follow-on interventions for career development of youth. The primary audiences for this evaluation are the USAID/Sri Lanka Economic Growth Office, IESC, and key Government of Sri Lanka (GSL) stakeholders who collaborate on the YouLead activity. The findings and recommendations from this evaluation will provide stakeholders with critical information to make adaptations in YouLead’s approaches or strategies for the remainder of the award and to design follow-on activities, and especially interventions related to career development for youth and human capital development. EVALUATION QUESTIONS This evaluation will address the following Evaluation Questions (EQs): 1. To what extent did YouLead achieve the following youth employment outcomes? a. Improving youth employability through skills gained from vocational training, particularly in terms of aligning skills to the emerging needs of the economy. b. Increasing entrepreneurial skills that position youth for realistic, gainful employment. c. Improving women's participation in the economy. d. Dispelling negative stereotypes that limit young men’s and women’s career choices. 2. Compare YouLead's youth skill training delivery approaches (in-person and virtual). a. What are the strengths and weaknesses of each approach in terms of effectiveness, efficiency and inclusiveness and suggest best ways forward? 3. What were the key advantages and challenges to working with each of the main YouLead stakeholders in achieving the activity objectives (including cross-cutting objectives)? SUMMARY OF EVALUATION METHODOLOGY The Evaluation Team (ET) utilized a mixed methods approach to answer the EQs. In addition to a document review, the ET conducted key informant interviews (KIIs), focus group discussions (FGDs), and a survey. KEY INFORMANT INTERVIEWS The ET conducted 69 KIIs with USAID/Sri Lanka staff members, IESC staff, GSL partners, private sector partners, youth, and parents. The ET selected a purposive sample of respondents, focusing on those most involved with YouLead. USAID.GOV YOULEAD EVALUATION FINAL REPORT | ii FOCUS GROUP DISCUSSIONS The ET conducted 28 FGDs with 215 respondents, comprised of trainers, career guidance practitioners, financial institution officers, National Apprentice and Industrial Training Authority (NAITA) inspectors, youth participants, and parents of youth participants. BENEFICIARY SURVEY The ET collected additional information from 83 of the youth FGD participants via a survey. This approach provided a way to gather more in-depth information about the YouLead participants and their training experience. CONCLUSIONS EQ1A: TO WHAT EXTENT DID YOULEAD IMPROVE YOUTH EMPLOYABILITY THROUGH SKILLS GAINED FROM VOCATIONAL TRAINING, PARTICULARLY IN TERMS OF ALIGNING SKILLS TO THE EMERGING NEEDS OF THE ECONOMY? YouLead supported curriculum revision and development processes that effectively aligned content to employer needs. The weaknesses in the GSL Technical and Vocational Education and Training (TVET) quality control systems, however, reduced the effectiveness and reach of these interventions. In particular, the ET found a lack of appropriate systems led to barriers in fully institutionalizing the improved curricula. Specifically, the absence of: 1) verification of new curricula usage, 2) whether instructors received appropriate orientation/training when Tertiary and Vocational Education Commission (TVEC) released new curricula, and 3) quality assurance beyond review of lesson plans. YouLead also developed pedagogical training that improved the instructional techniques of the trainers who were able to apply them in their classroom settings. However, the gap in responsibility and accountability for continuing education of instructors meant that it was difficult to embed this new training into current systems. It was not yet clear if the online option with an external assessor would be successful, as the primary barrier to professional training for instructors and career guidance (CG) practitioners was finding personnel to cover their normal duties while they attended training. The ET found that the training modality would not necessarily decrease the participants’ time away from other responsibilities (i.e., reduce the burden of finding coverage for their normal duties). Further, the inflexibility of some existing curricula (particularly in highly technical fields) prevented instructors from applying the student-centered pedagogical practices they learned through YouLead training, which might have otherwise improved the students’ experiences and outcomes. YouLead achieved some success in integrating CG training and the Career Key tool. CG efforts that are not embedded in the TVET system or schools – Career Bus and career fairs – found success in assisting youth to enroll in courses aligned to the interests and strengths they identified in Career Key. Within the schools and TVET system, however, the knowledge and understanding gaps among stakeholders enabled them to dismiss the need for CG and made the CG practitioners less effective than they might have been in some areas. The YouLead.lk portal was not well recognized, and the necessary upkeep required extensive effort beyond what YouLead could support. Despite these challenges, YouLead contributed to youth employability through skill alignment, but the investments’ impact to date limits the barriers in the TVET system that are unaddressed. Young women continued to face higher barriers to pursuing training and finding employment than young men. Youth with disabilities and youth in rural and remote areas did not appear to attain the same benefits as their urban and peri-urban peers and peers without disabilities. iii | YOULEAD EVALUATION FINAL REPORT USAID.GOV Finally, the YouLead.lk portal did not contribute substantially to bridging the gaps between youth and employers, as youth neither recognized nor used the portal. The extensive effort needed on YouLead’s part to keep it current exacerbated the lack of awareness. EQ1B: TO WHAT EXTENT DID YOULEAD INCREASE ENTREPRENEURIAL SKILLS THAT POSITION YOUTH FOR REALISTIC, GAINFUL EMPLOYMENT? YouLead’s entrepreneurship curriculum (and delivery in partnerships with the Samurdhi Authority) improved the entrepreneurial skills of most youth who completed the course. However, some students lacked familiarity with the basic concepts and could not apply the content to their own businesses. In addition, young entrepreneurs would have benefitted from a soft skills module analogous to what is in the revised YouLead technical curricula. YouLead entrepreneurship training contributed little to an increase of self-employment among youth, as most of the participants were entrepreneurs prior to enrollment. However, many of the technical training participants also started their own businesses. The difficulties youth face in starting up their businesses largely remained, as YouLead had little influence on the operating environment. Although not asked directly, youth FGD respondents identified five common challenges: insufficient income from their businesses, a lack of follow-on support after training, supply chain issues, price increases for inputs, and a lack of market linkages. Other issues that arose in KIIs included a lack of business registration (which limits their markets to their home region) and a lack of understanding of packaging, marketing, and value addition. While participants reported some openings for youth in financing and gratis membership in the Jaffna Chamber of Commerce, for example, the supply-side orientation of YouLead had little impact on the fragmented business development services environment that youth entrepreneurs must navigate. EQ1c: To what extent did YouLead improve women's participation in the economy? YouLead had no direct impact on women’s participation in the economy, but employment among graduates of YouLead-supported technical training was higher than overall labor market participation rates for young women. Despite that, the percentage of young women who never actually worked was almost double to that of young men (37 percent compared to 21 percent, respectively). The YouLead priority sector in which the plurality of young women (along with a substantial proportion of young men) reported working was in information and communication technologies (ICT). Fewer young women were employed in other sectors, and outside of ICT, employment patterns typically followed traditional gender norms. EQ1D: TO WHAT EXTENT DID YOULEAD DISPEL NEGATIVE STEREOTYPES THAT LIMIT YOUNG MEN’S AND WOMEN’S CAREER CHOICES? Gendered employment patterns are shifting in Sri Lanka, but YouLead did not make significant contributions to those changes. The construction industry in the western part of the country and several industries in the southern part of the country appeared to be more open to hiring qualified workers regardless of gender. The technical trainings YouLead revised or developed seemed to have been effective in providing young women with the skills that employers need (according to trainers and CG practitioners), but FGD participants indicated that many did not complete their courses – particularly in the central, northern, and eastern parts of the country. Dropouts occurred because the youth found employment for themselves or learned from others that employers were not interested in hiring them in male-dominated fields. USAID.GOV YOULEAD EVALUATION FINAL REPORT | iv EQ2: WHAT ARE THE STRENGTHS AND WEAKNESSES OF IN-PERSON AND VIRTUAL APPROACHES IN TERMS OF EFFECTIVENESS, EFFICIENCY, AND INCLUSIVENESS AND WHAT ARE THE BEST WAYS FORWARD? Both in-person and virtual training options had strengths and weaknesses to offer YouLead participants. Learning can be effective in both modalities, but respondents had more attentiveness with in-person approaches, and youth in particular found in-person instruction to be more effective. Some of the professional respondents preferred virtual training, particularly for refresher material or material that was largely theoretical for which they had a proper grounding. Where virtual was largely ineffective was in the practice of hands-on skills, which was relevant for most of the technical training but also to a lesser extent for the professional training (role playing in online platforms was difficult, for example). Professionals also valued the opportunities for networking that in-person training offered, which helped them build a network of peers on whom they can rely over time. In terms of efficiency, virtual training led to reduced costs and allowed geographically distributed learners to all work together, which was particularly useful for professional training. However, virtual trainers needed more preparation than in-person trainers, as many had less experience delivering using virtual platforms, and they needed to adjust their regular practice to be effective. In terms of inclusivity, neither modality of training offered a panacea. The ET found that many training facilities lacked accessibility to youth with disabilities, and youth who lived in rural or remote areas were often unable to reach them due to distance (travel time and lack of access to safe, reliable transportation). Some responses noted that virtual instruction also posed challenges due to the lack of devices, Wi-Fi access, and technological literacy required to follow a virtual course. EQ3: WHAT WERE THE KEY ADVANTAGES AND CHALLENGES TO WORKING WITH EACH OF THE MAIN YOULEAD STAKEHOLDERS IN ACHIEVING THE ACTIVITY OBJECTIVES (INCLUDING CROSS￾CUTTING OBJECTIVES)? YouLead worked with the public sector to ensure scale and institutionalization. However, reliance on public sector institutions revealed some of their weaknesses, including ck of support around dissemination of updated course materials, quality control, a diffuse group of stakeholders, and a lack of clear ownership on the part of some stakeholders. It also meant that YouLead did not direct its resources specifically to youth. Working with large corporations presented similar advantages and challenges. Scale could be achieved in a sense as a sector's largest actors adopt certain standards, pulling the rest of the sector in that direction. Corporate training systems also could represent a form of institutionalization, and they typically bring subject matter expertise to the fore as well. However, YouLead's contributions could then be used for any target group the private sector partner chooses, and some noted no intention to expand their hiring of youth. The flexibility that a cooperative agreement and USAID’s management of this activity offered have been crucial in keeping the project moving forward with both public and private sector partners despite the myriad disruptions that Sri Lanka experienced over the past several years. However, that flexibility also meant that YouLead dropped certain activities or left things unfinished, which damaged its relationships and brand with some stakeholders. The lack of follow-through in the wake of a disruption in particular has been most harmful. RECOMMENDATIONS The following recommendations outline ways in which USAID/Sri Lanka and other YouLead actors could use its remaining time and longer-term considerations for future programming. v | YOULEAD EVALUATION FINAL REPORT USAID.GOV RECOMMENDATIONS FOR YOULEAD Given the short time remaining on the YouLead activity, the ET recommends the following: ● IESC should focus on quality control within TVEC to ensure high quality delivery of services. ● IESC should deepen engagement with other organizations among the current partner pool that can extend delivery to those who are not served. ● IESC should continue to offer virtual and in-person training options but require an in-person training component for hands-on skill work and to support networking opportunities. ● IESC should revisit commitments that are not fulfilled and close out as many of those as is feasible, focusing first on those that have promise for youth employment. ● IESC should monitor future commitments to ensure that communication continues in the wake of any disruption. RECOMMENDATIONS FOR FUTURE PROGRAMMING For future programming, the ET recommends the following: ● USAID should work with partner TVET institutions to develop and implement effective quality control practices at the classroom level. ● USAID should consider how to support TVET institutions to improve their accessibility, particularly to youth who are in vulnerable situations. ● USAID should incorporate a stronger component on promoting inclusive and gender￾sensitive human resources policies within private sector partner organizations and employers working with the program. ● USAID should commission a study to identify key messages that increase acceptance of nontraditional career choices among parents and families. ● USAID should support a dialogue with the Sri Lanka Foundation and The State Ministry of Skills Development, Vocational Education, Research & Innovation (MoSD)/TVEC to create a CG diploma course. 1 | YOULEAD EVALUATION FINAL REPORT USAID.GOV INTRODUCTION The Youth Employment and Business Start-up Activity (YouLead) is a seven-year activity (June 2, 2017- May 31, 2024) funded by United States Agency for International Development (USAID)/Sri Lanka and implemented by the International Executive Service Corps (IESC) under a Leader with Associate award (Cooperative Agreement EEM-A-00-04-00002-00 Associate Cooperative Agreement No. AID-383-LA-17-00001). YouLead aims to create better employment – including self￾employment – outcomes for Sri Lankan youth. While primarily USAID-funded with a budget of US $17,999,861.00 the activity expanded in 2021 to include cooperation with the Swiss Development Corporation and European Union (EU). Along with IESC’s cost share the total budget for YouLead is US $23,312,118.84. USAID/Sri Lanka contracted SI to conduct a utilization-focused evaluation of YouLead as a service request under the Asia Learning Monitoring and Evaluation (ALME) contract (No. AID-486-I-14- 00001) Task Order (TO) 46 between February and August 2022. The purpose of this evaluation is to assess the performance of the YouLead activity on the achievement of its higher-level results to date and to identify potential follow-on interventions for career development of youth. The primary audiences for this evaluation are the USAID/Sri Lanka Economic Growth Office, IESC, and key Government of Sri Lanka (GSL) stakeholders who collaborate on the YouLead activity. The findings and recommendations from this evaluation will provide stakeholders with critical information to make adaptations in YouLead’s approaches or strategies for the remainder of the award and to design follow-on activities, and especially interventions related to career development for youth and human capital development. ACTIVITY BACKGROUND Youth unemployment is a major economic concern in Sri Lanka, resulting from skills shortages and labor market mismatches. Sri Lanka has one of the highest youth unemployment rates in the Asia￾Pacific region, with female youth unemployment rate of 34.4% compared to a male youth unemployment rate of 21.0% in 2020 (World Bank, 2022) and aligns with a common pattern of disproportionate care responsibilities falling to women and traditional gender norms discouraging their participation in the formal labor force. Further, some of the industries that have significant potential (e.g., tourism and hospitality, construction, and textiles) experienced significant contraction, including among jobs youth typically hold, during the coronavirus disease (COVID-19) pandemic (ADB 2020).1 DEVELOPMENT HYPOTHESIS The development hypothesis underpinning the YouLead was: 1) If young women and men can access high-quality, market-driven vocational education and training, and have support that addresses structural and cultural impediments to employment, as well as the gap in technical and life skills, they will be able to obtain suitable jobs; and 2) [If] Offering skilled youth a package of integrated entrepreneurial and business development services, including access to finance, will increase sustainable self-employment. [Then] Improving access to job opportunities will reduce inequality and promote social cohesion, citizen empowerment, and personal dignity necessary for sustained growth and development. 1 https://www.adb.org/sites/default/files/publication/626046/covid-19-youth-employment-crisis-asia-pacific.pdf USAID.GOV YOULEAD EVALUATION FINAL REPORT | 2 ACTIVITY OBJECTIVES The objectives of the activity are as follows: ● Increase youth employability skills in targeted sectors. ● Improve quality, relevance, and delivery of TVET. ● Increase prospects of successful self-employment. The YouLead activity intends to contribute to inclusive economic growth by increasing employability of young people aged 16-35. Its primary strategies for achieving these objectives are through the strengthening vocational training (curricula, pedagogy, and linkages with private sector) for students, teachers, institutions, and entrepreneurs; improving workforce capacity to meet market demand; and linking with the private sector to encourage employment opportunities for youth (see Table I for a summary of core activities). Through the project’s market assessment, YouLead identified and prioritized the districts with the highest unemployment among young women and men, including Jaffna, Galle, Kandy, Mannar, Kegalle, Badulla, Kilinochchi, Hambantota, Nuwara Eliya, Monaragala, Matara, and Matale. It also focused on key sectors positioned for growth, as its market assessment and a review of local stakeholders identified, including tourism and hospitality, construction, information and communication technologies (ICT), and healthcare/caregiving. Additionally, YouLead placed a strong emphasis on gender, disability, and ethno-religious inclusion and expediting positive behavior changes. Table I: YouLead Core Activities and Partner Organizations Activity Description Partner(s) Curriculum Development and Delivery – Pedagogy Training YouLead developed a three-level Master Teacher training for instructors that incorporated general instructional strategies, assessment, and cooperative learning (Level 1), soft skills training (Level 2), and designing training (Level 3). Department of Technical Education and Training (DTET), National Apprentice and Industrial Training Authority (NAITA), Vocational Training Authority (VTA), Tertiary and Vocational Education Commission (TVEC), Open University, private sector partners, technical colleges Curriculum Development – Business Start-up Training Program YouLead developed a five-day, four-module entrepreneurship course for youth that included 1) Practical Entrepreneurship; 2) Business Idea Generation, Screening and Evaluation, and Selection, 3) Business Idea Feasibility Testing and Marketing Strategy, and 4) Business Plan Development and Business Start-up.2 Samurdhi Authority Curriculum Revision/Development - Technical Training YouLead worked with public sector partners to develop or update 90 curricula in various technical fields to improve alignment with industry needs and incorporate soft skills. Courses ranged from 40 hours to three years in duration to be offered to students in TVET institutions throughout Sri Lanka. NAITA, TVEC, VTA 2 NAITA and TVEC have not yet validated or endorsed this curriculum, but the Samurdhi Authority has begun using it in a few communities. 3 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Curriculum Revision/Development - Technical Training With private sector partners, YouLead offered its pedagogy training, along with leadership training to strengthen internal corporate training capacity. YouLead also developed and delivered (or planned to develop and deliver) technical curricula food and beverage and healthcare that its private sector partner could submit to NAITA and TVEC for validation and endorsement (with YouLead support). Private sector partners Curriculum Development and Delivery – CG Practitioner Training In order to establish sustainable, youth-friendly career counseling services, YouLead developed virtual and in-person training curricula for CG practitioners, counselors, teachers, Human Development Resource Officers, Skills Development Officers, NAITA inspectors, and other youth service professionals. MOSD, National Youth Corps (NYC), National Youth Service Council (NYSC), Provincial and Zonal Education Offices, District Secretariats, NAITA, VTA, DTET, Department of Manpower and Employment, secondary schools, colleges, and universities Curriculum Development and Delivery – Financial Institution Officer Training YouLead developed training on Lending Practices and Credit Analysis and Appraisal in Microfinance for financial institution officers to improve their knowledge and skills related to assessing the credit-worthiness of microenterprises. Private sector partners, Bank of Ceylon Social Media Campaigns YouLead conducted 14 social media campaigns to encourage employment, entrepreneurship, and work in non-traditional sectors, particularly for young women. Private sector partners YouLead.lk Portal YouLead developed an online job search and learning management system portal to connect youth with job opportunities and training resources for individual, self-guided learning. Private sector partners Career Bus YouLead launched the Career Bus in 2021 to reach youth in rural and remote areas. The Career Bus offered CG services and helped youth identify training opportunities. VTA, NAITA CRITICAL ASSUMPTIONS The initial activity work plan identified the following critical assumptions: ● Beneficiaries, particularly youth, would be willing to participate and collaborate in program activities. ● Organizations both public and private sector would provide in-kind and private or public financial leveraging. ● MOSD and other government institutions would support, instill, and take ownership of program recommendations and would adapt market-based TVET courses and career guidance and development recommendations. ● Technical and vocational education training centers, such as VTA of Sri Lanka and NAITA, among others, would be willing to support mind-set change, career guidance, and existing teaching and learning practices. ● The political situation in Sri Lanka would remain stable and no significant changes to its policies are made that may prove detrimental to program core initiatives. The fiscal year 2021 and 2022 work plans identified specific assumptions related to a reduction in the potential for growth in the tourism sector in the wake of COVID-19, changes in the commitment of local actors to sustain work that YouLead initiated (due to changes in personnel, policies, and other factors outside of YouLead’s control), the limited availability and accessibility of USAID.GOV YOULEAD EVALUATION FINAL REPORT | 4 business development services in Sri Lanka, and reduced availability of financing for young entrepreneurs. In addition, based on the experience of YouLead to date, the Evaluation Team (ET) identified two other tacit assumptions that underpinned the activity. The activity assumed that the public sector would work largely uninterrupted and with an approved operating budget, and any changes in counterparts would occur on a schedule that would be known in advance (and not be crisis-driven). Another critical assumption identified the security situation (including public health) would remain stable to allow the uninterrupted running of YouLead activities, or that instability or violence would not disrupt service delivery and partnership activities. EVALUATION QUESTIONS To evaluate the YouLead activity, the ET focused on the following three Evaluation Questions (EQs): 1. To what extent did YouLead achieve the following youth employment outcomes? a. Improving youth employability through skills gained from vocational training, particularly in terms of aligning skills to the emerging needs of the economy. b. Increasing entrepreneurial skills that position youth for realistic, gainful employment. c. Improving women's participation in the economy. d. Dispelling negative stereotypes that limit young men’s and women’s career choices. 2. Compare YouLead's youth skill training delivery approaches (in-person and virtual). a. What are the strengths and weaknesses of each approach in terms of effectiveness, efficiency and inclusiveness and suggest best ways forward? 3. What were the key advantages and challenges to working with each of the main YouLead stakeholders in achieving the activity objectives (including cross cutting objectives)? EVALUATION METHODOLOGY EVALUATION DESIGN AND METHODS The ET utilized a mixed-methods evaluation methodology consisting of document review, key informant interviews (KII), focus group discussions (FGDs), and a short screener survey for youth participants. The ET collected data between May 12, 2022 and June 16, 2022 in nine sites: Colombo, Jaffna, Matara, Anuradhapura, Hambantota, Ampara/Batticaloa, Nuwara Eliya, Kurunegala, and Kandy. Many of the key stakeholders were in Colombo (including USAID, IESC, GSL, and private sector partners), while Jaffna, Kandy, and Hambantota served as regional office hubs for collaboration with government and the private sector. DOCUMENT REVIEW The ET conducted a comprehensive and structured document review to inform the evaluation design, develop data collection tools, and contextualize the evaluation findings. USAID and IESC provided a package of program descriptions, work plans, quarterly and annual reports, labor market assessments, gender assessments, special studies, success stories, curriculum documents, and participant information for their initial review (see Annex II for a list of the documents reviewed). The ET condensed key findings into an initial desk review matrix to allow quick reference to the underlying development challenges, program strategies, and results to date. KEY INFORMANT INTERVIEWS The ET conducted 69 KIIs with USAID, IESC, private sector partners, government partners, and youth graduates of YouLead training activities. Using tailored KII guides to the roles individuals 5 | YOULEAD EVALUATION FINAL REPORT USAID.GOV played, the ET conducted individual, in-person or remote interviews in English, Sinhala, or Tamil, depending on the preference and availability of the interviewee. FOCUS GROUP DISCUSSIONS The ET conducted 28 FGDs with 215 respondents to gather information from youth service provider professionals who participated directly in YouLead trainings (e.g., master trainers, training of trainer participants, career guidance (CG) practitioners, financial institution officers, NAITA inspectors), as well as those who benefitted from the upgraded YouLead training, or the youth who completed entrepreneurship or technical training under the auspices of the trained professionals. FGDs allowed the ET to gather both broad and in-depth information about the experiences of these stakeholders and their perceived results. FGDs took place in either Tamil or Sinhala, depending on location and preferred language of the respondents. SCREENER SURVEY The ET also surveyed 83 youth who participated in FGDs and agreed to also complete a short screener survey in either Sinhala or Tamil. Survey respondents completed self-administered, paper copy surveys. The survey responses contributed to answering EQ 1. More than half of the respondents, 59.0%, identified as female, 39.8% identified as male, and 1.2% reported their gender as other. The respondents also covered the full range of participant ages of YouLead, as shown in Table II. Table II: Survey Respondents by Age Age Group Number Percentage 18-19 20 24.1% 20-24 23 27.7% 25-29 15 18.1% 30-34 13 15.7% 35 and older 8 9.6% Did not respond 4 4.8% * The six respondents older than 35 were under the age of 35 when they completed their training, and the ET allowed them to respond to help capture the longer-term impacts of the YouLead activity. The other two respondents in the 35 and older category were 35. The survey respondents also represented the range of ethno-religious affiliations common across Sri Lanka. Figure I and Figure II below show the distribution of survey respondents by ethnicity and religious affiliations they reported. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 6 Ten youth respondents who participated in YouLead technical training also reported that they have some degree of disability that could impact their ability to work or successfully seek employment. Table III (on next page) shows the distribution of respondents by type of impairment. Table III: Survey Respondents by Reported Disability Type of Impairment Some difficulty (#) A lot of difficulty (#) Cannot do at all* (#) Total Number Total Percent Difficulty seeing (with glasses) 3 1 1 5 6.0% Difficulty hearing (with hearing aid) 1 1 1 3 3.6% Difficulty walking or climbing stairs 0 0 1 1 1.2% 58% 34% 7% 1% Sinhalese Tamil Moor Multi-ethnic 20 6 6 1 27 22 1 0 5 10 15 20 25 30 Sinhalese Tamil Moor Multi-ethnic Sinhalese Tamil Sinhalese Male Female Other Count Figure I: Survey Respondents by Ethnicity Figure II: Survey Respondents by Religious Affiliation 7 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Difficulty concentrating or remembering 2 0 1 3 3.6% Difficulty with self-care 0 0 1 1 1.2% Difficulty communicating 0 1 1 2 2.4% * It should be noted that a single respondent straight-lined all the ‘cannot do at all’ responses; based on the other information provided in the survey, this was likely done in error. SAMPLING The evaluation design included different sampling methods for the data collection methods and stakeholder groups. The ET paid close attention to and selected sites with concentrations of different ethno-religious and language groups, varied dominant economic activities, and therefore opportunities for youth employment and entrepreneurship. The ET drew a purposive sample for the KIIs with USAID, IESC, public sector, and private sector respondents, as these individuals held key positions and had specific insights on the program. As shown in Table IV, the gender balance among KII participants was uneven, which reflected the overall composition of key staff, particularly among IESC, GSL partners, and private sector partners, which were heavily male. Table IV: Distribution of KIIs Site USAID IESC GSL Partners Private Sector Partners Youth Parents Total Colombo 2 6 13 14 0 0 35 Jaffna 0 1 1 1 0 0 3 Matara 0 0 0 0 2 0 2 Anuradhapura 0 0 0 1 1 1 3 Hambantota 0 1 1 1 4 2 9 Nuwara Eliya 0 0 0 0 5 3 8 Kandy 0 1 1 1 4 0 7 Remote 0 2 0 0 0 0 2 Total 2 11 16 18 16 6 69 % Female 50.0% 18.1% 18.8% 33.3% 50.0% 100.0% 37.7% % Male 50.0% 81.8% 81.3% 66.6% 50.0% 0.0% 62.3% The ET selected FGD participants for the master trainer, training of trainer, CG practitioner, financial institution officer, and NAITA inspector groups on a random basis from a stratified sampling frame based on activity, location, and gender/sex (disability status and ethno-religious affiliation were not available in the administrative data). The ETidentified alternates where more than eight individuals had participated in the training. Despite the ET’s intention to maintain a gender balance in FGDs, a combination of gender imbalance in the sampling frame and limitations in availability resulted in imbalanced groups, particularly among master trainers, financial institution officer, training of trainer, and CG practitioners (see Table V for additional detail). Table V: Distribution of FGDs Site Master Trainers Training of Trainers Career Guidance Practitioners Financial Institution Officers NAITA Inspectors Youth Parents Total Colombo 1 0 1 1 0 1 0 4 USAID.GOV YOULEAD EVALUATION FINAL REPORT | 8 Jaffna 0 1 1 0 0 1 1 4 Matara 0 0 1 0 0 3 0 4 Anuradhapura 0 1 0 0 0 1 0 2 Hambantota 0 1 0 1 1 0 0 3 Ampara/Batticaloa 0 1 1 0 0 3 0 5 Nuwara Eliya 0 0 0 0 1 0 0 1 Kandy 0 0 1 1 0 0 0 2 Kurunegala 1 0 0 1 0 1 0 3 Total 2 4 5 4 2 10 1 28 % Female Respondents 23.1% (n=3) 30.6% (n=11) 33.9% (n=19) 26.3% (n=5) 45.5% (n=10) 63.1% (n=41) 100.0% (n=4) 43.3% (n=93) % Male Respondents 76.9% (n=11) 69.4% (n=25) 66.1% (n=37) 73.7% (n=14) 54.5% (n=12) 36.9% (n=24) 0.0% (n=0) 56.7% (n=122) The ET sampled youth who participated in the YouLead-Samurdhi entrepreneurship training in Matara in a similar fashion. Using the full list of participants YouLead provided, the ET filtered the list for participants outside of the YouLead age range, then selected three neighborhoods with high numbers of trainees (to address EQ 2, this included one where all training is provided virtually), and randomly selected youth to invite for participation in FGDs. In total, 21 youth (14 young women and seven young men) participated in these FGDs in Matara. The imbalance in youth participants partially reflected the gender imbalance in the sampling frame for the entrepreneurship training participants. The ET’s inability to access a sampling frame for the technical training YouLead partners provided constituted a key constraint, particularly when identifying and sampling youth who participated in other types of training. The MOSD collects and stores all administrative data on enrollees in technical trainings TVET institutions offered. Given the timeline for the evaluation and the crisis affecting the GSL during this time frame, the ET was unable to obtain a comprehensive list of youth who had participated in the YouLead-supported technical trainings. Therefore, the ET relied on the snowball sampling technique to identify its youth respondents from technical training courses. Trainers, CG practitioners, and NAITA inspectors who participated in FGDs recommended youth potentially willing to participate, and the ET contacted those youth for KIIs, or if enough could be brought together, an FGD. Using this approach, the ET conducted KIIs with a convenience sample of 16 youth who participated in YouLead-supported entrepreneurship or technical training, as well as seven FGDs with a total of 44 youth. The gender imbalance in youth FGD participants partially reflected the biases introduced by the youth sample approach. As the ET expected that no parent information would be readily available, the sampling frame for parents included contact information that youth FGD participants and survey respondents provided voluntarily. The parent sample was therefore a convenience sample, and this approach enabled the ET to complete six parent interviews and one parent FGD with four participants. Although the ET aimed for a gender balance in parent KIIs and FGDs, all of the parent respondents were women as no fathers were willing or able to participate. ETHICAL CONSIDERATIONS AND PROTECTIONS All members of the ET received guidance and sensitization to the importance of maintaining the privacy, confidentiality, and protection of the study participants. Overall, the ET had no reason to believe any evaluation participants would experience any negative consequences for participating in the evaluation. The ET took several measures to ensure that all study participants were comfortable. First, the sample did not include any minors. Although YouLead worked with individuals as young as 16, the ET determined that the process of obtaining parental consent would be overly burdensome 9 | YOULEAD EVALUATION FINAL REPORT USAID.GOV on the parents, as the ET mostly collected data from youth and other respondents in person. Second, at the beginning of each KII or FGD, the interviewer/moderator read a detailed informed consent statement that made clear the purpose of the evaluation, the use of the data, and how participant information is treated. The ET notified all respondents that they could skip questions they preferred not to answer or end their participation at any time. Third, the data protection practices ensured the confidentiality of any individual’s data or responses, with identification codes used in place of names in all files. Only the ET had access to raw data; notes and recordings is on SI’s secure server and is destroyed upon submission of the report. DATA ANALYSIS As the data collection plan yielded both quantitative and qualitative data, the ET conducted parallel streams of analysis, then triangulated and synthesized the information. DATA PREPARATION At the completion of each KII or FGD, the ET transcribed into detailed and summary notes and saved along with the recordings (when permission to record is granted) on SI’s server. Similarly, as youth participants completed their screener surveys prior to their KIIs or FGDs, a member of the ET reviewed each form to ensure clarity and completeness. The ET entered survey data into Kobo and exported it into Statistical Package for Social Sciences (SPSS) for analysis. CODING The ET conducted the qualitative data analysis using deductive coding in Microsoft Excel following key themes under each EQ and key category (role, location, ethno-religious affiliation, and gender). To ensure consistency of understanding and use of the codes, the ET coded segments of the KIIs and FGDs jointly in an internal data analysis workshop. Based on the coding results, the ET worked collaboratively to develop the findings and conclusions. GENDER AND SOCIAL ANALYSIS PLAN The ET used the coding scheme to conduct content analysis of all generated qualitative data. The ET reviewed the coded segments relevant to the evaluation question being answered and synthesized the findings, accounting for stakeholder grouping, gender/sex, disability status, ethno-religious affiliation, location, time frame of engagement, specific activity (i.e., type of training or engagement), and mode of participation. The ET analyzed the youth survey in SPSS and examined the data for central tendencies, correlations, and sub-group differences. However, as the survey sample was small, the ET used the data descriptively rather than inferentially. In addition, the ET reviewed the activity’s monitoring data, including a review of outcomes of interest by gender and disability status, and publicly available documents relevant to youth employability and employment in Sri Lanka. LIMITATIONS AND BIASES The ET noted several potential biases/limitations for this evaluation: lack of baseline data, selection bias, recall bias, framing bias, and the effects of economic dislocation. Lack of baseline data. As the activity did not provide training directly to youth, YouLead did not gather baseline information directly on the extent of participants’ knowledge of the technical or entrepreneurship content related to their training. GSL partners relied on their admissions criteria, as well as the completion of any prerequisite courses for the trainings, which therefore prevented the ET from measuring changes in knowledge as a result of specific training courses. Selection bias. The ET had no way of assessing the extent to which the evaluation participants were similar to or differed from others who participated in or benefitted from the YouLead activity. This was particularly true of the youth who participated in technical training and parents, as the USAID.GOV YOULEAD EVALUATION FINAL REPORT | 10 convenience sample was quite small compared to the total population, and the ET had no way of knowing why trainers or others who referred youth to participate in the evaluation remained in contact with them but not with other youth. The ET therefore treated this information carefully, triangulating it with other respondents’ perceptions and presenting conflicting information as transparently as possible without abrogating respondent privacy. Further, only nine of 16 sites are in the evaluation (representing approximately one half of participants), and respondents in other sites might have had different experiences, as the ET identified differences across the included sites. Finally, the ET is also unable to say how self-selection bias might have influenced the data, particularly among the parent sample in which no fathers are represented. Recall bias. The ET also could not account for how individuals’ recollections might have deviated from the actual events that occurred or others’ perceptions of a shared experience. Given that the ET asked respondents to recall their experiences from as many as five years ago, the recollections may have faded somewhat over time. Framing bias. Framing bias also may have affected some responses, in which participants shared largely positive perceptions and refrained from discussing negative experiences. The ET asked about negative stakeholder experiences from a position of what they would change in future programming. Most stakeholders shared perceived deficiencies in the YouLead activity; the ET therefore considered framing bias to be minimal. Economic dislocation. In addition to limitations related to the design and the availability of participant information, Sri Lanka’s economic crisis and unrest escalated throughout the data collection period. Although the ET traveled to each of the sites as planned, those factors may have impinged on some individuals’ ability to participate in the evaluation (contributing to further selection bias). Further, the results showed that the economic crisis affected some of the program outcomes, particularly around youth employment (discussed below). However, given data limitations, the evaluation could not determine if the economic crisis under- or overstated the perceived impact on youth who completed YouLead-supported training. FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1A To what extent did YouLead improve youth employability through skills gained from vocational training, particularly in terms of aligning skills to the emerging needs of the economy? FINDINGS To evaluate the extent to which YouLead improved youth employability through skills gained from vocational training and alignment to the emerging economy, the ET examined the ways in which the activity was able to respond and adapt. CURRICULUM DEVELOPMENT AND DEPLOYMENT A first step for YouLead in addressing the misalignment of training content and employer needs was to revise or develop curricula to reflect the current demands of the labor market. As such, YouLead collaborated with the TVEC and its partner agencies to update national competency standards and curricula under the National Vocational Qualification Framework of Sri Lanka (NVQ). KII respondents shared that the process for each curriculum revision or development entailed a series of public-private stakeholder meetings to ensure that the revised or new curriculum met industry 11 | YOULEAD EVALUATION FINAL REPORT USAID.GOV needs. YouLead supported the development/revision of 90 curricula, of which the relevant GSL approving body for classroom and/or virtual use endorsed 47. Overall, KII and FGD respondents from the GSL, private sector partners, trainers, and National Apprentice and Industrial Training Authority (NAITA) inspectors concurred that the new and revised curricula were significantly better aligned to employer needs, but most FGD respondents were unaware of YouLead’s role in the process of developing the curricula. In FGDs and KIIs, respondents described the process for implementing new and revised curricula. They shared that after accreditation and validation, TVEC published the revised or new curriculum on its website and mailed it to district-level technical institutions. There was typically no orientation or training on the new materials. At the district level, FGD respondents indicated that it takes three to six months or longer to implement a new or revised curriculum because instructors are not prepared to deliver the new materials using pedagogical approaches with which they are not familiar, lack of machinery or equipment (some of which is not updated since the aftermath of the 2004 tsunami), and a shortage of expendable supplies. Several trainers and CG practitioners indicated that technology-intensive fields are the most challenging, for example, FGD respondents in Matara indicated that the equipment available for Computer Numerical Control and Robotic Technology training was the contemporary equipment 18 revisions ago. Therefore, some new curricula is not offered at the district level. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 12 TRAINING OF TRAINERS AND MASTER TRAINERS A second step in addressing alignment issues was strengthening pedagogical skills of the instructors teaching the new and revised curricula. In partnership with the DTET, NYSC, Vocational Training Authority (VTA), NAITA, TVEC, and a range of private sector employers, YouLead has also trained approximately 850 instructors from both public and private sector institutions in pedagogical and technical training and use of psychometric tests. These trainings utilized an activity-based approach, and FGD participants from training of trainer courses were very positive about the training’s impact, as they reported that it strengthened their ability to effectively deliver courses for their students. Among the 850 trained, YouLead selected 52 to be Master Trainers who are tasked with coaching and mentoring others within their district’s technical institutions. In FGDs, the Master Trainers reported that they had cascaded their pedagogical training to others in their institutions. The ET found a key challenge in retraining the instructors across GSL partner institutions was getting instructors released from their daily duties to attend the pedagogy training. According to training of trainer and master trainer FGD respondents, some institutions – and VTA in particular – did not have staff to cover classes and other normal operations for instructors to attend the five-day training. KII respondents confirmed that the lack of a centralized system for maintaining, deepening, and upgrading instructor skills remained a gap in the system, and this gap caused challenges for the YouLead team in developing a sustainable approach to improving instructional skills. To address this gap in the short-term, KII respondents indicated that TVEC and YouLead have agreed to a virtual solution, with a plan to digitize the pedagogy training and have instructors complete it online with the support of an external assessor. KII respondents shared that some technical institutions – including DTET, NYSC, and NAITA – took initiative to institutionalize the new pedagogical methodologies by integrating them into existing systems and changing their student learning assessment systems. The new assessment system utilized rubrics that were simpler for the instructors to use and easier for the youth to understand than the prior assessment system. NAITA inspector FGD respondents reported that they also started to adopt market strategies and attitude-changing soft skills in their training modules. The respondents also indicated that DTET, NYSC, and NAITA further adopted the student-based, participatory methodologies that YouLead promoted to teach English language, food security, and baking. According to training of trainer respondents working in other fields, however, few instructors who were teaching subjects such as construction technology, software engineering, or plumbing adopted those methods because the competency-based learning procedures required in the TVEC-approved curricula do not allow flexibility in their teaching. Respondents also described how the mechanisms to verify the implementation of new/revised curricula at the district level were minimal. Respondents reported that what quality control does occur is largely a review of lesson plans submitted, and it does not include teaching observation. PUBLIC AND PRIVATE PARTNERSHIPS In addition to the training provided through the GSL system, YouLead simultaneously developed a broad range of partnerships with employers and strategic partners to strengthen the qualifications of youth and, ultimately, youth employment. YouLead worked with employers not only to provide input on the skills and knowledge needed in various careers but also to develop tailored training for private sector partners could be certified through TVEC. The initiatives with private sector partners are concentrated in the priority sectors that YouLead identified early in the project. For example, YouLead partnered with Sri Lanka Association for Software Services Companies (SLASSCOM) to develop the “Future Careers Bridge” platform, which tested youth ICT skills in one of five tracks: digital transformation, cyber security, customer service, business analysis, or quality assurance. If they successfully completed all the challenges, they received a SLASSCOM certificate of 13 | YOULEAD EVALUATION FINAL REPORT USAID.GOV achievement, which employers in the ICT field recognized. As a result, approximately 875 youth3 who had completed YouLead-supported training in ICT courses earned certificates of achievement and were hired in the sector. YouLead also collaborated with the Young Foundation to build 27 commercial kitchens in VTA or the University of Vocational Technology (UNIVOTEC) vocational training centers around the country, three of which are designed specifically to support learners with disabilities. These were not yet operational at the time of the evaluation data collection. YouLead also developed partnerships to assist youth in formalizing their credentials in a given field. Sri Lanka’s Recognized Prior Learning System created an opportunity for YouLead to work with NAITA and the Public Utilities Commission of Sri Lanka to certify 35,000 electricians around the country with professional qualifications (recognized under the NVQ system) and licenses, which will advance their opportunities for work and increased income. Career Guidance Supports One of YouLead’s major interventions was to develop and provide CG training for selected CG Officers, Human Resource Development Officers, and Skill Development Officers within the technical institutions under the purview of MOSD and TVEC. CG practitioner FGD respondents stated that, prior to YouLead, these institutions used a career interest test that was complex and poorly suited to help youth select courses of study in higher education or find suitable jobs. In 2018, YouLead piloted its first CG training, which the YouLead team refined based on the initial feedback. In 2019, YouLead launched the revised CG training with the Career Key component, a psychometric test to match youth personality and interests and career options and made the training available online in Sinhalese and Tamil. More recently, YouLead supported the localization and digitization of Career Key into the CareerMe platform (careerme.lk). YouLead trained almost 1,000 CG personnel across partner institutions. The training and platform allowed district-level officers to enhance their ability to assess youth interests and strengths and offer appropriate guidance. According to KII respondents, the MOSD Human Resource team and the technical institutions deployed their personnel to provide training to schoolteachers and principals nationwide. In total, they conducted 675 CG training sessions focused particularly on counseling students and using Career Key/Career Me. However, the ET found that implementation varied across the country. Each district had a district career guidance committee (DCGC) with seven sub-committees that represented various public and private sector stakeholders. According to CG FGD and a few KII respondents, due to overlapping roles and responsibilities, across many districts stakeholder engagement with the DCGCs was low, and their activities were minimal. As a result, the DCGCs’ contribution towards the effective implementation of CG practices was insignificant. In the FGDs, CG practitioners reported that they had gained both knowledge and skills through their YouLead training. However, they were not always able to use those skills as intended, as some principals and teachers did not understand or value the potential benefits to their students. The respondents noted that the most critical barrier to their work was their lack of professional qualifications, which meant that their colleagues, students, and parents sometimes failed to allocate time for their sessions. Youth FGD respondents who had participated in CG counseling sessions in Ampara, for example, expressed that the Career Key, counselling, and guidance from the CG practitioners helped inform their selection of technical courses after completing their general education. In Kandy, although Career Me is available to the CG practitioners, they indicated that they are not using it and they prefer Career Key, which some of the other CG practitioners alluded to in FGDs. One key informant suggested that Career Me lacks sustainability unless it is housed within a Ministry. 3 Disaggregation not available. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 14 Respondents also reported that highly vulnerable youth, youth in rural areas, and youth with disabilities did not have equal access to CG and learning opportunities, and it was not clear to what extent the training and materials are tailored for vulnerable youth or youth with disabilities. A related set of interventions, career fairs and the Career Bus program, conducted in partnership with VTA and NAITA, were also very helpful for youth. The Career Bus housed fixed tablets, TVs, Wi-Fi connections, and other communication equipment that youth could use for a period of time to incentivize youth attendance. It travelled from Matara to Jaffna covering nearly 13 districts with YouLead support in the course of a year. The activity intended for VTA to continue the Career Bus program, but postponed it due to an increase in fuel prices and lack of funds. The Career Bus reached youth who, according to key informants, would otherwise not have had CG services. For example, the Career Bus offered mobile CG services to around 1,500 youth in Polonnaruwa and Anuradhapura, approximately 875 of whom then enrolled in technical institutions in those districts. “The Career Bus program conducted with the support of the YouLead helped to reach the majority of the youth at divisional levels and get them to do the Career Key and support them to identify the correct career path. A larger number of youth got enrolled into our program after these initiatives” (Master Trainer FGD respondent). Career fairs also helped introduce youth to a range of careers and employers. For example, the career fair program conducted in Kegalle and Jaffna provided CG services to around 3,000 youth and engaged approximately 150 public and private partners. Youth learned about various careers, career path selection, on the job training, and internships and apprenticeships. However, some key informants (particularly in Kandy) expressed concern about a mismatch between the participating employers and the skill levels of the youth who attended. SKILLS GAINED AMONG YOUTH RESPONDENTS Technical Skills Given the extensive variation in technical skills youth gained via the 47 approved and disseminated YouLead-supported training courses, the ET had no way to assess skill uptake at the individual level. However, 89.3 percent of young men and 91.9 percent of young women survey respondents reported they felt they had the skills and knowledge needed to be competitive in the labor market. In addition, FGD respondents were generally positive about the skills they had gained through their training programs. One exception to this finding was related to youth with disabilities who tried to access training through GSL partner institutions. Accessibility and adaptive teaching methods are limited for youth with disabilities (electronic copy but no paper copies or large print for vision impairments, for example), who reported feeling marginalized in the process. CG practitioners and training of trainers FGD respondents indicated that many youth with disabilities therefore attended private institutions more responsive to their needs. Soft Skills Each of the revised curricula included three modules and competency standards central to fostering soft skills development among trainees: workplace communications and interpersonal relations, occupational literacy and numeracy, and teamwork. Youth respondents indicated that they spent approximately two instructional hours per week on soft skills. While most respondents reported that the soft skills were very useful, a few claimed that the instructors had only very shallow knowledge of soft skills and that they are not well positioned to coach the students. In Kurunegala in particular, where the security forces provided the training, the youth were largely uninterested in the soft skills components. Although YouLead did not collect baseline data assessing the participants’ skills, the youth screener survey included scales for leadership emergence (leadership skills in team settings), self-efficacy, 15 | YOULEAD EVALUATION FINAL REPORT USAID.GOV critical thinking, and perseverance. The figures below show the distribution of scores for each, with the columns representing the number of youth who scored at each interval on the indices. The curved line represents the normalized distribution of scores for young men and young women. As shown in Figure III, Figure IV, Figure V, and Figure VI, the majority of respondents’ answers placed them in the upper half of the scale for leadership emergence, self-efficacy, and perseverance. The only scale where the majority of both young men and young women scored below the mean was on critical thinking. Overall, young men’s mean scores were slightly higher than the young women’s except on self-efficacy, where young women scored slightly higher than the young men. Figure III: Survey Respondents’ Leadership Emergence Scores Figure IV: Survey Respondents Self-Efficacy Scores USAID.GOV YOULEAD EVALUATION FINAL REPORT | 16 Figure V: Survey Respondents Critical Thinking Scores Figure VI: Survey Respondents’ Perseverance Scores As the differences on all scales by gender were minor (and likely not significant given the small sample sizes), the differences by ethnicity were also minor on three of the four. The differences by ethnicity on the leadership emergence scale, however, were much larger than by gender. Tamil, Moor, and multiethnic respondents reported greater frequency of their leadership-related behaviors than their Sinhalese peers did. The key difference in their scores was related to the frequency with which the youth influenced group goals and decisions in their team’s work. One possible explanation is that the sociocultural norms and socialization processes for youth in Sri Lanka’s various communities produce different sets of behavior in group settings. Another explanation might be that the teamwork structure differed across technical training courses, which may account for some of the variance shown above. Based on the composition of the sample and information available to the ET, it was not clear if either of those possibilities were definitive factors in the results. Job Search Skills In addition to technical and soft skills, many youth reported their technical training included a job search skills module. Overall, survey respondents reported the job search skills they learned were highly useful, as shown in Figure VII below. Employment was high among those who reported the training was moderately useful (80.0% of male and 75.0% of female respondents), a little bit useful (80.0%, though it had been 100.0% prior to the economic crisis; all female respondents), and extremely useful (69.2% of male, 70.6% of female, and 100.0% of other respondents), but less so 17 | YOULEAD EVALUATION FINAL REPORT USAID.GOV among those who responded it was very useful (57.1% of male and 50.0% of female respondents). Youth who reported having a disability all found the job search skills to be extremely useful (50.0%) or very useful (50.0%), and all but one had gainful employment (the one who was unemployed had been hired for a new job but had not yet started). Figure VII: Survey Respondents’ Assessment of Utility of Job Search Skills Learned YouLead.lk Portal YouLead developed an online job search and learning management system portal, YouLead.lk, which was available in Sinhalese and in Tamil to support CG practitioners and youth to develop and follow a career trajectory. Although CG practitioners and private sector partners valued the concept of a job search and skills development portal, they found YouLead’s timelines to make changes to the portal in response to feedback (for example, resolution of functionality issues) to be very slow. They also noted that its utility is limited by the fact that available jobs are not updated frequently. The ET identified a lack of recognition among youth and trainers of the YouLead brand. As YouLead worked through its partners to develop and deliver curricula, YouLead generated very little name recognition for its work. While that approach had little negative impact on other areas of YouLead’s work, it was detrimental in gaining recognition for the YouLead.lk portal among youth. Figure VIII below shows that fully two-thirds of survey respondents were unaware of the portal and the resources contained therein. 0.0% 0.0% 19.2% 26.9% 50.0% 3.8% 0.0% 13.5% 10.8% 21.6% 43.2% 10.8% 0% 10% 20% 30% 40% 50% 60% Not at all useful A little bit useful Moderately useful Very useful Extremely useful Don’t know Women Men USAID.GOV YOULEAD EVALUATION FINAL REPORT | 18 Figure VIII: Survey Respondents’ Perceptions of Utility of YouLead.Ik Portal and Online Resources CONCLUSIONS Overall, the ET concluded that YouLead contributed to youth employability through the alignment of skills to employer needs in its development of curricula and training of trainers, NAITA inspectors, and CG practitioners. However, unaddressed TVET system barriers limited the activity’s impact to date, including limiting access to youth in rural and remote areas and youth with disabilities. YouLead supported curriculum revision and development processes that were effective in aligning content to employer needs. The weaknesses in the GSL TVET quality control systems, however, reduced the effectiveness, reach, and degree of institutionalization of these interventions. In particular, the lack of verifications of the usage of new curricula, whether instructors received appropriate orientation or training when TVEC released new curricula, and quality assurance only involved the review of lessons plans. YouLead also developed pedagogical training that improved the instructional techniques of the trainers who were able to apply them in their classroom settings. However, the gap in responsibility and accountability for continuing education of instructors meant that it was difficult to embed this new training into current systems. It was not yet clear if the online option with external assessor would be successful, as the ET found that the primary barrier to access training involved obtaining approval from their organizations because they lacked coverage while instructors attended training. The modality of training did not necessarily decrease the time away from other responsibilities. Further, the inflexibility of some existing curricula (particularly in highly technical fields) prevented trained instructors from applying the student-centered pedagogical practices they learned through YouLead training, which might have otherwise improved the students’ experiences and outcomes. CG practitioners used YouLead CG training and the Career Key as anticipated. CG efforts that were not embedded in the TVET system or schools – Career Bus and career fairs – found success in assisting youth to enroll in courses that Career Key/Career Me indicated would be a good fit for their interests and strengths. Within the schools and TVET system, however, the gaps in knowledge and understanding among stakeholders enabled them to dismiss the need for CG and decreased the effectiveness of the CG practitioners in some areas. YouLead interventions to support employability seemed to be effective for young men and young women, although young women continued to face higher barriers to pursuing training (and finding employment) than young men. Youth with disabilities and youth in rural and remote areas did not appear to attain the same benefits as their urban and peri-urban peers and peers without disabilities. 10% 6% 4% 14% 66% Good Very Good Excellent Did not experience Do not know 19 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Finally, the YouLead.lk portal did not contribute substantially to bridging the gaps between youth and employers, as youth neither recognized nor used the portal. The extensive effort needed on YouLead’s part to keep the portal exacerbates its lack of awareness. EVALUATION QUESTION 1B To what extent did YouLead increase entrepreneurial skills that position youth for realistic, gainful employment? FINDINGS To assess the extent to which YouLead increased entrepreneurial skills, the ET relied on the document review, KIIs, FGDs, and the screener survey data. YouLead utilized multiple strategies to increase entrepreneurial skills among youth, both small-scale and institutional. The institutional approaches encompassed the revision and restructuring of a business startup course that NYSC, NAITA, and VTA offered, and in partnership with the Samurdhi Authority, Sri Lanka’s primary poverty alleviation agency, leveraged its existing service delivery network to reach potential small￾scale entrepreneurs. The national-level trainers from each institute participated in the development of the new curriculum with the support of the YouLead/IESC focal point. Entrepreneurship training included key topics such as budget preparation, proposal preparation, and record keeping using local examples and participatory teaching approaches, but it did not include soft skills modules and competencies incorporated into the technical training courses with YouLead’s support. At the time of the evaluation, the training was available in Sinhalese (aligned to the Samurdhi Authority’s initial roll-out in the Southern Province), with the Tamil version under development. It is not clear, therefore, if the entrepreneurship training would have yielded similar results elsewhere in the country with different economic opportunities and socio-cultural norms. The partnership with the Samurdhi Authority enabled YouLead to reach a wide population of young people who otherwise might not have had the resources attend the technical training other GSL institutions offered. However, the partnership came with challenges. The Samurdhi Authority did not limit enrollment in its programs exclusively to youth, and only 17.2 percent of initial YouLead￾Samurdhi entrepreneurship trainees fell within the age range of 16-35 (340 of 1,979). Among FGD and survey respondents, it was also notable that the large majority of those trained already had a business of some sort, so there appeared to be little new self-employment generated. Through private sector partnerships, YouLead included boot camps for women entrepreneurs. For example, in 2020 YouLead partnered with Hutch Company to facilitate a series of five boot camps. Each camp had 60 women entrepreneur participants, with content focused on efficient accounting, production strategy (especially product placement), and personal development. With the support of the Board of Investment and Export Development Board (EDB), the boot camps also incorporated in-depth mentoring and coaching on business development, market linkages, market strategy, online marketing, and export marketing. These boards are the GSL’s agencies for attracting foreign direct investment and strengthening export opportunities for Sri Lankan businesses, respectively. ENTREPRENEURIAL SKILLS To youth YouLead offered entrepreneurial skills, including soft skills, specific technical knowledge, and skills related to entrepreneurship. Trainers and CG practitioners opined that the YouLead approach to entrepreneurship development was comprehensive, and it was the best model that Sri Lankan technical institutions use. In FGDs, most of the youth who participated in entrepreneurship training responded very positively about the training and the utility of the topics. Youth FGD participants reported that the budget, business proposal, and accounting modules were particularly useful to them. However, a small minority of participants shared that the training was too technical, USAID.GOV YOULEAD EVALUATION FINAL REPORT | 20 particularly around business proposal development, and that they did not really use YouLead acquired skills. A few also claimed instructors had little personal knowledge about entrepreneurship. The Samurdhi-YouLead partnership also offered the business startup training both as an in-person and a virtual course, depending on timing of the course.4 The online iteration is largely structured as independent learning with daily assignments which is submitted to the instructor. Youth and CG practitioner FGD participants noted that this format was challenging for youth who lacked a background in the basic concepts that the course covered, as there were few opportunities to engage the instructor with questions or discuss with peers. ENTREPRENEURSHIP OUTCOMES Overall, 58.1 percent of young men and 51.1 percent of young women reported they had started or continued a business since completing their YouLead training. This represents 81.8 percent of the entrepreneurship trainees and 44.4 percent of technical trainees surveyed. Most of those who started businesses reported that they were able to generate some income, as shown in Figure IX. Few, however (11.8 percent of young women and 12.5 percent of young men) indicated that the income they currently generate is sufficient to meet all their needs, as shown in Table VI . Despite how that their earnings did not meet all of their needs, most businesses did not suffer significant changes as a result of the economic crisis, as only two respondents (one woman and one man) noted a decrease in their ability to meet their needs over the past several months. Figure IX: Entrepreneurs Whose Businesses Successfully Generated Income by Gender Table VI: Extent to Which Current Business Income Meets Survey Respondents’ Needs by Gender Gender of Respondent None of my needs Few of my needs Some of my needs Most of my needs All of my needs Young Men* 0.0% 18.8% 43.8% 25.0% 12.5% Young Women 5.9% 29.4% 29.4% 23.5% 11.8% * Percentages total 100.1 due to rounding. Based on the survey data, the income stability appeared to reflect the hours youth reported they worked as opposed to the potential for income generation. All but one of those who reported that their business income met all of their needs also reported working more than 45 hours per week. Similarly, the majority of those who reported their income met most of their needs also worked more than 37 hours per week. A significant percentage of youth reported working more than 37 hours yet did not earn enough to meet their needs, as shown in Table VII. 4 Many institutions offered online courses during the COVID-19 pandemic, which necessitated the adaptation of materials developed for classroom-based learning. 88.9% 88.9% 95.7% 77.3% 0% 20% 40% 60% 80% 100% Business generated income in past four months Business currently generates income Male Female 21 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Table VII: Extent to Which Current Business Income Meets Survey Respondents’ Needs by Hours Worked Weekly Hours Worked None of my needs Few of my needs Some of my needs Most of my needs All of my needs 37-45 hours (n=10) 0.0% 20.0% 60.0% 20.0% 0.0% More than 45 hours (n=15) 0.0% 13.3% 26.6% 33.3% 26.7% The survey results appeared to indicate that YouLead had success in positioning entrepreneurship trainees and a substantial number of technical trainees for entrepreneurship with the skills gained in their course work. However, these findings differed somewhat from the tone of the youth FGDs, where youth entrepreneurs indicated that they were struggling. Financial institution officers indicated many small businesses had closed due to improper financial management either resulting from or coinciding with the COVID-19 pandemic and the economic crisis. They also reported loan approvals are concentrated in automotive service and beauty industries (not those YouLead prioritized). However, other stakeholders found creative ways to support youth. As one example, the Jaffna Chamber of Commerce amended its constitution to allow youth to join the chamber at no cost for two years, which can advance their networks and careers. Many youth FGD participants noted their disappointment that their training did not include follow￾up support after they completed course requirements. CG practitioners and trainers also observed that in light of various roles different organizations play in supporting new businesses, coordination among those different stakeholders was not evident at the district level, particularly among the Chamber of Commerce, Samurdhi Authority, business startup units at the district, divisional secretariates, and financial institutions. This lack of coordination hindered the ability of youth to get through the business incubation and initial growth period. Beyond limited income and lack of follow-on support, youth FGD respondents identified three more common challenges: supply chain issues, price increases for inputs, and a lack of market linkages. Other issues that arose in KIIs included a lack of business registration (which limits their markets to their home region) and a lack of understanding of packaging, marketing, and value addition. DEVELOPMENT OF SOCIAL NETWORKS In line with the literature, another important factor in the success of entrepreneurs (and for youth more generally) was the development of networks of people outside their families on whom they can rely for advice and support as important factors for youth success (Mason and Brown 2014; Silbereisen and Lerner 2007). These networks represented social capital that could translate into new opportunities and generally worked to reinforce the development of self-efficacy and other soft skills. While not an explicit strategy under YouLead beyond ensuring that CG practitioners were available in schools and technical institutions, other youth skills development programs tacitly achieved this result through placements in work-based learning and other apprenticeship type opportunities or facilitated market linkages for entrepreneurs. Although some youth reported several unrelated adults from whom they said they could seek advice, more than half of young men and almost half of young women indicated two or fewer adults, as shown in Figure X. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 22 The ability to rely on unrelated adults differed substantially by ethnicity, however, as shown in Figure XI. Sinhalese youth all had at least one adult other than family on whom they could rely for career advice, while about half of the Tamil and Moor respondents reported that they had no one outside of their families. It was not clear why there were differences by ethnicity, nor if youth had no or discounted experience with CG practitioners, but the youth did not seem to consider them as people from whom they could seek career advice. CONCLUSIONS YouLead’s entrepreneurship curriculum and delivery in partnership with the Samurdhi Authority improved the entrepreneurial skills of most of the youth who completed the course, the majority of whom were women (76.5 percent of all youth trainees). It was not clear if the training was equally effective for youth with disabilities, as no respondents in the evaluation sample reported having any type of impairment. However, according to FGD respondents, some students lacked familiarity with the basic concepts and were unable to apply the content to their own businesses. In addition, young 21.7% 17.4% 13.0% 8.7% 8.7% 8.7% 21.7% 29.4% 2.9% 14.7% 8.8% 8.8% 5.9% 29.4% 0% 5% 10% 15% 20% 25% 30% 35% 0 adults 1 adult 2 adults 3 adults 4 adults 5 adults More than 5 adults Men Women 18.5% 25.9% 11.1% 14.8% 7.4% 22.2% 47.8% 7.1% 3.6% 3.6% 28.6% 50.0% 16.7% 16.7% 16.7% 100.0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 adults 1 adult 2 adults 3 adults 4 adults 5 adults More than 5 adults Sinhalese Tamil Moor Multiethnic Figure X: Number of Unrelated Adults from Whom Survey Respondents Could Seek Career Advice by Gender Figure XI Number of Unrelated Adults from Whom Survey Respondents Could Seek Career Advice by Ethnicity 23 | YOULEAD EVALUATION FINAL REPORT USAID.GOV entrepreneurs who participated in FGDs stated that they would have benefitted from a soft skills module analogous to those included in the technical curricula that YouLead revised. YouLead entrepreneurship training contributed little to self-employment among youth, as most of its participants were entrepreneurs prior to training, but many of the technical trainees also started their own businesses. For example, among the respondents with disabilities, two who completed technical training (one female and one male - cooking and construction, respectively) operated their own businesses. YouLead had little influence on the operating environment, and as such the difficulties youth faced in starting up their businesses. While participants reported some openings for youth in terms of financing and gratis membership in the Jaffna Chamber of Commerce, for example, the supply-side orientation of YouLead had little impact on the fragmented business development services environment that youth entrepreneurs must navigate. EVALUATION QUESTION 1C To what extent did YouLead improve women’s participation in the economy? FINDINGS By design, YouLead prioritized system upgrades to improve youth employability and young women’s participation in the economy through: better aligning technical courses with employer needs, CG services, improved entrepreneurship training content, or improved training techniques among the cadre of YouLead trained professionals. Overall, key informants and FGD participants indicated that women’s employment remained lower than men’s employment – irrespective of training course, location, or ethno-religious affiliation. Several CG practitioners noted in their FGDs that their peers continued to recommend traditionally female-dominated sectors to young women. Among entrepreneurship trainees, most young women FGD participants reported that they already engaged in self-employment prior to their training and continued their business after the training. Their observations aligned with the survey responses from youth (both entrepreneurs and technical trainees). As shown in Figure XII, most youth who completed their training subsequently engaged in some form of remunerated work. Overall, 78.8 percent of young men and 63.3 percent of young women reported working in the last three months, and of those, most were working up through the beginning of the current economic crisis (75.2 percent of young men and 61.2 percent of young women had worked in the last three months). Given the timing of the data collection, the economic crisis led to a drop in the employment rate, as 57.6 percent of young men and 53.1 percent of young women reported working at the time. FGD respondents who reported losing their jobs or whose businesses closed were unsure of when or if they would be able to return to work. While this small sample was not representative of the larger population of training graduates, the data suggested that despite these losses, youth who completed YouLead-supported training were more economically active than their peers, as the overall Sri Lanka youth labor force participation rate was 27 percent (35 percent for male and 18 percent for female youth) and the overall labor force participation rate was 57.5 percent (68 percent for men and 38 percent for women) (World Bank 2022). USAID.GOV YOULEAD EVALUATION FINAL REPORT | 24 Figure XII: Work Experience among Survey Respondents Youth trainees and entrepreneurs worked across a range of sectors, as shown in Figure XIII, with significant concentrations in ICT and construction, two of YouLead’s high priority sectors. A few young women reported working in the hospitality industry (food and beverage). Only one respondent worked in a field related to caregiving (pharmacy), another YouLead high priority sector. Figure XIII: Sector of Work among Survey Respondents Young men and women reported somewhat different patterns of days worked per week, as shown in Figure XIV below. Fewer women worked six days per week and more worked two, three, or four days per week than their male peers. Some of the differences between young men’s and young women’s work patterns may have been related to the sectors in which they worked, but FGD participants suggested that other impediments were contributing factors to those differences. 78.8% 75.2% 57.6% 63.3% 61.2% 53.1% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% Worked at all after training Worked in last three months Currently working Male Female 28.6% 23.8% 4.8% 4.8% 9.5% 4.8% 9.5% 9.5% 4.8% 6.9% 41.4% 10.3% 3.4% 3.4% 10.3% 3.4% 6.9% 3.4% 10.3% 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% Men Women 25 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Figure XIV: Days Worked per Week by Survey Respondents IMPEDIMENTS TO WORK A key question that arose from the above survey responses on work experience is why 21 percent of young men and 37 percent of young women had not worked at all after their training, and why an additional 21 percent of young men and 26 percent of young women who reported working had a limited schedule. Young men and women identified impediments to their ability to work, as shown in Figure XV. For 40.8 percent of young women, personal and family responsibilities prevented them from working or working as much as they would have otherwise. Other significant factors for young women included their own illness or disability (22.4 percent), lack of suitable work (18.4 percent), and maternity leave (14.3 percent). While the issues were largely the same for young men, more young men reported lack of suitable work as an impediment to working (36.4 percent, almost double the rate for their female peers), followed by personal or family responsibilities (30.3 percent), lack of transportation (21.2 percent), and illness or own disability (15.2 percent). 16.7% 0.0% 4.2% 4.2% 37.5% 37.5% Men 1-9 hours/week 10-18 hours/week 19-27 hours/week 28-36 hours/week 37-45 hours/week More than 45 hours/week 11.4% 8.6% 5.7% 11.4% 34.3% 28.6% Women 1-9 hours/week 10-18 hours/week 19-27 hours/week 28-36 hours/week 37-45 hours/week More than 45 hours/week USAID.GOV YOULEAD EVALUATION FINAL REPORT | 26 Figure XV: Impediments to Work Reported by Survey Respondents As noted above, the small sample was not representative, however, it appeared that many of these youth live in households with other means of support beyond their own labor since personal and family responsibilities is cited a primary impediment to working for many. The lack of suitable work indicated that youth were unable to find work for which they were trained, which was true of many workers during COVID-19 pandemic-related shutdowns in 2020 and 2021. This may also speak to a continuing challenge in the alignment of courses and employer needs, but the ET did not identify any specific cases of misalignment as a primary issue. Maternity leave was (and will continue to be) a reason that female youth are out of the workforce for periods of time, and if paternity leave were to become a more common practice, more young men would likely take such time off as well. Finally, as both youth and parent FGD participants indicated that young women were discouraged from seeking work outside their district, young men’s greater challenges with lack of transportation reflected a larger geographic scope in their employment search. CONCLUSION YouLead had no direct impact on women’s participation in the economy, however based on the sample, the ET found that YouLead-supported technical training graduates had higher labor market participation rates than the national average for young women. Despite that, after completing YouLead, the percentage of young women who never actually worked was double to that of young men. The YouLead priority sector in which the plurality of young women (along with a substantial proportion of young men) reported working was in information and communication technologies (ICT). Fewer young women were employed in other sectors, and outside of ICT, employment patterns typically followed traditional gender norms. 15.2% 3.0% 30.3% 9.1% 6.1% 9.1% 21.2% 36.4% 6.1% 22.4% 14.3% 40.8% 2.0% 4.1% 12.2% 6.1% 18.4% 4.1% 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% Men Women 27 | YOULEAD EVALUATION FINAL REPORT USAID.GOV EVALUATION QUESTION 1D To what extent did YouLead dispel negative stereotypes that limit young men’s and young women’s career choices? FINDINGS YouLead utilized multiple strategies to begin to shift perceptions of gendered career choices. In the CG training, practitioners learned about the Career Key and Career Me assessment tools and how those help youth explore career options matching their personalities and interests. CG practitioners across the country reported that they had used the Career Key tool to good effect. In addition, YouLead produced a series of social media materials to break down labor market stereotypes, and one strategic partner reported that those materials were effective when paired with other gender￾focused interventions. In addition to the social media materials compiled around gender stereotypes, YouLead developed a more limited set of materials on the benefits of entrepreneurship due to parental perceptions of entrepreneurship as a risky option. CHANGES IN PATTERNS OF YOUTH CHOICES AND BEHAVIOR In focus groups, trainers, CG practitioners, and young women all indicated that in recent years more young women enrolled in YouLead-supported courses for traditionally male-dominated fields, although they could not link the increase to YouLead activities. In the Northwestern Province and Colombo, one key informant in the construction sector indicated that women employment was gradually increasing, with women now representing 20 percent of his company’s workforce (compared to 5 percent in 2018). Also, 39 percent of the company’s 2022 supervisor training course participants were women. However, youth and master trainers in Kurunegala reported that young women remained largely in traditional roles, such as secretarial work, beauty, tailoring, childcare, and food preparation, but they also saw some other emerging roles like screen printing, graphic design, and landscaping. Respondents also observed that a high concentration of young men in Kurunegala remained in automobile mechanic courses, but that they also sought migration opportunities to Japan or South Korea. In Hambantota, both male and female training of trainer and NAITA inspector FGD participants mentioned an increase in women’s enrollment in the automotive courses, hotel and tourism courses, ICT courses, and solar panel technician courses, noting that employers in those sectors were willing to recruit them. In Matara, youth FGD participants similarly reported increasing women’s enrollment in masonry courses, automotive courses, plumbing courses, and electronics courses. Young women entrepreneurs in Matara also reported a similar experience. Many of the female entrepreneurship graduates reported that they owned businesses that were not traditionally viewed as women’s work, such as operating a fish stall. They shared that they encountered no objections from their families, although some found that others in their communities vocalized their objections to women working in non-traditional fields, but those opinions had no effect on the participants. While participants in the western and southernmost sites remarked on the increasing opportunities for women in male-dominated sectors, participants’ experiences elsewhere differed. In Nuwara Eliya, for example, FGD participants noted that women’s enrollment in some courses increased, but few women completed the courses because of societal pressures. Also, employers’ lack of willingness to hire them limited what they perceived they could achieve with the training in the long run. This was also true in the plantation sector and in the central regions of the country. One key informant from the public sector reported that at least 20 percent of trainees in the plantation sector are women, but they discontinue their courses when on-the-job training begins. Parents in Nuwara Eliya agreed that government jobs offered youth greater security and prestige and discouraged entrepreneurship among youth. One key informant from the private sector in Colombo thought that many families USAID.GOV YOULEAD EVALUATION FINAL REPORT | 28 working in the plantation sectors largely share these views, saying that families feel being an entrepreneur was “a waste of time.” In Jaffna, parents indicated that families discouraged young women from going into caregiving work and encouraged secretarial or office work, even if it meant a lower-skilled and lower-paid option. In Anuradhapura, parents reported that young women, particularly those in rural areas, were self￾employed in careers such as tailoring, food manufacture, coir product manufacture, carpet making, or compost fertilizer production. Despite the concern in some areas of entrepreneurship risks, these parents indicated that expanding such small-scale businesses would be a welcome development with additional financial support, market access, and technical skills. In comparison, those same parents also noted that families encouraged young men, particularly those from lower socio￾economic backgrounds, to join the security forces to ensure they have a stable livelihood, some degree of respect in the community, and a pension. The stereotypes limiting youths’ choices were also not strictly reflective of traditional gender norms. Parents and youth in Jaffna reported families were not comfortable with youth and young women seeking work outside their district due to differences in language and culture, which limited options to the careers and sectors that are prevalent in that area. One exception to this sentiment was that families acknowledged how good job opportunities might be found in the EU if the youth had resources to migrate, but relocating to other areas of Sri Lanka was largely discouraged. A key informant in Kurunegala noted that parents there have similar concerns about youth seeking jobs outside the district. Their concerns were primarily around security, safety, language, and culture. Respondents in Ampara also concurred with these concerns, but they noted that families – and particularly Muslim families – recognized opportunities for youth to work in the Middle East. SOURCES OF INFLUENCE ON CAREER CHOICES Youth, CG practitioners, and trainers indicated that youth generally selected their technical or vocational courses of study. They said youth considered outcomes in terms of job placement and salary and looked to examples of older youth who pursued the same course of study. According to key informants and FGD participants (youth, parents, CG practitioners, and trainers), parents and spouses of youth participants, if applicable, exerted a great deal of influence on the career choices. One program manager reported that fathers and elder siblings accompany young women in CG discussions, which curtailed their ability to speak freely with the practitioner. In Jaffna in particular, youth FGD respondents indicated it was the father and father’s family that weighed most heavily on a youth’s choices, and medicine, engineering, and government were the most highly valued sectors. NAITA respondents felt differently; teachers had a large role to play in the selection of courses of study and careers among their students. One key informant indicated that youth in urban areas typically had somewhat more latitude than youth in rural areas, as they had greater access to information and exposed to more options in their immediate environments. VIEWS ON GENDER EQUALITY The qualitative data patterns align with views youth shared in their survey responses to questions around whether men and women should be equal in society in various ways (political representation and opportunities, labor market opportunities and work, unpaid care work responsibilities in the household, and social standing). Overall, the vast majority of youth, both male and female, indicated that men and women should be equal in the political, work, household, and social spheres. As shown in Figure XVI below, more young women disagreed that men and women should be equal than young men, and they typically “strongly disagreed” whereas most of the young men “disagreed.” 29 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Figure XVI: Survey Respondent’s Views of Whether Men and Women Should Have Equality Clear patterns emerged by ethnicity as well. In total, 8.4 percent of respondents strongly disagreed with all four statements; all but one of those reported their ethnicity as Tamil (five young women and one young man), and the exception reported their ethnicity as Moor. They were also mostly young, with ages ranging from 19 to 22 years old with one older respondent (27). In addition to those who disagreed with the statements about gender equality categorically, four Sinhalese respondents disagreed with individual statements (one regarding politics, one regarding labor market opportunities, and two regarding unpaid care work). CHANGES IN PERCEPTIONS AND BEHAVIORS AMONG PROFESSIONALS TRAINED UNDER YOULEAD In addition to the intentional work on breaking down stereotypes, stakeholders felt YouLead training improved the way that some of the trained professionals perceived and interacted with young women. Financial institution officers in Kurunegala reported that they observed young women to be more stable as entrepreneurs – when they start a business, they tend to stay in the same sector, which improves their likelihood of obtaining financing. They felt that young men tend to start businesses and then shift into something new every two to three years, which makes them less attractive business prospects than the young women.5 A similar shift occurred in Jaffna among Samurdhi personnel who supported young entrepreneurs in developing their business plans and applying for loans. A key informant observed that some of the personnel had poor communication and interaction skills, particularly with young women, prior to their YouLead training. After the training, however, they perceived a better appreciation for the perspectives of the youth, and those same personnel assisted 750 women in the past two years. CONCLUSIONS Gendered employment patterns are shifting in Sri Lanka, but YouLead did not make significant contributions to those changes. The construction industry in the western part of the country and several industries in the southern part of the country appeared to be more open to hiring qualified 5 It should be noted that the pattern of shifting from one business to another may be positive, if it reflects young men taking advantage of opportunities to move up the value chain or transition from a lower-value proposition to a higher-value proposition, but where lenders pay significant attention to duration of operations, those shifts seem to work against entrepreneurs who seek funding from the Sri Lankan banking system. 93.9% 90.9% 90.9% 93.9% 87.5% 89.8% 87.8% 89.8% 6.1% 9.1% 9.1% 6.1% 12.5% 10.2% 12.2% 10.2% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Equal representation and opportunities in politics Equal opportunities in the labor market and freedom to choose their work Equal responsibility for unpaid care work in the household Equal social standing Strongly Agree/Agree - Men Strongly Agree/Agree - Women Strongly Disagree/Disagree - Men Strongly Disagree/Disagree - Women USAID.GOV YOULEAD EVALUATION FINAL REPORT | 30 workers regardless of gender. The technical trainings YouLead revised or developed seemed to have been effective in providing young women with the skills that employers need (according to trainers and CG practitioners), but many did not complete their courses – particularly in the central, northern, and eastern parts of the country. This was because they found employment for themselves or learned from others that employers were not interested in hiring them in male-dominated fields. EVALUATION QUESTION 2 What are the strengths and weaknesses of in-person and virtual approaches in terms of effectiveness, efficiency, and inclusiveness and what are the best ways forward? FINDINGS YouLead designed training for two distinct audiences: TVET and financial system professionals and youth. Their needs and experiences differed significantly, and a discussion regarding each of those audiences follows below. PROFESSIONAL TRAINEES YouLead partner institutions identified professional trainees to participate in the YouLead trainings, and the ET received no information about the specific processes used to identify them other than KII respondents indicating that they selected the most relevant staff to attend. Only one KII respondent identified an issue with turnover of trained trainers (specifically at VTA) but suggested that it had little to do with YouLead or the trainings specifically. YouLead initially conducted all professional trainings in-person, then shifted its training offerings to virtual formats for the training of trainers, CG practitioners, and selected private sector partners due to COVID-19. Overall, the development of virtual courses enabled each one to be run at a lower unit cost (as YouLead incurred no expenses for travel or transit time). Experiences varied depending on the timing and mode of training, and each of the sections below discusses participant experience with a specific course. Training of Trainers Most training of trainers FGD respondents described the in-person trainings as well organized, efficient, and effective. They shared that the trainers were knowledgeable and that the flow of the training content was logical and easy to follow. The utilization of case studies, brainstorming sessions, and other adult learning methods kept participants engaged throughout the training. Participants across GSL partners (including VTA, NAITA, and Industrial Development Board (IDB)) shared that they used the knowledge they gained through the training. One GSL stakeholder in Colombo, however, critiqued the organization of the trainings for its lack of any formal evaluation (that could be shared with GSL partners). One group in Jaffna also criticized the training, saying that participants were too passive and that the three-day period was insufficient for practical, hands-on sessions related to the CareerMe process in particular. FGD and KII respondents indicated that their virtual trainings of trainers utilized Microsoft Teams. For most participants, this platform worked well. One NAITA respondent remarked that the virtual training had the benefit of exposing participants to new technology, which they could then adopt for their own use. Respondents indicated that the interactivity was somewhat limited using this platform, but it met their expectations. CG Trainings Most respondents found that the in-person trainings met their expectations. Good logistics ensured that the training proceeded smoothly, and the trainers were well prepared. One group in Colombo reported that the five-day format was insufficient to cover all the material. Another source of 31 | YOULEAD EVALUATION FINAL REPORT USAID.GOV frustration was that the language in which the training is offered is not the native tongue for some of the participants, making it harder for them to absorb the content. In Matara YouLead conducted virtual CG training over Zoom. Participants reported they found the training to be effective, learning about TVEC curricula revisions, new trends in vocational training, new tools (including Zoom), and presentation skills. They also felt that the training prepared them to conduct their own virtual trainings. Private Sector Trainings Some tailored private sector training also shifted from an in-person format to a virtual format as a result of the COVID-19 pandemic-associated lockdowns. One private sector representative shared that the original planning for in-person training had been a three-day duration, whereas the virtual training that replaced it required five days. Youth Trainees Youth trainees self-selected (with input from others as described above) into their entrepreneurship or technical skills training courses. Despite some opportunities, many young women continued to remain in traditional sectors. The fact that families in some regions reported feeling uncomfortable with youth (and particularly young women) leaving the district constrained youth employment and education opportunities. The sample of youth participants came from urban and peri-urban areas, but CG practitioners and training of trainer FGD participants identified gaps in the access of rural and remote youth to technical colleges and training centers, as they were too far away to be accessible for some. The ET found limited virtual accessibility, as FGD respondents said that often youth in rural and remote areas lacked Wi-Fi or had limited access to the devices needed for participation. They also cited the lack of accessibility for youth with disabilities – both physical accessibility of the training centers and a lack of consideration to the use of assistive devices. The youth with disabilities who participated in the evaluation did not identify these as significant issues, but their number was quite small, and most reported vision impairments, as discussed above. It was not clear whether youth with other kinds of impairments or more severe impairments would have been able to access the trainings. Most youth who participated in the FGDs and responded to the survey participated in both in￾person and online training. Around the country, those who participated in-person generally had positive remarks about their experiences. Youth in Jaffna described their in-person classes as being PowerPoint lecture-based, with some instructors using group assessments or question and answer sessions to liven up their lessons followed by practical training. Youth in Kurunegala described the same experience but noted specifically that they experienced no student-centered teaching methods because the instructors had a specific time slot for each unit they had to cover before the students’ work-based learning session. In contrast, youth in Hambantota noted enthusiasm about the new Front Office course offered at the technical college, saying it was really engaging, and they enjoyed it. Youth in Kurunegala who participated in virtual training described their learning process as two-hour class blocks three times per day. Each two-hour block included one hour of theory and one hour of practical classwork as a group through the Google Classroom online platform, but there was little opportunity for the youth to engage directly with the instructor. Each day, the instructor uploaded assignments, which had to be submitted the following day. The respondents explained that it was a stressful process because the instructors were very strict on their timelines, and the materials were sometimes hard to understand without additional support from the instructor. One youth FGD in Matara comprised of those who only participated in the virtual training expressed frustration with their experience, as they felt the trainer was insufficiently prepared to teach an entrepreneurship course. Other Matara youth who attended the in-person entrepreneurship training USAID.GOV YOULEAD EVALUATION FINAL REPORT | 32 also said that they were frustrated, as they expected the Samurdhi-YouLead training to encompass follow-on support after the training. This was not universally true, however, as one youth KII respondent said the training was highly effective in her case. She used the knowledge she gained to develop a business proposal, applied for a loan, and received the requested financial assistance. Evidently, the information around the expected benefits and limitations of the courses were not clearly communicated to youth in advance. The sentiments youth expressed about their training experiences aligned with the survey respondents’ feedback as well. As shown in Figure XVII below, few youth viewed the virtual training experience as poor, but most deemed the in-person training experience as substantially better. Figure XVII: Survey Respondents Assessment of In-Person versus Virtual Instructor In summary, both modalities of training functioned, particularly for professional training, but the in￾person trainings produced better learning outcomes for some people and stronger connections to peers and instructors/mentors, as shown in Table VIII. Table VIII: Summary of Strengths and Weaknesses of Virtual and In-Person Training Format In-Person Training Virtual Training Strengths -More active participation and engagement -Opportunities to network with peers -Opportunities for hands-on, practical learning (particularly for career guidance) -Continuity despite lockdown restrictions -Lower level of effort to deliver trainings -Effective refresher training platform -Opportunities to learn and adopt new tools for future iterations -Greater flexibility Weaknesses -Inability to operate during lockdown restrictions -Lack of devices, Wi-Fi access, and compatibility of platforms -Inability to engage in practical, hands-on training -Participant inattention and passive participation -Poor preparation/inexperience with content or delivery mode among some resource persons/trainers -Independent learning courses did not offer sufficient support to learners who were unfamiliar with key concepts 1.2% 6.2% 19.8% 22.2% 46.9% 3.7% 6.3% 10.0% 33.8% 11.3% 18.8% 20.0% 0% 10% 20% 30% 40% 50% Poor Fair Good Very Good Excellent Did not experience/do not know In-person instruction Virtual instruction 33 | YOULEAD EVALUATION FINAL REPORT USAID.GOV CONCLUSIONS Both in-person and virtual training options had strengths and weaknesses to offer YouLead participants. Learning can be effective in both modalities, but respondents had more attentiveness with in-person approaches, and youth in particular found in-person instruction to be more effective. Some of the professional respondents preferred virtual training, particularly for refresher material or material that was largely theoretical for which they had a proper grounding. Where virtual was largely ineffective was in the practice of hands-on skills, which was relevant for most of the technical training but also to a lesser extent for the professional training (role playing in online platforms was difficult, for example). Professionals also valued the opportunities for networking that in-person training offered, which helped them build a network of peers on whom they can rely over time. In terms of efficiency, virtual training led to reduced costs and allowed geographically distributed learners to all work together, which was particularly useful for professional training. However, virtual trainers needed more preparation than in-person trainers, as many had less experience delivering using virtual platforms, and they needed to adjust their regular practice to be effective. In terms of inclusivity, neither modality of training offered a panacea. The ET found that many training facilities lacked accessibility to youth with disabilities, and youth who lived in rural or remote areas were often unable to reach them due to distance (travel time and lack of access to safe, reliable transportation). Some responses noted that virtual instruction also posed challenges due to the lack of devices, Wi-Fi access, and technological literacy required to follow a virtual course. EVALUATION QUESTION 3 What were the key advantages and challenges to working with each of the main YouLead stakeholders in achieving the activity objectives (including cross cutting objectives)? FINDINGS COLLABORATION WITH USAID USAID structured YouLead as a cooperative agreement, which allowed the activity much flexibility in terms of how it achieved its goals. The USAID activity managers were also supportive of YouLead’s management approach, which intentionally sought opportunities to advance activities across its many partners. If for any reason one set of activities experienced a delay or setback, the YouLead team worked with USAID for approval to shift the focus of its activities, such as from GSL partners to private partners or from one sector to another sector. The YouLead team was also diligent in formalizing changes in the activity’s results framework, which they revised three times in the project period with USAID’s concurrence. This flexibility enabled YouLead to advance its goals despite the numerous shocks and disruptions that occurred in Sri Lanka during the life of the activity. NATIONAL-LEVEL GSL COLLABORATION YouLead faced a challenge of working with a complex and fragmented TVET system. Respondents at the Ministry-level acknowledged the gaps and difficulties of working across stakeholders (particularly between vocational education institutions and general/academic education institutions), and they indicated that YouLead was not able to bridge those gaps (and they had not expected the activity to do so). Despite these challenges, YouLead’s partnership approach had many positive impacts. Among initial impacts, respondents across GSL agencies noted that YouLead made substantial contributions to staff capacity, particularly Skills Development Officers, Human Resource Officers, and curriculum development professionals. TVEC stakeholders were among the most positive, USAID.GOV YOULEAD EVALUATION FINAL REPORT | 34 noting the important contributions YouLead made to coordination among TVET institutions. YouLead provided opportunities and a format (district-level stakeholder discussion meetings) to foster collaboration between GSL and private sector actors on both skill gaps and opportunities for work-based learning. In addition, because YouLead worked across GSL partners and through its existing systems, it largely achieved institutionalization of new and revised curricula (as discussed under EQ 1) plus new pedagogical training for instructors. However, one of the drawbacks to this approach was the relatively slow process – particularly early on in the activity – of achieving the intended results. As noted earlier, this collaboration removed YouLead from any influence over actual implementation of training beyond the preparation of youth-facing trainers and Master Trainers. Many trainees, particularly the CG practitioners and training of trainer participants, noted that the lack of follow-up in their training systems was an obstacle to sustainability. YouLead also followed a different pattern than some earlier TVET strengthening activities, in which GSL agencies received additional funding to support cascade training to their provincial, zonal, and divisional counterparts. MOSD did not clearly communicate this expectation to all agencies involved. As a result, some agencies had not allocated budgets or time for training their personnel, and some agencies delivered no training beyond what YouLead offered initially. A further result from coordinating across all agencies was the lack of depth of engagement and ownership among most of the individual agencies. Similarly, working through existing structures allowed for YouLead intervention benefits leaking to populations other than youth. For example, the Samurdhi Authority partnership expanded the ability of YouLead’s to reach youth in vulnerable situations, but as Samurdhi also serves older Sri Lankans, YouLead had no ability to limit the resources to youth participants. These challenges ultimately stemmed from YouLead’s design and intention to improve functioning of existing GSL institutions and systems. Not all challenges resulted from YouLead’s design, however. Some agencies (notably Divisional Secretariats, DTET, and NAITA) were critical of YouLead because as a result of the Easter attack and later COVID-19 restrictions, some trainings were never completed, nor did YouLead make an effort to allow the trainees to finish. The agencies that participated in those trainings found the waste of staff time very frustrating, and it compounded their frustrations related to the long lead time for meeting several of YouLead’s targets. Therefore, although respondents largely assessed their agencies’ relationships with YouLead as positive or helpful, only one respondent reported fully satisfaction with their partnership and its outcomes. Most, including TVEC, DTET, VTA, IDB, felt that the partnerships only partially achieved what they hoped or expected. DISTRICT-LEVEL GSL COLLABORATION The national-level complexity was mirrored in district-level institutions. YouLead’s three regional offices were responsible for coordination with the provincial and district level main GSL partners. Perceptions of district-level collaboration varied nationwide, partly as YouLead was not designed to replicate national-level trainings at the provincial and district levels. Rather, the expectation was GSL agencies would train their own personnel, which did not occur (i.e., with IDB or VTA). YouLead staff spoke highly of district-level partnerships with the District Coordinating Committees, Industrial Committees, and Educational Committees (the latter two on career fairs and guidance for students). There was less agreement on the effectiveness of collaborations with technical colleges, some of which (given their lack of ongoing contact with and feedback from the private sector) did not perceive a need to change teaching practices or update curricula. 35 | YOULEAD EVALUATION FINAL REPORT USAID.GOV Local level GSL partners had differing views of their partnerships with YouLead. GSL partners in Hambantota agreed YouLead was responsive to their needs for changes, particularly with training content. However, regional stakeholders in Kandy perceived that there was insufficient consultation with GSL counterparts around YouLead interventions and a lack of understanding of local context – despite extensive collaboration at the national level. That may have been a reflection of the distance and existing communication gaps between the center and the periphery in the TVET system, but it hindered YouLead’s ability to work effectively with district-level stakeholders in that area. Across sites at the district level, it appeared that YouLead trainees across organizations had little interest or organizational support to collaborate with each other. For example, one respondent indicated that Skills Development Officers and Human Development Officers did not work together at all. Although YouLead’s impacts would be stronger if the various partners collaborated effectively at the district level, YouLead had no method for incentivizing that cooperation. PRIVATE SECTOR COLLABORATION In parallel with its GSL partners, YouLead collaborated directly with Chambers of Commerce and individual firms. This created some tension in how partners perceived YouLead – some expected YouLead to behave as a donor and others expected YouLead to act as a direct service provider. It performed as both, depending on the partner, but it also may have contributed to misperceptions about the extent of its activities in both directions. Chambers of Commerce representatives praised YouLead’s initial outreach and communication, citing the development of memoranda of understanding that outlined their shared goals and the expected terms of their cooperation. They said in that stage, communication was clear and frequent. However, that changed over time. None of the Chamber of Commerce representatives felt the YouLead activity lived up to its potential. They indicated that it was not a true partnership, and that communication and collaboration rapidly dropped off after the kickoff. For example, in Hambantota, the Chamber of Commerce intended to train and provide startup funding to young entrepreneurs. It had expected YouLead to partner in that effort, but given YouLead’s approach to working through GSL institutions, it was not able to meet the Chamber’s expectations. The Chambers of Commerce did not have a clear understanding of how YouLead was designed to operate, and that gap in communication led them to have unrealistic expectations followed by disappointment in and disillusionment with the activity. As one respondent noted, “The only positive thing with YouLead is that the Chamber got rent from them - nothing else positive.” The ET found the Chambers of Commerce provided instrumental support in connecting YouLead with various private sector partners. In its first several years YouLead engaged with many firms, whose perspectives varied as to the value of the partnerships. Drawing on curricula GSL partners developed and approved, YouLead tailored training of trainer courses for individual firms’ internal training systems. Most private sector partners that conducted YouLead-supported training reported a good experience, commending YouLead’s logistical support, level of preparation of resource personnel, and the effectiveness and participatory nature of the trainings. Only one private sector respondent indicated that YouLead’s training of trainers was too basic and did not meet the organization’s expectations, despite being tailored for them. In some cases, YouLead worked with firms to develop new trainings that YouLead shepherded the firms through the GSL certification process, and those partners shared that the YouLead team performed exceptionally well in helping them navigate the GSL processes for certification. YouLead partnered with some private sector organizations that also delivered their own sets of services to youth, including training of trainers and providing materials. These partnerships largely succeeded as the partner organizations met expectations around the training, and they reached tens USAID.GOV YOULEAD EVALUATION FINAL REPORT | 36 of thousands of youth who previously did not access GSL TVET systems for one reason or another. Both partners in these models appreciated the support from YouLead, and the only critique they made was on the length of time required for updates to YouLead.lk and their online materials. Two private sector partners reported that after they signed initial memoranda of understanding, COVID-19 pandemic restrictions interfered with their planned activities, and the parties agreed to postpone their plans. In these two cases, however, the respondents had never heard any further from YouLead, and no activities had been reinstated since the restrictions had been lifted. Both respondents felt that YouLead had wasted their resources in setting up relationships and then failing to follow through with the terms of the agreements. CONCLUSIONS YouLead worked with the public sector to ensure scale and institutionalization. However, reliance on public sector institutions revealed some of their weaknesses, including supports around dissemination of update course materials, quality control, a diffuse group of stakeholders, and a lack of clear ownership on the part of some stakeholders. It also meant that YouLead did not direct its resources specifically to youth. Working with large corporations presented similar advantages and challenges. Scale could be achieved in a sense as a sector's largest actors adopt certain standards, pulling the rest of the sector in that direction. Corporate training systems also could represent a form of institutionalization, and they typically bring subject matter expertise to the fore as well. However, YouLead's contributions could then be used for any target group the private sector partner chooses, and some noted no intention to expand their hiring of youth. The flexibility that a cooperative agreement and USAID’s management of this activity offered have been crucial in keeping the project moving forward with both public and private sector partners despite the myriad disruptions that Sri Lanka experienced over the past several years. However, that flexibility also meant that YouLead dropped certain activities or left things unfinished, which damaged its relationships and brand with some stakeholders. The lack of follow-through in the wake of a disruption in particular was most harmful. 37 | YOULEAD EVALUATION FINAL REPORT USAID.GOV RECOMMENDATIONS The following recommendations are organized into those that should be considered for YouLead in its remaining time and those that are for USAID’s longer-term consideration in future programming. RECOMMENDATIONS FOR YOULEAD Given the short time remaining on the YouLead activity, the ET recommends that the IESC-led team focus on the following five recommendations. Unless otherwise noted, the IESC and the ET agreed on the recommendation when reviewed in the recommendations workshop. 1. Focus on quality control within existing systems to ensure high quality delivery of services. YouLead/IESC should work with at least one GSL partner in the time that remains in the activity to incorporate effective ongoing assessment into instructors’ pedagogical practices. In the recommendations workshop, IESC identified TVEC as the partner that would be both most receptive and most strategic. Working with TVEC would serve two purposes, keeping this task focused within the time that remains and setting the standard to which all agencies will operate in the future. 2. Deepen engagement with other organizations among the current partner pool that can extend delivery to those who are currently not served. As YouLead’s institutionalized approach has had little impact on improving access for young people to technical and entrepreneurship training (in rural or remote areas, for youth with disabilities, etc.), YouLead/IESC should be opportunistic in the time that remains to collaborate with other organizations to utilize the materials and approaches YouLead has developed to extend the reach and impact of the program to youth who are not served. YouLead could draw from several models (e.g., the CG steering committee, its collaboration with Save the Children) that might serve as a template for engagement. 3. Explore opportunities to work with private sector partners on promoting inclusive and gender-sensitive HR policies. As YouLead engages its private sector partners and offers its resources to support the development of their training units, YouLead/IESC should work with its partner to incorporate inclusive and gender-sensitive approaches into both the training and the policies of the partner organization. The ET does acknowledge how, in the recommendations workshop, IESC noted that a recent World Bank activity that tried to undertake a similar approach was unsuccessful. 4. Continue to offer virtual and in-person options for professional training but require an in-person training component for hands-on skill work and to support networking opportunities. For professional training of CG practitioners, training of trainers, master trainers, and NAITA inspectors, YouLead/IESC should continue to offer virtual or hybrid training options that contain costs and minimize the amount of time away from other duties. 5. Revisit commitments that have not yet been fulfilled and close out as many of those as is feasible, focusing first on those that show promise to increase youth employment. The many shocks that affected Sri Lanka over the past several years disrupted YouLead’s work plans and required the postponement of some activities. In the remaining time, YouLead/IESC should address the frustration expressed by some private sector and GSL partners. Not all of these activities are likely to be completed, and YouLead should prioritize those that are most likely to increase youth employment, particularly in light of the current economic crisis. 6. Monitor future commitments to ensure that communication continues in the wake of any disruption. Building on the prior recommendation, YouLead/IESC should USAID.GOV YOULEAD EVALUATION FINAL REPORT | 38 ensure that frequent communication supports any new commitments made in the remaining time available. If any new circumstances arise that would derail the commitment, the partners should ensure that all parties agree to the planned resolution and a clear timeline for any follow-up. RECOMMENDATIONS FOR FUTURE PROGRAMMING For upcoming activities, USAID might also consider the following recommendations. 1. Work with partner TVET institutions to develop and implement effective quality control practices at the classroom level. USAID should work with partner TVET institutions to establish criteria or set standards for continuing education of TVET instructors, as well as designate an agency to be responsible for their training and ensure that such training requirements are incorporated into the planning (time and budget allocation) processes within each institution. This should also include incentivizing the uptake of student-centered pedagogical practices. Another important aspect of such an initiative would be to conduct a landscape assessment to determine which institutions lack the equipment and expendable supplies to teach new and revised curricula as they are disseminated. On an ongoing basis, those curricula should come with a self-assessment checklist to be returned to district or national offices to alert agencies immediately if they lack the resources to implement a new curriculum. 2. Consider how to support TVET institutions to improve their accessibility, particularly to youth who are in vulnerable situations. One of the gaps in the YouLead programming was that it did not assist in overcoming the barriers youth encounter when they seek technical or entrepreneurship training. Future activities should explore a range of models that could include small-scale satellite learning centers, development of guidelines for accommodation or utilization of assistive devices for students with disabilities or expanded financial support through scholarship schemes or stipends. 3. Consider how to incorporate a stronger component on promoting inclusive and gender-sensitive human resources policies within private sector partner organizations and employers working with the program. In some regions, the norms that constrain young men and young women’s career choices have begun shifting, but in others, little change has occurred. USAID should look for opportunities to support industry clusters in setting new and inclusive standards in their practices. 4. Consider conducting a study to identify key messages that increase acceptance of nontraditional career choices among parents and families. In preparation for future activities, USAID should commission a study to capture those messages (and interlocutors) that can be influential in supporting greater choices for young men and young women in the workforce. 5. Foster dialogue with the Sri Lanka Foundation and MoSD/TVEC to create a CG diploma course. As practitioners cited the lack of professional qualifications as a barrier, future activities should explore the feasibility and utility of creating a diploma course in CG. 39 | YOULEAD EVALUATION FINAL REPORT USAID.GOV ANNEX I: SCOPE OF WORK SERVICE REQUEST USAID/Sri Lanka YouLead Performance Evaluation (February 2022) A. Purpose The purpose of this evaluation is to evaluate the performance of the YouLead activity on achieving their higher-level results and identify potential follow-on interventions for career development of youth. USAID/Sri Lanka Economic Growth Office, YouLead, and key Government of Sri Lanka (GSL) counterparts of the activity are the primary audiences of this evaluation. They will use the findings and recommendations to design follow-on activities; especially on career development for youth, and potential human capital development interventions and, make course corrections to the remainder of the YouLead Activity. B. Background Youth Employment and Entrepreneurship in Sri Lanka A 2018 Asian Development Bank (ADB)-funded study indicates that youth unemployment is a major economic concern in Sri Lanka due to skills shortages and labor market mismatches. Sri Lanka has one of the highest youth unemployment rates in the Asia-Pacific region and unemployment rates are significantly higher among young women than young men. The same study outlines how queuing for traditional careers, especially public sector jobs, aggravates youth unemployment and the importance of supporting programming to create gainful employment opportunities and increase awareness and communication to dispel gender and social biases for certain types of industries. Sri Lanka historically had low female labor force participation among peers, and it is currently at around 36%. A recent USAID/Sri Lanka gender assessment revealed that disproportionate care responsibilities and traditional expectations about women’s roles discourage their participation in the formal labor force. Given this context, YouLead was designed to improve technical and vocational education and training, create a more skilled workforce, and link young people aged 16-35 to productive job and entrepreneurship opportunities, with a strong emphasis on gender and expediting positive behavior changes. The COVID-19 pandemic is likely to have tightened constraints affecting youth unemployment due to the impact on the economy and prolonged disruptions to youth skills development and vocational training. YouLead Activity YouLead is a seven-year activity funded by USAID, that aims to create better employment including self-employment outcomes for the Sri Lankan youth. The objectives of the activity are as follows: 1. Increase youth employability skills in targeted sectors 2. Improve quality, relevance, and delivery of Technical and Vocational Education and Training (TVET) USAID.GOV YOULEAD EVALUATION FINAL REPORT | 40 Increase prospects of successful self-employment YouLead project supports inclusive economic growth by increasing youth employability. The project does this by strengthening vocational training for students, teachers, institutions, and entrepreneurs; improving workforce capacity to meet market demand; and linking with the private sector to encourage employment opportunities for youth. Increase youth employability skills in targeted sectors: YouLead aims to improve work readiness skills including technical and soft skills required to obtain and maintain jobs in the growth sectors of the economy. This involves a wide array of diverse partnerships. Two notable examples are 1) YouLead-piloted tourism sector soft skills and industry exposure program partnering with hotels. 2) YouLead’s partnership with Sri Lanka Association for Software Services Companies (SLASSCOM) and Open University of Sri Lanka are creating career pathways for non-ICT graduates to enter the growing ICT sector of Sri Lanka, opening ICT career opportunities for youth without conventional ICT related university degrees. Improve quality, relevance, and delivery of Technical and Vocational Education and Training (TVET): The main pillar of YouLead's activities under this objective is revision/development of new curricula for the TVET sector, and pedagogical and technical training of instructors of Sri Lanka's TVET sector. YouLead has thus far revised or developed 66 curricula of which 40 have been endorsed by the relevant approving body of the government of Sri Lanka for classroom use. YouLead has also trained around 850 instructors from both public and private sector in pedagogical and technical training. YouLead’s main partners for this objective are Vocational Training Authority, National Apprentice and Industrial Training Authority (NAITA), Tertiary and Vocational Education Commission (TVEC) and a range of private sector employers. Career guidance is a key component of YouLead’s youth employability skills and vocational training initiatives. The program aims to institutionalize career guidance through supporting to set up professional qualifications for career guidance practitioners. Various initiatives under YouLead have thus far delivered one-on-one career guidance to over 64,000 individuals in Sri Lanka. Increase prospects of successful self-employment: YouLead has implemented various initiatives to strengthen the entrepreneurial culture and to enable youth and other entrepreneurs to achieve more gainful self-employment opportunities. The initiatives include entrepreneurship training for youth and small and medium enterprises (SMEs), developing a common entrepreneurship training module jointly with 11 other public and private sector institutions, providing mentoring services, support for incubators and accelerators, and training of credit officers of banks and financial institutions on cash-flow based credit assessment techniques. YouLead partners with a host of public, private, and civil society stakeholders in pursuit of these objectives. YouLead works with following key public sector skills/vocational training that come under YouLead’s Government of Sri Lanka (GSL) line Ministry – State Ministry of Skills Development, Vocational Education, Research and Innovation: 41 | YOULEAD EVALUATION FINAL REPORT USAID.GOV 1. National Apprentice and Industrial Training Authority (NAITA) 2. Vocational Training Authority (VTA) 3. Tertiary Vocational Education Commission (TVEC) 4. Department of Technical Education and Training (DTET) 5. UNIVOTEC Other public sector institutions that YouLead works with includes (but not limited to) Department of Manpower and Employment (DoME), National Youth Service Corps (NYSC), NYC, Ministry of Education, Samurdhi Authority (GSL's premier anti-poverty agency) and Sri Lanka Foundation Institute. A snapshot of YouLead’s private sector, financial, and civil society partners are outlined below: Type of entity Number of partnerships Private Companies 14 Commercial Banks and Financial Institutes 11 Community Organizations, Social Ventures and Workforce Development programs 7 Non-Governmental Organizations 3 YouLead's activities are informed by two labor market surveys conducted in 2018 and 2020. YouLead initially prioritized 12 districts with highest youth unemployment rates and subsequently increased it to 16. However, YouLead retained the flexibility to operate in any district of Sri Lanka throughout the activity period. YouLead activities are focused on (but not limited to) sectors with high-growth potential i.e., tourism, ICT, construction, retail trade, and caregiving. YouLead places a strong crosscutting emphasis on gender to improve the labor force participation of women in Sri Lanka, which lags behind its regional peers. YouLead is also mandated to promote wider social inclusion including ethno-religious minorities and persons with disabilities. YouLead also engages in outreach and media campaigns to dispel negative social norms affecting career decisions of youth and to encourage them to chart their own path to success without being impeded by long held narrow career paths. YouLead monitors its performance through a range of indicators including indicators monitoring better employment outcomes (new or better employment including self-employment), participation and completion of workforce development program, gender and youth inclusion. The YouLead program has thus far supported 29,242 individuals to find new or better employment (including self￾employment). YouLead collects data using a variety of techniques including surveys, data submissions by partners, and existing government ministry data. COVID-19 has had a notable impact on YouLead activities, mainly due to the closure of training intuitions for prolonged periods and restrictions imposed on public gatherings. As a response YouLead managed to transfer most of its activities to virtual delivery mode and has also run several USAID.GOV YOULEAD EVALUATION FINAL REPORT | 42 online campaigns to encourage youth focus on skills development during downtime caused by COVID related lockdowns. C. Evaluation Questions For this evaluation an initial set of evaluation questions were identified during the Performance Management Plan development exercise. Those were further revised with the inputs from the USAID Sri Lanka Economic Growth office. These evaluation questions can be further refined based on the discussion between the USAID and the evaluation team (ET). The initial set of evaluation questions were identified during the Performance Management Plan development exercise Those were further revised with the inputs from the USAID Sri Lanka Economic Growth office. These evaluation questions can be further refined based on the discussion between the USAID and the evaluation team (ET). 1. To what extent did YouLead achieve the following youth employment outcomes? a. Improving youth employability through skills gained from vocational training, particularly in terms of aligning skills to the emerging needs of the economy. b. Increasing entrepreneurial skills that position youth for realistic, gainful employment. c. Improving women's participation in the economy. d. Dispelling negative stereotypes that limit young men’s and women’s career choices. 2. Compare YouLead's youth skill training delivery approaches (in-person and virtual). a. What are the strengths and weaknesses of each approach in terms of effectiveness, efficiency and inclusiveness and suggest best ways forward. 3. What were the key advantages and challenges to working with each of the main YouLead stakeholders in achieving the activity objectives (including cross cutting objectives). D. Evaluation Methodology The ET will determine a robust evaluation design that is appropriate for this evaluation. It is expected the evaluation will use mixed methods, as recommended by the Evaluation Policy[1]. Whether or not planning to use a mixed method, the ET should provide a clear motivation for the approach they recommend. The evaluation must clearly articulate the link between each evaluation question, the proposed data to address it and the analysis plan for these data. For example, the design may describe the statistics to be used in a quantitative analysis. For qualitative approaches, the design may detail each planned analytical step (eg. coding frame, how it was developed). The evaluation design must demonstrate that the proposed approaches are best practice (based on evaluation and research literature), that they are intended to provide robust answers to each evaluation question, and that they are suitable to the Sri Lankan context. The proposal must be specific and name each data collection method used, the reason to use it and the motivation for data sources for each method. For example, if Focus Group Discussions (FGD) are proposed, the ET must specifically describe what is meant by FGD, what is the motivation for the selection of its participants and why would FGD be more appropriate than a Group Discussion or a Key Informant Interview for a particular data source and a particular evaluation question. The evaluation must also examine differences by gender. If a detailed analysis by gender is not relevant for a question, the ET must explain why. The analysis may require more than simple 43 | YOULEAD EVALUATION FINAL REPORT USAID.GOV disaggregation of quantitative data. For example, analysis of gender dynamics is more than statistics by gender. The ET must refer to relevant USAID guidance on gender and inclusion and propose specific evaluation designs, as appropriate. The ET may propose to design quantitative data collection instruments and qualitative protocols to gather data as appropriate. Though such instruments and protocols may be based on existing tools, they must be adapted and tested so that they address the specific evaluation questions. Instruments must be translated and the language used must be tested to make sure that each question is understood and validated in the YouthLead and local context. The ET must include a detailed plan for relevant tests of such instruments. All quantitative data must be stored digitally and the proposal must demonstrate that proper data quality assurance systems will be put in place. The quantitative data must be properly coded and shared with USAID. All qualitative data must be recorded and transcribed. The qualitative data must be shared with USAID in a summary format (Example: Microsoft Excel qualitative data analysis matrix). This performance evaluation must adhere to relevant quality standards as provided in the USAID Evaluation Policy, as well as international standards for conducting social sciences research, especially the “Do No Harm” principles such as the principles for treatment of human subjects and collection and safeguard of personally identifiable information. The ET is required to follow USAID and/or host-country government protocols for research activity such as Internal Review Board requirements. E. Evaluation Team Composition The team will be composed of experts in conducting evaluations of civil society programs. The Team Leader will have ultimate responsibility for overall team coordination and development of the final report. Including the Team Leader, the ET will likely consist of three members. The team will have the following expertise or qualifications: Team Leader: 1. A masters degree in international development or any related field. 2. At least 10 years of experience in international development, especially with youth employment skill development and youth entrepreneurship development programs. 3. Previous experience performing related project evaluations particularly USAID funded youth employment and/or entrepreneurship programs. 4. Recent experience in and background knowledge of Sri Lanka is highly preferred but such experience and knowledge of South Asia or countries similar to Sri Lanka are required. 5. Fluent in English, particularly in report writing. Two Senior Researchers: 1. Bachelor’s degree in international development or related field. 2. At least 7 years of experience in the international development sector. 3. Experience in working with Sri Lankan youth and/or career development programs. 4. Fluent in English The team must be collectively fluent in Sinhala and Tamil. In addition to the above three core members, the team may consist of logisticians, research USAID.GOV YOULEAD EVALUATION FINAL REPORT | 44 assistants, data analysts/IT assistants. F. Schedule and Logistics The evaluation should take a maximum of 24 weeks to complete. Below are some key considerations when developing the schedule. The deliverables are listed in the next section. 1. Due to COVID pandemic, the ET should assess the situation in the country and make necessary changes to the data collection modes.[2] 2. US-based ET staff to adjust some of their work time to Sri Lankan friendly time (8 AM - 5:30 PM) 3. Time allotted in the U.S. to write the report and make revisions once comments on the draft are received from USAID. 4. Anticipate at least two rounds of revisions to the draft report. Consider local holidays, especially the April New Year and Wesak Poya in May. All logistical support will be provided by the ET, including any travel, transportation, secretarial, interpretation, report printing and communication, as appropriate. USAID and relevant IPs will provide all relevant documents as well as contact names and phone numbers to the ET. G. Deliverables and reporting requirements All deliverables must be submitted to USAID within the timeline indicated in below and by no later than the TOs end date. The ET must promptly notify the TO-Contracting Officer’s Representative (COR) of any problems, delays, or adverse conditions which materially impair the ET’s ability to meet the requirements of the Contract. The expected deliverables are as follows: Kick-off meeting/presentation Prior to the evaluation the ET shall meet with assigned USAID personnel for an in-briefing and planning session. Evaluation design Based on the Proposal from the ET, the final design for the evaluation must be approved by USAID. The required components of the evaluation design are: 1. Summary of desk review findings and references to the relevant literature: The desk review must include the available documents from the YouLead implementing partner, USAID, and other relevant secondary sources and literature. USAID/Sri Lanka will provide you USAID and YouLead documentation to the ET. 2. Details of the approach (see “methodology” above) to answer the evaluation questions including: 1. data collection methods, 2. data sources, selection criteria and sampling plan, 3. a description of data collection instruments, 4. an analysis plan 45 | YOULEAD EVALUATION FINAL REPORT USAID.GOV 3. An Evaluation Design Matrix linking each question to relevant data source, data type and data analysis, 4. A discussion potential risk and limitations of the proposed approach and mitigation strategies Work plan The ET must prepare a detailed work plan which will show how to operationalize the methodologies described in the evaluation design proposal. It must include: 1. Draft schedule and logistical arrangements; 1. Evaluation milestones (including, but not limited to, the evaluation deliverables listed in this SOW) 2. Anticipated schedule of ET data collection efforts; 3. Dates and arrangements for piloting data collection instruments; 2. Members of the ET, delineated by roles and responsibilities; 3. Description of evaluation dissemination products, including the evaluation report, presentation slides, infographics and briefs. 4. Dissemination plan (designed in collaboration with USAID) Data collection instruments and testing protocol The ET must submit a draft of data collection instruments/tools to USAID. The tools must match the Evaluation Design and include questionnaires, forms, and/or guides for data collectors as appropriate. Before data collection begins: 1. Data collection instruments and protocols must meet USAID technical requirements and be approved by USAID 2. Data collection instruments must be tested empirically (pilot interview with interviewer and interviewee debrief afterwards, cognitive testing, or other best practices for instrument testing) 3. Results of data collection tests must be shared with USAID. (USAID will add the modality). 4. The final data collection tools that incorporated piloting results must be approved by USAID prior to actual data collection. 5. Interviewers must be trained in appropriate qualitative methods In-brief presentation A meeting with USAID/Sri Lanka and YouLead before the start of field work to present and validate the evaluation methodology as presented in the evaluation design report. Findings and recommendations presentation to USAID Before beginning to draft the evaluation report (but after the data have been collected and analyzed), the ET must present the findings and recommendations to USAID Sri Lanks. This findings presentation allows the ET to demonstrate the robustness of its data collection and analysis and, given that the evaluation work was robust, to obtain USAID approval to conduct a recommendations workshop and to begin drafting the evaluation report. The ET should use this as another opportunity to collect feedback from the USAID on the findings and recommendations. Findings and recommendations virtual workshop(s) with stakeholders USAID.GOV YOULEAD EVALUATION FINAL REPORT | 46 The ET must organize and deliver a virtual workshop involving key stakeholders to: 1. Present and explain key evaluation findings, 2. Elicit feedback about the findings 3. Generate policy and programmatic recommendations based on the findings. The audience for the findings and recommendation workshop(s) will be determined in collaboration with USAID. Evaluation report outline In accordance with the USAID evaluation report template[3], the ET should share an evaluation report outline with USAID and get it approved. This report outline should help USAID to understand how the ET is going to present the findings, conclusions and recommendations related to evaluation questions. Draft evaluation report The ET must provide a draft of the full report to USAID. The draft must include all the required components (please refer to the Evaluation Report Template) of the final report. USAID and the ET may agree to submit and review the draft in sections if they jointly prefer an iterative process (for example, the Evaluation Purpose and Questions, the Project Background, and the Methods sections might be cleared first, as they should be easier to draft.) Final evaluation report The final report should be no more than 35 pages including Tables and Figures. The report should be written for a non-evaluation/non-M&E as appropriate for the audience. All technical details about the methodology should be included in the annexes. A comment log should list all the comments submitted by USAID on the draft and explain how they were addressed in the final report. Draft dissemination materials The ET must provide a draft of the dissemination materials to USAID. The draft must include all the required components of the final materials and must be properly copy edited. The dissemination material should include extended executive summary and simplified presentation slides as an annex. Final dissemination materials The ET must submit dissemination materials to the evaluation COR in the form of a 5-10-page summary (and an accompanying set of PowerPoint slides) of key findings, conclusions and recommendations. The ET shall submit an electronic version of the report (in Microsoft Word and in PDF format) in English (no translation required). The report shall also be submitted to the Development Experience Clearinghouse according to the ADS. Submission of the raw dataset to the Mission and the Development Data Library is not necessary for unstructured, qualitative data, and notes (redacted or unredacted). [1] https://www.usaid.gov/sites/default/files/documents/1870/USAIDEvaluationPolicy.pdf [2] As of Feb 2022, there is no travel or movement restriction imposed by the government, despite an increasing number of COVID patients. 47 | YOULEAD EVALUATION FINAL REPORT USAID.GOV [3] https://usaidlearninglab.org/library/evaluation-report-template USAID.GOV YOULEAD EVALUATION FINAL REPORT | 48 ANNEX II: DOCUMENTS CITED AND REVIEWED Abayasekara, A. and N. Gunasekara, “Determinants of youth not in education, employment or training: Evidence from Sri Lanka,” Review of Development Economics 23, no. 4 (2019): 1840-1862. Aboobucker, I. “Sustainable Youth Led Action Plans for Sri Lankan Youth Unemployment Issues.” (2019). Available at SSRN: https://ssrn.com/abstract=3344596 or http://dx.doi.org/10.2139/ssrn.3344596. IESC. “Annual Report - Year Four, October 1, 2020, to September 30, 2021.” (Colombo: IESC, 2021). IESC. “Annual Report - Year Three, October 1, 2019, to September 30, 2020.” (Colombo: IESC, 2021). IESC. “Potential Adjustments to Year Four Work Plan in Response to COVID-19 Impacts on In￾Person Program Activities.” (Colombo: IESC, n.d.). IESC. “Quarterly Progress Report – April 1, 2019-June 30, 2019.” (Colombo: IESC, 2019). IESC. “Quarterly Progress Report – April 1, 2020-June 30, 2020.” (Colombo: IESC, 2020). IESC. “Quarterly Progress Report – April 1, 2021-June 30, 2021.” (Colombo: IESC, 2021). IESC. “Quarterly Progress Report – January 1, 2019-March 31, 2019.” (Colombo: IESC, 2019). IESC. “Quarterly Progress Report – January 1, 2020-March 31, 2020.” (Colombo: IESC, 2020). IESC. “Quarterly Progress Report – January 1, 2021-March 31, 2021.” (Colombo: IESC, 2021). IESC. “Quarterly Progress Report – October 1, 2018-December 31, 2018.” (Colombo: IESC, 2019). IESC. “Quarterly Progress Report – October 1, 2019-December 31, 2019.” (Colombo: IESC, 2020). IESC. “Quarterly Progress Report – October 1, 2020-December 31, 2020.” (Colombo: IESC, 2021). IESC. “TVET Student Pvt Employee and CG Delivery.” (Colombo: IESC, n.d.). IESC. “YouLead Behavioural Change Social Media Outreach.” (Colombo: IESC, n.d.). IESC. “YouLead Behavioural Changes and Success Stories.” (Colombo: IESC, n.d.). IESC. “YouLead Current Program Description EU Activities Included April 2021.” (Colombo: IESC, n.d.). IESC. “YouLead Participants Training for Officers.” (Colombo: IESC, n.d.). IESC. “YouLead Program Description Original.” (Colombo: IESC, n.d.). IESC. “YouLead Technical Application for Extension.” (Colombo: IESC, 2020). IESC. “YouLead Work Plan Year 3 October 1, 2019-September 30, 2020.” (Colombo: IESC, 2019). IESC. “YouLead Work Plan Year 4 October 1, 2020-September 30, 2021.” (Colombo: IESC, 2020). 49 | YOULEAD EVALUATION FINAL REPORT USAID.GOV IESC. “YouLead Work Plan Year 5 October 1, 2021-September 30, 2022.” (Colombo: IESC, 2021). ILO and ADB. “Tackling the COVID-19 Youth Employment Crisis in Asia and the Pacific.” (Manila: ILO and ADB, 2020). Jayathilake, H.D. “Impact of unemployed youth on national development in Sri Lanka,” Journal of Emerging Issues in Economics, Finance and Banking 9, no 1 (2020): 2985-3005. Mason, C. and R. Brown. "Entrepreneurial Ecosystems and Growth Oriented Entrepreneurship.” Background paper prepared for the workshop organized by the OECD LEED Program and the Dutch Ministry of Economic Affairs on Entrepreneurial Ecosystems and Growth Oriented Entrepreneurship. (The Hague, Netherlands: OECD, 2014) Perera, M. “Ineffectiveness of Social Contacts and Alternate Job Search Methods for Unemployed Youth in Sri Lanka” [Working Paper]. (Colombo: Verite Research, 2020). Ramasamy, R. “Youth radicalization and violent extremism: a study of pathways in the post-war context of Sri Lanka,” Journal of Youth Studies (2021). DOI: 10.1080/13676261.2021.2010684. Seneviratne, WSA. “Training and Unemployment in Sri Lanka: Impact of Skills Training on Unemployment,” Asian Journal of Advances in Research 11, no. 4 (2021): 274-282. Silbereisen, R.K., and R.M. Lerner. "Approaches to Positive Youth Development: A View of the Issues." In Approaches to Positive Youth Development, edited by R.K. Silbereisen and R.M. Lerner, 3-30. (Thousand Oaks, CA: Sage Publications, 2007). VEGA and IESC. “Annual Progress Report - June 2017-September 2017.” (Colombo: VEGA and IESC, 2017). VEGA and IESC. “Annual Report Year One - October 1, 2017 to September 30, 2018.” (Colombo: VEGA and IESC, 2018). VEGA and IESC. Annual Report – Year Two – October 1, 2018 to September 30, 2019.” (Colombo: VEGA and IESC, 2019). VEGA and IESC. “Quarterly Progress Report – April 1, 2018-June 30, 2018.” (Colombo: VEGA and IESC, 2018). VEGA and IESC. “Quarterly Progress Report – January 1, 2018-March 31, 2018.” (Colombo: VEGA and IESC, 2018). VEGA and IESC. “Quarterly Progress Report – October 1, 2017-December 31, 2017.” (Colombo: IESC, 2018). VEGA and IESC. “YouLead Technical Application [NOFO Response].” (Washington, DC: VEGA and IESC, 2017). VEGA and IESC. “YouLead Work Plan Year 1 June 2, 2017 - September 30, 2018.” (Colombo: VEGA and IESC, 2017). VEGA and IESC. “YouLead Work Plan Year 2 October 1, 2018-September 30, 2019.” (Colombo: VEGA and IESC, 2018). USAID.GOV YOULEAD EVALUATION FINAL REPORT | 50 Verite. “Improving Youth Employment & Entrepreneurship in Sri Lanka: Insights and Strategies.” (Colombo: Verite, 2020). Verite. “Youth Labour Market Assessment.” (Colombo: Verite, 2018). Verite. “Youth Labour Market Assessment (Gender Assessment).” (Colombo: Verite, 2018). Weerasiri, ARP and GRSRC Samaraweera. “Factors influencing youth unemployment in Sri Lanka,” Asian Journal of Management Studies 1, no. 1 (2021): 49-72. Wimalarathna, B.M.R.M., P.H.T. Kumara, and S. Niroshini. “Causes of Unemployment among the Educated Youth in Sri Lanka.” International Research Conference of Uva Wellassa University, Sri Lanka. (2020). World Bank. “Labor Force Participation Rate.” (Washington, DC: World Bank, 2022). Retrieved from https://data.worldbank.org/indicator/SL.TLF.CACT.ZS?locations=LK. 51 | YOULEAD EVALUATION FINAL REPORT USAID.GOV ANNEX III: DATA COLLECTION PROTOCOLS KII GUIDE – USAID STAKEHOLDERS SI, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As a core member of the program management team, we believe you have unique insights into YouLead’s operations, management, and results. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. We do not believe your participation and any disclosures in this interview will have direct benefits or negative consequences for you, and the information you share with us will be treated as confidential. The transcript and notes from this interview will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at an aggregate level (for example, USAID stakeholders). You may choose not to answer any question, and you may choose to end the interview at any time. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If no, thank the respondent and end the interview. If yes, continue with script.] Do you consent to have this interview recorded for accuracy purposes? [Record response] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: ___________________ Background Questions 1. To start, I’d like to hear about your role with the YouLead program. Could you please tell us a bit about when and how you became involved in the program and the role(s) you have played over time? Program Activities 2. Would you please share with us a bit about your perceptions of the various YouLead activities? a. Effectiveness of integration of work readiness skills into existing firm-based private sector technical training (tourism and ICT) b. TVET curriculum revisions c. TVET training of trainers d. TVET career guidance center development e. Joint entrepreneurship curriculum development f. Business development supports (mentoring, cash-flow based assessment training for financial institutions, support for incubators, etc.) 3. For the TVET training of trainers that YouLead offered, what was your perception of the efficacy of the in-person training approach? (EQ2) USAID.GOV YOULEAD EVALUATION FINAL REPORT | 52 a. What, from your perspective was the level of effort to deliver training? b. What were the overall outcomes? c. To what extent did it provide strategies for the inclusion of young women, youth with disability, and youth from ethno-religious minorities? d. What barriers to access the in-person training was the project unable to overcome, if any? e. If outcomes of instruction were tracked, did young women/vulnerable youth have similar outcomes to the young men/non-vulnerable youth? 4. What was your perception of the efficacy of the virtual training approach for TOT? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What modifications were made to deliver the content in a virtual format? c. What were the overall outcomes? d. To what extent did it provide strategies for the inclusion of young women, youth with disability, and ethno-religious minorities? e. What barriers to access the virtual training was the project unable to overcome, if any? f. If outcomes of instruction were tracked, did young women/vulnerable youth have similar outcomes to the young men/non-vulnerable youth? 5. For the entrepreneurship training that YouLead offered, what was your perception of the efficacy of the in-person training approach? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What were the overall outcomes? c. To what extent did it include young women, youth with disability, and youth from ethno-religious minorities? d. What barriers to access the in-person training was the project unable to overcome, if any? e. Did young women/vulnerable youth have similar outcomes to the young men/non￾vulnerable youth? 6. What was your perception of the efficacy of the virtual training approach for entrepreneurship? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What modifications were made to deliver the content in a virtual format? c. What were the overall outcomes? d. To what extent did it include young women, youth with disability, and ethno￾religious minorities? e. What barriers to access the virtual training was the project unable to overcome, if any? f. Did young women/vulnerable youth have similar outcomes to the young men/non￾vulnerable youth? Partnerships Although YouLead directly delivered some training to youth, the team more frequently partnered with local organizations to strengthen their training curricula, teaching practices, and work-based learning experiences. 7. To what extent was USAID involved in facilitating relationships with key government stakeholders, such as NAITA, VTA, TVEC, etc.? (EQ3) 8. How effective were these partnership in achieving the intended goals of the program? 53 | YOULEAD EVALUATION FINAL REPORT USAID.GOV 9. What were the key advantages to working with the key stakeholders this way? (EQ3) [Probe on National Apprentice and Industrial Training Authority (NAITA), Vocational Training Authority (VTA), Tertiary Vocational Education Commission (TVEC), Department of Technical Education and Training (DTET), UNIVOTEC, private sector employers, Sri Lanka Association for Software Services Companies (SLASSCOM) and Open University of Sri Lanka] 10. There are also potential challenges in working through partnerships as well. How would you describe the challenges YouLead faced, if any, in partnering with these key organizations? (EQ3) [Probe on National Apprentice and Industrial Training Authority (NAITA), Vocational Training Authority (VTA), Tertiary Vocational Education Commission (TVEC), Department of Technical Education and Training (DTET), UNIVOTEC, private sector employers, Sri Lanka Association for Software Services Companies (SLASSCOM) and Open University of Sri Lanka] [Probe on the type of engagement (duration, quality, nature, etc.)] 11. To what extent were youth and youth-led organizations engaged directly as key partners? [Probe on how if yes, and why they were/are or were/are not engaged as partners] Program Outcomes 12. To what extent has YouLead met your expectations of performance to date? a. Aligning skills to emerging needs? [Probe on differences between sectors selected, geographic regions, partners (if not already covered), and youth characteristics] b. Increasing vocational skills among diverse youth? c. Increasing entrepreneurial skills among diverse youth? d. Improving participation of young women (and young men) in the economy? e. Shifting cultural norms around appropriate careers for men and women? 13. What have been the critical successes for YouLead thus far in your opinion? a. Probe for articles, social media postings, other hard evidence as-needed. 14. What, from your perspective, have been YouLead’s critical challenges or failures thus far? 15. What aspects of YouLead could be (or are likely to) be sustained after the program ends? Recommendations 16. Based on what you have learned over the course of the YouLead program, what would you continue in future program efforts? 17. What would you do differently in future program efforts? 18. What have I not asked you about YouLead that I should have asked? What else should I know about your experience with the program? If you think of anything that might further improve YouLead that we did not have a chance to discuss, please call or email… THANK YOU FOR YOUR TIME AND COOPERATION! Time Completed: ___________________ Interviewer: ___________________ USAID.GOV YOULEAD EVALUATION FINAL REPORT | 54 KII GUIDE – IESC TEAM MEMBERS SI, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As a core member of the program implementation team, we believe you have unique insights into YouLead’s operations, management, and results. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. The information you share with us will be treated as confidential. The transcript and notes from this interview will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, program management stakeholder). You may choose not to answer any question, and you may choose to end the interview at any time. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If no, thank the respondent and end the interview. If yes, continue with script.] Do you consent to have this interview recorded for accuracy purposes? [Record response.] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: ___________________ Background Questions 1. To start, I’d like to hear about your role with the YouLead program. Could you please tell us a bit about when and how you became involved in the program and the role(s) you have played over time? Program Activities 2. Would you please share with us a bit about the YouLead activities you have led/been involved with from an implementation or design perspective? [Probe on the options below based on the response to Q1 above.] a. Effectiveness of integration of work readiness skills in existing firm-based private sector training (tourism and ICT) b. TVET curriculum revisions c. TVET training of trainers d. TVET career guidance center development e. Joint entrepreneurship curriculum development f. Business development supports (mentoring, cash-flow based assessment training for financial institutions, support for incubators, etc.) g. Partnership management h. YouLead web portal design or content management i. Monitoring, evaluation, and learning 3. [If relevant] For the TVET training of trainers that YouLead offered, how effective was the in-person training approach? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What barriers to access the in-person training did you identify? Was the project able to overcome those, if any? 55 | YOULEAD EVALUATION FINAL REPORT USAID.GOV c. To what extent did it provide strategies for the inclusion of young women, youth with disability, and youth from ethno-religious minorities? d. What were the overall outcomes, and how did you approach measuring them? e. If outcomes of instruction were tracked, did young women/vulnerable youth have similar outcomes to the young men/non-vulnerable youth? 4. [If relevant] How effective was the virtual training approach? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What modifications were made to deliver the content in a virtual format? c. What barriers to access the virtual training did you identify? Was the project able to overcome those, if any? d. To what extent did it include young women, youth with disability, and ethno￾religious minorities? e. What were the overall outcomes, and how did you approach measuring them? f. Did young women/vulnerable youth have similar outcomes to the young men/non￾vulnerable youth? 5. [If relevant] For the entrepreneurship training that YouLead offered, how effective was the in-person training approach? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What barriers to access the in-person training did you identify? Was the project able to overcome those, if any? c. To what extent did it include young women, youth with disability, and youth from ethno-religious minorities? d. What were the overall outcomes, and how did you approach measuring them? e. Did young women/vulnerable youth have similar outcomes to the young men/non￾vulnerable youth? 6. [If relevant] How effective was the virtual training approach? (EQ2) a. What, from your perspective was the level of effort to deliver training? b. What modifications were made to deliver the content in a virtual format? c. What barriers to access the virtual training did you identify? Was the project able to overcome those, if any? d. To what extent did it include young women, youth with disability, and ethno￾religious minorities? e. What were the overall outcomes, and how did you approach measuring them? f. Did young women/vulnerable youth have similar outcomes to the young men/non￾vulnerable youth? 7. What type of data do you collect and how have you used this information over the course of the program to make adaptations over the course of the project? [Probe on specific feedback loops/Collaboration, Learning, and Adaptation practices established, utility thereof, and lessons learned on data use.] Program Outcomes 8. How would you describe the outcomes of the YouLead program to date in terms of… a. Aligning skills to emerging needs of employers? [Probe on differences between sectors selected, geographic regions, partners (if not already covered), and youth characteristics.] b. Increasing vocational skills among diverse youth? c. [If not addressed in Qs 5 and 6] Increased entrepreneurial skills among diverse youth? d. Improving participation of young women (and young men) in the economy? USAID.GOV YOULEAD EVALUATION FINAL REPORT | 56 e. Shifting cultural norms around appropriate careers for men and women? 9. What have been the critical successes for YouLead thus far in your opinion? 10. What, from your perspective, have been YouLead’s critical challenges or failures thus far? 11. What, from your perspective, is needed to achieve program outcomes and sustainability? Partnerships Although YouLead directly delivered some training, partnering with local organizations was a key strategy to strengthen training curricula, teaching practices, and work-based learning experiences. 12. How would you describe the process of identifying and recruiting key stakeholders to participate in the program? 13. What were the primary advantages to working with the key stakeholders this way? (EQ3) [Probe on National Apprentice and Industrial Training Authority (NAITA), Vocational Training Authority (VTA), Tertiary Vocational Education Commission (TVEC), Department of Technical Education and Training (DTET), UNIVOTEC, private sector employers, Sri Lanka Association for Software Services Companies (SLASSCOM) and Open University of Sri Lanka] 14. There are also potential challenges in working through partnerships as well. How would you describe the challenges YouLead faced in partnering with these key organizations? (EQ3) [Probe on National Apprentice and Industrial Training Authority (NAITA), Vocational Training Authority (VTA), Tertiary Vocational Education Commission (TVEC), Department of Technical Education and Training (DTET), UNIVOTEC, private sector employers, Sri Lanka Association for Software Services Companies (SLASSCOM) and Open University of Sri Lanka] 15. What is the level of partnership engagement at this point in the program? Has the level of engagement evolved over time? Lessons Learned 16. Based on what you have learned over the course of your involvement with the YouLead program, what would you continue in future program efforts? 17. What would you do differently in future program efforts? 18. What have I not asked you about YouLead that I should have asked? What else should I know about your experience with the program? If you think of anything that might further improve YouLead that we did not have a chance to discuss, please call or email… THANK YOU FOR YOUR TIME AND COOPERATION! Time Completed: ___________________ Interviewer: ___________________ 57 | YOULEAD EVALUATION FINAL REPORT USAID.GOV KII GUIDE – GOVERNMENT PARTNERS Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. Given the mandate of your organization, we believe you have unique perspectives on the processes used to strengthen institutions and expand the training available to youths through the YouLead program. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. The information you share with us will be treated as confidential. The transcript and notes from this interview will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, program management stakeholder). You may choose not to answer any question, and you may choose to end the interview at any time. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If no, thank the respondent and end the interview. If yes, continue with script.] Do you consent to have this interview recorded for accuracy purposes? [Record response] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: _______________ Background Questions 1. To start, I’d like to hear about how you first became connected with the YouLead program. Could you please tell us a bit about when and how you came to be involved in the program? 2. Could you please share with us what your organization or department hoped to achieve in the partnership with YouLead? Program Activities 3. Would you please share with us a bit about any YouLead activities in which you have [your department has] participated and the nature of the participation? [Probe on the options below based on the response to Q1 above. Probe on the type of engagement (duration, quality, nature, etc.)] a. Integration of work readiness skills in private sector training (tourism and ICT) b. TVET curriculum revisions c. TVET training of trainers d. TVET career guidance center development e. Joint entrepreneurship curriculum development f. Business development supports (mentoring, cash-flow based assessment training for financial institutions, support for incubators, etc.) Partnerships Although YouLead directly delivered some training to youth, it more frequently partnered with national or local organizations to strengthen their training curricula, teaching practices, and work￾based learning experiences. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 58 4. How effective do you think the partnership model was for achieving the intended goals of the program? 5. What were the key advantages to YouLead’s approach to working through partnerships? (EQ3) 6. What challenges were there, if any, in YouLead’s approach to working through partnerships? (EQ3) 7. How would you describe your partnership with USAID as it relates to this program? 8. In your opinion, how sustainable are the partnerships facilitated by YouLead? Program Outcomes 9. To what extent has YouLead met your expectations of performance in terms of… (EQ1) a. Aligning skills to emerging needs? [Probe on differences between sectors selected, geographic regions, partners (if not already covered), and youth characteristics] b. Increasing vocational skills among diverse youth? [Probe on work readiness, technical skill levels, and performance.] c. Increasing entrepreneurial skills among diverse youth? [Probe on work readiness, entrepreneurship skill, and performance.] d. Improving participation of young women (and young men) in the economy? e. Shifting cultural norms around appropriate careers for men and women? 10. What have been the critical successes for YouLead thus far in your opinion? [Probe for articles, social media postings, other hard evidence as needed.] 11. What, from your perspective, have been YouLead’s critical challenges or failures thus far? 12. What aspects of YouLead could be (or are likely to) be sustained after the program ends? Lessons Learned 13. Based on your experience with the YouLead program, what should they continue to do in future program efforts? 14. What should they do differently in future program efforts? 15. What have I not asked you about YouLead that I should have asked? What else should I know about your experience with the program? If you think of anything that might further improve YouLead that we did not have a chance to discuss, please call or email… THANK YOU FOR YOUR TIME AND COOPERATION! Time Completed: ___________________ Interviewer: ___________________ 59 | YOULEAD EVALUATION FINAL REPORT USAID.GOV KII GUIDE – PRIVATE SECTOR PARTNERS Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As an employer, we believe you have unique perspectives on the utility of the skills training supported by the YouLead program. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. The information you share with us will be treated as confidential. The transcript and notes from this interview will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, program management stakeholder). You may choose not to answer any question, and you may choose to end the interview at any time. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If no, thank the respondent and end the interview. If yes, continue with script.] Do you consent to have this interview recorded for accuracy purposes? [Record response] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: ___________________ Background Questions 1. To start, I’d like to hear about how you first became connected with the YouLead program. Could you please tell us a bit about when and how you came to be involved in the program? Program Activities 2. Would you please share with us a bit about any YouLead activities in which you have [your firm has] participated? [Probe on the options below based on the response to Q1 above.] a. Integration of work readiness skills in existing firm-based private sector training (tourism and ICT) b. TVET curriculum revisions c. TVET training of trainers d. TVET career guidance center development e. Provision of work-based learning (job placements and internships) f. Hiring of YouLead graduates g. Joint entrepreneurship curriculum development h. Business development supports (mentoring, cash-flow based assessment training for financial institutions, support for incubators, etc.) Program Outcomes 3. What has your experience been with graduates of YouLead-supported training? [Probe on work readiness, technical skill levels, and performance.] (EQ1) a. [If relevant] How prepared were YouLead graduates to enter the work place? [Probe on time to onboard and how they compare to other young employees who did not participate in YouLead.] USAID.GOV YOULEAD EVALUATION FINAL REPORT | 60 b. To the best of your knowledge, did those graduates participate in their YouLead training in person or virtually? Have you observed any differences in the work readiness, technical skills, or performance between those graduates? (EQ2) c. To what extent did your experience with graduates include young women, youth with disability, and/or youth from ethno-religious minorities? [If yes] Have you observed any differences in the work readiness, technical skills, or performance between those groups of graduates? (EQ1) d. To what extent, if at all, have you noted any shifting in ideas or norms about what kinds of careers are appropriate for men and women in your firm? (EQ1) [Probe on shifts in attitudes in sector/value chain.] Partnerships 4. How would you describe the partnership experience with YouLead? (EQ3) a. What went well? b. What were points of difficulty or frustration, if any? 5. To what extent do you believe the activities initiated under the YouLead program will be sustainable beyond the end of its funding from USAID and other donors? [Probe on whether partnerships with TVET institutions will continue, etc.] Lessons Learned 6. Based on your experience with the YouLead program, what should they continue to do in future program efforts? 7. What should they do differently in future program efforts? 8. What have I not asked you about YouLead that I should have asked? What else should I know about your experience with the program? If you think of anything that might further improve YouLead that we did not have a chance to discuss, please call or email… THANK YOU FOR YOUR TIME AND COOPERATION! Time Completed: ___________________ Interviewer: __________________ 61 | YOULEAD EVALUATION FINAL REPORT USAID.GOV DRAFT FGD GUIDE – TRAINERS Hello, everyone, and thank you for agreeing to participate in today’s discussion group. My name is [state name], and I will be leading today’s discussion. I am working on this project for Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As participants in one of the professional training streams, we believe you have unique perspectives on the impacts of the program’s various offerings. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. This focus group discussion will last approximately ninety minutes. There are no right or wrong answers to the questions I am going ask you. We want to capture the full range of opinions that you may have, so please feel free to disagree. However, please speak one person at a time, because we plan to record the focus group session. Recording the focus group will allow me to concentrate on your responses and ensure that I do not miss anyone’s input during the discussion. If at any time, anyone feels that they would prefer to speak with the recorder off, please let me know and I will stop the recorder. The information you share with us will be treated as confidential. The transcript and notes from this focus group will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, YouLead trainer). You may choose not to answer any question. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If anyone replies no, thank the respondent and ask him/her to excuse him/herself from the room. If yes, continue with script.] Do you consent to have this focus group recorded for accuracy purposes? [If yes, record response, turn on the recorder and continue. If no, do not record the FGD.] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: ___________________ Background Information 1. To start, kindly introduce yourself with your first name, how long you have you been in the training/teaching field, how long you have been involved with the YouLead program, and what are the YouLead assisted courses you teach? Program Activities - Collaborating with YouLead – (EQ1) 2. When and how did you first hear about the YouLead program? 3. How were you selected to participate in the YouLead training of trainers? Did you wish to participate or were required to by your supervisor/employer? Did you receive sufficient USAID.GOV YOULEAD EVALUATION FINAL REPORT | 62 communication from the organizers prior to the training? Did you complete your training in person or virtually? 4. What were the best aspects of participating in person or virtually? What were the worst aspects? 5. Did you learn anything new during the training? What training methodologies did you find most useful? [Probe on what and why those were the most engaging.] Program Outcomes - Effectiveness of the Training – (Learning) - (EQ1) 6. In your opinion, how relevant was the YouLead training to develop your knowledge and skills related to (career counselling, entrepreneurship, appraising credit applications, etc.)? 7. One of the objectives of YouLead was to make revisions to the existing curricula used for the skill development trainings and to develop new training curricula to more closely align with the demands of the labor market. To what extent do you believe these efforts were successful or not? [Probe on specific evidence that supports their opinions either way and the benefits or drawbacks to this approach (including allocation of time and resources.] 8. How would you characterize the quality of the content and the instructional materials for the YouLead curricula, whether it was revised or newly created? Adaptation 9. What new knowledge and skills that you gained from the YouLead training have you adapted into your training/teaching methods? Have these modifications led to more or less work for you? If more, in your opinion, has it been worth it? Has your supervisor/administrative staff required you to incorporate anything in particular? [Probe on gender-sensitive and inclusive approaches and any innovations in their teaching approaches.] 10. What impacts, if any, have you observed in your students’ participation in the classroom and their technical skills, and/or work readiness skills? [Probe on differences between young men and young women, youth with disabilities, and youth from ethno-religious minority communities.] 11. As a YouLead trainer, how have you been supporting and coaching other colleagues who have not participated in the training? 12. What measures have been taken by your school to institutionalize new or revised curricula, new or inclusive teaching approaches, or other YouLead-supported? Do you feel your colleagues and school administrators are supportive of these adjustments? [Probe on successes and challenges or obstacles to integrate them into the present system (I.e., training equipment/facilities and approvals/validation/approvals)]. 13. In your opinion, how do gender norms or stereotypes limit young men’s and women’s participation in the classroom and career choices? Have you observed any shifting in perceptions or attitudes of your students’ parents and other important figures in their lives that would allow youth more choice and agency? 14. How are youth guided into work-based learning (job placement and internship) opportunities or entrepreneurship opportunities? [Probe on whether gender-sensitive guidelines are used.] Recommendation for Further Improvements and Developments 15. Have you had any contact with the YouLead team since you completed the program? If so, did you provide follow-up and feedback? How was it received? 16. Based on your experience with the program, what recommendations would you make? [Probe on recommendations to strengthen the training of trainers, the follow-on support, the institutionalization of good practices.] 17. Would you recommend YouLead to your colleagues and students? If not, why? 63 | YOULEAD EVALUATION FINAL REPORT USAID.GOV THANK YOU FOR YOUR TIME AND COOPERATION! Time Ended: __________________ Interviewer: __________________ USAID.GOV YOULEAD EVALUATION FINAL REPORT | 64 KII GUIDE –YOULEAD PARTICIPANTS PLACED IN JOBS OR HAVE STARTED BUSINESSES Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. Given your personal experience with [work-based learning (job placement or internship) or starting a business] through the YouLead program, we believe you have unique perspectives on the program’s effectiveness. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. The information you share with us will be treated as confidential. The transcript and notes from this interview will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, program management stakeholder). You may choose not to answer any question, and you may choose to end the interview at any time. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If no, thank the respondent and end the interview. If yes, continue with script.] Do you consent to have this interview recorded for accuracy purposes? [Record response] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: __________________ Background Information 1. To start, could you please tell us a little about how you became involved with YouLead or your skill development course? [Probe on when he/she enrolled, training completed (technical training curriculum or entrepreneurship skills development training), mode of delivery (in-person or virtual), and language.] Program Activities (Collaborating with YouLead) 2. How efficient was the enrollment process, communication and other arrangements before the training started? [Probe on Pre-Assessment / Approval/ Communication in timely manner/travel arrangements, as relevant.] 3. Was a resource person (other than your instructor) available to you during your training? If so, how effective was the resource person during the training program and how relevant was the language used? 4. How knowledgeable was your trainer(s)? Was their presentation style engaging? 5. Do you think the training program was the appropriate length of time? Was the schedule convenient or did you have to make special arrangements in order to participate? [Probe to ascertain whether the duration was too long/too short and whether gender/traditional familial responsibilities caused challenges with the schedule of training.] 65 | YOULEAD EVALUATION FINAL REPORT USAID.GOV 6. How did you learn about the opportunity for this job placement/internship/entrepreneurship opportunity? [Probe for a description of the selection/matching process from the youth’s perspective and initial and ongoing supports from the trainer or instructor.] 7. Did you participate in any other activities or supports offered by the YouLead Team, such as Career Counselling/Career Fair /Career Me /YouLead Mentoring Platform? If so, what was your experience? [Probe on what he/she learned beyond the training course content.] If not, why? 8. Have you used any of the YouLead web portals (YouLead.lk / YouLead Facebook page/lovesrilanka.org) to identify skills development opportunities, jobs opportunities, and entrepreneurship opportunities? If so, did you find it useful? How might it be improved? If you did not access any of the portals, why not? 9. When choosing your training specialty and employment/entrepreneurship opportunity, did you feel you made the choice yourself, or did you feel pressure from others to decide or that you might face negative reactions from your family or others if you made certain selections? [Probe on how gender stereotypes, stereotypes around disability, or attitudes towards minority populations influenced the decision, if at all.] 10. Overall, did you find your [in person or virtual] training effective? What were the positive aspects and what would you have changed based on your experience? (EQ 2) Program Outcomes - Effectiveness of the Activity – (Learning) – (EQ 1) 11. Could you tell us about your present job/your business? [Probe on perception of success in that job/business, challenges or obstacles faced, changes in status to job/business as a result of COVID and/or current crisis.] 12. In your opinion, what are the knowledge and skills you learned in your course that help you carry out your job/business? [Probe on technical skills, soft skills, and work readiness skills.] 13. Working (Job Placement/Internship) Interviewees Only a. How is the working environment? What do you like and dislike in your working environment? [Probe on impacts of norms or stereotypes that might influence how others see the respondent.] b. What types of issues or problems have you come across when you work and who do you seek out to help resolve them? c. Are you satisfied with the support provided to you by your employer or do you feel that there should be improvement? 14. Entrepreneurship Interviewees Only a. When did you start your business and what did you do beforehand? [Probe on pre￾preparation, and whether they had the business or a similar one prior to the program.} b. If you started the business before enrolling in YouLead, what changes did you make to the business since participating in the program? c. How did you obtain financial support to start the business? Have you received additional financial supports since participating in YouLead? [Probe on personal savings, family savings, youth/village savings and loan association, financial institutions, government funds, etc.] 15. What other YouLead support or guidance have been most important for you in overcoming the challenges you have faced? 16. To what extent has your job/business helped you achieve your goals for improving your/your family’s well-being? [Probe on concrete changes to standard of living, status in the community, or other marker or success.] 17. As you consider your future, what do you hope to do? [Probe on career advancement, finding better employment, expanding current business, shifting into new business, additional education, etc.] Do you feel those are real possibilities, or might you face challenges, including negative reactions from your family or others if you follow those aspirations? USAID.GOV YOULEAD EVALUATION FINAL REPORT | 66 [Probe on how gender stereotypes, stereotypes around disability, or attitudes towards minority populations influenced the decision, if at all.] Recommendation for Further Improvements and Developments 18. Have you had any contact with the YouLead team since you completed the program? If so, did you provide follow-up and feedback? How was it received? 19. Based on your experience with the program, what recommendations would you make to improve the program for other young people like yourself? [Probe on recommendations to strengthen the training of trainers, the follow-on support, the institutionalization of good practices.] THANK YOU FOR YOUR TIME AND COOPERATION! Time Ended: __________________ Interviewer: ___________________ 67 | YOULEAD EVALUATION FINAL REPORT USAID.GOV FGD GUIDE – YOULEAD PARTICIPANTS (YOUTH GRADUATES) Hello, everyone, and thank you for agreeing to participate in today’s discussion group. My name is [state name], and I will be leading today’s discussion. I am working on this project for Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As participants in one of the training streams supported by YouLead, we believe you have unique perspectives on the impacts of the program’s various offerings. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. This focus group discussion will last approximately ninety minutes. There are no right or wrong answers to the questions I am going ask you. We want to capture the full range of opinions that you may have, so please feel free to disagree. However, please speak one person at a time, because we plan to record the focus group session. Recording the focus group will allow me to concentrate on your responses and ensure that I do not miss anyone’s input during the discussion. If at any time, anyone feels that they would prefer to speak with the recorder off, please let me know and I will stop the recorder. The information you share with us will be treated as confidential. The transcript and notes from this focus group will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, YouLead youth training participant). You may choose not to answer any question. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If anyone replies no, thank the respondent and ask him/her to excuse him/herself from the room. If yes, continue with script.] Do you consent to have this focus group recorded for accuracy purposes? [If yes, record response, turn on the recorder and continue. If no, do not record the FGD.] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: __________________ Background Information 1. To start, with the discussion kindly introduce yourself with your first name, the institution where you enrolled in your skill development course, when you completed training, and whether you did your training in person or virtually? Program Activities (Collaborating with YouLead Project) 2. Why did you enroll in a course with the training institution or provider you chose? USAID.GOV YOULEAD EVALUATION FINAL REPORT | 68 3. In your opinion, how relevant was the curriculum used by your instructors/trainers supported to develop your technical skills, soft skills and work readiness skills? [Probe on utility and relevance of learning materials.] 4. Did you find your in-person or virtual training effective? What were the positive aspects and what would you have changed based on your experience? (EQ 2) [Probe on teaching methods, duration, time and resources allotted, convenience of schedule. Capture differences in responses by in-person and virtual participants.] 5. In your opinion, how engaging was your instructor(s)/trainer(s)? What did they do differently than other teachers? 6. Were any YouLead training supports offered to you as part of your participation in training (Resource person, Career Counselling/Career Fair /Career Me /YouLead Mentoring Platform, YouLead web portals (YouLead.lk / YouLead Facebook page/lovesrilanka.org)? If so, did you participate and what was your experience with them or why didn’t you participate? [Probe on utility, relevance, ease of use.] 7. When choosing your training specialty, did you feel you made the choice yourself, or did you feel pressure from others to decide or that you might face negative reactions from your family or others if you made certain selections? [Probe on how gender stereotypes, stereotypes around disability, or attitudes towards minority populations influenced the decision, if at all.] Program Outcomes - Effectiveness of the Activity – (Learning) – (EQ 1) 8. What has been your work experience since enrolling in the program? Have you done anything different based on what you learned in the program? [Probe on businesses started, jobs started, apprenticeships started, additional education pursued, challenges and obstacles identified, etc.] 9. To what extent did the knowledge and the skills you gained from your training assist you in finding a job, internship, or starting your own business? Did you find you had any gaps in your knowledge or skills? 10. To what extent have you received ongoing support after the completion of your training (through your trainer/instructor, YouLead Mentoring Platform, or YouLead web portals)? [Probe on type of support that has been most useful and what is lacking.] Recommendation for Further Improvements and Developments 11. Have you had any contact with the YouLead team since you completed the program? If so, did you provide follow-up and feedback? How was it received? 12. Based on your experience with the program, what recommendations would you make to improve the program? [Probe on recommendations to strengthen the trainings, the follow-on support, and other aspects of the program.] 13. Would you recommend the training you completed to your friends? THANK YOU FOR YOUR TIME AND COOPERATION! Time Ended: __________________ Interviewer: __________________ 69 | YOULEAD EVALUATION FINAL REPORT USAID.GOV FGD GUIDE – PARENTS Hello, everyone, and thank you for agreeing to participate in today’s discussion group. My name is [state name], and I will be leading today’s discussion. I am working on this project for Social Impact, an independent third-party evaluation firm, has been contracted by USAID/Sri Lanka to conduct a midterm evaluation of the YouLead program to assess the effectiveness of program interventions to date and strengthening future program interventions. As parents of YouLead participants, we believe you have unique perspectives on the impacts of the program’s various offerings. The evaluation will be used to make informed decisions about mid-course corrections and future program efforts. This focus group discussion will last approximately one hour. There are no right or wrong answers to the questions I am going ask you. We want to capture the full range of opinions that you may have, so please feel free to disagree. However, please speak one person at a time, because we plan to record the focus group session. Recording the focus group will allow me to concentrate on your responses and ensure that I do not miss anyone’s input during the discussion. If at any time, anyone feels that they would prefer to speak with the recorder off, please let me know and I will stop the recorder. The information you share with us will be treated as confidential. The transcript and notes from this focus group will not be shared with any other organization. Reported information will be aggregated, and statements made will not be attributed at an individual level, but at a role level (for example, YouLead participant parent). You may choose not to answer any question. Do you have any questions for me about the evaluation, its purpose or use, or how we will handle the information you share with us today? [Answer any questions that are asked.] Do you consent to participate in this evaluation? [If anyone replies no, thank the respondent and ask him/her to excuse him/herself from the room. If yes, continue with script.] Do you consent to have this focus group recorded for accuracy purposes? [If yes, record response, turn on the recorder and continue. If no, do not record the FGD.] We greatly appreciate your cooperation and time. Thank you for answering each question to the best of your ability, and please don’t hesitate to ask for clarification if a question is unclear to you. Time Started: __________________ We greatly appreciate your cooperation and time. To start us off, would you each please share your first name, the training course your child completed, and when they completed the course? General Questions – (EQ1) 1. What was your overall opinion of your child’s participation in the skill development training? 2. Whose idea was it to enroll in the program? Were you supportive of their participation or did you have any concerns? 3. After completing the skill development training, what type of job/business is your child doing? 4. How did they select this particular job/business? USAID.GOV YOULEAD EVALUATION FINAL REPORT | 70 5. Did you have any concerns about whether this career choice was appropriate for your child? [Probe on gender norms or prejudices that might affect them, perceptions of standards of behavior, etc.] 6. Has your child discussed with you any issues that they experienced in the workplace? [Probe on what the problem was and whether there was a successful resolution.] 7. What type of support has your child needed to do their job/business effectively? Who provides those supports? [Probe on parental support, wider family support, community support, financial institution support, etc.] 8. What is the opinion of the community and other family members about their job/business? 9. In your opinion, what – if anything – would change those opinions? 10. To what extent did the skill development training yield the results you and your child expected? [Probe on benefits or unexpected negative consequences for the participants.] 11. Have you noticed any differences in your child after participating in the program? THANK YOU FOR YOUR TIME AND COOPERATION! Time Ended: __________________ Interviewer: __________________ 71 | YOULEAD EVALUATION FINAL REPORT USAID.GOV ANNEX IV: EVALUATION TEAM TEAM COMPOSITION Dr. Christine Allison, Team Leader: Dr. Christine Allison, Team Leader: Dr. Christine Allison has more than 25 years’ experience conducting research studies and evaluations across five continents and 30 countries. She brings a plethora of work in youth livelihoods that aligns with industry best practices, which is particularly relevant for this evaluation. With key expertise in areas such as most significant change, mixed-methods and meta-analyses; social justice and the human condition; outcome mapping and overall data visualization; Dr. Allison has a well-established track record of delivering maximum utility for all the clients she has served. In the last 10 years, Dr. Allison has delivered nearly a dozen such utilization-focused performance evaluations centering around youth employment and empowerment, including multiple for USAID and in South Asia. Indrani Rajendran, Senior Local Researcher: Indrani Rajendran has more than 20 years of experience as a Monitoring and Evaluation consultant in the youth development sector in Sri Lanka. Specifically, she supported Sri Lankan youth programming with organizational improvement plans, developed monitoring frameworks, provided technical support with survey implementation, and planned and facilitated workshops on organizational internal analysis and strategic planning. Ms. Rajendran has participated in dozens of evaluations, produced data collection mechanisms and tools, participated in KIIs and FGDs, and contributed to final reporting writing. Ms. Rajendran’s strong research skills, familiarity with USAID programs, and strong background in the youth development sector will bring immense value to the team. She is proficient in English, Tamil, and Sinhala. Anupama Dias, Senior Local Researcher: Anupama Dias has over 15 years of experience in monitoring and evaluation, private sector, and research within multiple developmental organizations and sectors of Sri Lanka. She has worked organizations such as World Vision, United Nations, Save the Children, and UNDP. Ms. Dias has extensive experience developing data collection tools, extensive qualitative data analysis, and interpretating results for finalizing. Additionally, Ms. Dias has experience participating in evaluations that focus on youth, specifically with the Child Fund Sri Lanka, and providing detailed analyses of child labor while working at Institute for Participatory Interaction in Development. She has a Masters in Disaster Management and a Bachelor of Science in agriculture. She is proficient English and Sinhala. Yalini Sivasubramaniyam, Junior Local Researcher: Yalini Sivasubramaniyam has more than 12 years of experience in research, implementation, and evaluation within multiple development sectors of Sri Lanka. Ms. Sivasubramaniyam has worked with over 250 youth interns and 100 Civil Society Organizations nationwide in support of youth empowerment and education. She has a Masters Degree in Human Rights, and is particularly adept at facilitating KIIs and FGDs with youth participants, on sensitive subjects, and with great respect for diversity. She is a native Tamil speaker with full fluency in Sinhala, and whose interpretation skills the Government of Sri Lanka has used for official state visits – including that of Prime Minister of India Sri Narendra Modi. Cherisma Rajaratnam, Logistician: Cherisma Rajaratnam has more than seven years’ experience in project management within multi-cultural teams. Ms. Rajaratnam participated in a number of studies in the role of logistician, notetaker, and junior researcher. In 2020 and 2021, Ms. Rajaratnam assisted and supported the Social Impact team with the data collection process and evaluation for the Gender and Social Inclusion Analysis and Civil Society Organization Evaluation, respectively. Before that, she had assisted the team of researchers on Political Economy Analysis for Improving USAID Engagement for Efficient and Effective Governance in Sri Lanka. Ms. Rajaratnam has native knowledge of Tamil and good knowledge of Sinhala. USAID.GOV YOULEAD EVALUATION FINAL REPORT | 72 EVALUATION MANAGEMENT Amy Hardeman Zangari, Project Director: Conducted quality assurance review on deliverables and provided high-level guidance on youth livelihoods, evaluation methodology, major deliverables, and risk mitigation strategies. Dominick Margiotta, Project Manager: Managed contractual aspects of the evaluation – including adherence to deadlines and requirements of the SOW; coordinated meetings and interactions among SI management and evaluation teams; and ensured quality and compliance of deliverables with contractual requirements. Kwabena Boampong, Project Assistant: Provided administrative and logistical assistance, and supported deliverable development, including formatting, copyediting, and data visualization.