Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1 MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY PERFORMANCE EVALUATION: HONDURAS EMPLEANDO FUTUROS WORKFORCE DEVELOPMENT ACTIVITY SUBMISSION DATE: November 20, 2020 Contract Number: AID-OAA-I-15-00019 – Task Order Number: AID-522-TO-16-00002 This publication was produced for review by the United States Agency for International Development. It was prepared by Dexis Consulting Group. ABSTRACT This report evaluates the performance of the Empleando Futuros workforce development (WFD) Activity in Honduras. The WFD Activity aimed to increase citizen security for vulnerable populations in urban, high-crime areas in Honduras by supporting workforce development to increase income-generating opportunities for youth who are the most at risk of being perpetrators of violence. The performance evaluation examines two questions: • To what extent are quality WFD services increased, and to what extent do quality WFD services protect against violence? • To what extent do WFD actions produce a workforce with relevant skills to support country development? Evaluators collected participant data using surveys at program intake and upon completion, followed up with program deserters, and conducted key informant interviews with participants, implementing partners, and employers. The evaluation found that adjustments made during the course of the WFD Activity resulted in greater participant retention in the program. Important in the journey to self￾reliance, implementing partners are incorporating components of the WFD Activity into other programs. Two thirds of youth participants reported receiving job placement services, but only a small fraction of those resulted in (mostly temporary) employment. Participants did note that the training they received aided them in their own job search. The key outcomes of the WFD Activity for participants are increases in employability, job search skills, “soft skills,” and protective factors (resilience, emotional regulation, positive attitude/confidence, and interpersonal skills). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) ii MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY PERFORMANCE EVALUATION: HONDURAS EMPLEANDO FUTUROS WORKFORCE DEVELOPMENT ACTIVITY ¿ VERSIONS: • Draft I – August 18, 2020 • Draft II – September 25, 2020 • Draft III – November 20, 2020 SUBMITTED TO: USAID/Honduras Prepared by: Dexis Consulting Group DISCLAIMER: The authors’ views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) iii TABLE OF CONTENTS Table of Contents...................................................................................................................................................... iii Table of Figures .......................................................................................................................................................... iv Acronym List .............................................................................................................................................................. vii 1 Executive Summary ............................................................................................................................................ 1 1.1 Overview..................................................................................................................................................... 1 1.2 Country Context ....................................................................................................................................... 2 1.3 Key Findings................................................................................................................................................ 3 1.3.1 Employment and Employability ........................................................................................................... 3 1.3.2 Violence .................................................................................................................................................. 5 1.3.3 Migration ................................................................................................................................................. 5 2 Evaluation Purpose & Evaluation Questions .................................................................................................. 6 2.1 Evaluation Purpose .................................................................................................................................... 6 2.2 Evaluation Questions ................................................................................................................................ 7 3 Activity Background ........................................................................................................................................... 8 4 Evaluation Methods ............................................................................................................................................ 9 4.1 Limitations.................................................................................................................................................11 4.2 Data Collection Tools ............................................................................................................................13 4.2.1 Violence Involved Persons-Risk Assessment and Employability Survey.....................................13 4.2.2 Survey Data Collection Methods......................................................................................................14 4.2.3 KII Guides .............................................................................................................................................14 5 Findings ...............................................................................................................................................................15 5.1 Learning Question 1................................................................................................................................15 5.1.1 Sub-Question 1.1................................................................................................................................. 15 5.1.2 Sub-Question1.2 ..................................................................................................................................21 5.1.3 Sub-Question 1.3................................................................................................................................. 24 5.1.4 Sub-Question 1.4................................................................................................................................. 29 5.2 Learning Question 2................................................................................................................................32 5.2.1 Sub-Question 2.1................................................................................................................................. 32 5.2.2 Sub-Question 2.2................................................................................................................................. 35 5.2.3 Sub-Question 2.3................................................................................................................................. 42 5.3 Additional Learning Questions – Migration ........................................................................................47 5.3.1 Sub-Question 3.1................................................................................................................................. 47 5.3.2 Sub-Question 3.2................................................................................................................................. 48 5.3.3 Sub-Question 3.3................................................................................................................................. 51 6 Discussion..........................................................................................................................................................53 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) iv Annex 1. Survey Sample Demographics ................................................................................................................56 Annex 2. Limitations .................................................................................................................................................58 Annex 3. KII Guides..................................................................................................................................................61 Annex 4. Employability by Type of Technical Training .......................................................................................65 Annex 5. Employability Score Improvement ........................................................................................................65 Annex 6. Effect of Sociodemographic conditions on Risk Level .......................................................................66 Annex 7. Detail of Employability Improvements ................................................................................................. 67 Annex 8. Additional Protective factor Analysis ...................................................................................................68 Annex 9. Additional Migration Analysis.................................................................................................................78 Annex 10. Violence-Involved Persons Risk Assessment and Employability Survey .......................................83 TABLE OF FIGURES Figure 1: Sample distribution, by program completion and program modality........................10 Figure 2: The six-month follow-up sample is comparable to all EF enrollees ............................10 Figure 3: Completion rate (%) among those enrolled in 2019, by sex, age, and education level ............................................................................................................................................................................16 Figure 4: Percentage of participants employed or returned to school, by time of observation..............................................................................................................................................................17 Figure 5: Explanatory factors affecting the probability of being employed at the six-month follow up...................................................................................................................................................................19 Figure 6: Participants perceived changes in their employment conditions between the six￾month follow up and before entering EF, by program completion.................................................20 Figure 7: Reasons for perceiving an improvement in employment conditions between before EF and the six-month follow up .......................................................................................................20 Figure 8: Dropout rates by baseline risk level ..........................................................................................21 Figure 9: Change in risk level between baseline and six-month follow up, among those who completed the program ....................................................................................................................................23 Figure 10: Characteristics of good and bad mentors as identified by participants...................26 Figure 11: EF's most useful results, as identified by participants .....................................................27 Figure 12: What participants find most useful of EF, by sex..............................................................29 Figure 13: Participant recommendations to improve EF....................................................................30 Figure 14: Type of enterprise of employment, by sector of training..............................................32 Figure 15: Employment search experience after EF .............................................................................33 Figure 16: Plans for implementing the lessons learned from EF in the organizations interviewed..............................................................................................................................................................38 Figure 17: Interaction between lessons learned, challenges, and plans for sustained implementation.....................................................................................................................................................42 Figure 18: Most common words used by employers to describe the experience with EF graduates..................................................................................................................................................................43 Figure 19: Main reasons for stagnated or increased hiring of EF graduates................................46 Figure 20: Sociodemographic factors associated with the odds of having attempted to migrate to the US in the last year at baseline..........................................................................................48 Figure 21: Percentage of participants who at baseline said that they had the intent to migrate to the US, by period of enrollment .............................................................................................49 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) v Figure 22: Percentage of participants with the intention to migrate to the US, by program completion and time of response ..................................................................................................................49 Figure 23: Reasons for wanting to migrate in the next three years, by time of observation50 Figure 24: Perception of a good future in Honduras among program completers, at baseline and six-month follow up...................................................................................................................51 Figure 25: Perception of a good future in Honduras among program completers, at baseline and six-month follow up, by sex ...................................................................................................52 Figure 26: Reasons provided for disagreeing with the idea of a good future in Honduras, at baseline and six-month follow up...................................................................................................................52 Figure 27: Factors associated with agreeing that there is a good future for the participant in Honduras, at six-month follow up ............................................................................................................53 Figure 28: Socio-demographic characteristics of sample at baseline.............................................56 Figure 29: Socio-demographic characteristics of sample at endline and six-month follow up . ......................................................................................................................................................................................57 Figure 30: Percentage employed at the baseline, endline and six-month follow up, by type of technical training.............................................................................................................................................65 Figure 31: Percentage of participants who completed the program, with improved employability scores between baseline, endline and six-month follow up ...................................65 Figure 32: Explanatory factors of the probability of decreasing risk levels between baseline and the six-month follow up, according to multivariate logistic regression................................66 Figure 33: Effect of sociodemographic conditions on the probability of decreasing the risk level, based on a logistic multivariate regression that omits the effect of psycho-emotional protective factors .................................................................................................................................................66 Figure 34: Percentage of participants who reported an improvement in employability skills between baseline, endline and six-month follow, by program completion. ................................67 Figure 35: Regression coefficients of multivariate regressions explaining the standardized scales associated with the protective factors ...........................................................................................68 Figure 36: Differences in baseline distribution of the protective factors between participants with and without a six-month follow up............................................................................69 Figure 37: Mean values at baseline of the scales associated to protective factors, and standardized changes between baseline and endline, and baseline and six month follow up, by program completion.....................................................................................................................................71 Figure 38: Effect of program completion and sociodemographic characteristics on the change in protective factor scales between baseline and six-month follow up, based on DID models.......................................................................................................................................................................75 Figure 39: Changes in the distribution of the scales associated with the protective factors between baseline, endline and six month follow up, by program completion. ..........................76 Figure 40: Effect of employment and six-month follow up on the protective factors, as estimated by fixed effects regressions .........................................................................................................77 Figure 41: Percentage of participants who attempted to migrate to the US the last year, by time of observation..............................................................................................................................................78 Figure 42: Percentage of participants who attempted to migrate to the US the last year, by program completion (baseline estimates)................................................................................................. 78 Figure 43: Percentage of participants who attempted to migrate to the US the last year, by selected sociodemographic characteristics...............................................................................................78 Figure 44: Migration intentions at six-month follow up among those with migration intentions at baseline, by selected sociodemographic characteristics...........................................79 Figure 45: Migration intentions at six-month follow up among those who expressed to have migration intentions at baseline, by psycho-emotional protective factors ..................................80 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) vi Figure 46: Factors associated with the odds of having migration intentions at baseline and six month follow up, based on multivariate logistic regressions (OR presented).....................81 Figure 47: Reasons for having migration intentions, by gender........................................................82 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) vii ACRONYM LIST ARM Adult Resilience Measure CARSI Central American Regional Security Initiative CBP Customs and Border Patrol CBT Cognitive Behavioral Therapy CCES Community Cohesion and Efficacy CI Confidence Interval DERS Difficulties in Emotional Regulation EF Empleando Futuros EFIS Empleando Futuros Information System ISRD International Self-Report Delinquency Study KII Key Informant Interview MEL Monitoring Evaluation and Learning MESCLA M&E Support for Collaborative Learning and Adapting Activity METAS Mejorando la Educación para Trabajar, Aprender y Superarse Programa de Capacitación PTSD Post-traumatic Stress Disorder PE Performance Evaluation SPS San Pedro Sula USAID United States Agency for International Development VIP-RA Violence-Involved Persons Risk Assessment WFD Workforce Development Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1 1 EXECUTIVE SUMMARY 1.1 OVERVIEW USAID/Honduras contracted Banyan Global (Banyan) to implement the Empleando Futuros (EF) Workforce Development (WFD) Activity in Honduras. This Activity aimed to increase citizen security for vulnerable populations in urban, high-crime areas in Honduras by supporting workforce development to increase income-generating opportunities for youth who are the most-at-risk of being perpetrators of violence. The Activity strengthened comprehensive workforce readiness services, including for job linkage and self-employment, to benefit at-risk youth, including those who qualify for secondary and tertiary violence prevention services. The Activity contributed to the USAID/Honduras Country Development Cooperation Strategy Development Objective 1 (DO1), Sub-Intermediate Result 1.1.2, “Quality services that protect against violence increased”. All participants were from communities with medium to high levels of violence in Tegucigalpa, San Pedro Sula (SPS), Tela, Choloma, La Lima, Villanueva and La Ceiba. The WFD Activity, implemented 2016-2021, focuses primarily on young males ages 16-30, given their prevalence in national violence statistics, both as perpetrators and victims of violence. At the time of enrollment, target participants were out of school, with no less than a sixth￾grade education and no more than a high school degree; and unemployed, underemployed, or with inconsistent or low-skill employment (“NiNis”). The WFD Activity sought to insert 3,750 of those trained into the workforce. Due to funding cuts in 2019, the program revised training targets to 4,600 and insertion targets to 2,500 during the period of evaluation (see Country Context below). To provide lessons learned to the WFD Activity to make any adjustment necessary to reach their goal, USAID/Honduras requested the Monitoring and Evaluation Support for Collaborative Learning and Adapting (MESCLA) Activity to conduct a performance evaluation (PE) of the Activity. The mid-term evaluation included participants enrolled from program inception in October 2017 through June 30, 2018. This second and final PE focuses on those participants enrolled in the WFD Activity from April 2019 to December 2019. Together with WFD Activity implementing partners, MESCLA collected participants data using surveys at program intake and upon completion. MESCLA also followed up with program deserters and conducted key informant interviews (KIIs) with participants, implementing partners, and employers. MESCLA examined the results to identify areas for adaptation and learning for future WFD programs. According to program reports, by the end of December 2019, 3,421 participants had completed the Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 2 WFD program, and 1,137 participants were employed or had returned to formal education any time during the 12 months following completion of training. 1.2 COUNTRY CONTEXT The WFD Activity operates in a country with high migration rates and difficult economic conditions for youth. After a dramatic increase in migration attempts leading up to May 2019 (more than 40,000 attempted migrants), migration attempts decreased dramatically, falling by more than 90% to less than 2,000 per month by May 2020 (CBP/DHS 2013-2020). Employment during the same period for those between 19-24 and 25-29 was 87.8 and 91.7 percent respectively, while official unemployment was only 12.2 and 8.3 percent (INE, 2019). Meanwhile, those in underemployment or working for less than the minimum wage were 66 percent for both groups. This demonstrates the fact that, despite low official unemployment, there is a great number of youths who work but do not earn enough to sustain themselves or their families.1 The implementation of this performance evaluation measurement was marked by a series of complications due to the country context, including funding cuts (and changes in activities), and the global COVID-19 pandemic. Following the March 30, 2019 USG announcement that there would be funding cuts to the Central American region in response to high rates of illegal migration to the United States, USAID/Honduras requested scaling back training and activities, therefore impacting the WFD Activity. Scheduled funding was ultimately withheld during the remainder of FY19 which resulted in changes to the training targets. Finally, the COVID-19 pandemic began as data collection for this performance evaluation was beginning. The MESCLA team made changes to the data collection instruments to capture activities and employment before the national lockdown on March 15, 2020. Forecasts about the impact of the pandemic on the labor market were negative,2 however the exact medium- and long-term impact of the pandemic and lockdown on responses cannot be known with the 1 In 2010 a “transitory” decree was issued for a special temporary employment program No. 230-2010 (Gazette No. 32, 358 of 11/05/2010) establishes temporary employment for specific work or services. The duration of the contracts can be up to 36 months and are renewable. At the beginning of 2014, Congress issued the Hourly Employment Law making temporary employment modalities included in the 2010 program permanent. According to a 2018 UNAH report on the employment situation in Honduras, the hourly employment law - instead of fulfilling its purpose of reducing unemployment -caused an increase in underemployment in the country. The visible underemployment rate increased from 4.6% in 2009 to 14.2% in 2018. Visible underemployment by definition includes people who worked less than 36 hours in the reference week and at the time of the interview expressed a desire to have been able to work longer and is used as a proxy for temporary employment. 2 https://forbescentroamerica.com/2020/05/19/cohep-alerta-de-posible-desaparicion-de-400000-empleos-en-honduras-por￾covid-19/ Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 3 data collected. (Participants were asked about employment before the lockdown in an attempt to minimize the impact on responses. See further discussion in Limitations section). 1.3 KEY FINDINGS The MESCLA team used a mixed methodology for the PE, including review of WFD Activity reports; data collection via a six-month follow-up survey and KIIs; data analysis of enrollment, program progression, and socioeconomic data in the Empleando Futuros Information System (EFIS); data analysis of existing baseline, endline, and new follow-up data. According to the MESCLA EF Technical Proposal (9/2017), the PE process was to answer two learning questions: • Q1: To what extent are quality WFD services increased, and to what extent do quality WFD services protect against violence? • Q2: To what extent do WFD actions produce a workforce with relevant skills to support country development? Some findings referenced here and throughout the report use dropouts (did not complete 80 percent of course, n=54), basic training completers (n=58), and technical training completers (basic + additional technical training, n=466) for comparison purposes only when results are statistically significant. When comparisons are made, those comparisons are only suggestive of difference/no difference between groups given that there was no way to control for potential selection bias. Pre-post results without a comparison group should also be interpreted with the normal caution for performance evaluations, as it is not possible to attribute changes to the WFD Activity alone since no impact evaluation was undertaken. 1.3.1 EMPLOYMENT AND EMPLOYABILITY • The improvement in completion rates indicates that the WFD Activity modifications to recruitment and training improved the retention of participants across all socio-demographic characteristics (sex, age, or education level) since the mid-term PE. In the mid-term PE completion rates differed by gender (women completed at higher rate than men), age (lower completion for 16-17-year-olds, highest for 18-19-year-olds), and education level. • Seven in ten completers who were unemployed when they started the program were employed or had returned to school in the six-month follow up. • Nearly half of those who completed technical training and engaged in some kind of economic activity after the WFD Activity indicate that their employment was better (in terms of pay, stability, hours, or location) than before. This improvement is especially notable given the high Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 4 rates of underemployment in Honduras. This improvement is statistically significant when compared to less than a quarter of those who did not complete technical training reporting better employment after the WFD Activity. • About one quarter of those who were in informal employment or self-employed when the program started had moved to a formal position in the six-month follow up, and an additional one in ten returned to school (this was true of dropouts as well, meaning this move cannot be attributed solely to the WFD Activity). • Eight in ten dropouts who were unemployed when they started the program were employed or had returned to school in the six-month follow up. One in every three dropouts left the program because they found a job or to return to school. • Despite inconsistent direct job placement support from EF reported by participants, participants noted that the training they received from EF improved their job search and interview skills, improved their entrepreneurial abilities, and provided them with a recognizable certification that aided in their own job search. • Among program completers, women are significantly less likely than men to be employed or to have returned to school at the six-month follow up (76.4 percent versus 63.4 percent, p<0.05). However, at the national level, women are also less likely to be employed or in school than men: 89.7 percent of men versus 56.8 percent for women which reveals structural barriers that were beyond the scope of the WFD Activity3. • Older youth (26-30 years old) had 53 percent lower odds of being employed at the six month follow up than younger youth (16-20 years old), while individuals with higher resilience levels were 24 percent more likely to be employed than those with lower resilience levels. • Women, those with less education at baseline, those who completed only basic training, and younger participants reported more improvement in employability skills than men, those with more education at baseline, those who completed technical training, and older participants. This is not surprising given that the employability skills measured are basic labor competencies. • A large majority (13 out of 17) of employers interviewed expressed satisfaction with EF employees and cited better attitudes/soft skills among EF graduates as a reason for hiring or desiring to hire from the program in the future. 3 These numbers, estimated using INE’s 2018 household survey, consider those who are not working for pay or looking for employment as not working. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 5 • Service providers cite poor attitudes and the vulnerable situation of youth as a major challenge to program completion. This stigmatization is associated with youth in marginalized communities in Honduras with limited opportunity to develop soft skills. • Employers and service providers note that soft skills and positive attitudes are key for success in the program and in securing and maintaining employment (while employers mentioned appropriate technical skills as an area for improvement). • Service providers are using or plan to use the lessons learned, materials (especially life skills), and relationships (with other service providers and employers) in other programs. • The most common area for improvement mentioned by participants was clarity in the services offered. Participants felt cheated because, when enrolling, they understood the WFD Activity would help them find employment or a job interview, and that did not happen in all cases. 1.3.2 VIOLENCE • Participants with secondary and tertiary risk of engaging in violent behaviors at the baseline had a higher probability of dropping out of the WFD Activity than those with primary risk. • Nearly six in ten participants at secondary or tertiary risk levels at baseline decreased their risk between baseline and the six-month follow up. • Protective factors significantly improved between baseline and the six-month follow up regardless of program completion. However, technical training completers showed the most consistent and statistically significant improvements in emotional regulation and resilience. 1.3.3 MIGRATION While the original aim of the WFD Activity was not to reduce migration to the U.S., the PE included migration questions to increase understanding of the youth employment and migration dynamic. The sample size was small (n=344 at baseline and follow-up) and questions did not separate between regular and irregular migration, however the learning that came out of this initiative includes the following findings: • Participants with secondary education and those in the primary risk category were less likely (statistically significant) to have attempted to migrate in the year prior to enrolling in EF than those with no secondary education and those in the secondary risk category. Meaning more education and lower violence risk levels are predictive of lower migration. • Migration intentions decreased by nearly two thirds between baseline and six-month follow up, regardless of program completion. (Note that over this same period, US Customs and Border Protection (CBP) apprehensions at the southwest border also declined significantly). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 6 • Migration intentions decreased for dropouts and those who completed basic training, but not for those who completed technical training, which were already substantially lower at baseline than overall trends (all statistically significant). • The reason for having migration intentions remained stable between baseline and six-month follow up with lack of employment as the most common reason for having migration intentions. • Migration intentions increase with past migration attempts and decrease with perception of a good future in Honduras. At the six-month follow up, difficulty with emotional regulation also increases the odds of having migration intentions. • About three out of four participants agree that there is a good future in Honduras. This perception increases among those who completed basic or technical training compared to dropouts and decreases by age group. • Similar to those with migration intentions, nearly 9 in 10 of those who do not see a good future in Honduras cited the economic situation as the primary reason for that negative perception. 2 EVALUATION PURPOSE & EVALUATION QUESTIONS 2.1 EVALUATION PURPOSE The Empleando Futuros WFD Activity in Honduras began in FY2016 under Development Objective 1 Sub-Intermediate Result 1.1.2, “quality services that protect against violence increased”. USAID/Honduras contracted Bayan Global to implement the WFD Activity from FY2016-2021 4 .The Activity is part of the Central America Regional Security Initiative (CARSI). As such, it aims to support workforce development that will increase income-generating opportunities for youth who are the most at risk of being perpetrators of violence in urban, high-crime areas in Honduras. The WFD Activity will 4 In response to cuts to United States foreign assistance to Honduras, in July 2019 Banyan Global submitted a contingency plan to USAID outlining adjustments in activities (reductions in scale and scope) and modified projections due to the change in available funds. Formally accepted by USAID in August 2019, that plan indicates that project activities will continue through November 2020. Theory of Change At-risk youth who are provided with high-quality, comprehensive, and market-driven vocational training services and assistance (life skills, basic competencies, psycho-social support, technical skills and job placement support) will both increase their employment opportunities and reduce their risk factors; therefore, increasing their household incomes and reducing their incidence for being victims or perpetrators of violence and crime. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 7 strengthen comprehensive workforce readiness services, including job linkage and self-employment, to benefit at-risk youth, including those who qualify for secondary and tertiary violence prevention services. The WFD Activity goal in Honduras is to “increase employment and protective factors for at-risk youth living in the targeted USAID/Honduras DO1 high crime municipalities of Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, and Tela.” The WFD Activity seeks to achieve the following three primary results: • Result 1: Access to high quality, comprehensive workforce development services for at-risk youth increased, with an emphasis on youth who qualify for secondary prevention services. • Result 2: INFOP’s institutional capacity to deliver high-quality, market-driven services improved. • Result 3: Access to workforce-related services, including income-generating activities, increased for youth who have been in conflict with the law, including former gang members. At-risk youth eligible to participate in the WFD Activity were defined by the following criteria: (1) living in selected high-crime communities in the DO1 target municipalities of Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, La Lima, Villanueva and Tela; (2) being out of school, with at least a sixth-grade education and no more than a high school degree; and, (3) being unemployed, under-employed, or with inconsistent or low-skill employment. The criteria also specify that at least 30% of the participants will be at-risk youth in need of secondary prevention services. The scope of this PE covers Result 1 exclusively. The PE will serve to support USAID’s decision-making process and inform delivery strategies to ensure outcome effectiveness in subsequent activities. It should be noted that a pilot impact evaluation was conducted in 2017 but was terminated due to challenges with consistency in implementation and high dropout rates in the first full year of the WFD Activity. While improvements have been made in both areas, the PE is expected to provide further input for future adaptation. Absent a control group, however, the PE will not be able to answer the impact evaluation questions originally presented by USAID. 2.2 EVALUATION QUESTIONS According to the MESCLA EF Technical Proposal (9/2017), the PE process was to answer two learning questions: • Q1: To what extent are quality WFD services increased, and to what extent do quality WFD services protect against violence? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 8 • Q2: To what extent do WFD actions produce a workforce with relevant skills to support country development? The mid-term PE (2018/19) focused on the service delivery and violence portions of Q1 given that the status of program implementation did not allow for a review of employment/employability. The current PE will therefore focus on Q1 sub-questions dealing with employment/ employability (i.e. special focus on employment/employability in Q1.2-1.4) and Q2. The scope of this PE is limited to the following sub-questions: • Q1.1: To what extent has the WFD Activity reached the targets of Result 1? Why have these targets been or not been reached? • Q1.2: How does the risk of violence and employability change after program completion and six months after finishing the program? • Q1.3: What are the perceptions of beneficiaries about the people and services that help them succeed? • Q1.4: How do stakeholders perceive the quality of the WFD Activity deliverables? • Q2.1: To what extent is the WFD Activity delivering job placement services to participants? • Q2.2: What are the perceptions among service providers about the services of the WFD Activity? • Q2.3: What are the perceptions among employers about the services of the WFD Activity? 3 ACTIVITY BACKGROUND WFD Activity participants are drawn from the following target population: • Males (at least 60 percent) between the ages of 16 and 30; • Living in specifically identified communities within the following high-crime municipalities: Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, Villanueva, La Lima, or Tela; • Currently out of school, with no less than a sixth-grade education and no more than a high school degree; and • Currently unemployed, underemployed, or with inconsistent or low-skill employment. Prior to the period of evaluation, the WFD Activity in Honduras was comprised of three separate phases in a Basic Training Model: 104 hours of life skills and cognitive behavioral therapy (“phase 1”), followed by basic labor competencies (80 hours), and 24 hours each of “capstone” labor orientation and Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 9 customer service (“phase 2”), and an internship or job placement and six months of observation and mentoring (“phase 3”). Mentors accompanied the youth during phase 1, and labor advisors accompanied the youth during phases 2 and 3. Technical Training Models: In earlier phases of the WFD Activity, the Basic Training was completed as a requirement for entry into the technical training component. Some implementing partners maintained this model during the period of evaluation, while many other implementing partners moved to an Integrated Service Model that added technical/vocational training to the basic training model. In this model youth were recruited for a specific occupation, for example restaurant services or sales associate. Training was then integrated where youth spent one half day on the phase 1 activities, and one half-day on phase 2 activities (the same total number of hours per topic was maintained from the Basic Training Model). Mentors accompanied the youth during phase 1, and labor advisors accompanied the youth during phases 2 and 3. In some cases, the integrated training model was used in direct response to a company’s immediate job insertion needs, in which case it is referred to as a Specialized Integrated Service Model. Finally, a Customized Dual Model was also introduced, which was offered in the same way as the Integrated Service Model, with the customer service training provided by the employer. In all models, training was conducted by local service providers (“implementing partners”) and considered “complete” for youth who finished 80 percent of the hours in phases 1 and 2.5 4 EVALUATION METHODS A mixed methodology was used for the PE, including: • Document review of WFD Activity reports; • Data collection via a six-month follow-up survey and KIIs; • Data analysis of enrollment, program progression, and socioeconomic data in the Empleando Futuros Information System (EFIS); • Data analysis of existing baseline, endline, and new follow-up data. According to EFIS data, between April 2019 and December 2019, 1,615 individuals enrolled in EF. Of those, 85 percent (1,370 participants) had a baseline Violence-Involved Persons Risk Assessment (VIP￾5 Given the change in models, the WFD Activity also attempted to recover youth who had dropped out after completing phase 1 of the Basic Training Model by offering them a chance to “complete the program” by completing the customer service and capstone courses. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 10 RA), and 54 percent of the total enrolled (869 participants) had either an endline, six-month follow up conducted for the PE, or both. This last group constitutes the primary population for this PE (Figure 1).6 MESCLA conducted the six-month follow-up survey data for a sample of 578 of the 1,370 participants with a baseline, and KIIs with 41 participants in Tegucigalpa, San Pedro Sula, Choloma and La Ceiba. MESCLA also conducted KIIs with 18 implementing partner representatives, from seven of the eight implementing partner organizations, and 17 employers. Figure 1: Sample distribution, by program completion and program modality The socio-demographic characteristics at baseline of the sample were not significantly different from all EF enrollees in terms of sex or age, but there was a difference in education levels with an under sampling of primary to 8th, over sampling of secondary to 12th (see Figure 2). The sample of dropouts, completers of basic training, and those who complete technical training were also not significantly different from each other in terms of sex, age, or education level. The endline and six-month follow up samples for dropouts and those who only completed basic training are also not significantly different in terms of sex, age, or education level from those who originally enrolled (see Annex 1 for all sample characteristics). This means that for these populations the three samples are comparable. However, in the case of those who completed technical training, those in the endline sample were slightly less educated than those who originally enrolled, and those who answered the six-month follow up were slightly older and more educated (p<0.05). Figure 2: The six-month follow-up sample is comparable to all EF enrollees 6 Dropout cases are used for comparison purposes only. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 11 Male 45.5% 43.4% Age 16-20 50.7% 50.4% Age 21-25 35.5% 35.2% Age 26-30 14.1% 14.2% Primary, 6-8th grade 11.0% 6.4%** Primary, 9th grade 11.8% 9.5% Secondary, 10-11th grade 18.5% 18.2% Secondary, 12th grade 58.7% 65.8%** 4.1 LIMITATIONS While MESCLA anticipated collecting the survey and KII data in person, the beginning of data collection coincided with the outbreak of the COVID-19 pandemic and subsequent stay-at-home orders in March 2020. Surveys were therefore conducted online, and KIIs by phone. Given that there is no single source that could be used to answer all the questions posed in this report, we draw from different, complementary, data sources. Therefore, reference populations and samples differ between questions and figures in the following manner: • Data on enrollment, dropout, and completion are from the WFD Activity information system called the EFIS. • WFD Activity implementers collected VIP-RA and Employability data at baseline and endline, and MESCLA conducted follow up surveys. Analysis of characteristics of participants at baseline refers to all participants who responded to the VIP-RA and Employability survey within three weeks of enrollment. Comparison between baseline, endline and the six-month follow up, however, depend on those who responded in two or three observations. This decreases the sample sizes. Dropouts did not respond to an endline, and the six-month follow up was only applied to a statistical sample of all participants. • Response numbers in some of the analyses and figures may be smaller because participants may not have answered all questions necessary to calculate risk scores and protective factors. In each of the analyses we try to specify, as clearly as possible, the data used and the population it refers to. There are also several issues and limitations about data to consider that arose during the PE. Key among them are the following: • Due to COVID-19, rather than applying the survey in face-to-face interviews or with the assistance of an enumerator, respondents filled in the survey online. KIIs were also completed EF Enrollees Six-month follow up Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 12 telephonically with youth participants, service providers and employers. The response rate was lower than anticipated, and the team was only able to reach 71% of the target sample size in the survey, and 88% of planned qualitative interviews. There may have also been an unknown impact on responses to the survey given the difference in data collection techniques between baseline and follow up. • Per agreement between MESLCA and the WFD Activity, the VIP-RA was initially collected at intake or during the first three weeks of beneficiary enrollment as a baseline. However, MESCLA learned from EF staff that WFD Activity implementers had sometimes been administering the VIP-RA as late as the fifth week of enrollment. Late application of the VIP-RA limits the ability to identify change given that there is no real baseline and that application during week five of a eight week program provides little time for meaningful change to occur, or may already be affected by changes that occurred between enrollment and the fifth week. • Migration questions were added to the baseline in April 2019 and to the endline in July 2019. Given this timing, the analysis is based on 806 individuals who answered migration questions in the baseline survey, and 344 who answered migration questions at both baseline and the six￾month follow up. Data from the endline are not used, as only 115 participants had migration data at all three points in time. • The six-month follow up was conducted between March and April 2020 among participants that enrolled in the program between April and December 2019. On average, they had completed the program six months prior to the six-month follow up. However, some had enrolled up to a year before, while others had enrolled only four months before. Locating WFD participants several months after they had concluded the program was a challenge and required the use of a replacement list to achieve the necessary sample size. This challenge was due to outdated contact information, and a reported unwillingness to engage with the WFD Activity again. Methodological limitations include: • This is a PE, not an impact evaluation. In some sections of the report we compare the situation of WFD Activity participants at baseline, endline and approximately six months after completion. We provide these as evidence of results, but in a strict sense, we cannot know if the changes observed were a result of the program, or if they were a product of the natural evolution of participants over time. In order to know this, an impact evaluation would be necessary. • The results may not be representative of the change in all WFD Activity participants, but only of those who answered the VIP-RA and Employability Survey. This is due to the following: Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 13 ◦ 85 percent of all WFD Activity participants were interviewed at the beginning of the program, 65 percent had an endline, and of these, 64 percent were interviewed for a follow up. ◦ Among program completers, those willing to answer the follow-up survey may represent those most satisfied with the program, possibly indicating they have achieved better results. There may also be a selective bias among drop-outs. Finally, given that the VIP-RA and employability survey asks about sensitive topics, participants may have lied in some of their answers, especially when interviewed at the beginning of the WFD Activity, when they were not familiar with the program and hence did not trust EF. If their lying decreased over time, we may be reporting results that underestimate the true positive effect of the program or that even show a move towards riskier attitudes. There is no way we can assess the size of this effect. 4.2 DATA COLLECTION TOOLS 4.2.1 VIOLENCE INVOLVED PERSONS-RISK ASSESSMENT AND EMPLOYABILITY SURVEY The VIP-RA, employment, and employability instruments are standardized questionnaires (collectively “VIP-RA and employability survey”) collected among WFD Activity participants at enrollment, completion, and, for the sample above, six months after either dropout or program completion. The questionnaires are completed together as part of a joint instrument (full instrument in Annex 9). Employment information was captured using the questionnaire used for the Honduras National Household Survey. Some modifications were made to the questionnaire so that it would follow the Colombia National Household Survey questionnaire, which goes further in depth in several areas of employment and entrepreneurship. Employability information was captured using the Youth Employability Survey employed by the USAID Activity Mejorando la Educación para Trabajar, Aprender y Superarse Programa de Capacitación (METAS). Small modifications were made based on the WFD Activity Monitoring Evaluation and Learning (MEL) Plan, but the METAS tool was largely replicated verbatim.7 At the request of USAID, faculty and staff at the University of Notre Dame developed and validated the VIP-RA tool for the WFD Activity in order to distinguish between primary and secondary risk, and to track change in risk for WFD Activity impact and performance evaluations.8 The VIP-RA served several 7 More on the tool can be found in Abdalla, Barth, Dunn, Holter, Ortega, and Tinta (2013) and USAID Honduras (2014). 8 For more information on the tool, see the “Development and validation of the violence-involved persons risk assessment: Honduras” report submitted to USAID September 2017. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 14 purposes: (1) establish empirically defensible risk of violence among young adults ages sixteen to thirty,9 (2) help target interventions for the secondary risk population, and (3) measure change in risk during and after program participation via an impact and/or performance evaluation. 4.2.2 SURVEY DATA COLLECTION METHODS When enrolling participants, the WFD Activity collects baseline data using the VIP-RA and employability survey. Banyan training subcontractors, called implementing partners, apply the VIP-RA and employability survey to all new enrollees at the latest by the third week of the WFD Activity in order to have a baseline observation that is comparable among participants.10 The implementing partners also committed to collect data on a rolling basis as participants finished the program. In order to have enough follow-up information on WFD Activity participants who either dropped out before a second data collection and those who had completed at least six months ago, MESCLA collected 578 follow-up VIP-RA and employability surveys. The sample selection was drawn from a file that MESCLA constructed matching VIP-RA and employability baseline data and enrollment information provided by the WFD Activity. (For more on sampling, see Annex 1.) Following up with participants who had not been in contact with the WFD Activity for some time was complicated by the fact that one in three participants was unreachable through the contact information provided (i.e. no answer to three attempted calls, or disconnected numbers). Among those reached, the refusal rate for the survey varied by age groups, from 54 percent among 16-19-year-olds, to 30 percent among 20-24-year-olds, and 16 percent for 25-31-year-olds. 4.2.3 KII GUIDES The team conducted KIIs with participants, service providers, and employers following a guide (Annex 3). KIIs were held by phone given COVID-19 restrictions. The team followed a standard question and response format, including follow-up questions for clarification and greater depth. The WFD Activity provided names and contact information for all KII subjects. Of the 190 potential KIIs subjects, 76 interviews were conducted. Only 6 KII subjects refused the interview, while the remainder were either unreachable at the contact information provided by the WFD Activity or did not answer at the time of the programmed interview. 9 The term “young adult” or “adult” is used to distinguish the VIP-RA from other tools that target younger age groups commonly referred to as “youth.” 10 This, however, is not always the case. Of 1,615 participants in the sample, only 1,370 had a baseline VIP-RA. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 15 5 FINDINGS This section provides the main findings for the two learning questions and each of the sub-questions from the evaluation. Some findings referenced here and throughout the report use dropouts (n=54), basic training completers (n=58), and technical training completers (n=466) for comparison purposes and only when results are statistically significant. It is not possible to attribute changes to the WFD Activity alone without an impact evaluation. Therefore, when comparisons are made, it is done to emphasize that changes cannot be attributed to the program alone. Results without a comparison group should also be interpreted with caution for the same reason. The only time results can be attributed to the program is when statistical significance is indicated, meaning the difference was large enough to detect with the available data. 5.1 LEARNING QUESTION 1 • Q1: To what extent are quality WFD Services increased, and to what extent do quality WFD services protect against violence? 5.1.1 SUB-QUESTION 1.1 • Q1.1: To what extent has the WFD Activity reached the targets of Result 1 (Access to high quality, comprehensive workforce development services for at-risk youth increased, with an emphasis on youth who qualify for secondary prevention services.)? Why have these targets been or not been reached? 5.1.1.1 HOW MANY INDIVIDUALS HAS THE WFD ACTIVITY BENEFITED TO DATE? According to WFD Activity reports, 6,750 have been enrolled from the start of the WFD Activity to December 31, 2019. From April 2019 to December 2019 – the period under consideration in this PE – 1,615 participants enrolled in the program. 5.1.1.2 HOW MANY PARTICIPANTS HAVE COMPLETED THE WFD ACTIVITY? WFD Activity reports indicate that of the 6,750 enrolled throughout the life of the project to December 31, 2019, 3,421 or 50.7 percent completed the program. This is 74.4 percent of the target of 4,600 completions. “Completion” is at least 80% of the hours under the Basic Training Model (see section 3 for more details on hours and models). EFIS data indicate that 87.2 percent (1,408) of the 1,615 participants who enrolled between April and December 2019 completed training. (See Figure 1 above in Section 4 for breakdown of these participants by training type). 5.1.1.3 ARE THERE ANY SEX, AGE OR EDUCATION DIFFERENCES IN COMPLETION RATES? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 16 Figure 3 shows completion rates by sex, age, and education level. No significant differences in completion rates were found by sex, age, or education level. This differs from the results of the mid￾term PE where there was a difference by gender (women completed at higher rate than men), age (lower completion for 16-17 year-olds, highest for 18-19 year-olds), and education (higher completion for those with secondary education in phase 1, higher completion for those with primary education in phase 2). The improvement in completion rates indicates that WFD Activity modifications improved the retention of participants across all socio-demographic characteristics. Figure 3: Completion rate (%) among those enrolled in 2019, by sex, age, and education level 5.1.1.4 HOW DO ENROLLMENT AND INSERTION RATES COMPARE WITH TARGETS? The original enrollment, completion, and insertion targets for the WFD Activity were 11, 250, 7,500 and 3,750 respectively. Due to funding cuts, targets were changed so that the target for youth completing the program was reduced to 4,600 and those securing employment to 2,500. As of December 31, 2019, the WFD Activity had enrolled 6,750 participants. Of those enrolled, 3,421 or 74.4 percent of the target of 4,600 had completed WFD Activity training. In the same period, the WFD Activity had inserted11 2,481 participants, or 99 percent of the target of 2,500. According to program reports, of the 1,615 WFD Activity participants enrolled during the April￾December 2019 PE analysis timeframe, 1,504 (93.1 percent) had found employment or returned to 11 “Inserted” refers to those employed or returned to formal education once in the year after completing or dropping out per the revised EF PIRS section 1.3. For the PE, an individual is considered “employed” if they worked for at least one hour during the last week (Instituto Nacional de Estadísticas standard) or the week prior to the COVID quarantine, including those self￾employed or who worked without pay in a family business. Return to formal education is only available in the six-month follow up as it refers to return to school or training after leaving EF. 91.0 91.0 91.2 90.9 90.6 89.3 90.1 90.9 91.6 75 80 85 90 95 Male Female Age 16-20 Age 21-25 Age 26-30 Primary 6, 7, 8th Primary 9th Secondary 10th, 11th Secondary 12th+ Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 17 formal education at any time during the 12-month period after completing or dropping out of the program. To answer a key question of this PE – whether participants had obtained new or improved employment – we analyzed data collected at baseline and at the six-month follow up. As shown in Figure 4, the percentage of individuals who were working or studying was significantly larger in the six-month follow up than at baseline (n=524). This result holds for both those who only completed basic training (67.2 percent) and for those who completed the technical training (69.3 percent). There is no significant difference between these two groups at follow up, which indicates that the type of training completed does not make a difference in the chance of having employment at the six-month follow up.12 (See Annex 4.) About one in three participants who left the program before completing basic training did so because they found a job or went back to school, which contributes to the higher percentage working among dropouts. (However, the completers significantly reported securing better jobs post program Figure 4). Figure 4: Percentage of participants employed or returned to school, by time of observation Note: Bars indicate 95% confidence intervals. To make these comparisons consistent, the analysis is restricted to the 524 individuals who enrolled from April-December 2019 that had a baseline and six-month follow up. 12 The percentage employed at baseline in this subgroup (25.7%) is not significantly different from those who do not have an endline (26.8%) or those who do not have a six-month follow up (31.5%), indicating that the results are not biased, at least in terms of the original employability. 26.5% 37.9% 25.1% 27.8% 69.1% 67.2% 69.3% 83.3% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% All completers (n=524) Completed basic training (n=58) Completed technical training (n=466) Dropouts (n=54) % employed Enrollment Six-month follow up Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 18 Nearly two-thirds of those who were unemployed when the program started were employed or back in school in the six-month follow up (40 percent had returned to school, 60 percent were employed). There was also some success in securing new and improved employment among those who started in informal or who were self-employed. About one quarter of those who were in informal employment or self-employed at baseline had moved to a formal position (including permanent and temporary), and another one in ten of those who were in informal employment or self-employed had returned to school. However, this result is similar to that of those who dropped out. Among program completers, women are significantly less likely than men to be employed or to have returned to school at the six-month follow up (76.4 percent versus 63.4 percent, p<0.05). This difference, however, disappears once one controls for age, education level, municipality, and psycho￾emotional protective factors.13 (At the national level, women are also less likely to be employed or in school than men: 89.7 percent of men versus 56.8 percent for women.)14 Figure 5 shows that those in the age group 26 to 30 had 53 percent lower odds of being employed at the six month follow up than those in the age group 16-20, whereas individuals with higher resilience levels were 24 percent more likely to be employed than those with lower resilience levels. 13 Protective factors are measured by the prevalence of Post-traumatic Stress Disorder (PTSD), Difficulties in Emotional Regulation (DERS), Adult Resilience Measure (ARM), and Community Cohesion and Efficacy (CCES). Section 5.1.2.2. provides more detail on protective factors. 14 These numbers, estimated using INE’s 2018 household survey, consider those who are not working for pay or looking for employment as not working. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 19 Figure 5: Explanatory factors affecting the probability of being employed at the six-month follow up Notes: The figure presents odds ratios and the 90% confidence interval. As the effects on odds ratios are multiplicative, effects larger than one increase the odds of being employed, while effects smaller than one decrease the odds. In the figure, significant effects (p<0.10) are marked with a black dot and are those where the confidence intervals do not cross the 1.0 value. The sample for this figure is individuals who completed basic or technical training, had a baseline and six-month follow up observation (n=354). Males, 16-20-year-olds, primary 6-8th grade, primary risk, and La Ceiba are the comparison group (whose odds of being employed are represented by the red line). Another important indicator is how participants perceived their employment conditions six months after leaving the program. As shown in Figure 6, participant perceptions depend on whether they completed the technical training or not. Nearly half of those who completed technical training and mentioned having engaged in some kind of economic activity after EF indicate that their employment was better after EF than before. This improvement is statistically significant when compared to those who did not complete technical training.15 15 Improvement in perceived quality of employment is important given the estimated 50 percent underemployment rate in Honduras in 2019 (Instituto Nacional de Estadística. Encuesta Permanante de Hogares de Propósitos Múltiples, 2019). Underemployment is estimated as the number of individuals working below the minimum wage and is often used as a proxy for quality of employment (https://repositorio.cepal.org/bitstream/ handle/11362/5341/1/S2011956_es.pdf). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 20 Figure 6: Participants perceived changes in their employment conditions between the six-month follow up and before entering EF, by program completion Note: *p<0.10. This question was asked in the six-month follow up VIP-RA to those who mentioned that they had worked after EF, including those who had not worked for pay (included in figure as “has not worked after”). When asked why they felt their employment conditions had improved, over half indicated that their pay had improved, nearly half reported that their new employment offered prospects for advancement, and one in five stated that the new job offered opportunities to learn more (multiple responses accepted, Figure 7.) Figure 7: Reasons for perceiving an improvement in employment conditions between before EF and the six-month follow up Note: Multiple responses accepted. N=140 participants who said that their employment conditions had improved between the beginning of EF and the six-month follow up. 5.1.1.5 ACCORDING TO THE VIP-RA CALIBRATION, DO THOSE IDENTIFIED AT SECONDARY AND TERTIARY RISK FINISH/DROPOUT OF THE PROGRAM AT THE SAME RATE AS PRIMARY RISK PARTICIPANTS? 23.5 21.2 43* 38.2 39.4 22.5 2.9 3.0 3.1 14.7 18.2 20.8 20.6 18.2 10.6 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Dropouts (n=34) Completed basic training (n=33) Completed technical training (n=293) Better The same Worse Had not worked before Has not worked after 20.0% 25.0% 28.6% 30.0% 33.6% 40.0% 48.6% 54.3% 0% 10% 20% 30% 40% 50% 60% Job is closer to youth's career goals Job is more stable Better benefits (i.e. insurance, vacation, etc.) Reduced or improved commute time Better or more convenient hours or work schedule Opportunities to study while employed Prospects for advancement Improved pay Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 21 Participants with secondary and tertiary risk of engaging in violent behaviors at the baseline had a higher probability of dropping out of the WFD Activity than those with primary risk (Figure 8). Nearly twice as many participants at the secondary risk level dropped out compared to those at primary risk. The dropout rate among those with tertiary risk was less than those at secondary risk, but given the small sample size of participants with tertiary risk (4 out of 53 participants with tertiary risk dropped out), it cannot be established if this result is statistically different from those with primary risk. Figure 8: Dropout rates by baseline risk level Note: *** p<0.001 5.1.2 SUB-QUESTION1.2 • Q1.2: How does the risk of violence and employability change after program completion, and six months after finishing the program? 5.1.2.1 WHAT IS THE PERCENTAGE OF INDIVIDUALS WITH IMPROVED EMPLOYABILITY SKILLS, AND WHAT IS THE VARIATION IN EMPLOYABILITY BY BENEFICIARY CHARACTERISTICS? EF participants were asked about their perception of their ability to perform employability skills such as money management, handle problems at work, use basic math, complete job application, communicate with potential employers, use a computer. Responses are compared here between baseline and endline to show change during enrollment, however these improvements were also sustained in the six-month follow up as shown in Annex 7.16 16 These questions on perception of ability are different from EF pre-post skill assessments. 7.5 14.1*** 11.8 0.0 5.0 10.0 15.0 20.0 25.0 30.0 Primary Secondary Tertiary Drop-out rate Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 22 Of the six employability skills, participants reported an average of 1.42 more skills between baseline and endline. The three skills with greatest improvement among all program completers at endline were ability to communicate with potential employers, fill out a job application, and use a computer. There was a notable difference between those that completed only basic training compared to those who complete basic and technical training: • Technical completers reported more improvement in the ability to communicate with potential employers than basic completers (33.5 and 27.3 percent respectively). • Basic completers reported more improvement in the ability to fill out a job application than technical completers (38.6 and 28.7 percent respectively). • Basic completers reported more improvement in the ability to use a computer than technical completers (25 and 22.3 percent respectively). Women, those with less education at baseline, those who completed only basic training, and younger participants, improved in more employability skills than men, those with more education at baseline, those who completed technical training, and older participants. The fact that those who only completed basic training improved more than those who completed technical training is not surprising given that the employability skills measured are basic labor competencies. The fact that those with more education and more training improved less is potentially due to their starting with more skills and/or believing they had the skills already at baseline and then learning they don’t have as much ability as they originally believed once provided additional training (for example, the starting levels of reported skills for those with a 12th grade education were higher than those who had not completed primary education). 5.1.2.2 WHAT IS THE PERCENTAGE OF INDIVIDUALS WITH A DECREASED RISK OF VIOLENCE, AND WHAT IS THE VARIATION OF THE VIOLENCE BY BENEFICIARY CHARACTERISTICS? The classification of participants who completed the program according to their initial risk level reveals a reduction in risk between baseline and endline for those who were initially classified as having elevated (secondary and tertiary) risk. About half of those in both secondary and tertiary risk at baseline moved to primary risk at endline, and nearly twenty percent of those in tertiary risk at baseline moved down to secondary risk (Figure10). The decrease in risk level is sustained and even more accentuated in the six￾month follow up, where over half of those who were originally in secondary risk moved to primary risk (Figure 10). Among those at tertiary risk at baseline, ninety percent decreased to secondary or primary risk at the six-month follow up (though it is worth noting that the tertiary group was very small at baseline). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 23 Figure 9: Change in risk level between baseline and six-month follow up, among those who completed the program Baseline risk Six-month follow up risk Primary Secondary Tertiary Primary (n=467) 79.1% 20.3% 0.6% Secondary (n=155) 54.3% 44.0% 1.7 % Tertiary (n=16) 50.0% 40.0% 10.0% Note: Sample sizes reflect the number of participants who completed the program and had data at both points in time and had enough information to estimate their risk level. The change between baseline and six-month follow up expressed as percentage of each baseline risk who ended in six-month follow-up risk level. In total, nearly six in ten participants at secondary or tertiary risk levels at baseline decreased their risk between baseline and the six-month follow up. A multivariate logistic regression (Annex 6) shows that: • The probability of decreasing risk level is lower for those in the 26-30 age group than among younger participants. • Initial levels of education levels and baseline protective factors (stress coping, emotional regulation, resilience, community cohesion) did not negatively affect the results of the program on risk level (primary, secondary, tertiary). See Annex 6 for additional analysis. • The probability of decreasing risk level is greater for those living in San Pedro Sula and Choloma than among those living in La Ceiba (most likely due to higher original risk scores in San Pedro Sula [0.19] and Choloma [0.18] than La Ceiba [0.14]). Risk in the VIP-RA risk assessment is based on past engagement in delinquent behavior using questions from the International Self-Report Delinquency (ISRD) study and dynamic psycho-emotional and social factors common in the protective factors literature. The ISRD study has been validated in over 30 countries globally. Questions range from property damage and petty theft to assault and drug use and sale. Overall, self-report measures show ‘a moderate and consistent correlation between police data on offenders known to the police and ISRD data on self-reported offending (for robbery, assault and theft)’.17 Protective factors are measured by the prevalence of Post-traumatic Stress Disorder (PTSD), Difficulties in Emotional Regulation (DERS), Adult Resilience Measure (ARM), and Community Cohesion and 17 Enzmann, D., I. H. Marshall, M. Killias, J. Junger-Tas, M. Steketee, and B. Gruszczynska. 2010. “Self-Reported Youth Delinquency in Europe and beyond: First Results of the Second International Self-Report Delinquency Study in the Context of Police and Victimization Data.” European Journal of Criminology 7 (2): 159–183. In Honduras, with almost complete impunity for violent crimes and widespread use of pre-trial detention, even official criminal records do not accurately reflect an individual’s past delinquency. Both sources, official and self-report, have their own sources of bias. Fears of reprisals even among friends for reporting the actions of the other are well founded and might lead to under-reporting. Mistrust in authority and outsiders runs deep, leading to potential under-reporting as well. Despite these issues, the self-report methodology is the best available means to get at whether or not an individual or his/her peers have been involved in delinquent behavior. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 24 Efficacy (CCES).18 Addressing these factors was not the primary focus of the WFD Activity, however the association between protective factors, completion and employment rates was a learning question. Changes in protective factors do indeed vary by program completion, sex, age and education (see Figure 35, Annex 8). Most importantly, as dynamic characteristics, one would expect to see some change among protective factors over time. In general, protective factors improved for both dropouts and those who completed the technical training (see Figure 37, Annex 8 for data): • For dropouts (n=54), post-traumatic stress and difficulties in emotional regulation improved, but community cohesion and efficacy worsened. • For basic training completers (n=58), only post-traumatic stress decreased between baseline and endline, but this effect was not sustained through the six-month follow up. • Technical training completers (n=466) showed the most consistent improvements: ◦ Between baseline and endline, post-traumatic stress decreased, and this change was sustained through the six-month follow up. ◦ Difficulties in emotional regulation decreased between baseline and endline, and between baseline and six-month follow up. ◦ Technical completers were the only group that, on average, significantly increased their resilience between baseline and the six-month follow up. • Protective factors at six-month follow up are not statistically associated with having obtained employment or not (see Annex 8). 5.1.3 SUB-QUESTION 1.3 • Q1.3: What are the perceptions of beneficiaries regarding people and services that helped them succeed? 5.1.3.1 IN THE OPINION OF PARTICIPANTS, WHAT DISTINGUISHED SUCCESSFUL MENTORS AND ASESORES LABORALES? Some of the participants interviewed were confused regarding the roles and titles of the EF contributors they interacted with. This information is important in itself, as it indicates that participants were not aware of the structure of the program, and the roles that different contributors played. It also suggests 18 All are measured in a 0 to 10 scale: an increase on the PTSD and DERS scale indicates more probability of PTSD and more difficulties with emotional regulation, which indicates a lack of protective factors, while increases in the ARM and CCES indicate that the protective factor increases. For more information on these factors, see the “Development and validation of the violence-involved persons risk assessment: Honduras” report submitted to USAID September 2017. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 25 that the opinions participants expressed regarding the mentors and labor market advisors should be read with caution, as they may refer to a different role. Of the 40 participants interviewed: • Eight participants were confused when identifying their mentor, and one said that they did not have a mentor. This response, from a male participant (undisclosed age, La Ceiba) illustrates the confusion regarding mentors (referring instead to the facilitators): “Bueno pues excelente, con las dos primeras excelente, hubo una que aparte de las dos, porque después de ellas vinieron otras que hubo una, no sé cómo se llama, pero era una negrita que era muy enojada.” • Seven participants said that they had not had a labor market advisor (“asesor laboral”). Another 11 of the 40 participants interviewed confused the role of the labor market advisor with that of the mentors or the facilitators that taught them about job skills. When asked to describe the support they had received from their labor market advisor, participants responded with tasks that were the responsibility of the facilitators or mentors. For example, 10 of the 40 participants said that the labor market advisor accompanied them through the process, listening to them and monitoring their progress (the role of the mentor); eight participants said that the labor market advisor trained on job skills (the role of the facilitator), seven said that they trained them on the skills needed for a job interview (the role of the facilitator). Only eight mentioned that labor market advisors were responsible for helping to find them job interviews. Of the 27 participants who answered to the question of what kind of support they received from their mentors: • 21 participants gave answers indicating that the mentor accompanied them and followed their progress closely, eight explained that the mentors motivated them, and four explained that the mentor was personally concerned about their wellbeing. • The same participants were later asked to describe what they found most useful or liked most of their relationship with their mentors (more than one answer was accepted). Their answers indicate that they particularly valued mentors who behaved professionally or had a good personality (16 answers), that accompanied and listened to them (12 answers), that they helped them to better themselves (10 answers) and that they motivated them (10 answers). Men found the accompaniment more useful than women, as 9 men mentioned this aspect, whereas only 3 women did. In the rest of the comments there were no gender differences. Figure 10 below summarizes the key characteristics that participants listed as remarkable or poor in their mentors. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 26 Figure 10: Characteristics of good and bad mentors as identified by participants Good mentors • Accompaniment • Motivation • Personal involvement • Professional, respectful • Listened to them • Helped them improve themselves • Created safe spaces Bad mentors • Intolerant • Not understanding women’s family restrictions • Did not create enough trust • Did not get too involved with participants The following quotes summarize the various positive qualities that participants identified in their mentors: Nos trataba muy bien, era una persona que escuchaba. (Female, 22 years old, Tegucigalpa) Pues él nos trataba de dar la confianza para que nosotros pudiéramos comentar tal vez lo que nos pasaba o como nos sentíamos en qué nos había servido esto y lo otro y pues sí darnos una palabra un apoyo allí. (Female, undisclosed age, San Pedro Sula) Sí porque, sí porque ella durante, sí, estoy viendo aquí, en cualquier momento, perdón, a cualquier momento que usted necesita, un momento para hablar o para agarrar aire, vengan y hablen conmigo y pues ella siempre nos acompaña en el camino. Male, 24 years old, La Ceiba) Tuve pues el acompañamiento verdad con Ericka, Ericka Martínez me parece que es el apellido, estuve presente en este proceso de Mentoría con ella, te puedo decir que fue una fuerza, una persona que es muy abierta al diálogo, tiene la capacidad de escucha y a partir de ello verdad, ella tiene su manera de poder acompañarlo a uno y de que uno no se sienta solo en el proceso verdad de Mentoría, entonces e incluso verdad me ayudó a crear ese proyecto de vida, ella siempre me preguntaba ¿usted como se ve de aquí a cinco años? fue como ¿qué metas tiene? ¿Qué va a ser para lograrlas? Entonces ella al estarlo escuchando a uno, ella como que le daba uno el aliento para poderlo lograr verdad, para poderlo lograr. (Male, undisclosed age, San Pedro Sula) Not all participants reported good experiences with their mentors. One woman (19 years old, San Pedro Sula) said that the program had not adequately understood her personal problems, even when she asked for help. The same participant complained that their mentor had bullied several of her classmates, who as a result abandoned the program. The following experience illustrates how mentors were sometimes unresponsive to women’s particular needs: Bueno yo tengo una hija, entonces a veces llegaba unos minutos tarde porque yo tenía quien me la cuidara verdad y siempre me tocaba ir a dejarla, entonces a veces llegaba 5 minutos después y él no me comprendía y entonces es ahí donde yo venía y me alteraba y entonces sí es cierto que más de alguna vez le falté el Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 27 respeto por lo mismo, porque yo me sentía ahogada, pero nunca dejaba de ir a la clase, si yo hablaba con él, que mire, que en ese momento estaba atravesando el problema, entonces quizás ahí fue cuando sentí que no tenía ese apoyo de ellos y no, y bueno quizás me entendieron. (Female, 26 years old, Tegucigalpa). When asked whether their mentors could have done more for them, most participants (27 of 40) expressed that they were completely satisfied, but five said that they wished their mentors had paid more attention to the participants, advised them more and giving them more time while three expressed that the mentors could have created an environment where participants expressed themselves more freely and felt more trust. 5.1.3.2 IN THE OPINION OF PARTICIPANTS, WHAT ARE THE KEY SERVICES THAT EF PROVIDED THAT HELP BENEFICIARIES SUCCEED? WHAT DID PARTICIPANTS FIND MOST USEFUL? Figure 11: EF's most useful results, as identified by participants When asked what they got out of the program, responses fall into four general categories (multiple responses were allowed): 1. Psycho-emotional resilience and coping skills (30 of 40 participants); including improving general communication skills (11), learning how to get along with others (9) and improved self￾confidence (8) 2. Job skills (20 of 40 participants); including technical skills (7) 3. Job search skills, including those necessary for interviewing (8) 4. Employment (4) An example of psycho-emotional resilience improvement is a woman in Choloma who expressed that she could not express herself before entering the program, but then, thanks to it, she and her classmates had left their shyness behind: Más que todo, a mí como persona la verdad no me podía expresar, nos daba pena hablar en público. Sí, la verdad que sí, pero fíjese que ellos nos daban la confianza para que nosotros nos expresemos, nos Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 28 desenvolvamos bien al momento de hablar y habíamos varias personas calladas, calladas, pero ahí ya después hasta hablamos y además queríamos más bien alargar el programa porque no nos queríamos salir, no queríamos que se terminara. (Female, Choloma, 26 years old). This example from a male participant in La Ceiba illustrates the job skills participants gained: Antes no tenía mucho conocimiento verdad de lo que era una empresa y cosas así y de las charlas que ellos me dieron me ayudaron bastante, nos explicaron cómo tratar al cliente, cómo tratar a las personas, cómo tratar de conversar con ellos, entonces eso me ayudó bastante. (Male, La Ceiba, unrecorded age). When asked what they found most useful, the tools to increase their psychoemotional resilience and job skills were again mentioned as the most common answers (by 24 and 16 of the 40 participants, respectively). The job skills that participants found especially useful were: 1. Training in client services (5) 2. Learning how to behave at work (4) 3. Technical training (3) Job search skills were also useful as five interviewees mentioned that EF had provided them with tools to conduct a job search, including presenting themselves in interviews and CVs. For instance, a male participant mentioned: “Con las capacidades que nos ayudó a desarrollar Empleando Futuros era prácticamente un éxito asegurado porque llegábamos a las entrevista de trabajo y a la mayoría pues cumplía con los requisitos y se quedaba trabajando, entonces desde ahí fue un gran beneficio.” (Male, Tegucigalpa, unrecorded age). Men and women differed in terms of what they found most useful of the program, as men were much more likely to mention benefits that had to do with improved job skills or improved abilities to do a job search (nine out of the 19 men mentioned job skills, whereas only five of the 21 women did; and seven men mentioned improved job search skills while only one woman did). On the other hand, women were more likely to mention psycho-emotional skills such as how to get along and work with others (eight out of the 21 women mentioned this, versus five out of the 19 men). See Figure 12 below. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 29 Figure 12: What participants find most useful of EF, by sex Note: Figure 12 shows the number of mentions, based on 21 female respondents and 19 male respondents. 5.1.4 SUB-QUESTION 1.4 • Q1.4: How do stakeholders perceive the quality of the WFD Activity? During the follow-up VIP-RA and employability surveys, participants were asked if they had any recommendations for EF. Most participants did not have any recommendations, saying that they had liked everything in the program. Of those who had recommendations, the greatest number said that EF should increase job offers to participants, making concrete suggestions such as linking with more employers or offering informal jobs in addition to formal ones. One in ten respondents made suggestions related to increasing the timing of the training, either in terms of the length or the hours offered. They also suggested expanding the topics covered, to areas such as English and computers. Others recommended improving the trainers and their training methods, so that they were more dynamic, but also more responsible and respectful of participants, and about the same number recommended having better training spaces, adding more classrooms, and moving to different locations. See Figure 13 below. 0 5 10 15 Job skills Job search and interview skills Psycho-emotional resilience Learned how to better get along with other Found a job What did you find most useful about EF? Female Male Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 30 Figure 13: Participant recommendations to improve EF Note: n=483 5.1.4.1 HOW DO PARTICIPANTS PERCEIVE THE DURATION AND USEFULNESS OF THE DIFFERENT PHASES? This question was addressed in the mid-term PE and the “phases” were not relevant for this cohort. However, at the end of the interview, participants were asked to give final comments and specific recommendations to the WFD Activity. These comments were not impacted by Covid-19 lockdowns as all participants had completed the WFD Activity at least three months prior to the lockdown. Of the 40 interviewees, 25 mentioned that they had a very good experience, but participants also identified two areas for improvement. The most common area for improvement (mentioned by 10 participants, five male and five female) was clarity in the services offered. Participants felt cheated because, when enrolling, they understood the WFD Activity would help them find employment or a job interview, and that had not happened.19 They also suggested that the program should be clearer with what they would and would not do for participants. The following quotes exemplify the shortcomings that participants found in this aspect, and some of their recommendations: Mire; los cursos son excelentes, no me quejo, no me quejo para nada, pero lo que sí me gustaría es que les dieran seguimiento a las personas, porque vaya, hasta el sol de hoy ya no me han vuelto a llamar, ni a mis compañeros de hecho, si tienen trabajo, porque a muchos ante esta situación (COVID19) han perdido su trabajo y nadie se ha reportado, entonces creo que eso sería lo más importante que le dieran seguimiento a la persona. (Male, 19 years old, Tegucigalpa). 19 The mid-term PE included a similar finding and recommendation for greater clarity during recruitment. 1.9% 2.3% 2.5% 3.3% 3.5% 5.0% 6.2% 6.7% 10.0% 15.6% 41% Improve the follow up with exbeneficiaries Improve transportation and meal support Improve the way participants are treated More flexible schedule/decrease hours Better organization/More discipline Other Better training spaces Better trainers/Better training methods Increase the time/trainings Increase job offers Liked everything, change nothing Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 31 Interviewer asks: ¿Y ellos no les hicieron cómo conseguir un trabajo o referir en una empresa? ¿No? Response: No, no, no, solamente un diploma que dieron pero digamos, eso era lo que verdad, nosotros yo creo que al final nosotros esperábamos, porque ese fue el ofrecimiento que nos dieron, yo en eso no tengo quejas porque me ha servido ¿verdad? lo que me enseñaron, pero también me hubiera gustado que me hubieran dado la oportunidad de un empleo. (Female, 22 years old, Tegucigalpa). No, sinceramente no, nosotros salimos de la capacitación en lo que fue septiembre, que teníamos pues como le digo, nosotros fue por parte de Diunsa que también nos iban a dar la oportunidad de trabajo para trabajar se suponía que tres meses, empezamos hasta en noviembre el cuatro, el treinta y uno de diciembre se nos terminó el contrato y pues en el grupo que teníamos de la capacitación todos empezamos a, porque nos dijeron que teníamos apoyo hasta, teníamos apoyo un año y pues hasta el día de hoy sinceramente no se ha visto el apoyo de ellos en lo que es lo que nos prometieron, de que era, de que nos iban a hacer todo lo posible por conseguir una estabilidad laboral. (Female, 29 years old, Tegucigalpa). The second most common recommendation (mentioned by five participants) had to do with extending the courses, the time that EF followed participants, and allowing participants to take other vocational training courses, if interested. That is clearly expressed in the opinions of this participant: Bueno, a mí me gustaría que no solo fuera una vez la oportunidad, que hubieran más oportunidades, para los que hemos ya salido como, como quien dice bueno yo ya, ya saqué cocina, púchica, pero me gustaría como sacar el curso para banquetes, me gustaría sacar el curso para, un curso de panadería, que hubiera como más chance, para poder seguirnos como quien dice relacionando más, que no se quede hasta ahí, porque como yo ya fui seleccionada ya no puedo seguir aplicando. (Female, undisclosed age, Tegucigalpa). Finally, three participants (two males and one female) suggested more flexibility in the schedule, as they had difficulties getting to the classes on time or had to leave early. One male participant even mentioned that he had to skip the whole life skills training because the schedule was not compatible with other obligations: A todo eso fue lo que casi no asistí como le digo, porque es que eso los daban por, como estábamos casi todo el día allí entonces eso lo daban por la mañana y lo demás, lo que es las horas prácticas nos lo daban en las tardes y como le digo por los mismos problemas personales yo no podía, se me hacía difícil asistir en la mañana que era lo de Habilidades para la vida y todo eso. (Male, undisclosed age, San Pedro Sula). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 32 5.2 LEARNING QUESTION 2 • Q2: To what extent do WFD actions produce a workforce with relevant skills? 5.2.1 SUB-QUESTION 2.1 • Q2.1: To what extent is the WFD Activity delivering job placement services to participants? 5.2.1.1 DID THOSE WHO COMPLETE THE PROGRAM CONSISTENTLY RECEIVE JOB PLACEMENT ACTIVITIES? One way to analyze job placement is the extent to which participants found employment in the same sector in which they were interested and trained. Using data from EF follow-up calls with participants who had completed technical training, were employed at the time of the calls, and who had been trained in services or manufacturing and textiles (n=573).20 As Figure 14 shows, most participants found employment in their sector of training, however one in five participants who had been trained in manufacturing and textiles were working in a services position, and about one in ten of those who had been trained in services were working in a textiles position. Figure 14: Type of enterprise of employment, by sector of training Note: Authors estimates using EFIS and EF job insertion data. Another way to analyze job placement is with responses from the 40 participant interviewees. Of those participants, 27 benefited from EF’s job placement services, either in the form of getting temporary jobs through EF, obtaining job interviews through EF, or from being referred to job fairs by EF. Of these, 23 found a job through these EF efforts (Figure 15). 20 5 participants had been trained in construction and technology. Given this limited sample, they were not included in the analysis. 20.4 90.9 78.6 8.7 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Manufacturing and textiles (n=103) Services (n=470) % employed in sector Sector of training Services Manufacturing and textiles Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 33 Figure 15: Employment search experience after EF Below are some examples of the different paths to employment participants followed: • Yo estoy trabajando prácticamente desde que terminamos el curso hubo una feria de empleos y ahí me pude conseguir un empleo, me dieron una entrevista y actualmente estoy trabajando. (Female, La Ceiba, Undisclosed age) • Gracias a Dios por Empleando Futuros verdad, entonces lo que le digo es que yo no terminé todo el programa porque entre medio del programa yo conseguí el trabajo, porque yo llevé una carta de recomendación de ustedes. (Male, 24 years old, La Ceiba) • A todos los que nos enseñaron, me ha sido muy útil, trabajé en Mendels y ahorita en Campero y todo lo he puesto en práctica lo que ellos me enseñaron. (Male, 24 years old, Tegucigalpa) Of the participants who did not receive employment placement through EF or who did not find a job through EF, 12 ultimately found a job on their own after completing EF and four had not actively looked for a job. Although some participants were frustrated with the lack of direct support from EF in the job search process, these participants recognize that the training they received from EF had been very useful in their own job search process, either by improving their job search and interview skills, by improving their entrepreneurial abilities, by providing them with concrete job skills, or because of the recognition the program has among certain employers. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 34 These testimonials exemplify their experiences: • (When asked how what she learned on the program helped him find a job) Bueno la verdad yo digo que sí, porque bueno yo estuve trabajando ahí en ¿cómo es que se llama? hice la entrevista con él y empezamos hablar, pero yo digo que si también por que uno se sabe expresar más, no hay que tener miedo, siempre mirarlo a los ojos a las personas, cosas así pues, yo digo que sí. (Male, 26 years old, La Ceiba). • (When asked what of what she had learned from the program had helped her) Bueno en la parte del currículum últimamente hice un currículum y lo fui a dejar a Walmart porque vi que estaban pidiendo impulsadoras, entonces ahí pude ir, pude recabar esa información que tenía, de cómo hacer un currículum y todo eso y lo pude, lo pude hacer gracias a Dios y poderlo meter a Walmart, y de lo de atención al cliente estaba trabajando en una clínica de vacunación de aquí de la Ceiba, entonces en papelería, recepción prácticamente es todo atención al cliente. (Female, undisclosed age, La Ceiba). • Cuando entre en Empleando futuro ya tenía mi negocio verdad, pero no lo tenía poco avanzado, no tenía, no sabía un poquito, si lo tenía solo así, pero no sabía que era tener un negocio, plan de negocio, plan de marketing, todo eso no lo sabía, que era segmento de mercado, proveedores, todo eso no lo sabía, entonces hubo un tema que se trató de emprendimiento, se trató de esa de emprendimiento, la maestra nos explicó, que es eso, plan de marketing, profesores, quienes son nuestros segmentos de mercado, quienes son nuestros nichos y aprendí bastante, ya después como administrar nuestro dinero, hacer tanto y tanto y aprendí, ahora tengo anotado en mi cuaderno, un cuaderno de libreta tengo anotado donde pongo todas mi ganancias, las de en un mes, las ganancias de un mes y todo, todo me enseñaron ahí, el tema de, la tutora Isabel me enseño todo eso, como hacer cada paso, todo eso. (Choloma, undisclosed age, male). One challenge participants encountered after finding employment was that most jobs were temporary. Of the 35 participants who had worked at least once between program completion and the interview, only 18 were still working. This challenge was the same for those who found employment on their own (seven of the 12 who were still working when we interviewed them), as well as for those who found employment through EF (11 of the 23 who had found employment through EF were still working when we interviewed them). The experience a participant who found employment in Diunsa via EF is representative of other cases: Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 35 • Por parte de Diunsa que también nos iban a dar la oportunidad de trabajo para trabajar se suponía que tres meses, empezamos hasta en noviembre el cuatro, el treinta y uno de diciembre se nos terminó el contrato. (Female, 29 years old, Tegucigalpa). Temporary work experience could be beneficial in finding a new job. However, short-term employment has not proved useful for some of them when looking for permanent employment. The following experience is an example of this: • (When asked why she has not found a job) Pues yo creo que es la experiencia porque ahora en cualquier trabajo dicen; tantos años de experiencia, entonces y mientras tanto yo solo tuve, ya mi primer trabajo pero solo fueron dos meses entonces creo que es eso. (Female, 22 years old, La Ceiba). 5.2.1.2 ARE THERE DIFFERENCES IN THE RECEPTION OF JOB PLACEMENT ACTIVITIES BETWEEN PARTICIPANTS WITH DIFFERENT CHARACTERISTICS (SEX, AGE, SECTOR)? There were no differences in receipt of job placement activities between males and females, and the selection of the qualitative interviews does not allow for analysis based on age. The probability of being employed in the six month follow up did not depend on the sector of employment for which participants completed training, as just over two-thirds of those trained in each sector were employed at the time of the follow up. 5.2.2 SUB-QUESTION 2.2 • Q2.2: What are the perceptions among service providers about the services of the WFD Activity? 5.2.2.1 OF THE SUPPORT THAT EF PROVIDES, WHAT HAS BEEN MOST USEFUL TO THEM? There were eight service provider organizations who were the implementing partners of the WFD Activity and provided training, mentorship and/or job placement services to the participants. We interviewed 18 representatives from seven service provider organizations to determine how their organization had benefited from participation in the WFD Activity. The three most common responses were (multiple responses accepted): 1. An increase in activities or improvement in the quality of their services (nine interviewees) 2. Organization became better organized and trained (nine interviewees) 3. Organization gained strategic alliances (eight interviewees) Examples of benefits that service providers gained through the WFD Activity include budget management, expanding to new geographical areas, assistance to be able to deliver activities that the Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 36 organizations already had in place such as life skills, and extending their services to link participants with the labor market. The following two testimonials summarize these perceptions: • Son dos, como (organization) ya teníamos unos, más de 10 años de tener lo que es una bolsa de empleo verdad, pero entonces ahora pues el proyecto Empleando a futuro lo que hizo es que se fortaleció el servicio de bolsa de empleo, nosotros como (blanked organization) siempre estábamos enviando personal a las empresas y ese personal que nosotros enviábamos, teníamos personas que se nos regresaban verdad ¿porque? Porque no pasaban las pruebas básicas, las pruebas de conocimientos básicos que era de Matemática, Español, pruebas psicométricas básicas y entonces cuando ya nosotros adoptamos el programa y le damos reforzamiento en estas áreas entonces teníamos mejores resultados con nuestros jóvenes y nosotros podíamos insertarlos también. (Coordinator of EF, Organization in Choloma). • Logramos fortalecimiento institucional verdad, logramos un fortalecimiento porque siento que como parte del proceso, verdad, nuestra oferta formativa aumentó, aumentamos también la capacidad de los chicos beneficiarios del proceso y esto también dio un paso a que nuestro personal, eh, fuese capacitado por ejemplo, proceso de capacitación, eh, también se implicó en compra de materiales, equipos, herramientas, eh, porque hubieron áreas donde ya contábamos con un equipo pero hubieron áreas que se abastecieron de equipos y herramientas.(Inaudible) pues tuvimos la oportunidad de abrir una gama en nuevas empresas para la, la cartelera de empresas aumentó porque eso permitió ir conociendo nuevas, nuevas empresas, nuevos mercados…. También, otro de los beneficios que podemos ser partícipes, de, de procesos de mejora verdad, del cual nosotros le denominamos el sistema de mejora continua, eh, ¿en qué consiste esto?, pues consistía en que, en cada cosa que hacíamos el proyecto nos solicitaba, era como un, una estrategia pero al mismo tiempo era un compromiso que nosotros, toda cosa que hacíamos teníamos que sistematizarlas, teníamos que documentarla, teníamos que evidenciar y eso nos fue permitiendo tener también como aprendizajes. (Technical coordinator of formative programs, organization in Tegucigalpa). When asked about what they found most useful, service providers mentioned (multiple responses permitted): 1. The training provided to their technical staff (10 of 18 interviewees) 2. New methodologies learned (10) 3. Training materials received (10) Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 37 4. Accompaniment and the constant documentation and evaluation by EF (4) In terms of the components of the WFD Activity that service providers found particularly relevant, the two most mentioned were life skills (5 interviewees) and mentorship (5 interviewees). The perceptions regarding these components vary across organizations according to their specialization and the components of the program they delivered. For example, the coordinator of youth programs in an organization in Tegucigalpa that originally specialized in vocational training found the training on client services very useful because it is necessary for a lot of occupations. He also thought that the behavior and cognitive components were important, but he regretted that they did not implement it consistently. In his words: Todo para mí muy importante lo del servicio al cliente así es, porque se da bastante lo de las ventas entonces para nosotros incluir eso porque antes no se incluía en la primera, en el primer corte que nosotros tuvimos. Entonces yo considero que servicio al cliente y cognitivo conductual verdad que lastimosamente se dio hasta el final muy bien y ya teníamos un gran porcentaje de jóvenes insertados laboralmente que al final iban los que podían ir porque tal vez hoy tenían su día libre o algo, pero si realmente hubiéramos dado, es algo que, yo creo que podríamos mejorar verdad. (Coordinator of youth programs, Tegucigalpa). Meanwhile, the executive director of an organization in Choloma that specializes in job placements recognized that they did not provide the mentorship as mandated because they had to adapt to their time and needs: Nos quedamos con todos los manuales, las metodologías, el tema de mentoría si lo cambiamos un poco, porque realmente no lo está pasando, bueno nos encantaría tener un mentor a tiempo completo porque realmente mirábamos mucho valor en eso, pero realmente hemos hecho como una versión de mentoría grupal, para trabajar exactamente el plan de acción a corto, mediano y largo plazo del joven verdad, que nos lo permite la metodología trabajar. (Executive Director, organization in Choloma). On the other end of the spectrum, the supervisor of an organization in Tegucigalpa that worked on integral youth programs found the life skills component, which they implemented even before the WFD Activity, as the most useful: Hay una parte de formación que se le llama fase 1 que era la parte de habilidades para la vida, eso ya es un proceso que ya viene desde ya días entonces la implementación de ese proceso.” (Supervisor, organization in Tegucigalpa). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 38 5.2.2.2 WHAT ARE THE POSSIBILITIES THAT SERVICE PROVIDERS CONTINUE IMPLEMENTING THE SERVICES THAT THEY PROVIDE WITH EF AFTER THIS ACTIVITY ENDS? To assess the possibilities of continuing the implementation of components of the WFD Activity model after the activity ends, the service providers interviewed were asked three questions: what lessons they had learned from EF, whether they had any plans to put these lessons into practice, and to provide an example of how their organization is implementing these lessons. The answers to these questions show that organizations gained resources that may help them implement similar programs in the future and allow them to continue their operations: 1. Service providers have better capabilities and methodologies (14 of 18 interviewees) 2. Service providers have better experience (8) 3. Service providers now have better methods to teach (5) 4. Now recognize need to be flexible when implementing these kind of activities (2) 5. New alliances with other organizations (2) 6. New managerial and evaluation skills (3) Of the 18 interviewees, 15 were asked directly whether they had plans to put these lessons into practice in the future. With the exception of one interviewee who was not working in the organization anymore, all answered that they did. Their responses, however, indicate that the components and the way they are implementing the components varies, as Figure 16 below shows. Figure 16: Plans for implementing the lessons learned from EF in the organizations interviewed 0 1 2 3 4 5 6 Actively seeking manners to continue using EF components Using EF components in other activities Will use personal experience in EF in other projects Kept the evaluation and continuous improvement component Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 39 Of the six providers who mentioned that their organization is actively seeking ways to implement the lessons learned from the WFD Activity, three explained that they are talking to other potential donors and partners and proposing an adaptation of the model. An additional two explained that they are thinking of ways in which their organization could use some components of the model developed by EF in future activities. Three interviewees highlighted that they asked EF to transfer all materials and methodologies. The following quotes illustrate these different plans: • (Interviewer asked whether they have plans for implementing the lessons learned in the future) Sí, inclusive ahorita estamos en negociaciones con Empleando Futuros para que ellos nos pasen todo el material, las metodologías y estamos en negociaciones también y al mismo tiempo con otros organismos internacionales para ver si ellos sistematizan el proyecto y que nos quede a nosotros ya como institucionalizado. (EF coordinator, organization in Tegucigalpa). • De hecho, pensamos que eventualmente nosotros pudiéramos vender este servicio a las empresas, capacitar a sus colaboradores, el tema de las habilidades blandas es algo que no lo recibimos muchas veces en un sistema educativo formal y que con esta, con estos conocimientos que han adquirido nuestros facilitadores podemos pues colaborar para cerrar esas brechas. (EF technical officer, organization in Choloma). One third of service providers indicated that their organization is already implementing some components of the WFD Activity in other programs, including: 1. Changing the way in which they connect their beneficiaries to potential employers 2. Keeping the evaluation and continuous improvement methodology they learned from EF 3. Adapting the soft skills components of EF to other youth programs. Some examples of this continued implementation include: • Ahorita tenemos algo que se llama Unidad técnica de Formación e inserción, ya no solo es inserción pura como la hacíamos antes de los buscadores con ofertas, si no que ahora hemos implementado todo el modelo que piloteábamos con Empleando Futuro como respuesta a las empresas. (Executive Director, organization in Choloma). • Sí de hecho, parte de lo que yo le mencionaba del sistema de mejora continua, verdad, surge del otro acceso que desarrollamos con el proyecto y que hoy en día estamos aplicando pues. (Coordinator of technical formation, organization in Tegucigalpa). • Como le dije soy el coordinador de un convenio de la AECID, que es Cooperación española, donde me encargo de lo que es desarrollo juvenil y emprendimiento, en donde se están desarrollando ya dos proyectos de empleabilidad juvenil y desarrollo de negocios para jóvenes, entonces hay ciertas Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 40 cosas que nosotros estamos utilizando de Empleando Futuros aquí. (Coordinator of Youth development programs, organization in Tegucigalpa). Service providers were also asked what challenges they had faced when implementing the WFD Activity and how they had solved them. Interviewees identified challenges in two broad areas: working with vulnerable youth and organizational difficulties associated with implementing new activities and coordinating across institutions. Although not explicitly linked to their plans for continued implementation, the challenges they identified may threaten the sustainability of the activities, particularly if they were to continue the focus on vulnerable youth. In terms of the challenges that service providers found when working with vulnerable youth, nine of the 18 providers indicated that they had difficulties recruiting participants, either because the requirements of the program were difficult to meet (e.g. finding enough men), or because unemployed and out of school youth were not interested in training programs. The coordinator of youth programs in an organization in Tegucigalpa expressed this: • Para nuestra organización básicamente lo que fue la parte de reclutamiento de jóvenes... Porque, por el tipo de población, como es una población acomodada por decirlo así que son los niños, que tienen un estilo de vida bien arraigado de no estar haciendo nada y están cómodos, entonces poder convencer a esa población y más las otras características que quería el proyecto, ese fue el mayor reto que nosotros. • Another provider, an advisor from an organization in Choloma expressed, “Primero es la búsqueda del joven, el joven todavía tiene la perspectiva prácticamente de querer un empleo, pero prácticamente con la idea de no encontrarlo también verdad entonces.” Additionally, six of the 18 service providers explained the challenge of keeping participants in the program and accompanying them through the process, because the participants had little interest in lengthy programs and prioritized finding a job quickly. This can be seen in the statements from an advisor in Choloma and a supervisor in Tegucigalpa: • Porque ellos lo que andan buscando es algo más rápido, es decir yo ando buscando un trabajo, porque teníamos nosotros cuando llegamos a las comunidades buscar al chico, para ver si lo hacían; tenemos este programa pero tenés que capacitarte entonces decían; No, no, ahorita no puedo porque yo necesito trabajar, necesito llevar dinero a la casa y es todo, pero nosotros le estábamos dando una buena opción, le estamos dando de que no vas a andar solo buscando un empleo, vas a Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 41 tener un aleado con nosotros o teníamos jóvenes que entraban al programa que a los días desertaban también. (Advisor, organization in Choloma). • (Entre las dificultades) Cambiar esa mentalidad del joven que se dedicara completamente al trabajo o al estudio. (Supervisor, organization in Tegucigalpa). Another difficulty associated with the target population mentioned by five of the 18 service providers was that it had been problematic to find jobs for the participants because there was a prevailing stigma against youth from their background. This is explained by the EF coordinator in an organization in Tegucigalpa: • (When speaking of the challenges they faced) Crear esa apertura de parte del sector empresarial para que le generara una oportunidad laboral a cada uno de los chicos y chicas considerando que venían de zona de altos niveles de violencia, quitar algunos estigmas de parte del sector empresa hacia esa población, romper esquemas de la población porque al inicio nosotros creíamos que todo se iba ser una mentira. (EF coordinator, organization in Tegucigalpa). In terms of the organizational challenges, eight of the 18 service providers mentioned the planning and implementation of new activities, and three service providers mentioned the coordination across different actors. The advisor of an organization in La Ceiba indicated the difficulties of coordinating across organizations with the following words: • Había fase uno, fase dos fase tres, del proyecto entonces la fase uno fue manejada por otras organizaciones en ese caso eran dos organizaciones, y entonces yo tuve la oportunidad y la bendición de estar en la primera fase también de cuando ya llegue, ya llega la segunda fase me di cuenta realmente que sin duda alguna hubiera sido muy bueno de que una sola organización manejara todo ese proceso, de implementación en sí de manera general de todo el proyecto tanto de parte uno, fase dos, ya por qué, porque los jóvenes primero reciben una formación con una x organización esa organización aunque haya una sola metodología aunque haya todo pero cada individuo cada profesional es diferente verdad, entonces cuando ya llegan a la fase dos, no es que a mí me enseñaron que la hoja de vida era de esta manera o a mí me enseñaron a hacer esto de esta manera verdad. (Coordinator, organization in La Ceiba). The EF coordinator in an organization in Tegucigalpa elaborated further, explaining that working with different organizations implied that they had different priorities and did not always teach the youth what they perceived was important: Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 42 • Otro de los retos que para mí fue grande es que como la formación humana …no es formación humana, las habilidades para el empleo o para el éxito laboral se les daba de inicio con otra organización y la parte de CLB, no, la de formación, de habilidades para el éxito laboral, que es por lo general habilidades para el éxito laboral es con juegos y en las otras organizaciones no les daban el valor de someterse a horarios, de someterse a normas, ya cuando entraron a la organización, para ellos era bien difícil que entraran donde nosotros ya con la formación técnica donde sí se da valor al tiempo, al respeto de reglas, a la hora en que llegaban, o sea ya hay normas que van de acuerdo con lo que en realidad ellos se van a encontrar en el mundo del trabajo y a algunos jóvenes no les gustaba eso. (EF coordinator, organization in Tegucigalpa). Figure 17 depicts the interaction between the lessons learned, challenges, and plans for sustained implementation. Figure 17: Interaction between lessons learned, challenges, and plans for sustained implementation 5.2.3 SUB-QUESTION 2.3 • Q2.3: What are the perceptions among employers about the services of the WFD Activity? 5.2.3.1 HOW SATISFIED ARE EMPLOYERS WITH THE EMPLOYEES THAT WERE TRAINED IN EF? Thirteen of the 17 employers interviewed expressed satisfaction with the experience they had employing EF graduates, mainly because they had a good attitude and other soft skills that characterized them as good workers. Eight employers mentioned the importance of a positive attitude and soft skills, Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 43 and eight of employers also mentioned one or more attitudinal advantages of EF graduates. The positive attributes that employers found in EF graduates include that they are willing to work hard, engaged, committed, adaptable, and have good training (see Figure 18 for additional attributes). The following comments show this, both among employers in large and small enterprises: • Le voy a ser muy sincero, la verdad que todos han demostrado pues lo que hemos buscado verdad, la eficiencia, la facilidad, han sido muy buenos sinceramente, y más que todo se mira la diferencia porque hemos tenido, de todos los que hemos tenido, hablando de gente fuera, contratados que no sean del proyecto, que a veces hemos contratado ya personas de cierta edad, de repente 38, 40, 45 años y se les mira la diferencia verdad, entonces nos hemos enfocado más en contratar personal joven, por la misma situación porque se les mira más, mayor eficiencia, más movilidad, entonces de los que hemos tenido, de los cinco que hemos tenido, estas últimas dos personas son las que, son las que más satisfechos nos hemos sentido la verdad. (Small food production company in La Ceiba, less than 10 employees). • Pues muy bien, la verdad que, la gente pues, viene con esa actitud, con esa actitud, no tanto con la, con las aptitudes pero sí con la actitud positiva de querer un lugar de trabajo entonces eso les ayuda a ellos a tener una, un desempeño donde ellos preguntan, donde ellos quieren conocer, donde adquieren como le diría como no desaprovechar la oportunidad entonces tratan de dar la milla a extra, y lo digo porque, en base a opinión de los jefes que hemos tenido pues, ellos dicen, no, realmente la persona, es alguien que no se queja, es alguien que se ve que está enamorado de lo que hace, pero nosotros hemos pensado que es porque en realidad quieren esa oportunidad y al tenerla no la quieren desaprovechar. (Home appliances store, national, more than 1,000 employees). Figure 18: Most common words used by employers to describe the experience with EF graduates When praising the soft skills of EF graduates, employers recognized that the EF training had been very useful, making youth more adaptable and responsible. The quotes below, which mirror those of 11 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 44 employees who explicitly praised EF soft skills training, show that employers sometimes have an original negative perception of the youth, and that the program helped them change some of those stereotypes. • Fíjese que le diré que yo prefiero a este grupo Empleando Futuro porque son personas que llegan ya como le digo, ya filtradas, que llevan ese deseo de trabajar y hay personas que nosotros traemos de afuera que hay que hacer lo que hay que ver actitudes, si hay gente que llega, ¿me entiende? Y esta gente ya está capacitada a lo que va. Como le digo hay una profesora que les inculca esta disciplina porque ella ya conoce qué es lo que (enterprise) quiere de estos jóvenes, entonces esta gente ya llega con disciplinados, llega con buenas maneras de trabajar en la parte que no tienen buena actitud y con buena actitud ellos y son honestos, honrados, son disciplinados y eso es lo bueno de Empleando Futuro, porque vienen ya ellos filtrados a nuestra empresa. (Manufacturing, Tegucigalpa, more than 500 employers). • Yo sí creo que tal vez estos muchachos por el hecho de ir a través del proyecto, tienen un poquito de mayor conciencia, nosotros contratamos personal de afuera que solo leen rótulo y llegan y estas personas pareciera como que fueran un poco menos responsables, así que los, que nos han llegado del proyecto hemos notado que tal vez porque llegan con la influencia del proyecto, a través del proyecto y por el proceso que tuvieron tienden a ser más estables y a permanecer más tiempo. (Furniture manufacturing enterprise, approximately 200 employees, La Ceiba). Despite the praise for EF graduates among employers, four employers also found some shortcomings in the training, especially in terms of a mismatch between the technical training and the specific skills required for the job. The following comments show that, despite finding many positive traits in EF graduates, employers felt that they still had to provide additional practical, on-the-job training the graduates. • Nosotros aplicamos una encuesta en, de seguimiento con las alianzas que hemos tenido y todos los jefes de área comparten que son jóvenes que brindan o tienen muy buena actitud, compromiso, quizás lo único que les hace falta es la experiencia técnica, pero de esa parte ya nos encargamos nosotros, ya nosotros debemos capacitarlos a como operar una máquina o algún proceso pero de lo contrario son personas bien comprometidas y responsables.” • The same employer later explained: “Al inicio cuando nosotros estuvimos recién, recibiendo candidatos, quizás con la demora nosotros mirábamos que eran personas que no eran capacitadas, que eran personas que tenían la base, porque USAID está aliada con la cámara de comercio de Choloma, pero venían por parte de USAID y mirábamos la diferencia de cuando solo nos mandaban electricistas a cuando eran capacitados como electricistas entonces, yo les decía a ellos, Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 45 mándenos electricistas y nos los mandaban y se descartaban en el proceso porque no estaban capacitados. (Manufacturing company, San Pedro Sula, more than 1,000 employees). When asked about what may make EF more useful to their firm: • Primero que todo que las prácticas que ellos hicieran fueran en máquinas de coser industriales (al parecer la empresa opera digitalmente su equipo) y otra de las situaciones es también prepararlos o sea para cuando ya estuvieran en el ámbito laboral… Para el segundo grupo, para asegurarlos que profesionalmente que ellos pues, porque lo que queríamos era ayudarles a que tuvieran trabajo verdad, entonces necesitábamos un poquito de reforzamiento en esa área, en el área de costura, en el área de usar las máquinas, también le voy a decir que es bien difícil porque ellos pasan pegados a las máquinas, solo se levantan a comer por poco tiempo y luego otra vez en la máquina y ellos son muchachos jóvenes, entonces también el hecho de seguir reglas, ayudarles un poquito a entender cuáles son las responsabilidades y objetivos y también los reglamentos de la compañía verdad. (Manufacturing company, Choloma, 1,500 employees). The suggestions to the program of three participants and two service providers support these employers’ opinion that technical training could be complemented with more practical, updated practices. For instance, one participant who received training to repair motorcycles suggested that they be given live practices, with real motorcycles instead of using photographs and books “explicarle así pieza por pieza porque solo así por fotos es más difícil, así cuesta más aprender” (male, San Pedro Sula), while another participant explicitly requested more practical training “Es que mucha teoría y creo que en esa área es más práctica que teoría porque la teoría uno puede verla en cualquier lugar pero lo práctico no, entonces creo que deberían haber hecho más cosas prácticas que teoría”. In terms of the comments from the service providers regarding the suggestion to update the technical training, the following quote is very illustrative: “Todavía tenemos feedback de parte de las empresas que hay que mejorar el tema de formación técnica, que la oferta de los Institutos, que los colegios no responde a lo que las empresas están requiriendo”. Finally, four employers expressed that the neighborhoods from which EF graduates come are a problem because they must commute during daylight hours for security reasons, and two employers explained that they had experienced attitude problems with some EF graduates. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 46 5.2.3.2 WHAT DETERMINES HOW MANY YOUTHS TRAINED BY EF ARE HIRED? Of the 17 employers interviewed, eight said that the number of EF graduates they hire has increased in the last year, while eight said they had not increased (and one employer did not answer this question). Figure 19: Main reasons for stagnated or increased hiring of EF graduates Stagnated hiring • EF stopped contacting/no more graduates • Limited hiring capability • Company requirements stricter Increased hiring • Positive past experiences • New training areas A comment from the representative from a medium-sized (less than 200 employers) company that manufactures furniture in La Ceiba illustrates the shared feeling that EF was not contacting the employer as much: No, no ha crecido, yo más bien sentí últimamente que el proyecto decreció, cayó, dejaron de visitarnos, dejaron, la comunicación se cayó y creo que no sería que no tienen presupuesto, pero se dejó de fortalecer el proyecto y ya no tuvimos más muchachos llegando. On the other hand, the main reason provided for increasing the number of EF employees (given by the eight employers who increased their hiring) was the good previous experience. Two employers explained that the good performance of past EF hires led them to consider them as their first option when they needed to employ new workers, and one said that they even worked with EF to develop new training areas. The following comments describe these experiences: • Sí por el buen desempeño de ellos, porque nosotros por ejemplo al salir a alguien de la empresa, ya sea por los motivos que sea, nosotros tenemos una plantilla de reemplazo y la plantilla de reemplazo pues igual teníamos, contábamos con jóvenes del, eran jóvenes del proyecto, a los cuales nosotros ya teníamos una evaluación previa a los jefes y ellos dicen contratar plantilla de reemplazo entonces nosotros ya teníamos, tenemos nuestra base de datos. (Home appliances retail, national, more than 1000 employees). • Ya había más expansión de joven en cuanto a los requerimientos que nosotros solicitábamos, entonces el jefe cuando lo contrataba miraba de que sí, eh, sí era lo que esperábamos. (Manufacturing, Tegucigalpa, more than 1000 employees). • Fíjese que en este período puede ser que tal vez ellos al inicio algunos verdad puedan buscar un trabajo permanente de inicio. Que tal vez, que tal vez ellos de inicio anden buscando eso, pero nosotros la verdad, hemos tenido muy buenos resultados en jóvenes tal vez, pues hubo por ejemplo en La Ceiba entraron 29 jóvenes al proyecto y ninguno desertó y fue un proyecto nuevo en La Ceiba por ejemplo, ninguno desertó, se les explicó; miren el proyecto consiste en esto y esto y ninguno Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 47 desertó, todos terminaron y más bien nosotros estábamos como, como preocupados por decirlo así, o porque los jóvenes que en ese momento teníamos 26 plazas para temporales entonces íbamos a tener tres como en una plantilla de reemplazo, pero resulta que hubo un incremento de plantillas y los pudimos contratar a todos de forma temporal, entonces la verdad que el proyecto en sí es muy bueno, los jóvenes saben de qué si ellos se esfuerzan, ponen mucho interés al proceso de formación y dan buenos resultados a las empresas van a, van a tener éxito. (Home appliances retail, national, more than 1000 employees). 5.3 ADDITIONAL LEARNING QUESTIONS – MIGRATION While the original aim of the WFD Activity was not to reduce migration to the U.S., the PE included some migration questions to improve learning on youth and migration. The sample size was small (n=344 at baseline and follow-up) and questions did not separate between regular and irregular migration, however the learning that came out of this initiative includes an analysis of the association between migration and employment, employability, and risk of violence.21 We address those associations in turn below. 5.3.1 SUB-QUESTION 3.1 • Past attempts to migrate Participants were asked whether they had attempted to migrate to the US within the last year. We present the answers given at baseline, which are similar to the responses given at endline and six-month follow up (see Annex 9).22 Overall, the proportion of participants who reported that they had attempted to migrate to the U.S. within the last year decreased over time from 8.9 percent in FY2019- Q3 to 6 percent in FY2019-Q4, and 4.6 percent in FY2020-Q1. However, these differences are not statistically significant. Participants who did not complete primary education were significantly more likely to have attempted to migrate to the US than those who had completed the 12th grade (19.3% vs 3.1%, p<0.05, see Annex 9). Similarly, participants who were in the primary risk category were significantly less likely to have 21 Migration questions were added to the baseline in April 2019 and to the endline in July 2019. Given this timing, the analysis is based on 806 individuals who answered migration questions in the baseline survey, and 344 who answered migration questions at both baseline and the six-month follow up. Data from the endline are not used, as only 115 participants had migration data at all three points in time. 22 No statistical differences were found in the responses given at baseline, endline and six-month follow up. We also did not find differences at baseline between those who dropped out of the program, those who completed basic training, and those who completed technical training. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 48 attempted to migrate than those who were in the secondary risk category (4.4% vs. 11.2%). A regression indicates that these differences are also present when other socio-demographic characteristics are controlled (see Figure 20). Having secondary education decreases the odds of having attempted to migrate compared to only having primary education while being at secondary risk of violence involvement increases the odds of past migration attempts compared to being at primary risk. Figure 20: Sociodemographic factors associated with the odds of having attempted to migrate to the US in the last year at baseline Notes: The figure presents odds ratios and the 90% confidence interval. As the effects on odds ratios are multiplicative, effects larger than one increase the odds of having attempted to migrate, while effects smaller than one decrease the odds. In the figure, significant effects (p<0.10) are marked with a black dot and are those where the confidence intervals do not cross the 1.0 value, represented by the red line. N=749 participants observed at baseline answering all relevant questions. The comparison group is males, 16 to 20 years old, those with up to 8th grade of primary education, and who are categorized as being in primary risk level. 5.3.2 SUB-QUESTION 3.2 • Migration intentions Overall, the percentage of participants who said that they had the intention to migrate to the U.S. in the next three years substantially decreased during the PE period. (Note that over this same period, US Customs and Border Protection (CBP) apprehensions at the southwest border also declined significantly.) Participants who enrolled in FY2019-Q3 were significantly more likely to express intent to migrate than participants who enrolled in FY2019-Q4 and FY2020-Q1 (Figure 21). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 49 Figure 21: Percentage of participants who at baseline said that they had the intent to migrate to the US, by period of enrollment Note: Results based on 806 participants observed at baseline answering relevant questions. Figure 22 shows the percentage of participants who said they had the intention to migrate to the U.S. in the next three years, by program completion and time of response. Participants who dropped out of the program and those who only completed basic training were more likely to have migration intentions at the time of baseline than those who completed technical training. At the six-month follow up, migration intentions had significantly (p<0.10) decreased for dropouts and basic training completers, but not for technical training completers (who had much lower migration intentions than overall trends to begin with).23 Figure 22: Percentage of participants with the intention to migrate to the US, by program completion and time of response 23 Again, this mirrors migration rates nationally, which declined beginning May 2019 and suggests an overall decline in migration intentions that is also reflected in these data. 35.8% 19.9% 12.5% 0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0% FY2019-Q3 (n=159) FY2019-Q4 (n=559) FY2020-Q1 (n=88) %with migration intentions Percentage of enrollment 53.1% 48.3% 12.7% 18.8% 20.7% 15.2% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% Dropouts (n=32) Completed basic training (n=29) Completed technical training (n=283) Baseline Six month follow up Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 50 Note: Figure based on participants who responded to migration questions both at baseline and the six-month follow up. Given the small sample size, 90% confidence intervals are presented. Migration intentions decreased by 45 percent (from 67 to 37 participants) between baseline and six￾month follow up for those participants who originally intended to migrate. Exploratory, not statistically significant analysis suggests that the probability of change from having migration intentions in baseline to having no migration intentions in the six-month follow up was better for women, older participants, those who obtained employment, those who at the six month follow up perceive that they have a good future in Honduras, and those who increased their resilience and their perception of community cohesion and efficacy (see Annex 9, figures 44 and 45 for data). Lack of employment was the most common reason for having migration intentions at both baseline and six-month follow up (Figure 23). Nearly twice as many participants mentioned not having a job as the reason for their migration intentions at baseline than any other reason. Concerns about security became a close second reason to migrate at follow up. Over a quarter of participants at that time said that, even when they were employed at the time of the interview, they could earn more in the U.S. Figure 23: Reasons for wanting to migrate in the next three years, by time of observation Note: Percentages do not add to 100% because multiple responses were accepted. Figure refers to participants who said they had migration intentions at baseline, and at the six-month follow up. There are no statistical differences between men and women in the reasons they give for having migration intentions (see Annex 9). Potentially due to the small sample size compared to other studies that focus on migration, migration intentions are not significantly associated with sex, age, education, program completion, employment, and most psycho-emotional protective factors at baseline or six-month follow up (se Annex 9, figure 46). However, past migration attempts increase probability of migration intentions and perception of a good future in Honduras decreases migration intentions at both baseline and six-month follow up. At 53.1 28.5 11.2 20.7 23.5 57.0 47.5 27.7 21.8 13.9 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 I don't have a job I want to live in a safer place Even when I have a job, I'd earn more there To study Family reunification % of participants who have migration intentions Baseline (n=179) Six month follow up (n=101) Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 51 the six-month follow up, difficulty with emotional regulation also increases the odds of having migration intentions. 5.3.3 SUB-QUESTION 3.3 • Perception of good future in Honduras Most participants agree or completely agree with the statement that there is a good future for them in Honduras. There are no significant differences in their responses between baseline and the six￾month follow up (Figure 24). Figure 24: Perception of a good future in Honduras among program completers, at baseline and six￾month follow up Note: Figure refers to 312 participants who answered the migration questions at baseline and the six-month follow up and who had completed baseline or technical training. Men and women have similar perceptions of a good future in Honduras, with the exception that women become less optimistic between baseline and the six-month follow up. For men, there are no significant differences between baseline and the six-month follow up in any of the responses. In the case of women, however, between baseline and the six-month follow up the percentage who answered that they were in agreement increased, while the percentage that indicated complete agreement decreased. Complete disagreement Disagree Agree Complete agreement Baseline 5.8% 21.5% 46.5% 26.3% Six month follow up 5.8% 19.9% 54.2% 20.2% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 52 Figure 25: Perception of a good future in Honduras among program completers, at baseline and six￾month follow up, by sex Note: 90% Confidence Interval (CI) shown, p<0.10. As with those who have migration intentions, the economic situation, including the lack of employment, was the most common reason for not seeing a good future in Honduras (Figure 26). The second and third most common reasons at both times were corruption and violence, including delinquency. The percentage of participants who mentioned gangs and domestic violence as an explanation to not expect a good future in Honduras decreased significantly at follow up. Figure 26: Reasons provided for disagreeing with the idea of a good future in Honduras, at baseline and six-month follow up Note: Figure refers to participants who said that they disagreed or disagreed completely with the idea of a good future in Honduras. Multiple answers were accepted. Male Complete disagreem ent Male Disagree Male Agree Male Complete agreement Female Complete disagreem ent Female Disagree Female Agree Female Complete agreement Baseline 4.4% 21.3% 51.5% 22.8% 6.8% 21.6% 42.6% 29.0% Six-month follow up 5.9% 19.9% 52.2% 22.1% 5.7% 19.9% 55.7% 18.8% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% Economic situation Violence Gangs Domestic violence Political situation Corruption Baseline (n=102) 87.1% 68.3% 63.4% 19.8% 60.4% 71.3% Six month follow up (n=173) 89.0% 59.0% 35.8% 2.9% 49.7% 59.5% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0% Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 53 Perception of a good future at the six-month follow up decreases with age, as participants in the 26- to 30-year age group have lower odds of having such a positive opinion than those in the 16- to 20-year group (Figure 27). Participants who finished technical training are 176% more likely to perceive a good future in Honduras than dropouts. Sex, education level, and being employed at the time of the survey do not have an effect on perception of a good future in Honduras. Figure 27: Factors associated with agreeing that there is a good future for the participant in Honduras, at six-month follow up Notes: The figure presents odds ratios (p<.05) and the 95% confidence interval. As the effects on odds ratios are multiplicative, effects larger than one increase the odds of agreeing that there is a good future, while effects smaller than one decrease the odds. In the figure, significant effects are marked with a black dot, and are those where the confidence intervals do not cross the 1.0 value, represented by a red line. N= 537 participants who answered migration questions the six-month follow up. Male, primary up to 8th grade, primary risk, unemployed and dropout are reference categories. 6 DISCUSSION Some findings referenced here and throughout the report use dropouts (n=54), basic training completers (n=58), and technical training completers (n=466) for comparison purposes only when results are statistically significant. When comparisons are made, those comparisons are only suggestive of difference/no difference between groups given that there was no way to control for potential selection bias. Pre-post results without a comparison group should also be interpreted with the normal caution for performance evaluations, as it is not possible to attribute changes to the WFD Activity alone since no impact evaluation was undertaken. The adjustments made during the course of the WFD Activity (decreased time-to-completion and integrated and customized models) resulted in greater participant retention in the program from mid- Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 54 term PE to final PE. Service providers (IPs) are also incorporating components (employer engagement, evaluation and learning methods, and soft skills) of the WFD Activity into other programs or have plans for doing so, which is important uptake in the journey to self-reliance. Two thirds of youth participants reported receiving job placement services, so this element of the Activity was not provided as consistently as intended. And while many participants received job￾placement services (calls with labor advisors, notices of job fairs, invitations to interviews), only a small fraction of those resulted in (mostly temporary) employment. Despite this inconsistency in direct job placement support, participants noted that the training they received from the WFD Activity improved their job search and interview skills, improved their entrepreneurial abilities, and provided them with a recognizable certification that aided in their own job search. While the WFD Activity faced challenges in finding employment for youth directly, the WFD Activity appears to aid youth in finding their own employment. This is especially important given that one challenge participants encountered after finding employment was that most jobs were temporary, and they found themselves looking for work and interviewing multiple times whether or not the WFD Activity placed them in employment or not. The key outcomes of the WFD Activity for participants therefore are increases in employability, job search skills, “soft skills” and protective factors (resilience, emotional regulation, positive attitude/confidence, and interpersonal skills). Participants, service providers and employers cite those outcomes as key to completing the program, attaining employment, maintaining employment, and seeking new employment. Employers also cite the need to provide additional practical, on-the-job training in enterprise-specific technical skills no matter past technical training, which limits the impact of technical training (with the exception of the employer-specific specialized model that has challenges of scale and replicability). Resilience, emotional regulation, and employment also decrease migration intentions and increase perception of a good future in Honduras among those surveyed. Based on these findings, the theory of change for a future program could be expressed as follows: If participants receive soft skills training and job placement assistance (in the Honduras labor market context), then they will: a) Attain multiple short-term jobs and have the soft skills to manage transitions. OR b) Attain permanent employment and have the soft skills to retain that employment. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 55 Either path will result in lower migration intentions and/or seeing a better future in Honduras. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 56 ANNEX 1. SURVEY SAMPLE DEMOGRAPHICS The three samples - dropouts, completers of basic training, and those who complete technical training - are comparable. Figure 28 shows that the socio-demographic characteristics at baseline of those dropouts, completers of basic training, and those who complete technical training are similar. Figure 28: Socio-demographic characteristics of sample at baseline Variable Dropped out Completed basic training Completed technical training Male (%) 45.5% 41.7% 45.8% Age 16-20 49.6% 56.8% 50.2% Age 21-25 35.8% 29.5% 35.8% Age 26-30 14.6% 13.7% 14.0% 6, 7 and 8th year of primary school 13.1% 14.0% 10.4% 9th year of primary school 13.1% 14.0% 11.5% 10th and 11th year of secondary school 18.9% 21.3% 18.1% 12th grade and more 54.9% 50.7% 60.0% The endline and six-month follow up samples for dropouts and those who only completed basic training are also not significantly different in terms of sex, age, or education level from those who originally enrolled (Figure 29). In the case of those who completed technical training, however, those in the endline sample were slightly less educated than those who originally enrolled and had a baseline VIP-RA (14.4% had an education level of 6th, 7th or 8th grade, versus 10.4% among those with a baseline VIP-RA, p<0.010), whereas those who answered to the six-month follow up were slightly older (17.2% in the 25 to 30 age group, as opposed to 14.0% in the baseline VIP-RA, p<0.0.10), and more educated (66.9% with 12th grade or more, versus 60.0% in the baseline). Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 57 Figure 29: Socio-demographic characteristics of sample at endline and six-month follow up. Program completion and modality Variable Baseline VIP-RA Endline Six-month follow up Dropped out Male (%) 45.5% 40.7% Age 16-20 49.6% 37.0% Age 21-25 35.8% 50.0% Age 26-30 14.6% 13.0% 6, 7 and 8th year of primary school 13.1% 9.3% 9th year of primary school 13.1% 13.0% 10th and 11th year of secondary school 18.9% 16.7% 12th grade and more 54.9% 61.1% Primary risk 56.6% 66.0% Secondary risk 41.5% 30.0% Tertiary risk 1.9% 4.0% Has migration intentions 53.1% 18.5% Completed basic training Male (%) 41.7% 41.9% 44.8% Age 16-20 56.8% 53.8% 46.5% Age 21-25 29.5% 32.7% 34.5% Age 26-30 13.7% 13.5% 19.0% 6, 7 and 8th year of primary school 14.0% 14.7% 5.3% 9th year of primary school 14.0% 11.8% 12.3% 10th and 11th year of secondary school 21.3% 20.6% 21.5% 12th grade and more 50.7% 52.9% 51.4% Primary risk 70.2% 62.9% 55.4% Secondary risk 23.4% 31.4% 37.5% Tertiary risk 6.4% 5.7% 7.1% Has migration intentions 48.3% 26.% 20.7% Completed technical training Male (%) 45.8% 43.2% 43.8% Age 16-20 50.2% 51.3% 41.2%* Age 21-25 35.8% 34.2% 41.6% Age 26-30 14.0% 14.5% 17.2%* 6, 7 and 8th year of primary school 10.4% 14.4%* 6.2%* 9th year of primary school 11.5% 13.2% 8.8% 10th and 11th year of secondary school 18.1% 18.9% 18.1% 12th grade and more 60.0% 53.4% 66.9%* Primary risk 71.7% 78.8% 74.4% Secondary risk 25.8% 19.1% 24.7% Tertiary risk 2.5% 2.1% 0.9% Has migration intentions 12.7% 12.1% 17.0% Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 58 ANNEX 2. LIMITATIONS Findings referenced throughout the report use dropouts (n=54), basic training completers (n=58), and technical training completers (n=466) for comparison purposes only. There was no control group as a part of this performance evaluation, and results should be interpreted as suggestive of difference/no difference between groups given that there was no way to control for potential selection bias nor to overcome the small sample size for dropouts and basic training completers. The only exception to this is when statistical significance is indicated, meaning the difference was large enough to detect despite the often-small sample size. Given that there is no single source that could be used to answer all the questions posed in this report, we draw from different, complementary, data sources. This causes sample numbers to differ between questions and figures in the following manner: a) Data on enrollment, dropout, completion, and employment are from EFIS and EF surveys. b) WFD Activity implementers collected data at baseline and endline, and MESCLA conducted follow up surveys. Analysis of characteristics of participants at baseline refers to all participants who responded to the VIP-RA and Employability survey within three weeks of enrollment. Comparison between baseline, endline and the six-month follow up, however, depend on those who responded in two or three observations. This decreases the sample sizes. Dropouts did not respond to an endline, and the six-month follow up was only applied to a statistical sample of all participants. c) Response numbers in some of the analyses may be smaller because participants may not have answered all questions necessary to calculate risk scores and protective factors. In each of the analyses we try to specify, as clearly as possible, the data used and the population it refers to. There are also several issues and limitations about data to consider that arose during the PE. Key among them are the following: • Due to COVID-19, rather than applying the survey in face-to-face interviews or with the assistance of an enumerator, respondents filled in the survey online. KIIs were also completed telephonically with youth participants, service providers and employers. The response rate was lower than anticipated, and the team was only able to reach 71% of the target sample size in the survey, and 88% of planned qualitative interviews. There may have also been an unknown impact Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 59 on responses to the survey given the difference in data collection techniques between baseline and follow up. • Per agreement between MESCLA and the WFD Activity, the VIP-RA was initially collected at intake or during the first three weeks of beneficiary enrollment as a baseline. However, MESCLA learned during the mid-term PE that WFD Activity implementers had been administering the VIP-RA during the fifth week of enrollment. Late application of the VIP-RA limits the ability to identify change given that there is no real baseline and that application during week five of a ten week program provides little time for meaningful change to occur (especially since early activities such as Cognitive Behavioral Therapy (CBT) are expected to have the greatest impact on risk), or may already be affected by changes that occurred between enrollment and the fifth week. • Migration questions were added to the baseline in April 2019 and to the endline in July 2019. Given this timing, the analysis is based on 806 individuals who answered migration questions in the baseline survey, and 344 who answered migration questions at both baseline and the six￾month follow up. Data from the endline are not used, as only 115 participants had migration data at all three points in time. • The six-month follow up was conducted between March and April 2020 among participants that enrolled in the program between April and December 2019. On average, they had completed the program six months prior to the six-month follow up. However, some had enrolled up to a year before, while others had enrolled only three months before. Locating WFD participants several months after they had concluded the program was a challenge and required the use of a replacement list to achieve the necessary sample size. This challenge was due to changed contact information, migration, and a reported unwillingness to engage with the WFD Activity again. Methodological limitations include: • This is a PE, not an impact evaluation. In some sections of the report we compare the situation of WFD Activity participants at baseline, endline and approximately six months after completion. We provide these as evidence of results, but in a strict sense, we cannot know if the changes observed were a result of the program, or if they were a product of the natural evolution of participants over time. In order to know this, an impact evaluation would be necessary. • The results are not representative of the change in all WFD Activity participants, but only of those who answered the VIP-RA and Employability Survey. This is due to the following: ◦ Not all WFD Activity participants were interviewed at the beginning of the program. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 60 ◦ Only a fraction of those participants who had an endline VIP-RA were interviewed for a follow up. This threatens the conclusions that can be achieved from the comparisons of participants situation over time in several ways: • The participants who had a follow-up interview can be selective towards having better results given that: ◦ They were more accessible and willing to be in touch with the WFD Activity, which may also be indicative of higher satisfaction. This is particularly true for those who had dropped out of the program or were willing to continue with the program but had not been called back. ◦ Given that the VIP-RA and employability survey asks about sensitive topics, participants may have lied in their some of their answers, especially when interviewed at the beginning of the WFD Activity, when they were not familiar with the program and hence did not trust EF. If their lying decreased over time, we may be reporting results that underestimate the true positive effect of the program or that even show a move towards riskier attitudes. There is no way we can assess the size of this effect. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 61 ANNEX 3. KII GUIDES Guía 1. Guía para las entrevistas con los exparticipantes de Empleando Futuros (Q.1.5). Datos generales de la entrevista Fecha: _____________________________ Hora: __:___ Tipo de exparticipante: [no preguntar, ver muestra] ___ Terminó la capacitación básica. No siguió a la capacitación especializada ___ Terminó la capacitación técnica en el modelo integrado ___ Terminó la capacitación técnica en el modelo especializado Sexo: ____ Hombre _____ Mujer _______ Edad: ____ Años Escolaridad completada: ______ Grado _____ Nivel Municipio: ____ Tegucigalpa _____ San Pedro Sula ____ Choloma _____ La Ceiba Nombre del/la entrevistador/a: _______________________________________ Presentación Buenos días. Mi nombre es ____________________ y trabajo en ______________. Como le explicamos al contactarlo, estamos haciendo un estudio sobre Empleando Futuros, el proyecto que busca enseñar habilidades para mejorar sus oportunidades de empleo en el que usted participó. Como parte de este estudio, estamos conversando con algunas personas que han participado en el programa. La conversación va a durar aproximadamente 1 hora. Durante ella, vamos a platicar de su experiencia en Empleando Futuros y de lo que pasó una vez que terminó el programa. Para guiar la conversación, le voy a hacer algunas preguntas, pero lo más importante es que usted me cuente de su experiencia con toda libertad. Lo que usted me diga se va a analizar en conjunto con otras entrevistas similares que haremos y se va a utilizar para escribir un reporte. Ninguna de las opiniones y experiencias que me comparta se van a vincular a usted en particular o a su nombre, pues en el reporte no se va a listar a los entrevistados o a identificar por su nombre. Durante la conversación voy a tomar notas, pero me gustaría también grabar nuestra conversación para poder concentrarme lo más posible en nuestra plática y para, al integrar su experiencia con la de otros entrevistados, poder ser fiel a lo que me diga. ¿Tengo su autorización para grabar la entrevista? ____ Sí ____ No Cuándo y por qué se matriculó en el programa Para empezar a hablar sobre Empleando Futuros, me gustaría que me contara: • ¿Cuándo se matriculó en el Programa? • ¿Por qué se matriculó en el Programa? Posibles preguntas de seguimiento: ✔ ¿De qué forma se enteró del programa? ✔ ¿Qué le llamó la atención de Empleando Futuros? Beneficios de EF • En su experiencia, ¿participar en Empleando Futuros le dejó algún beneficio? ¿Cuáles? Preguntas de seguimiento: ✔ ¿Puede darme un ejemplo de cómo ha usado lo que aprendió en Empleando Futuros? ✔ ¿Algo cambió en su vida después de que terminó su participación en Empleando Futuros? ¿Qué? Componentes que más le sirvieron de EF • De todos los servicios y temas que recibió de Empleando Futuros, ¿hay alguno que le pareció más útil y por qué? [Sólo mencionar si no se acuerdan: los componentes incluyen habilidades de vida, Competencias Laborales Básicas, formación técnica básica, formación técnica en una ocupación especializada, mentoría, asesoría laboral] ◦ Por favor deme un ejemplo de cómo ha usado las habilidades que obtuvo de ese componente en su vida. Papel y características de los mentores • ¿Qué tipo de acompañamiento recibió de su mentor/a durante su formación en Empleando Futuros? • ¿Más o menos cuantas horas por semana pasó con su mentor/a?, [¿Si no se acuerdan, pregunta “más o menos cuantas veces estuvo con su mentor?”] • ¿Qué es lo que le pareció más útil de su relación con el/la mentor/a con el que trabajó? Por favor deme un ejemplo concreto. • ¿Pudo, en su opinión, haber hecho algo el/la mentor/a que le sirviera más para mejorar sus oportunidades de aprovechar las enseñanzas de Empleando Futuros? ¿Qué? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 62 Papel y características de los asesores laborales • ¿Qué tipo de acompañamiento recibió del/la asesor/a laboral con el que trabajó durante su formación en Empleando Futuros? • ¿Qué es lo que le pareció más útil de su relación con el/la Asesor/a Laboral con el que trabajó? Por favor deme un ejemplo concreto. • ¿Pudo, en su opinión, haber hecho algo el/la Asesor/a Laboral que le sirviera más para mejorar sus oportunidades de aprovechar las enseñanzas de Empleando Futuros? ¿Qué? Experiencia de trabajo previa y después de EF • ¿Había trabajado alguna vez antes de entrar a Empleando Futuros? ¿En qué y cuándo? [Seguimiento: ¿Trabajo remunerado? ¿Para algún familiar? etc.] • Después de haber terminado su formación en Empleando Futuros, ¿ha tenido algún trabajo? Por favor cuénteme todos los trabajos que ha tenido después de salir de Empleando Futuros. [Detalle la duración y el tipo de trabajo] ◦ En caso afirmativo: ¿Le sirvió Empleando Futuros para poder conseguir esos trabajos? ¿Cómo? Por favor deme un ejemplo concreto. ◦ ¿Le sirvió Empleando Futuros para poder desempeñarse bien en esos trabajos? ¿Cómo? ¿Por favor deme un ejemplo concreto? ◦ En caso negativo: ¿Por qué no ha podido encontrar un trabajo? ¿Qué problemas ha tenido para hacerlo? ¿Qué podría haber hecho Empleando Futuros para ayudarle a superar estos obstáculos? ◦ Después de salir de Empleando Futuros, ¿ha regresado a la escuela? ¿a qué grado y cuándo? Conclusiones Esas son todas las preguntas que tengo para usted. • ¿Hay algo más que le gustaría compartir de su experiencia en Empleando Futuros que no me haya dicho? • ¿Tiene preguntas para mí? Muchas gracias por su tiempo. Me gustó mucho platicar con usted y sus opiniones van a ser muy útiles para el Programa. Guía 2. Guía de entrevista con prestadores de servicios (Q2.2). Datos generales de la entrevista Fecha: __________________________ Hora: __:___ Tipo de entrevistado: Prestador de servicios Organización: __________________________________ Municipio: ____________________________ Nombre del/la entrevistador/a: _______________________________ Introducción Buenos días. Mi nombre es _________________ y trabajo en _________________. Como le expliqué al solicitar esta entrevista, estamos haciendo un estudio sobre el programa Empleando Futuros. Como parte de este estudio, estamos conversando con algunas personas que han colaborado con el Programa. Esta entrevista va a durar cuando mucho 1 hora. Durante ella, vamos a platicar de su experiencia con Empleando Futuros. Me gustaría grabar nuestra conversación, porque no quiero perder detalle de sus comentarios. Al escribir nuestro reporte, es posible que citemos algunas de sus respuestas sin hacer referencia a su nombre, organización o posición concreta. ¿Tengo su autorización para grabar la entrevista? ___ Sí ___ No Presentación – papel en EF Para comenzar, le pediría que me diga: • Su nombre y posición. • Sabemos que su organización participó en la capacitación de jóvenes para el programa Empleando Futuros recientemente. ¿Cuál fue su papel específico durante esta colaboración? Logros y retos operativos Pensando en la experiencia de su organización con el programa Empleando Futuros, • ¿Qué ganó su organización al participar en Empleando Futuros? • ¿Cuáles fueron los mayores retos, para su organización? • ¿Qué hicieron para superar estos retos? • ¿Cuáles fueron los componentes o servicios más útiles del Programa para su organización? Logros y retos para los beneficiarios Y en términos de los resultados para los beneficiarios, • ¿Cuáles fueron los mayores logros del Programa para los beneficiarios? • ¿Cuáles fueron los mayores retos para los beneficiarios? • ¿Hicieron algo para superar estos retos? ¿Qué? Retroalimentación de los empleadores Durante la experiencia en Empleando Futuros: Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 63 ¿Se enteró, a través de comentarios directos de los empleadores, por los asesores laborales o los jóvenes, qué opinaban los empleadores respecto a los jóvenes entrenados / contratados? ¿Qué le dijeron? Enseñanzas Para la organización en que trabaja, • ¿Les dejó alguna enseñanza participar en Empleando Futuros? ¿cuáles? • ¿Tiene planes para poner estas enseñanzas en práctica en el futuro? ¿Cúales y cómo? Por favor deme al menos un ejemplo concreto. [Trate de obtener más de un ejemplo y detalles de cúal componente va a continuar y como lo van a realizar, incluyendo financiamiento.] • ¿Volvería a participar en un programa similar en el futuro? Conclusiones Esas son todas las preguntas que tengo para usted. • ¿Hay algo más que le gustaría compartir de su experiencia en Empleando Futuros que no haya dicho? • ¿Tiene preguntas para mí? Muchas gracias por su tiempo. Me gustó mucho platicar con usted y sus opiniones van a ser muy útiles para el Programa. Guía 3. Guía de entrevista con empleadores (Q2.3). Datos generales de la entrevista Fecha: ___________________________ Hora: ____:____ Tipo de entrevistado: Empleador Modelo: ___Integrado ___Especializado [no preguntar, ver muestra] Empresa: ___________________________________ Municipio: ________________________________ Nombre del/la entrevistador/a: ___________________________ Introducción Buenos días. Mi nombre es ________________ y trabajo en _________________. Como le expliqué al solicitar esta entrevista, estamos haciendo un estudio sobre Empleando Futuros, el Programa para mejorar las habilidades de trabajo de los jóvenes como los que usted ha empleado. Como parte de este estudio, estamos conversando con algunas personas que han colaborado con el Programa. Esta entrevista va a durar aproximadamente 1 hora. Durante ella, vamos a platicar de su experiencia empleando egresados de Empleando Futuros. Voy a grabar nuestra conversación, porque no quiero perder detalle de sus comentarios. Al escribir nuestro reporte, es posible que citemos algunas de sus respuestas sin hacer referencia a su nombre, empresa o posición concreta. ¿Tengo su autorización para grabar la entrevista? ___ Sí ___ No Características de la empresa Para comenzar, le pediría que me diga su nombre, organización en la que trabaja y su puesto. • ¿A qué se dedica la empresa en la que trabaja? • ¿Cuántos empleados tienen? • ¿De qué forma se enteró del programa? ¿Por qué decidieron participar? Nivel del informante • ¿Qué relación tuvo o qué papel jugó su empresa con Empleando Futuros [ej. Sólo empleador, participaron en reuniones, etc.]? ¿Por cuánto tiempo jugó este papel? Presencia de trabajadores jóvenes • En esta empresa, ¿típicamente emplean jóvenes de entre 16 y 30 años? ¿qué puestos desempeñan estos jóvenes, comúnmente? • ¿Del total de empleado aproximadamente cuantos están entre 16 y 30 años de edad? • ¿Qué capacidades busca en los jóvenes que contrata? Origen e interés de la empresa de colaborar (participar en) con Empleando Futuros • ¿Hace cuánto tiempo su empresa colabora (participa) con Empleando Futuros? • ¿Cómo surgió la colaboración de su empresa con Empleando Futuros? • A su empresa, ¿qué fue lo que la motivó a colaborar con Empleando Futuros? Experiencia en la colaboración • ¿Cuántos jóvenes de Empleando Futuros han contratado en su empresa? • ¿Cuántos de los contratados de Empleando Futuro siguen trabajando actualmente? • En términos de su desempeño, ¿cuál ha sido su experiencia con los jóvenes de Empleado Futuros que han trabajado en su empresa? ◦ ¿Cuáles son las mayores ventajas que encuentra en los egresados de EF respecto a otros jóvenes que haya contratado? Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 64 ◦ ¿Y las mayores limitaciones? • ¿En caso de que haya contratado jóvenes de Empleando Futuros que ya no trabajan en la empresa, cuáles han sido las razones para su salida? (En caso de que tengan problemas para contestar la pregunta, pregunte por razones concretas, como ¿Nuevas oportunidades para ellos? ¿Dificultades de desempeño? ¿Otros?) • ¿En general, como se comparan en términos de capacidades y desempeño los jóvenes contratados de Empleando Futuros con jóvenes contratados de otra forma? Factores asociados al número de jóvenes de EF que contrata • ¿Ha crecido el número de jóvenes de EF que ha contratado en el último año? • ¿Qué explica este crecimiento? ¿O qué explica que no haya contratado más jóvenes de EF? • ¿Mirando al futuro, si tuviera que contratar más jóvenes daría prioridad a los de Empleando Futuro? ¿Por qué sí o por qué no? Oportunidades de Mejora • ¿Qué aspectos habría que cambiar para hace más provechosa para su empresa la participación en Empleando Futuros o en programas similares? Conclusiones Esas son todas las preguntas que tengo para usted. • ¿Hay algo más que le gustaría compartir de su experiencia en Empleando Futuros que no haya dicho? • ¿Tiene preguntas para mí? Muchas gracias por su tiempo. Me gustó mucho platicar con usted y sus opiniones van a ser muy útiles para el Programa. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 65 ANNEX 4. EMPLOYABILITY BY TYPE OF TECHNICAL TRAINING Figure 30: Percentage employed at the baseline, endline and six-month follow up, by type of technical training Type of training Baseline Endline Six-month follow up A la medida (n=83) 24.1 16.9 69.9 Técnica básica (n=65) 17.2 14.1 65.6 Técnica especializada (n=140) 25.7 31.4 65.0 Note: The analysis is restricted to the 578 participants who had a six-month follow up survey. ANNEX 5. EMPLOYABILITY SCORE IMPROVEMENT Figure 31: Percentage of participants who completed the program, with improved employability scores between baseline, endline and six-month follow up Notes: Results are based on 524 participants who completed the program (those who completed at least basic training) and had a baseline, endline and six-month follow up observation, and 54 participants who dropped out before completing basic training and have a six-month follow up. A participant is considered to have improved an employability score if their answer in the endline or six-month follow survey denotes that s/he feels confident more often (on a never, sometimes, always scale) for a particular ability. Those who said that they “Did not know” whether they had the ability were considered to be between “Never” and “Sometimes”. 19.39 22.12 20.61 30.21 32.52 22.56 27.00 23.34 30.89 29.57 31.76 23.60 36.54 30.77 40.38 36.54 32.69 28.85 Can manage their money Can handle problems at work Can use basic mathematics to solve problems at work Can fill a job application Can communicate with potential employers Can use a computer All completers endline All completers 6 months Dropouts 6 months Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 66 ANNEX 6. EFFECT OF SOCIODEMOGRAPHIC CONDITIONS ON RISK LEVEL Figure 32: Explanatory factors of the probability of decreasing risk levels between baseline and the six-month follow up, according to multivariate logistic regression Notes: The dependent variable takes the value of 0 when participants increased or did not change their risk level between baseline and the six-month follow up, and the value of 1 when they decreased their risk level (red line). The bars indicate 90% confidence intervals. Results are based on 147 participants who were secondary or tertiary risk at baseline and had a six-month follow up. Figure 33: Effect of sociodemographic conditions on the probability of decreasing the risk level, based on a logistic multivariate regression that omits the effect of psycho-emotional protective factors Basic training Technical training Female 16-20 (ref) 21-25 26-30 Primary, 9th completed Secondary 10th and 11th Secondary 12th and more La Ceiba (ref) San Pedro Sula Choloma Tegucigalpa -2 -1 0 1 2 3 4 5 6 7 8 9 Dropouts R2=0.1029 n=147 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 67 ANNEX 7. DETAIL OF EMPLOYABILITY IMPROVEMENTS Figure 34: Percentage of participants who reported an improvement in employability skills between baseline, endline and six-month follow, by program completion. Time of observation and type of participant Skill All completers endline Basic training endline Technical training endline All completers 6 months Dropouts 6 months Basic training 6 months Technical training 6 months Can manage their money 19.39 25.00 18.60 27.00 36.54 36.21 24.78 Can handle problems at work 22.12 25.00 21.75 23.34 30.77 22.41 22.63 Can use basic mathematics to solve problems at work 20.61 20.45 20.35 30.89 40.38 29.82 29.96 Can fill a job application 30.21 38.64 28.67 29.57 36.54 39.66 27.53 Can communicate with potential employers 32.52 27.27 33.45 31.76 32.69 43.10 30.24 Can use a computer 22.56 25.00 22.26 23.60 28.85 29.31 22.29 Note: Results are based on 524 participants who completed the program (those who completed at least basic training) and had a baseline, endline and six-month follow up observation, and 54 participants who dropped out before completing basic training and have a six-month follow up. A participant is considered to have improved an employability score if their answer in the endline or six-month follow survey denotes that s/he feels more confident with a particular ability. Those who said that they “Did not know” whether they had the ability were considered to be between “Never” and “Sometimes”. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 68 ANNEX 8. ADDITIONAL PROTECTIVE FACTOR ANALYSIS Controlling for other socioeconomic factors, those who completed technical training had less difficulty with emotional regulation and more resilience, but perceive less community cohesion and efficacy than those who dropout and those who only complete basic training. This indicates a selective enrollment and continuation in technical components of the program based on protective factors. Figure 35 shows that, in terms of the effect with other characteristics, women have more difficulty with emotional regulation and less resilience than men. Post-traumatic stress, difficulty with emotional regulation, and resilience are significant protective factors for older age groups while perceived community cohesion and efficacy decrease with age. Figure 35: Regression coefficients of multivariate regressions explaining the standardized scales associated with the protective factors Post￾traumatic stress (n=535) Difficulty in emotional regulation (n=491) Resilience (n=503) Community cohesion and efficacy (n=504) Dropouts (ref.) Completed basic training 0.03 -0.20 0.49 -0.30 Completed technical training -0.20 -0.54*** 0.62*** -0.45** Male (ref.) Female 0.08 0.27** -0.21** 0.11 16-20 years old (ref.) 21-25 years old -0.26** -0.26** 0.21** -0.19** 26-30 years old -0.52*** -0.45** 0.22 -0.13 Primary up to 8th grade (ref.) Primary, 9th grade completed 0.07 -0.22 0.18 0.05 Secondary up to 11th grade 0.07 -0.37* 0.22 0.32 Secondary, 12th grade -0.00 -0.36* 0.37 -.25 Constant 1.31*** 4.07*** 6.30*** 3.34*** R2 0.04 0.08 0.06 0.04 Note: *p<0.10, **p<0.05, ***p<0.001. Each column shows the results of the regression on a scale associated to a particular protective factor. Regressions were run on the standardized values of the scales and are interpreted in terms of changes in the standard deviations. The analysis in this section is restricted to the 578 individuals who had a six month follow up. Green indicates improvement (i.e. less difficulty in emotional regulation, greater resilience, etc.), red indicates worsening, and white indicates no significant difference. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 69 Figure 36 shows how the baseline distribution of the protective factors differs between those individuals who did not have a six-month follow up and those who did. Figure 36: Differences in baseline distribution of the protective factors between participants with and without a six-month follow up Note: Figure based on 742 participants who did not have a 6-month follow up and answered all PTSD questions and 536 participants who did have a 6 month follow up and answered all PTSD questions Note: Figure based on 654 participants who did not have a 6-month follow up and answered all DERS questions and 492 participants who did have a 6 month follow up and answered all DERS questions. Note: Figure based on 712 participants who did not have a 6-month follow up and answered all ARM questions and 505 participants who did have a 6 month follow up and answered all ARM questions. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 70 Note: Figure based on 682 participants who did not have a 6-month follow up and answered all ARM questions and 505 participants who did have a 6 month follow up and answered all ARM questions. Mean baseline value of the scales associated to the protective factors, by different sociodemographic characteristics For PTSD and DERS, the distribution of those who did not have a six month follow up appears slightly more skewed toward low levels of the difficulty than those who did have a six month follow up. However, the differences in the mean baseline values of the protective factors between those who did have a six month follow up and those who did not are not significant. Mean baseline value of the scales associated to the protective factors, by program completion Scale associated to protective factor Dropouts (n=54) Completed basic training (n=58) Completed technical training (n=466) PTSD 1.89 1.91 1.54 DERS 4.21* 3.95 3.56* ARM 7.96* 8.58 8.74* CCES 5.56* 5.11 4.73* *p<0.05. Indicates that the mean value of the scale is different between the two groups marked with a star. Note: Table refers to baseline levels of those who had a six-month follow up. The sample sizes of each scale varies, as not all interviewees responded to all the questions necessary for estimating each scale. Mean baseline value of the scales associated to the protective factors, by sex Scale associated to protective factor Male (n=251) Female (n=327) PTSD 1.53 1.67 DERS 3.48 3.78* ARM 8.80 8.55 CCES 4.83 5.05 *p<0.05. Indicates that the mean value of the scale is different between males and females. Note: Table refers to baseline levels of those who had a six-month follow up. The sample sizes of each scale varies, as not all interviewees responded to all the questions necessary for estimating each scale. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 71 Mean baseline value of the scales associated to the protective factors, by age groups Scale associated to protective factor 16-20 (n=291) 21-25 (n=205) 26-30 (n=82) PTSD 1.88* 1.47 1.04* DERS 3.85* 3.54 3.29* ARM 8.52 8.78 8.82 CCES 5.10 4.80 4.83 *p<0.05. Indicates that the mean value of the scale is different between the two groups marked with a star. Note: Table refers to baseline levels of those who had a six-month follow up. The sample sizes of each scale varies, as not all interviewees responded to all the questions necessary for estimating each scale. Mean baseline value of the scales associated to the protective factors, by education level Scale associated to protective factor Primary up to 8th grade (n=37) Primary 9th grade (n=55) Secondary up to 11th grade (n=105) Secondary 12th grade and more (n=379) PTSD 1.54 1.72 1.69 1.58 DERS 4.07 3.80 3.61 3.61 ARM 8.24 8.48 8.56 8.74 CCES 4.60 4.69 5.09 4.98 *p<0.05. Indicates that the mean value of the scale is different between the two groups marked with a star. Note: Table refers to baseline levels of those who had a six-month follow up. The sample sizes of each scale varies, as not all interviewees responded to all the questions necessary for estimating each scale. As indicated in Figure 37 below, protective factors improved for both dropouts and those who completed the technical training. Figure 37: Mean values at baseline of the scales associated to protective factors, and standardized changes between baseline and endline, and baseline and six month follow up, by program completion Scale Time and type of measurement Dropouts Basic training completers Technical training completers PTSD Baseline mean 1.89 1.91 1.54 Change to endline (in SDs) -0.53** -0.31** Change to 6 month follow up (in SDs) -0.28** -0.33 -0.33** DERS Baseline mean 4.22 3.95 3.56 Change to endline (in SDs) -0.19 -0.21** Change to 6 month follow up (in SDs) -0.55** -0.42 -0.29** ARM Baseline mean 7.96 8.58 8.75 Change to endline (in SDs) -0.01 0.17 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 72 Scale Time and type of measurement Dropouts Basic training completers Technical training completers Change to 6 month follow up (in SDs) 0.30 0.22 0.18** CCES Baseline mean 5.56 5.11 4.86 Change to endline (in SDs) 0.19 0.06 Change to 6 month follow up (in SDs) -0.56** -0.43** -0.07 Note: ** p<0.05 Indicates that the change between the score at endline or six month follow up and baseline is significantly different from cero. The table refers to those who had a baseline and six-month follow up observation. Sample sizes vary by scale, as the number of observations who answered to all questions needed to estimate a score was not constant across groups. Green indicates improvement, red indicates worsening, and white indicates no significant difference. To identify whether program completion is associated with changes in the protective factors between baseline and the six-month follow up, and whether these changes are similar or not for both sexes, ages and educational levels, we ran two difference-in-difference (DID) models for each one of the scales associated with the protective factors. The first DID model identifies whether, once baseline characteristics are controlled for, completing basic or technical training have an effect different to that of dropping out of the program on the scale at the six-month follow up. In these models, labelled as (Model 1) for each scale, the analysis should focus on the completing basic training and completing technical training variables. If these are statistically significant, results indicate that completing basic training or completing technical training have a different effect to that of dropping out of the program in the change in the scale. The second set of models (labelled Model 2) is meant to identify whether, for a particular scale, completing basic or technical training have a different effect than that of dropping out of the program, for a particular sex, age group or education level. In these models, focus should be placed on the interaction effects between program completion and the sociodemographic characteristics. For ease of interpretation, Figure 38 presents the effects of interest discussed below in bold. The six-month follow up coefficients in the PTSD, ARM and CCES model indicate that, once baseline characteristics have been controlled for, participants who dropped out of the program decreased their PTSD and perception of community cohesion between baseline and the six Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 73 month follow up, while the ARM model indicates that their resilience increased. With the exception of CCES, all these changes are similar between dropouts, basic training and technical training completers, as is indicated by the lack of significance of the coefficients for basic training and technical training. In the case of CCES, the change between baseline and six month follow up is lower among those who completed technical training than among those who dropped out of the program. The set of Models 2 had the objective of exploring whether the effect of completing basic training or technical training (as opposed to dropping out of the program) had a differential effect in certain groups of participants. The interaction coefficients between the sociodemographic characteristics and basic and technical training completion in these models confirm that is the case, but that the variation differs according to the scale. In the case of PTSD, among those in the oldest age group the change in scores was less for those who completed the basic and technical components of the program than among dropouts. To understand this, one should notice that for PTSD a negative coefficient indicates that post￾traumatic stress disorder decreases, whereas a positive coefficient indicates that it increases. The interaction of the age group 26-30 and basic training (2.41) and technical training (2.07) are both positive and significant, indicating that in these age group the decrease in PTSD is less than what would be expected for dropouts. For DERS, the changes between baseline and the six month follow up are similar between dropouts and those who completed the basic and technical components of the program for all the groups, except for women. The negative and significant (-0.70, p<0.10) coefficient for the interaction of female and technical training indicates that, among women, those who completed the technical component of the program decreased their difficulties with emotional regulation even more than what would be expected for dropouts. Finally, in the case of CCES, community cohesion and efficacy decrease with time for all participants, but these changes are even more exacerbated than expected for those who completed the basic and technical components of the program and are in the 21 to 25 year old group. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 74 Estimate of the mean protective factors at the six-month follow up of those who were employed, and those who were not, among those who had improved their risk scores. mean(PTSD) mean(ARM) mean(DERS) mean(CCES) No (n=20) 1.0 8.8 3.3 5.3 Yes (n=52) 1.0 9.1 3.5 4.5 Those who found an employment have higher resilience and more difficulties with emotional regulation. However, none of those differences are significant given the small sample sizes. It is also unknown whether the changes in the protective factors are what lead to the employment, or if the employment is what led to changing levels of protective factors. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 75 Figure 38: Effect of program completion and sociodemographic characteristics on the change in protective factor scales between baseline and six-month follow up, based on DID models Variable Model 1 Model 2 PTSD DERS ARM CCES PTSD DERS ARM CCES Follow up among dropouts -0.43* -0.61 0.37** -0.95*** -0.94 -1.46* 0.73 -1.74* Difference in follow up effects among: Basic training -0.07 0.15 -0.10 0.28 -2.66 -0.55 -0.09 -1.55 Technical training -0.08 0.29 -0.15 0.84** 0.04 0.98 -.21 1.91* Sex (reference=Male, dropouts) Female among dropouts -.56 0.61 -0.12 -0.16 Female – additional effect among basic training -.39 -0.65 0.37 0.30 Female – additional effect among technical training -.55 -0.70* 0.45 0.01 Age (reference=16-20, dropouts) 21-25, dropouts -0.14 -0.12 0.35 1.36** 26-30, dropouts -1.35 0.11 -0.11 -0.29 21-25, additional effect among basic training 1.13 0.61 -0.63 -1.68** 26-30, additional effect among basic training 2.41* 0.24 -0.19 -1.98 21-25, additional effect among technical training 0.42 0.29 -0.48 -1.22** 26-30, additional effect among technical training 2.07** 0.28 0.20 -0.12 Education (baseline=primary up to 8th grade, among dropouts) Primary, 9th grade, among dropouts 1.20 -0.49 -0.46 -0.02 Secondary, up to 11th grade, among dropouts -0.35 0.74 -0.36 1.87* Secondary, 12th grade, among dropouts 0.30 0.73 -0.55 0.09 Primary, 9th grade, additional effect among basic training 1.54 1.38 0.00 1.59 Secondary, up to 11th grade, additional effect among basic training 3.23 1.39 -0.38 0.59 Secondary, 12th grade, additional effect among basic training 2.19* 0.60 0.22 3.43** Primary, 9th grade, among technical training -1.31 0.84 -0.23 0.14 Primary, up to 11th grade, among technical training -0.27 -0.63 -0.07 -2.22* Secondary, 12th grade, among technical training -0.37 -0.67 0.09 -0.35 Constant 1.58*** 3.63*** 8.72*** 4.95*** 1.59*** 3.64*** 8.71*** 4.95*** Rho (fraction of the variance due to the individual component) 0.63 0.65 0.69 0.61 0.65 0.66 0.70 R2 within 0.10 0.11 0.05 0.05 0.16 0.17 0.09 0.14 Number of individuals 570 559 544 553 568 557 542 552 *p<0.10, **p<0.05, ***p<0.001 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 76 Figure 39: Changes in the distribution of the scales associated with the protective factors between baseline, endline and six month follow up, by program completion. Changes in PTSD Changes in DERS Changes in ARM Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 77 Changes in CCES To explore whether protective factors are associated with employment, Figure 40 shows the results of fixed-effects regressions. Fixed effects models control for all variables that are fixed between the two observations, such as age, sex, and program completion, while allowing for a measurement of the effect of variables that change between observations (such as employment status). Figure 40 shows the results of these models for each protective factor. The results of these regressions indicate, as already noted, that protective factors significantly improved between baseline and the six-month follow up (first row). They also indicate that, once time of observation and sociodemographic characteristics, program completion, and other unchanged characteristics have been controlled for, protective factors are not statistically associated with employment (second row). Figure 40: Effect of employment and six-month follow up on the protective factors, as estimated by fixed effects regressions PTSD DERS ARM CCES Six-month follow up (vs baseline) -0.545*** -0.361*** 0.276*** -0.254** Employed (vs. unemployed) 0.091 -0.007 -0.862 0.019 Constant 1.562*** 3.637*** 8.740*** 4.944*** n 570 559 544 553 Within R2 0.103 0.102 0.049 0.022 rho 0.635 0.645 0.695 0.595 ***p<0.001 **p<0.05 Note: All participants who had both a baseline and six-month follow up were included in the regressions and answered the relevant questions (thus N varies by factor as shown above). Green indicates improvement (i.e. less difficulty in emotional regulation, greater resilience, etc.) red indicates worsening, and white indicates no significant difference. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 78 ANNEX 9. ADDITIONAL MIGRATION ANALYSIS Figure 41: Percentage of participants who attempted to migrate to the US the last year, by time of observation Note: Percentages refer to 578 participants who answered to the baseline and had a six-month follow up Figure 42: Percentage of participants who attempted to migrate to the US the last year, by program completion (baseline estimates) Note: Estimates based on the baseline responses of 792 participants. These include participants who had an endline and six-month follow up VIP-RA. No statistical differences were found between those who had endline and six￾month follow up and those who did not. Figure 43: Percentage of participants who attempted to migrate to the US the last year, by selected sociodemographic characteristics Sociodemographic characteristic Percentage who attempted to migrate the last year Sex Male (n=363) 7.99 Female (n=429) 4.89 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 79 Sociodemographic characteristic Percentage who attempted to migrate the last year Age 16-20 years old (n=396) 4.50 21-25 years old (n=258) 7.78 26-30 years old (n=75) 10.34 Education Primary, up to 8th grade (n=66) 19.31 ** Primary, completed 9th grade (n=75) 8.43 Secondary, 10th and 11th grade (n=128) 7.86 Secondary, 12th grade and more (n=456) 3.14 ** Risk of violence Primary (n=527) 4.36** Secondary (n=214) 11.21 ** Tertiary (n=16) 6.25 ** p<0.05 Note: The figure presents responses at baseline. Figure 44 explores the association between sociodemographic variables and migration intentions at the six-month follow up, for those who mentioned migration intentions at baseline (n=67).24 With the exception of PTSD and DERS, the association between the variables analyzed and the change in migration expectations is as expected. These results are exploratory and should be taken with caution given the small sample size (n=67) resulting in no statistically significant results Figure 44: Migration intentions at six-month follow up among those with migration intentions at baseline, by selected sociodemographic characteristics Sociodemographic characteristics Migration intentions at baseline Migration intentions at six-month follow up Change in migration intentions baseline to follow up Sex Male 28 12 -57.14% Female 39 14 -64.10% Age 16-20 years old 37 14 -62.16% 21-25 years old 23 10 -56.52% 26-30 years old 7 2 -71.43% Employment Did not gain employment from baseline to six-month follow up 34 16 -52.94% Gained employment from baseline to six-month follow up 33 10 -69.70% Perception of a good future Does not perceive a good future in Honduras at six-month follow up 25 13 -48.00% Perceives a good future in Honduras at six-month follow up 42 13 -69.05% Note: N=67 participants who had migration intentions at baseline and responded to the migration questions at the six-month follow up. Sample was not large enough to disaggregate by education groups, nor to determine statistical significance of change. 24 Given this small number that answered all relevant migration questions, it is not possible to do a multivariate analysis that simultaneously controls for different characteristics to analyze the relative importance of variables on these changes. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 80 Figure 45: Migration intentions at six-month follow up among those who expressed to have migration intentions at baseline, by psycho-emotional protective factors Factor No migration intentions at six￾month follow up Migration intentions at six-month follow up PTSD No change between baseline and six month follow up 28.6 71.40 100 Increased 1 SD 43.28 56.72 100 DERS No change between baseline and six month follow up 29.68 70.32 100 Increased 1 SD 32.06 67.94 100 ARM No change between baseline and six month follow up 37.53 62.47 100 Increased 1 SD 45.21 54.79 100 CCES No change between baseline and six month follow up 32.99 67.01 100 Increased 1 SD 38.34 61.66 100 Note: Table refers to 67 participants who had migration intentions at baseline and responded to the migration questions at the six-month follow up. Estimates presented are marginal effects from bivariate logistic regressions. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 81 Figure 46: Factors associated with the odds of having migration intentions at baseline and six month follow up, based on multivariate logistic regressions (OR presented) Note: 95% The figures present odds ratios and the 95% confidence interval. As the effects on odds ratios are multiplicative, effects larger than one increase the odds of having attempted to migrate, while effects smaller than one decrease the odds. In the figure, significant effects (p<0.05) are marked with a black dot, and are those where the confidence intervals do not cross the 1.0 value Regression at baseline based on n=412, regression at six-month follow up based on n=302 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 82 Figure 47: Reasons for having migration intentions, by gender 44.4% 15.6% 4.4% 27.8% 22.2% 17.8% 54.5% 29.5% 6.8% 43.2% 13.6% 18.2% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% I don't have a job Even when I have a job, I'd earn more there My life has been threatened I want to live in a safer place Family reunification To study Men Baseline (n=90) Six month follow up (n=44) 61.8% 6.7% 1.1% 29.2% 24.7% 23.6% 59.6% 26.3% 3.5% 50.9% 14.0% 1.8% 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% I don't have a job Even when I have a job, I'd earn more there My life has been threatened I want to live in a safer place Family reunification To study Women Baseline (n=89) Six month follow up (n=57) Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 83 ANNEX 10. VIOLENCE-INVOLVED PERSONS RISK ASSESSMENT AND EMPLOYABILITY SURVEY Placeholder for VIPRA and Employability Survey to be added in PDF. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 84 U.S. Agency for International Development/Honduras Avenida La Paz, P.O. Box 3453 Frente a la Embajada Americana Tegucigalpa, Honduras www.usaid.gov