Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1 MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY EMPLOYMENT, EMPLOYABILITY AND VIOLENCE IN HONDURAS: PILOT IMPACT EVALUATION BASELINE REPORT FOR THE WORKFORCE DEVELOPMENT ACTIVITY December 2017 Contract Number: AID-OAA-15-00019 – Task Order Number: AID-522-TO-16-00002 This publication was produced for review by the United States Agency for International Development. It was prepared independently by Dexis Consulting Group and the University of Notre Dame Initiative for Global Development and authored by Juan Carlos Guzman and Tom Hare. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) i MONITORING AND EVALUATION SUPPORT FOR COLLABORATIVE LEARNING AND ADAPTING (MESCLA) ACTIVITY Employment, Employability and Violence in Honduras: Pilot Impact Evaluation Baseline Report for a Workforce Development Activity VERSIONS: • Draft 1 – December, 20, 2017 SUBMITTED TO: USAID/Honduras Prepared by: Dexis Consulting Group DISCLAIMER: The authors’ views expressed in this document do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 1 TABLE OF CONTENTS Table of Contents.............................................................................................................................................................. 1 Acronym List ..................................................................................................................................................................... 2 1 Executive Summary.................................................................................................................................................. 4 2 Evaluation Purpose & Evaluation Questions...................................................................................................... 4 2.1 Evaluation Purpose.......................................................................................................................................... 4 2.2 Evaluation Questions...................................................................................................................................... 5 3 Project Background.................................................................................................................................................. 5 3.1 Figure 1: WFD Activity Phases..................................................................................................................... 7 4 Evaluation Methods & Limitations........................................................................................................................ 7 4.1 Data Collection Tools: Employment and Employability......................................................................... 7 4.2 Data Collection Tool: Violence-Involved Persons Risk Assessment.................................................. 7 4.3 Data Collection Methods.............................................................................................................................. 8 4.4 Issues and Limitations..................................................................................................................................... 9 5 Findings, Conclusions & Next Steps.................................................................................................................. 10 5.1 Baseline Findings............................................................................................................................................ 10 5.2 Conclusions .................................................................................................................................................... 14 5.3 Next Steps...................................................................................................................................................... 14 6 Annex 1: Sources of Information........................................................................................................................ 16 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 2 Acronym List USAID United States Agency for International Development ARM Adult Resilience Measure CDC Centers for Disease Control CES-D Center for Epidemiological Studies Depression Scale CFA Confirmatory Factor Analysis CFI Comparative Fit Index CSES Community Cohesion and Efficacy Survey DERS Difficulties in Emotion Regulation Scale GRYD Gang Reduction and Youth Development ISRD International Self-Report Delinquency Survey MEL Monitoring Evaluation and Learning MESCLA M&E Support for Collaborative Learning and Adapting NDIGD Notre Dame Initiative for Global Development PCL-5 Posttraumatic Stress Disorder Checklist PTSD Posttraumatic Stress Disorder RMSEA Root mean squared error of approximation RSES RCT Rosenburg Self-Esteem Scale Randomized Controlled Trial TLI Tucker-Lewis Index UNAH-IUDPAS Universidad Nacional Autónoma de Honduras – Instituto Universitario en Democracia, Paz, y Seguridad UNODC United Nations Office on Drugs and Crime USAID United States Agency for International Development VIP-RA Violence-Involved Persons Risk Assessment VRAG Violence Risk Appraisal Guide Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 3 WFD Workforce Development WHO World Health Organization YSET Youth Service Eligibility Tool Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 4 1 EXECUTIVE SUMMARY The Empleando Futuros Workforce Development (WFD) Activity in Honduras contributes to the USAID/Honduras’ Country Development Cooperation Strategy Development Objective 1 (DO1), Sub￾Intermediate Result 1.1.2, “Quality services that protect against violence increased” (USAID, 2015). Given the dearth of literature on the link between workforce development and violence, USAID/Honduras commissioned the Monitoring and Evaluation Support for Collaborative Learning and Adapting (MESCLA) Activity to perform an impact evaluation of the WFD Activity. However, given the lack of references available to justify and effectively plan a large-scale impact evaluation, a pilot impact evaluation on a small subset of the beneficiary population was established. The findings of the pilot impact evaluation will be used to inform the decision of whether USAID/Honduras will invest in a full impact evaluation of the WFD Activity in 2018 To measure employment and employability, MESCLA and the Notre Dame Institute for Global Development (NDIGD) used the Employability and Evaluation Tool, which was developed for use in the USAID METAS project in Honduras. To measure risk of violence involvement, MESCLA and NDIGD developed and validated a Violence-Involved Persons Risk Assessment (VIP-RA) tool. This tool is used to distinguish between primary and secondary risk and scores will be tracked in impact and performance evaluations. For the pilot impact evaluation, MESCLA and NGIGD implemented a randomized controlled trial (RCT) including a treatment group for the initial participants in the WFD Activity, and a control group for those participants that would enter during a subsequent round of trainings. The results were examined to determine whether youth in the treatment and control groups would show differences in post-program employability or risk of violence. The randomization was successful in its goal of creating two groups of respondents who are comparable on average, as the team found a balance across most variables between treatment and control areas in the baseline among demographics and responses to the employability and violence-involvement tools. There are several considerable issues involved in the pilot impact evaluation, namely consistent implementation based on the protocols necessary to ensure validity of an impact evaluation, attrition, and security. 2 EVALUATION PURPOSE & EVALUATION QUESTIONS 2.1 EVALUATION PURPOSE The Empleando Futuros Workforce Development Activity (WFD) in Honduras began in 2016 under Development Objective 1 Sub-Intermediate Result 1.1.2, quality services that protect against violence increased. The need for an evaluation arose from the dearth of literature on the link between WFD and violence. Internationally, there have been mixed results among WFD projects that aim to impact violence. Several studies fail to demonstrate a link between employment and gang violence among youth in the United States (Millensky, Bloom, Muller-Ravett and Broadus, 2011; Uggen, 2000). Some suggest job training may increase violence risk in youth (Bloom, Orr, Bell, Cave, Doolittle, Lin and Bos, 1997), yet others demonstrate a positive impact but at a great investment (Schoschet, Burghardt and McConnell, 2008; Cave, Bos, Doolittle and Toussaint, 1993). In Honduras specifically, the METAS basic labor competencies program increased confidence among youth, but did not ultimately impact employment (USAID, 2014). Similarly, among young adults there have been mixed results in the United States (Raphael, 2011), but in this area no previous studies that evaluate results among young adults in Honduras were found. To address this evidence gap, USAID/Honduras engaged MESCLA to assess the possibility of conducting an impact evaluation of the WFD Activity. A pilot impact evaluation on a subset of the beneficiary population was agreed upon, and its findings will be used to inform the decision of whether USAID Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 5 should invest in a full impact evaluation of the WFD Activity using an experimental design. Issues such as attrition, security, and program coherence will be tracked throughout the pilot to determine the feasibility of a full impact evaluation. In addition, the phased start-up of the WFD Activity with approximately 1,500 beneficiaries in the first year of training (~2017-18,) and 5,000 by the second year of training (~2018-19) allows an opportunity to determine effect size and establish project and data collection implementation protocols ahead of any full-scale impact evaluation. It is worth noting that a performance evaluation is also being conducted on the full WFD Activity. The performance evaluation will be useful in determining whether WFD targets are being met among the beneficiaries. However, absent a control group, the performance evaluation will not be able to answer the impact evaluation questions originally presented by USAID. 2.2 EVALUATION QUESTIONS The questions that inform the pilot impact evaluation are as follows: 1. Do youth in treatment group and youth control group, selected on same basis and assigned randomly, show different post-program employability? 2. Do youth in treatment group and youth control group, selected on same basis and assigned randomly, show different post-program risk of violence? 2.a. Do the final treatment group risk levels vary by employment status and type of employment, referral status, sex, age, education, location, initial risk level, duration and composition of the intervention? 3 ACTIVITY BACKGROUND USAID outlined the following development challenges related to young adult employment in Honduras in the WFD Activity Statement of Work (2016): • Almost two-thirds of Hondurans live in poverty; the urban unemployment rate is 7.5 percent and the urban underemployment rate is 32.4 percent. 1 • Within Latin America, Honduras has one of the highest percentages of youth (27.5 percent) that are neither studying nor working (known as NiNis for the Spanish ni estudia ni trabaja). The 2014 census identified 846,186 NiNis, the majority of which (62 percent) were between 15 and 24 years old. • Honduran youth face a challenging labor market, further compounded by widespread reports of job discrimination against youth living in at-risk communities. • Youth between the ages of 15 and 29 represent more than 70 percent of all unemployed workers in the country. The 19-24 age groups have the highest unemployment rate of all age groups, standing at 10.5 percent in 2014, the worst rate since 2010. • For the 19-24 age group underemployment is at 42.6 percent.2 Further, USAID perceives that: “High youth unemployment and underemployment rates are driven by a core factor of lack of education and training for young people of legal working age[…]. Less than half (42 percent) of 1 Honduras Labor Market Observatory 2014-2015 2 Servicio Nacional de Empleo de Honduras, Secretaria del Trabajo, 2014. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 6 the youth searching for employment have completed their secondary education, 15 percent have some secondary education, and 15 percent have completed only primary education.”3 Compounding those issues, over the last decade Honduras has consistently ranked as one of the most violent countries in the world. While some improvement has been made to reduce the number of homicides in the country, the homicide rate remains high at 63.75, or 13 times the rate in the United States and four times the rate of Latin America as a whole (UNODC, 2016). Efforts to reduce violence are numerous and focus primarily on gang prevention among youth up to age 18. Although these efforts are important to prevent future offenses, these programs often do not include those that are most likely involved in violence – young adults ages 20-29. According to the National Autonomous University of Honduras Institute for Democracy, Peace and Security (UNAH-IUDPAS), a 24-year old male is nearly twice as likely to be murdered as an 18-year old male, and nearly twenty times more likely to be murdered as a 14-year old (2016). Targeting these young adults at highest risk of violence involvement is challenging among a population that lives in high crime, impoverished, urban communities. In this context, the WFD Activity is a five-year activity that: “Aims to increase citizen security for vulnerable populations in urban, high-crime areas in Honduras by supporting workforce development that will increase income-generating opportunities for youth [ages 16-30] who are the most at risk of being perpetrators of violence. The activity will strengthen comprehensive workforce readiness services, including for job linkage and self-employment, to benefit at-risk youth, including those who qualify for secondary and tertiary violence prevention services.” (USAID, 2016). WFD Activity beneficiaries are drawn from the following population: • Males between the ages of 16 and 30; • Living in specifically identified communities within the following high crime municipalities: Tegucigalpa, San Pedro Sula, Choloma, La Ceiba, or Tela; • Currently out-of-school, with no less than a sixth-grade education and no more than a high school degree; and • Currently unemployed, underemployed, or with inconsistent or low-skill employment. The WFD Activity in Honduras is comprised of three main phases (see Figure 1): ten weeks of life skills and cognitive behavioral therapy, followed by up to nine months of job skills training in a specific trade, and job placement and six months of observation and mentoring. All communities for the pilot impact evaluation share RETE as the community -level implementing partner responsible for recruitment, selection and mentoring of youth in year one in all the pilot communities. 3 Ibid. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 7 3.1 FIGURE 1: WFD ACTIVITY PHASES Source: USAID Workforce Development Activity 4 EVALUATION METHODS & LIMITATIONS 4.1 DATA COLLECTION TOOLS: EMPLOYMENT AND EMPLOYABILITY Employment was captured using the questionnaire used for the Honduras National Household Survey. Some modifications were made to the questionnaire so that it would follow the Colombia National Household Survey questionnaire, which goes further in depth in several areas of employment and entrepreneurship. Given the utility and validity of the Youth Employability and Evaluation Tool employed by the USAID METAS activity in Honduras (Abdalla, Barth, Dunn, Holter, Ortega, Tinta, 2013), this existing tool was utilized for the WFD Activity pilot impact evaluation. Small modifications were made based on the WFD MEL Plan, but the METAS tool was largely replicated verbatim.4 4.2 DATA COLLECTION TOOL: VIOLENCE-INVOLVED PERSONS RISK ASSESSMENT NDIGD and faculty in the Notre Dame Department of Psychology developed and validated the Violence-Involved Persons Risk Assessment (VIP-RA) tool for the WFD Activity in order to distinguish between primary and secondary risk, and for the impact and performance evaluations.5 The VIP-RA serves several purposes: (1) establish empirically-defensible risk of violence among young adults ages 16- 4 More on the tool can be found in Abdalla, et al. (2013) and USAID Honduras (2014). 5 For more information on the tool, see the “Development and validation of the violence-involved persons risk assessment: Honduras” report submitted to USAID September, 2017. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 8 30,6 (2) help target interventions for the secondary risk population, and (3) measure change in risk during and after program participation via an impact and/or performance evaluation. The VIP-RA seeks to determine which potential program beneficiaries are “the most at risk of being perpetrators of violence” as the mandated target population of the WFD program. This allows the program to focus on the specific needs of high-risk individuals in a targeted fashion. The VIP-RA must reliably distinguish between “primary” and “secondary” risk among young adults. Specifically, individuals at primary risk are those that, despite living in areas with high levels violence, exhibit few signs in thoughts or actions related to violent behavior, or that have many protective assets such as a strong family or positive peer group. Individuals at secondary risk are those that exhibit signs in either thoughts or actions related to violent behavior, or that do not have protective assets. This categorization of risk corresponds to the USAID theory of change that, “providing a combination of primary (community￾based infrastructure), secondary (focused on individuals at risk of becoming perpetrators), and tertiary (focused on those who have already committed crimes) prevention programs at the individual, family, relational, and community levels will lead to a reduction in violence” (USAID, 2015; p. 12). The VIP-RA was required to help the WFD Activity tailor its recruitment and programmatic activities to better reach those at secondary risk; and to measure changes in risk levels pre-post participation in the Activity. The tool does not rely on static risks (age, ethnicity, local demographics, personal history) that could lead to social profiling. Rather, it focuses on many of the “dynamic characteristics, attitudes, circumstances, and behaviors” that are related to violence and that can be modified (Hennigan, Maxon, Sloane, Kolnick, and Vindel, 2014). The VIP-RA covers domains – personal history, emotional dimensions, relationship/community dimensions, and past deviance – similar to past tools used to determine primary/secondary risk. The tool places these domains in the social-ecological framework of violence prevention. The VIP-RA was validated using a methodology similar to that used in the validation of past risk assessment tools (Hennigan, et al., 2014). 4.3 DATA COLLECTION METHODS The pilot impact evaluation is a randomized controlled trial (RCT) with a phase-in design. The pilot impact evaluation created a treatment group for the initial participants in the WFD training, and a control group for those that would enter during a subsequent round of training one year later. Randomization was performed at the individual level among all young adults who completed the WFD Activity “expression of interest” form, met minimum qualifications and who returned for the administration of the employability and VIP-RA tools as program intake. Three neighborhoods in or near Tegucigalpa were ultimately chosen for the pilot - Villafranca, Campo Cielo, and Los Pinos. Community selection was based on the experience of the implementing organization working in the community, availability of facilitators and mentors, sufficient young adult population from which to draw the sample, and similar socio-demographic features. Given the exploratory nature of the pilot impact evaluation, a target sample size of 200 participants in the treatment groups and 100 participants in the control group was established. This number was determined to be sufficient to calculate effect size for any future impact evaluation (as opposed to determination by power calculations). In addition, according to the WFD ME&L plan, the implementers anticipated a drop-out rate of 37.5%, and the team also anticipates an attrition rate (people who cannot be re-contacted) of less than 10%. Random assignment into treatment and control was conducted via public lottery in Villafranca and Los Pinos, and remotely using a random assignment in Campo Cielo. 6 We use the term “young adult” or “adult” to distinguish the VIP-RA from other tools that target younger age groups commonly referred to as “youth.” Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 9 Public lotteries were conducted to increase confidence in selection where possible, and were attended by approximately one-third to half of those who completed intake in both locations. All names were assigned a number, and community members drew the numbers from a box. Numbers were read aloud by a different community member and multiple lists were tracked to ensure accuracy. Final lists were posted for reference and those selected for treatment were later called to inform of the start date. All selected individuals were notified by phone with information on class start date. Baseline data collection was performed in September and October 2017 by volunteers from RETE, under the direction of technical experts from NDIGD and MESCLA. Volunteers were trained on the data collection tools during preliminary and refresher workshops. A manual was created to help guide the enumerators through the employment and income section of the questionnaire, given the importance of definitions contained in the survey. The survey was conducted on tablets using ODK software. During the first round of data collection in Villafranca and Campo Cielo, it was determined that the young adults themselves could complete the VIP-RA sections using the tablets, and that the extra confidentiality afforded by auto-filling responses might yield more reliable results. Thereafter, individuals were given the choice to complete the VIP-RA section alone or with an enumerator. It was also noted during data collection in the first two neighborhoods that some of the information requested for the impact evaluation was repetitive of information collected for program intake. These repetitive questions were reviewed and removed where possible. The following table shows the breakdown of treatment and control by community: Treatment and Control by Community Community Treatment Control Total Villafranca 100 55 155 Campo Cielo 30 25 55 Los Pinos 42 22 64 Total 172 102 274 4.4 ISSUES AND LIMITATIONS The objectives of the pilot impact evaluation included determining the feasibility for a large-scale impact evaluation. There are several issues to consider that arose during the pilot. Key among them are 1) consistent implementation based on the protocols necessary to ensure validity of an impact evaluation, 2) attrition, and 3) security. 1) Consistency of implementation is a major challenge in a program of this size, spread among several municipalities and implementing organizations. One community originally slated for inclusion in the pilot impact evaluation is an example of the challenges that would be faced at scale. In El Pedregal, baseline data was collected and participants were assigned randomly to treatment and control. However, local partners, without informing MESCLA, decided to assign to treatment all individuals who were available to be in the program during the weekdays and to control all individuals that can only participate during the weekends. This decision invalidated the impact evaluation protocol. Once this was discovered, El Pedregal had to be dropped from the pilot impact evaluation and a new community identified. Los Pinos was identified as the best replacement using the community selection criteria identified above. The schedule was also Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 10 modified in Villafranca, with only half of the treatment group starting as planned. This change did not pose threats to the validity of the evaluation, but it is suggestive of the challenges for implementation of any larger study. 2) The second issue to consider is attrition. Of the individuals assigned to treatment in Villafranca and Campo Cielo (data is not yet available for Los Pinos for the issue noted below), 12.5% and 23% respectively did not show for the first class, and 32% and 38% had dropped out after the first month. This represents a 34% attrition rate before the midway point of the first phase (of three phases). 3) Finally, security is an issue of concern in all communities, but has impacted Los Pinos in particular. The presence of gang leadership and gang members during application of the VIP-RA and the lottery process was noted, but did not present obstacles at that time. After training started in Los Pinos, however, the implementing organization was forced to stop classes due to communication from the gangs. As of this writing, the WFD program is awaiting the results of a security review to determine the way forward in Los Pinos. Together with these issues, it is worth noting again that this pilot impact evaluation should not be expected to or used to determine the impact of the WFD Activity. Its purpose is to determine the effect size and feasibility for any complete impact evaluation. As with any impact evaluation using an experimental design, it is also important to remember that this study has limited generalizability, or low external validity. The results of this study may have internal validity (comparisons between the treatment and control groups are unbiased for the population being studied), but not external validity (results do not necessarily apply to other populations, communities or changes of circumstances). 5 FINDINGS, CONCLUSIONS & NEXT STEPS 5.1 BASELINE FINDINGS The following tables present demographic characteristics and outcome variables. Each table measures whether the mean shown is statistically significant at the 5 percent level. Because the sample size varies for each characteristic, it will not be uncommon to find higher levels to be not statistically significant for one group (e.g. control group) while a lower level for the same variable would be significant for another group (e.g. treatment group). For each demographic characteristic and outcome indicator, the team compared means across treatment and control groups to look for differences between groups. Table 1 below presents the results of the balance test of the baseline. Overall, there are low mean differences between treatment and control groups, and only one statistically significant (at the p<0.05 level) difference across the two groups. This suggests that the treatment and control groups are balanced on nearly all observable characteristics (sex, age, marital status, parent status, education and work). Meanwhile, there is a significant difference among the outcome variables (VIP-RA score and components). Those differences are explored in more detail below and do not impact the sample balance. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 11 Then variables of interest among the sample population were explored, including sex, age, education, and work experience. Table 2 shows the sample characteristics by sex. Notably, women in the sample are, on the average, slightly older than the men, much less likely to be single, and much more likely to be a parent. Women are much more likely to be a single parent, and much less likely to be studying than men. The differences between women and men in work and employability are large, but not statistically significant. Scores on the VIP-RA are nearly identical between women and men, while women have a slightly higher rate of secondary risk (though that difference is not statistically significant). There are no significant differences between women and men on the component scores of the VIP-RA. Table 1: Initial Balance Test Treatment Control Difference Sex (% female) 65.7* 68.6* -2.9 Age 20.8* 21.0* -0.2 Single 76.2* 73.5* 2.6 Household size 5.9* 5.9* 0.0 Parent 36.0* 35.3* 0.8 Single parent 14.0 12.7 1.2 Study 9.9 9.8 0.1 Work 25.0* 30.4 -5.4 “NiNi” (neither work nor study) 67.4* 61.8* 5.7 Last attended Primary 39.5* 41.2* -1.6 Last attended Secondary 58.1* 56.9* 1.3 VIP-RA score 10.6* 9.6 0.9 Secondary risk 11.0* 7.8 3.2 Employability 62.2* 56.6* 5.6 Emotional regulation score 37.3* 38.4* -1.1 Depression score 12.6* 14.1* -1.4 PTSD score 20.5* 24.0* -3.6* Resilience score 89.3* 87.5* 1.8 * p<0.05 Table 2: Sample Characteristics by Sex Women Men Difference Age 21.7* 19.1* 2.5* Single 66.1* 93.4* -27.3* Household size 5.7* 6.2* -0.5 Parent 49.2* 8.8 40.4* Single parent 19.1* 2.2 16.9* Study 7.7 14.3* -6.6* Work 21.9* 37.4 -15.5 “NiNi” (neither work nor study) 72.1* 51.6* 20.5 Last attended Primary 35.5* 49.5* -13.9 Last attended Secondary 62.3* 48.4* 13.9 VIP-RA score 10.0 10.7* -0.7 Secondary risk 10.9 7.7* 3.2 Employability 65.0* 50.1* 14.9 Emotional regulation score 37.8* 37.4* 0.4 Depression score 14.7* 10.1* 4.5 PTSD score 22.4* 20.6* 1.8 Resilience score 87.8* 90.4* -2.7 * p<0.05 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 12 Table 3 shows the sample characteristics by age group. There are many more statistically significant differences among the age groups than by sex. As seen above, women in the sample tend to be older than men, and as demonstrated here, they occupy the vast majority of the 21-31 range. The single population tends to be younger, while those that are parents tend to be older. The younger aged group tends to be studying more than the older groups. Meanwhile, the working population in this sample is about the same across age groups, while those that are neither working nor studying tend to be older. The older age group tend to have slightly higher scores on the VIP-RA and thus fall into the secondary risk category. Scores on the components of the VIP-RA also differed slightly by age group. Table 4 shows the sample characteristics by education level. There is only one statistically significant difference, the emotional regulation score, between those who last attended primary and those who last attended secondary or beyond. Scores on the VIP-RA do not differ depending on education level. Table 3: Sample Characteristics by Age Years 16-21 21-26 26-31 Sex (% female) 54.3* 73.8* 97.7* Age 17.9* 22.7* 27.7* Single 88.7* 65.0* 46.5 Household size 6.3* 5.6* 5.2* Parent 15.2* 47.5* 86.0* Single parent 6.6 15.0* 34.9 Study 13.2* 7.5 2.3 Work 25.8* 30.0* 25.6* “NiNi” (neither work nor study) 63.6* 63.8* 74.4* Last attended Primary 47.7* 25.0* 41.9* Last attended Secondary 51.7* 71.3* 53.5* VIP-RA score 6.3* 14.3 16.4* Secondary risk 4.6 13.8 20.9* Employability 60.8* 57.5* 62.6* Emotional regulation score 39.8* 35.1* 35.2* Depression score 13.6* 11.3* 15.3* PTSD score 24.3* 18.8* 18.8* Resilience score 89.3* 89.4* 85.0* * p<0.05 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 13 Table 5 shows the sample characteristics by whether the young adult is working/studying or neither. Here there is only one statistically significant difference, where parents are much less likely to be working or studying. Scores on the VIP-RA are identical between groups. There are no significant differences between those that work/study and those who do neither on the component scores of the VIP-RA. Table 5: Sample Characteristics by Work/Study or Neither NiNi Work/Study Difference Sex (% female) 73.7* 53.7* 20.1 Age 21.1* 20.3* 0.8 Single 70.9* 83.2* -12.2 Household size 6.0* 5.8* 0.2 Parent 42.5* 23.2* 19.3* Single parent 16.8* 7.4 9.4 Study 0.0 28.4 -28.4 Work 0.0 77.9* -77.9* Last attended Primary 36.9* 46.3* -9.4 Last attended Secondary 60.9* 51.6* 9.3 VIP-RA score 8.7 13.1* -4.5 Secondary risk 9.5 10.5* -1.0 Employability 82.0* 84.77 -2.7 Emotional regulation score 38.6* 36.0* 2.6 Depression score 12.9* 13.6* -0.7 PTSD score 22.0* 21.3* 0.7 Resilience score 88.5* 88.9* -0.4 * p<0.05 Table 6 shows the sample characteristics by employability score. Of note here are the large, statistically significant percentages of NiNis that score high in employability. Scores on the VIP-RA are identical 7 Among those who study only. Table 4: Sample Characteristics by Education Level Primary Secondary + Difference Sex (% female) 59.1* 72.0* -12.9 Age 20.1* 21.3* -1.2 Single 79.1* 72.6* 6.5 Household size 6.2* 5.7* 0.5 Parent 32.7* 37.8* -5.1 Single parent 12.7 14.0* -1.3 Study 10.0* 9.8 0.2 Work 33.6* 22.6* 11.1 “NiNi” (neither work nor study) 60.0* 68.9* -8.9 VIP-RA score 10.5* 10.0 0.5 Secondary risk 10.9 9.1 1.8 Employability 54.2* 64.0* -9.9 Emotional regulation score 39.7* 36.3* 3.4* Depression score 16.4* 11.0* 5.5 PTSD score 26.0* 19.0* 7.0 Resilience score 86.9* 89.9* -3.0 * p<0.05 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 14 between groups. There are no large significant differences between employability levels on the component scores of the VIP-RA. 5.2 CONCLUSIONS The goal of this baseline report was twofold: first, to confirm that respondents from the treatment areas were, on average, similar to those in the control group; and second, to present baseline values for all major outcome variables, disaggregated by both treatment and control areas. A balance was found across most variables between treatment and control areas, which demonstrates that randomization was successful in its goal of creating two groups of respondents who are comparable on average. The balance tests hold for most of the important demographic, employability and violence￾involvement variables with a few notable exceptions. A higher proportion of women in the sample are parents and single parents, and women in the sample tend to be older than men. Parents in the sample also tend to be NiNis. However, both women and parents are balanced between treatment and control. Therefore, it is safe to conclude that, while interesting to track for any difference in impact these variables may have, they will not influence the results. Given these few differences between treatment and control groups, it seems that the randomization worked, insomuch that it resulted in two groups of people who are not different on existing characteristics. 5.3 NEXT STEPS Given that all communities did not start on the same timeline, and some have experienced delays, data collection going forward will be on a rolling basis. This will require close coordination between MESCLA and the implementing partners to ensure that treatment and control populations are surveyed at nearly the same time in each location. The following data collections remain: Table 6: Sample Characteristics by Employability Employability Low Medium High Sex (% female) 52.6* 70.0 72.3* Age 20.9* 19.3* 20.9* Single 81.6* 80.0* 72.3* Household size 5.9* 7.6* 5.8* Parent 26.3* 40.0 39.4* Single parent 9.2 20.0 14.9 Study 7.9 10.0 10.6* Work N/A N/A N/A “NiNi” (neither work nor study) 2.6 90.0* 89.4* Primary 48.7* 20.0 37.8* Secondary 51.3* 70.0 59.6* VIP-RA score 13.4* 9.4 9.0 Secondary risk 10.5* 10.0 9.6 Emotional regulation score 35.2* 41.5 38.5* Depression score 13.1* 18.0 12.9* PTSD score 21.6* 29.8 21.5* Resilience score 88.6* 85.9* 88.8* * p<0.05 Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 15 1) End Phase 1 (basic labor competencies, life skills) 2) End of Phase 2 (professional and technical training) 3) After insertion and during mentoring Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 16 6 ANNEX 1: SOURCES OF INFORMATION Abdalla, M., Barth, A., Dunn, A., Holter, A., Ortega, A., & Tinta, P. (2013). Youth Employability Evaluation Tool Validation. Education Development Center and Proyecto METAS Honduras: Washington, DC. Bloom, H., Orr, L., Bell, S., Cave, G., Doolittle, F., Lin, W., Bos, J. (1997). The benefits and costs of JTPA Title II-A programs: Key findings from the National Job Training Partnership Act study. Journal of Human Resources 32, 549–576. Cave, G., Bos, H., Doolittle, F., & Toussaint, C. (1993). JOBSTART. Final Report on a Program for School Dropouts. Manpower Demonstration Research Corporation: New York. Hennigan, K. H., Maxson, C. L., Sloane, D. C., Kolnick, K. A., & Vindel, F. (2014). Identifying high-risk youth for secondary gang prevention. Journal of Crime and Justice, 37(1), 104-128. Millenky, M., Bloom, D., Muller-Ravett, S., & Broadus, J. (2011). Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation. Manpower Demonstration Research Corporation: New York. Raphael, S. (2011). Controlling Crime: Strategies and Tradeoffs, P. J. Cook, J. Ludwig, J. McCrary, Eds. Chicago: University of Chicago Press. 521–565. Schochet, P., Burghardt, J., & McConnell S. (2008). Does Job Corps work? Impact findings from the National Job Corps Study. American Economist Review 98, 1864–1886. Uggen, C. (2000). Work as a turning point in the life course of criminals: A duration model of age, employment, and recidivism. American Sociological Review 65, 529–546. UNAH-IUDPAS (2016). Boletín Nacional Observatorio de la Violencia. Tegucigalpa. http://www.iudpas.org/pdf/Boletines/Nacional/NEd43EneSep2016.pdf UNODC (2016). UNODC Statistics. https://data.unodc.org. USAID Honduras. (2014). Employability Study: An Evaluation of METAS’ Basic Labor Competences training and Certification Program [Draft]. Washington: Education Development Center, Inc. USAID Honduras. (2015). Country Development Cooperation Strategy. USAIUSAID Honduras. (2016). Workforce Development Activity – Statement of Work. Monitoring & Evaluation Support for Collaborative Learning and Adapting (MESCLA) 17 U.S. Agency for International Development/Honduras Avenida La Paz, P.O. Box 3453 Frente a la Embajada Americana Tegucigalpa, Honduras www.usaid.gov